I’m going on my 40th year as a Data Guy. I was an Oracle DBA for 20 years, then Data Engineer, followed by Data Architect and now I’m drifting into the last role and chapter of my career – Data Management.
When I was younger, all I cared about was the Data Technologies – data stores and data processing, and even data visualization.
HOWEVER, along the way I started getting really cynical – I knew scores and scores of data was useless and even bullshit. Either a system was slapped together where the data couldn’t physically be used, or correlations between different data was impossible. One company I worked for had no way of knowing how to find ALL customer data – they literally has about 7 different systems that viewed the customer differently, and I barely was able to design something that attempted to bridge the gaps. The funniest thing was that we had data scientists who used this shitty data and made big proclamations of this and that business improvement – and I knew it was totally impossible to discern any of that – and that they were just complete bullshitters.
More importantly, besides bad data quality, data security, the cost of cloud computing and personal information protection are very serious business. Its taken all these years for the IT Industry to finally get serious about this – probably driven by GDPR, CCPA and just plain “Total Cost of Ownership”. Besides – knowing more about your data means you can use it for a business advantage.
I’m right in the drivers seat of this new movement – and I figure I’ll code a little less – well – maybe do more SQL where I am doing more data analysis and less data “plumbing”. I’m much more interested in ensuring Data Quality and making sure the data is useful to the business than the technology, but even in this regard I still love the tech part – in fact, I’m loving Snowflake – surely the next Data Warehouse technology that I will be using.
Anyway, it feels great to have this last major push and “theme” in my career – I don’t know when I will retire, but I do know what I want to do “When I grow up”.