At Credit Karma, I’m designing and developing a Python-MySQL Flask based REST API. It will be used by a web browser app called the CK Data Dictionary Navigator, and it will be Open Sourced. I won an employee award for designing the first version, and now we are adding the GUI and REST API after last year’s successful “socialization” of the Data Dictionary – which is a Distributed Data Dictionary and which catalogs every data source at CK – regardless of where it lives – locally, at a Co-lo or in the Cloud.
In years past I have designed such API’s using Stored Procedures – usually Oracle PL/SQL or the MySQL or PostgreSQL equivalent, but while SP’s do leverage the power of the database server, it makes the code database specific and requires that the user log into the database. With REST – its basically an HTTP call to get data.
The Data Dictionary is an internal product – and I have been very pleased at how important Enterprise Data Governance has become – in light of so many data breeches that we have seen of late – including the hacking of the US Democracy – and the violation of peoples personal information. The Data Dictionary lets security set alerts and alarms for any suspect data access, and for finance – it also lets people find duplication of effort and expensive wasteful data practices. Most companies now rent time on AWS, Google or others clouds and they pay by the terabyte. While disk is cheap – creating big data stores with no oversight can lead to security and run away cost issues.
I’m very happy and proud to have won an award for my work – and also to be right in the thick of building something that Credit Karma will Open Source most likely (hopefully at least) this year.