By Johan Snels – Data Engineer
Hello fans of the farm! In our previous blog, we used librarians and heroine to explain what a data platform actually is. I’m convinced every company could use a data platform in some sense. Easy for me to say since we are in the data platform selling business. That’s why I’ll let you make up your own mind and give you some indicators which, if present, should convince you to go to our website and order one of those state-of-the-art one day data platforms.
Mind-numbing manual reporting slavery
You know the feeling on a Friday afternoon and your boss Steve comes in: “Hi Chelsea, could you please quickly update our financial reports and include an update on our running projects? I have a management meeting at 4 pm”. Steve is a great guy, but Steve does not know the amount of manual exports and repetitive clicks you need to perform in order to provide him with these reports. After the tenth time this year, you start doubting whether you should give him the reports or your six weeks’ notice. And rightly so. People should not be asked to perform this kind of slavery. Imagine a tool of some sort that automatically gathers the necessary information from the different data sources. Additionally, it has built-in data pipelines that combine all relevant data points. Finally, all this is fed into your real-time dashboards. You guessed it! That’s what a data platform can do for you. Reports built once, available always.
No overview, no party
If you work in a company that likes to make a bit of profit once in a while, you’ll probably have multiple projects running. Let’s take the example of a construction company which has several ongoing building sites. Each site with its own customer data, invoicing, subcontractors and planning. Most of the time, the person responsible for invoicing is not the same as the one who is in charge of planning. This can become quite a mess if you’re working in a mid-sized company, let alone in a multinational enterprise environment. If it is your job to keep an overview of who is responsible for which data input and which person has access to what type of information, I invite you to read my upcoming blog: “10 signs of burnout”. Well-designed data platforms automatically scan your data and keep a data catalog. This basically is an organized inventory of all data assets in your organization. Easy peasy, no reason to burn anything.
Excel rules your world
Although bashing Microsoft Excel is one of my favorite spare time activities, I have to put some things in perspective first. Excel is the most successful tool in history to bring data analysis to the general public. Kudos to my friend Bill. The concept of performing dynamic calculations in an accessible way has dramatically increased efficiency for almost every company in existence. Hoooooowever, some people are just taking it too far. A stockpile of Excel files saved on a cloud storage (or worse, your local drive) must not serve as the company’s database. This is detrimental to keeping an overview of your data. No overview, no party.
Nor should any business-critical processes depend on Excel files. Every company has a Macro Mike nowadays. Macro Mike is able to write fancy macros in VBA. Production planning rolls out with the push of a button. Dandy! But Macro Mike forgets to put in place rigorous testing for his VBA logic so your company’s processes depend on untested software. Excel is not an IDE (Integrated Development Environment) and is not made for writing enterprise software. Nevertheless, Excel is great for exploratory data analysis. Use it for that purpose and nothing else. Data platforms not only combine all of your data in the same place, but also provide an environment which can process and transform your data in any way you want (and yes, it also spits out production plans if you ask nicely).
A plethora of API connections
We’ve already referred to this problem in our previous blog. We notice that the majority of companies find themselves in a situation where most of their processes are digitized in different software packages. Nearly all companies have an ERP system, CRM tools, accounting software, a planning solution etc. The default decision is to let these software packages exchange data via underlying API connections.
On the one hand, this poses a risk since there’s no master which guards a single version of the truth. If all software packages ingest data sources and transform them in their own way, it’s going to be very difficult to maintain proper data definitions. On the other hand, this system only works for companies that are stagnating or shrinking in size. If your ambition is to grow, your IT architecture needs to be able to scale accordingly. With your own dark web of API connections, you’ll soon lose oversight of what is what. Not convinced? Let’s talk again in five years!
In our next blog, we will give you more information on the concept “serverless” and discuss the advantages of a serverless Data Platform. If you’re excited about our content, make sure to drop us a follow on LinkedIn and stay informed about the next blog in this series. Interested in how a data platform would look like for your organization? Book a meeting with one of our data architects and we’ll tell you all about it!