Building a Data Platform on AWS from Scratch: Part 1
The CIO of a Chicago-based global consultancy contacted SPR to build out a data platform. While the core platform was to be built by the corporate IT organization run by the CIO, the goal was that this platform would be used by other organizations outside corporate IT as well to centralize data assets and data analysis across the company using a common architecture, building on top of it to meet the use case needs of each organization.
From the perspective of the client CIO, what we were to build is a data insights platform. In other words, a platform that enables data analysis leading to actionable, data-driven findings which create business value, a step beyond the given of analytics which simply makes sense of data by uncovering meaningful trends, but does not necessarily lead to business value. As such, from the perspective of our team, the purpose of this platform was to enable such insights to take place.
As with many established firms, use of Microsoft Excel was commonplace, with spreadsheets commonly distributed within and across organizations, as well as between the firm and external clients. Additionally, business units and consultancy practices had become siloed, each making use of disparate processes and tooling. So in addition to centralizing data assets and data analysis, another goal was to implement the concept of data ownership, and enable the sharing of data across organizations in a secure, consistent manner.