Business Case: Data Lake for a chemical company
Company-wide data in one place
If countless data are stored in different places, the overview is quickly lost. And their usability. A chemical company has changed all that and commissioned us with the architecture of a Data Lake. The result is a central data storage location for optimal internal collaboration, evaluation and data analysis.
© Marco Verch, Drohnenfoto: Marina und Northerly Island, Licence CC BY 2.0
Information from very different sources flows into the Data Lake, where it can be retrieved in a meaningful bundled and structured manner.
Project requirements and tasks
Our client is a large corporation in the chemical industry. Here, huge amounts of data of all kinds accumulate - from sales figures and production costs to master data or weather statistics from the agricultural sector. To centralize this and many other data from countless sources in a unified storage system makes it much easier to work together within the company and to evaluate data across systems. A central data lake is the solution, from which every department can make use of appropriate reporting tools. Of course, data security and data governance are the top priority. Planning and operating such a data architecture is a challenge even for a large corporation. We help with our expertise in the areas of architecture and Big Data and with our hands-on mentality.
This is our contribution
In a broad-based cooperation, we have designed, planned and provided the Data Lake for our customer. In doing so, we not only designed the Data Lake according to the customer's requirements, but also tested the needs-based use within the framework of PoCs, hackathons and workshops and developed role-related training concepts. With this project our versatility once again paid off: The technical implementation required the use of different technologies such as Microsoft Azure, Amazon AWS, Cloudera or Snowflake - no problem for our architects and data engineers. For example, the Data Lake we designed is also compatible with SAP HANA.
The project at a glance
Project title:
-
Enterprise Data Lake
Industry:
-
Chemistry
Project partners:
-
Microsoft
-
Hortonworks
-
Consulting companies
Term:
-
20 months
Unser Beitrag:
- Beratung
- Analyse, Konzeption und Implementierung einer Data-Lake-Architektur in der Cloud
- PoCs, Hackathons, Schulungskonzepte
- Testing, Optimierung und Übergabe an den Kunden
Technologien:
- Microsoft Azure
- Amazon AWS
- SAP HANA
- Hadoop
- Cloudera
- Snowflake
- Microsoft SQL-Server
- Microsoft SQL-Data-Warehouse