Fielmann AG, Controlling Analytics – Definition and implementation of an AWS Analytics Lakehouse architecture.

The challenge:

Via an omnichannel business model with digital sales channels, the Central European market leader supplies both brick-and-mortar and online retail. In different departments across the entire customer organization, different product teams work on the collection, evaluation and provision of data from the sales and eCommerce systems.

The Fielmann Group has established a central data analytics team that provides various evaluations and dashboards within the customer organization in an easily accessible and usable way. The aim is to enable the business to make its decisions data- and insight-driven. Here, the “one-stop store” idea is pursued of making all relevant data available at a central location. Depending on the skills of the analysts, there are opportunities here to prepare data themselves, to carry out efficient ad-hoc evaluations, to create management reports or to gain new insights with machine learning.

The omni-channel business model and the associated digital transformation require an agile, scalable and high-performance analytics architecture. In addition to new data sources (data mesh architectures, streaming data, APIs), legacy applications (classic databases) must also be integrated efficiently as part of mapping the entire customer journey on- and offline. Furthermore, the requirements for the availability of information from a wide range of stakeholders (controlling & finance, sales, marketing, logistics, etc.) must be mapped.

TheImplementation:

Implementation of the Architecture: “Analytics Lakehouse”

PROTOS Technologie GmbH supports the Analytics division in planning and automated provisioning of the AWS infrastructure for various ETL pipelines, provisioning of ready-to-consume data sources as well as with explorative data analyzes in order to optimally provide stakeholders with enriched data. PROTOS supports the migration of existing infrastructure components, data services, pipelines, and data artifacts for the new architecture.

In order to implement the processing with Big Data frameworks and modern database technologies sustainably and for the most diverse data producers or consumers, previous analytical data pipelines are migrated and converted from a data warehouse concept to an “analytics lakehouse architecture” based on AWS services (AWS S3, AWS Redshift, AWS Glue, AWS Lambda). In addition, PROTOS supports with components for an easy and efficient handling of data across departments, such as dbt for data provisioning and state-of-the-art reporting tools for visualization and adhoc analysis.

Provision of the Infrastructure: “Automation”

The modernized infrastructure is deployed using Infrastructure as Code (IaC), making it consistent and under version control. AWS CDK and hashicorp Terraform are used for this purpose. By using AWS managed services such as AWS Codebuild, the CICD infrastructure does not need to be manually maintained.

The automated deployments for the individual components as well as the customization of infrastructure and ETL jobs are built using DevOps best practices (AWS code build + code pipeline, GitHub Actions). The high level of automation and potential of serverless/elastic cloud architectures is reinforced by the use of AWS Elastic Container Service (ECS), AWS Lambda in conjunction with AWS StepFunctions.

Implementation of data pipelines

To enable different data consumers to work efficiently with the data, the focus is on high quality data by enriching it and putting it into an analytics-ready structure. PROTOS supports the analytics team at Fielmann, particularly in the development of data pipelines and ETL jobs. Spark is used on AWS Glue, AWS Redshift and in combination with AWS Lambda. This provides the product teams with ready-to-consume data sources.

Further Information:

For more information on cloud, infrastructure-as-code, terraform, serverless and DevOps, feel free to check out the PROTOS technology blog.

Your PROTOS team

More Reading

Post navigation