Implementation of the SZKHT data warehouse and data marts

Client Description

SZKHT (Szeged Environmental Management Nonprofit Ltd.) is a public service company dedicated to environmental protection and sustainable development in Szeged and its surroundings. Its main responsibilities include waste management, maintaining public cleanliness, and caring for green spaces. SZKHT aims to provide a clean and livable environment for residents while promoting environmental awareness and sustainable living. The company also actively participates in environmental education programs within local communities and schools.

Project General Description

The SZKHT launched a comprehensive data warehouse and data mart implementation project aimed at managing and analyzing environmental management data. This large-scale initiative integrates data from five companies into a unified controlling database, enabling more efficient decision-making and cost optimization. The project's goal is to create a flexible and scalable data infrastructure that not only meets current business needs but is also prepared for future challenges.

Let's talk!
Urban Management Sector
​ / ​
Environmental Management
​ / ​
Waste Management

Challenge

The project's goal is to create flexible, easily parameterizable data structures quickly by automating time-consuming tasks.

The system should be capable of automatically generating and configuring ETL (Extract, Transform, Load) processes, ensuring efficient data loading and updates. This significantly reduces the need for manual coding and minimizes the potential for errors.

A key feature of the project is the seamless integration of over 10 different source systems. This complex data integration process ensures that all relevant data sets are available on a central platform for the controlling department.

How we helped?

  • Design Phase
  • In the initial stage of the project, the focus was on easy parameterization, rapid development, and cost-effective operation. This led to the development of a unique data warehouse and data mart generator.
  • Data Warehouse and Data Mart Generator
  • This advanced tool enables the rapid and flexible creation of data structures by automating numerous time-consuming tasks. The generator employs intelligent algorithms to design the optimal data model, taking into account the characteristics of the input data and user requirements.
  • Data Structure Design
  • The generator analyzes input data sources and suggests the optimal data model to suit the business needs.
  • ETL Process Generation
  • Automatic creation of ETL scripts for efficient data loading and transformation, reducing manual intervention.
  • Creation of Data Marts
  • Specialized data marts are generated to meet the specific requirements of different business areas, ensuring targeted and relevant data availability.
  • Dependency Management
  • Intelligent algorithms determine the optimal sequence for loading processes, ensuring data integrity throughout the pipeline.
  • Parallel Processing
  • The system handles multiple data streams concurrently, minimizing load times with a scalable architecture that supports large volumes of data.
  • Resource Monitoring
  • Real-time monitoring and dynamic resource allocation maintain system stability and performance, ensuring smooth operation.
  • Scheduled Launch
  • Flexible scheduling options support both time-based and event-based triggers to meet various business requirements, allowing for automation and timely execution of tasks.
  • Pentaho Data Integrator
  • Java
  • Microsoft SQL Server