Data Engineer (Real-Time Big Data)

Описание GiG’s Data department develops and supports real-time big data projects, using information to support strategic decision-making and organisational growth.

Задачи • Design, create, and maintain ETL processes, data integrations, and data pipelines • Ensure seamless handling of large volumes of data • Develop high-performance services to process, store, and analyse large datasets using optimised code • Design and implement data validation and quality checks tailored to business needs • Troubleshoot live production systems and resolve issues • Test and implement new technologies relevant to current and future projects • Review code, promote best practices, and implement robust solutions across big data projects

Требования • Strong expertise in big data technologies such as Spark, Kafka, Nifi, ClickHouse, and big data datastores • Strong understanding of schema design methodologies and data models • Expertise in extracting large datasets using SQL and familiarity with SQL databases • Object-oriented programming skills, ideally in Python and/or Java • Strong experience with cloud computing and/or distributed systems

Условия • Great career development opportunities • International health insurance • Health and Wellbeing Package (350 EUR per year) • Birthday day off • Me Time: 1 day off per year