Data Engineer

Summary

Andersen is hiring a Data Engineer for a project migrating data pipelines to a new platform, building analytics solutions, and ensuring secure data processing.

The customer is an international technology company focused on developing digital products and services for end users. The organization operates in a dynamic environment, leveraging modern technologies to create scalable, secure, and user-friendly solutions with a strong emphasis on innovation, automation, and customer experience.

The project is focused on migrating legacy pipelines to a new data platform and implementing an on-premise LakeHouse solution. It includes developing analytics applications and ensuring secure processing of sensitive data through anonymization and tokenization of PII.

Responsibilities

• Designing and maintaining end-to-end data pipelines for batch and real-time processing.
• Integrating data from multiple heterogeneous sources into a unified data platform.
• Developing scalable data ingestion and transformation processes.
• Optimizing data pipelines to improve performance, stability, and efficiency.
• Implementing large-scale data migration and data loading solutions.
• Ensuring data quality, validation, and compliance with defined data standards.
• Building data delivery layers to support downstream systems and applications.
• Collaborating with cross-functional teams to define data requirements and support analytics use cases.

Requirements

• Experience with on-premises and cloud infrastructure.
• Hands-on experience with Kubernetes and Docker.
• Strong SQL and Python/PySpark skills.
• Experience with Apache Spark, Spark Structured Streaming, and Kafka.
• Familiarity with JupyterHub or similar development environments.
• Experience with AWS (S3, Lambda, DynamoDB, API Gateway, Athena) and GCP BigQuery.
• Experience with object storage solutions (AWS S3, Dell ECS).
• Strong knowledge of RDBMS (PostgreSQL, MS SQL, Oracle, StarRocks) and NoSQL databases (DynamoDB).
• Understanding of relational vs non-relational data models (ACID vs BASE).
• Experience with Delta Lake, Apache Iceberg, HDFS, and Hive Metastore.
• Knowledge of partitioning and data lakehouse design patterns.
• Experience with Apache Airflow and DAG-based pipeline orchestration.
• Understanding of data modeling concepts (Star Schema, SCD Type 2).
• Familiarity with data quality, validation, schema enforcement, and data governance principles.
• Level of English – from Intermediate+ and above.

Desired skills

• Exposure to Elasticsearch and Grafana.
• Exposure to dbt.
• Familiarity with CDC pipelines (Debezium + Kafka) as part of event-driven architecture.
• Familiarity with YAML-based infrastructure configuration (e.g., Kubernetes manifests).
• Basic understanding of GDPR, PII handling, and banking/compliance regulations (e.g., Central Bank requirements).

Reasons to join us

• Experience in teamwork with leaders in FinTech, Healthcare, Retail, Telecom, and others. Andersen cooperates with such businesses as Samsung, Siemens, Johnson & Johnson, BNP Paribas, Ryanair, Mercedes, TUI, Verivox, Allianz, T-Systems, etc.;
• The opportunity to change the project and/or develop expertise in an interesting business domain;
• Job conditions – you can work both fully remotely and from the office or can choose a hybrid variant;
• Guarantee of professional, financial, and career growth! The company has introduced systems of mentoring and adaptation for each new employee;
• The opportunity to earn up to an additional 1,000 EUR per month, depending on the level of expertise, which will be included in the annual bonus, by participating in the company's activities;
• Access to the corporate training portal, where the entire knowledge base of the company is collected and which is constantly updated;
• Bright corporate life (parties / pizza days / PlayStation / fruits / coffee / snacks / movies);
• Certification compensation (AWS, PMP, etc);
• Referral program;
• English courses;
• Private health insurance and compensation for sports activities.

Join us!

Lokalizacje

Worldwide