Data Engineer (Abu Dhabi, UAE)

Summary

Andersen is hiring a Data Engineer in Abu Dhabi (on-site) for a project developing scalable data platforms, pipelines, and real-time processing to support digital transformation and analytics.

The customer is a public-sector organization responsible for advancing digital innovation and modernizing services across multiple domains. It supports strategic initiatives in emerging technologies, collaborates with industry and research partners, and develops long-term technology frameworks for improving operational efficiency and service accessibility. The organization focuses on secure, scalable, and sustainable digital development to drive continuous improvement within the broader ecosystem.

The project focuses on digital transformation in the UAE to develop and support digital solutions, enhance social services, strengthen public engagement, advance international relations, and optimize administrative processes.

Responsibilities

• Designing and managing data pipelines, ETL processes, and data storage solutions.
• Working with SQL, NoSQL, and big data technologies like Hadoop, Spark, and Kafka.
• Ensuring data quality, governance, and compliance with security standards.
• Collaborating with AI Engineers to prepare and process datasets for machine learning.
• Developing and optimizing scalable data architectures to support large-scale processing and analytics.
• Automating ETL/ELT workflows using tools like Apache Airflow, dbt, or Prefect.
• Implementing data streaming solutions for real-time processing using Kafka, Flink, or Spark Streaming.
• Ensuring data security and compliance by implementing access controls, encryption, and monitoring.
• Monitoring and maintaining data pipeline performance, troubleshooting issues and optimizing workflows.
• Building and maintaining data warehouses and data lakes using Snowflake, BigQuery, Redshift, or Delta Lake.
• Enabling self-service data access by designing and managing data catalogs and metadata systems.
• Optimizing query performance and indexing strategies for efficient data retrieval.
• Managing data versioning and lineage tracking using tools like Apache Atlas or DataHub.
• Implementing CI/CD for data pipelines to ensure rapid and reliable deployments.
• Collaborating with DevOps teams to containerize and deploy data services using Docker and Kubernetes.
• Supporting data scientists and analysts by providing well-structured, accessible, and reliable datasets.

Requirements

• Experience as a Data Engineer for 4+ years.
• Strong experience with SQL and NoSQL databases (PostgreSQL, MySQL, MongoDB, Cassandra, Redis).
• Expertise in big data technologies (Hadoop, Spark, Kafka, Flink).
• Hands-on experience with ETL/ELT tools (Apache Airflow, dbt, Talend, Informatica).
• Proficiency in Python for data processing.
• Experience with cloud data platforms (Azure Synapse).
• Understanding of data modeling, indexing, and partitioning for performance optimization.
• Knowledge of data lake architectures (Delta Lake, Iceberg, Hudi).
• Familiarity with data security best practices (RBAC, encryption, GDPR, HIPAA compliance).
• Experience with data lineage, metadata management, and observability tools.
• Understanding of MLOps and data preparation for AI/ML applications.
• Level of English – from Intermediate+ and above.

Reasons to join us

• Experience in teamwork with leaders in FinTech, Healthcare, Retail, Telecom, and others. Andersen cooperates with such businesses as Samsung, Siemens, Johnson & Johnson, BNP Paribas, Ryanair, Mercedes, TUI, Verivox, Allianz, T-Systems, etc..
• The opportunity to change the project and/or develop expertise in an interesting business domain.
• Guarantee of professional, financial, and career growth! The company has introduced systems of mentoring and adaptation for each new employee.
• The opportunity to earn up to an additional 1,000 USD per month, depending on the level of expertise, which will be included in the annual bonus, by participating in the company's activities.
• Access to the corporate training portal, where the entire knowledge base of the company is collected and which is constantly updated.
• Bright corporate life (parties / pizza days / PlayStation / fruits / coffee / snacks / movies).
• Certification compensation (AWS, PMP, etc).
• Referral program.
• English courses.
• Private health insurance and compensation for sports activities.

Join us!

Lokalizacje

The United Arab Emirates