Data Engineer for SaaS solutions

Oferty pracy

Summary

The project is the creation of new solutions, such as intelligent personal assistants for enterprise engineers. These are SaaS solutions that operate with data and knowledge sources, analyzing them and using them to solve specific problems.

The customer is a company that provides the financial sector and various business areas with necessary analytical services, including data, expertise, and technology for making daily decisions.

Tech stack on the project: Python, Kafka, JSON, Protocol Buffers, Apache Avro, AWS, Google Cloud Platform, Apache Airflow, Luigi, Apache Oozie, Apache NiFi, Apache Beam, Make, Bash, CL tools.

Tasks:

  • Designing and building scalable and reliable data processing and transformation pipelines (designing ETL system for NLP/ML/DL and other projects);
  • Owning Data Engineering in projects in Deep Learning, Natural Language Processing, Data Capturing, Information Retrieval, and other areas;
  • Working together with Data Scientists, ML Engineers, and developers on building the intelligent capabilities for the company’s products;
  • Creating a robust design and best practices in Data Engineering.

Requirements

  • Experience as a Data Engineer for 3+ years;
  • Hands-on experience in building data processing systems using Kafka for 1+ years;
  • Experience with various data formats (JSON, Protocol Buffers, Avro, etc.) and schema validation tooling;
  • Experience with cloud platforms (AWS, GCP, etc.);
  • Experience with data processing automation tools, schedulers, and pipelines (Apache Airflow, Luigi, Apache Oozie, Apache NiFi, Apache Beam, Make, etc.);
  • Experience with Linux (Bash, CL tools);
  • Strong coding and software engineering skills;
  • Experience in Python programming for 2+ years;
  • Level of English – Intermediate.

Desired skills

  • Experience with Big Data processing tools (Spark, etc.);
  • Experience with search systems (Elasticsearch, Apache Lucene, Apache Solr);
  • Experience with RDF or OWL;
  • Experience with Big Data processing and analytics stack in AWS: EMR, S3, EC2, Athena, Kinesis, Lambda, QuickSight, etc.;
  • Experience in building distributed high-volume data services;
  • MS degree related to Computer Science, Data Science, or Statistics;
  • Experience in projects with Deep Learning, Natural Language Processing, or Information Retrieval;
  • Experience in ML.

Reasons to join us

Andersen is a pre-IPO software development company that provides a full cycle of services. For over 14 years, we have been helping enterprises and middle-sized firms worldwide transform their businesses by creating effective digital solutions using innovative technologies.

We welcome true specialists no matter what country they live in. Salaries at Andersen are pegged to the USD, and employees are provided with a social package and an extensive set of bonuses.

  • Cooperation with such businesses as Samsung, Johnson & Johnson, Ryanair, Europcar, TUI, Verivox, Media Markt, Shypple, etc. This project is just your beginning here — working with us means reliability and prospects;
  • Excellent teams with streamlined processes and an opportunity to change the project. There are also systems of mentoring and adaptation for each new employee;
  • Many different ways to grow: you can develop expertise in different business domains and improve as a specialist or a manager. Transparent performance review and assessment systems will allow you to determine your development path and plan your growth;
  • Flexible start of the working day: from 7 AM to 11 AM. You can telecommute, work at the office, or opt for a hybrid schedule — whatever is convenient for you;
  • Referral programs and an opportunity to additionally earn up to $1,500 per month by participating in the company's activities;
  • Access to the corporate training portal, where the entire knowledge base of the company is collected and which is constantly updated;
  • Such perks as private health insurance, English language courses, and certification compensation (AWS, PMP, etc.).

Będzie nam miło widzieć Cię w naszych szeregach

Nadal szukasz wymarzonej pracy w IT? Kliknij tutaj, aby zobaczyć pełną listę ofert pracy w firmie Andersen. Stale otwieramy nowe stanowiska!