Data Engineer CoE

Job Description

Data Engineer is required to join a team of experts to work with industry-leading technology within fast growing eCommerce company, Dubai office.- Data Engineer will work closely with stakeholders and Data Scientists.- He/she will be responsible for the quality, development and performance of data pipelines.- Data Engineer should be able to define the best way to ingest and aggregate required data and process it efficiently.- Should have excellent communication, analytical and problem-solving skills. REQUIREMENTS: Responsibilities:- Create and maintain optimal data pipeline architecture- Assemble large, complex data sets that meet functional / non-functional business requirements- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data technologies- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs- Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leaderQualifications:- 4+ years experience in similar role- MSc degree in Computer Science, Information Systems or another quantitative fieldREQUIREMENTS:- Working with data pipelines and architectures- Expertise with following management tools: Airflow, Luigi, etc.- Experience with stream-processing systems: Hadoop/Spark-Streaming, Kafka / Flink- Strong knowledge with scripting languages: Python, Java, Scala, etc.- Strong knowledge with automation tools e.g. Docker, Kubernetes, Openshift- Knowledge of search engines such as Elastic search- Cloud platforms and services: AWS or Azure, or GCP- Architecting on cloud computing environments (AWS, or Azure, or GCP)- Security, authentication and authorization- DevOps / DataOps and CI / CD frameworks

Keyskills :

About Company

HR Source Consulting is a premier Human Resources Consultancy that provide clients across the Middle East and North Africa (MENA) with expert human resources planning and solutions. By inference, we have the expertise to attract high calibre candidates that fill specialist roles in the key industry sectors with focus in:

• IT (Information Technology)


• Media & Communications


• FMCG and many others

We deliver to a multitude of client organisations across the MENA region from leading multi-nationals (MNCs) to SMEs and boutique businesses. We are your Regional HR partner with Global Reach .

We adopt tailor made services and solutions to ensure that our clients need is fully met .

We provide a multi-step consultancy approach to sourcing the ideal candidate, to ensure that your requirement is fully fulfilled and delivered to. Furthermore we offer tailor made HR Services and Solutions in the additional following areas:

• Permanent Recruitment Solutions


• Interim & Contract Recruitment Solutions


• Managed Services


• Salary Surveys


• Outplacement Services


• Relocation and Company setup advice

In short, HR Source Consulting is dedicated to delivering human resources solutions at every level (from Junior, Mid to Senior and C-Level Executives). This includes successful placements for every type and size of client organisation by sourcing candidates at all levels of seniority. We like to go the extra mile and serve our clients every need of HR and provide additional solutions and services from salary surveys, market intelligence, outplacements services and relocation and company set up advice.