Data Engineer

Job Description

Qualification (minimum):?Master of Science in Data Analytics or bachelor?s degree in computer science/engineering,?Certified Data Engineer with IBM/Google Cloud Certified/Azure?AWS Certified Solutions Architect?Databricks Certified Data Engineer Associate.Experience (minimum):?10+ years of experience in designing in Data Architectures, Big Data Analytics, ETL processes, Data Governance and Security.& Responsibilities:?Design and implement efficient database structures to store and manage exploration, production, and operational data.?Design, develop, and maintain scalable data pipelines to handle large volumes of data from various sources such as drilling rigs, production facilities, and sensors, Implement ETL (Extract, Transform, Load) processes to ensure data quality and integrity.?Integrate data from multiple sources including OT System, IoT devices, SCADA systems, and geospatial data with business data.?Ensure data consistency, accuracy, and availability across different platforms and systems., Translate business requirements into technical specifications for data infrastructure.?Implement best practices for Digital solution, data governance and compliance, Analyze the risks, benefits and opportunities associated with Digital architectural strategies, and balanced approach with a focus on operation functionality.?Introduce Digital solution with advanced analytics, machine learning, and AI initiatives by ensuring that data is well-prepared and accessible for these advanced use cases.?Align technology solutions with the technical architecture, operational processes and lifecycle processes.?Work closely with IT analysts, business stakeholders to understand data needs and provide technical solutions.?Work closely with IT Team to advance their application strategies by identifying and managing application dependencies, aligning architecture to cross business demands, and identifying opportunities to leverage technology and applications data across company.?Work on a scalable data versioning system to manage datasets efficiently for historical analyses and compliance.?Supervise the development and launch of an interactive data exploration tool that served 500+ internal users, increasing productivity and insights.?Conceived and enforced data governance policies, making compliance with UAE IA and ADHICS seamless, mitigating risk and improving data security posture.?Accomplishes organizational goals by accepting ownership for accomplishing new and different requests and exploring opportunities to add value to job accomplishments.Skills:?SQL for data manipulation and retrieval.?Python or R for data analysis and scripting and Apache Hadoop for handling big data.?Apache Spark for big data processing. ETL (Extract, Transform, Load) tools knowledge.?Data warehousing solutions like Redshift, Big Query, or Snowflake?Machine learning algorithms understanding Data pipeline and workflow management tools like (Airflow or Luigi)?Cloud services like AWS, Google Cloud Platform, Microsoft Azure and Oracle.?Docker and Kubernetes for containerization and orchestration.?Strong problem-solving skills and ability to apply these skills to identify and resolve anomalies in system functionality and performance.Key area of experience:?Programming & Scripting: Python (Pandas, NumPy), Java, Scala, Bash scripting, SQL, NoSQL.?Data Engineering Tools: Apache Hadoop, Apache Spark, Apache Kafka, Apache Airflow, Docker.?Data Visualization & BI Tools: Tableau, Power BI, Looker, QlikView, D3.js, Google Data Studio.?Database Management: MySQL, PostgreSQL, MongoDB, Microsoft SQL Server, Redis.?Cloud Computing: AWS (EC2, S3, RDS, Redshift), Google Cloud Platform (Big Query, Compute Engine), Azure Data Factory.?Ensure security practices are followed throughout the acquisition process and produce technical documentation.?Communicate complex information, concepts, or ideas in a confident and well-organized manner through verbal, written, and/or visual means.?Function in a collaborative environment, seeking continuous consultation with other staffs and experts.?Implement new process Company Profile TYCHE HR Consultancy is a leading human resource consultancy specializing in recruitment, sourcing, headhunting for permanent placements, outsourcing, local, and overseas hiring. Our team of experienced professionals works tirelessly to match the right talent with the right opportunities, helping businesses thrive and individuals succeed. With a strong network and an innovative approach to talent acquisition, we provide top-notch HR solutions tailored to our clients unique needs. Whether you are looking to build your dream team or take the next step in your career, TYCHE HR Consultancy is here to support you every step of the way.

Keyskills :
Implement ETL (Extract Transform Load) processes to ensure data quality and integrity. Integrate data from multiple sources including OT System IoT devices SCADA systems and geospatial data with business data