Work with Head of Data Management to ensure integrity and consistency of data and that underlying data infrastructure is up to the standard.
Support Head of Data Management with collating data for proposals and recommendations.
Understand complexity of operations and data collected and propose methods to improve architecture gaps for data collection, aggregation, and exposition for different analytics purposes.
Work with vendors and consultants to understand their infrastructure tools and methodologies.
Integrate and maintain data sources into the Company Data Lake.
Create optimised Data Marts for specific business use cases that will be feed Company BI platform (e.g. Tableau, Looker, Power BI)
Create data models that are required for the business
Work with Company IT Operations to maintain the data infrastructure (e.g. Data Lake)
Ensure all data pipelines are working and data lineage is accurate.
Ensure all SLAs for the data pipelines are all met in time.
Deploy ML models in production environment.
Ensure all new and old queries within Data Lake pipelines are optimised for efficient performance.
Ensure that all automated reports (daily, weekly and monthly) are all up and running.
Perform detailed research on latest ETLs/ELTs frameworks and languages that can support business requirements.
Design, build, and maintain communication between services using APIs and REST APIs.
Test and maintain data infrastructure by performing stress tests, UATs, etc.
