Senior Data Quality Engineer

About the Role

The Emirates Group’s Analytics Centre of Excellence (ACoE) serves as a centralized hub providing robust data and analytics support to various Emirates Group divisions. Their mission is to empower better decision-making through insightful data analysis on customers, operations, and market trends. This unit is integral to the Group’s digital transformation journey, aiming to become a data-centric organization.

As a Senior Data Quality & Automation Engineer, you will be an active contributor within a cross-functional team, taking autonomous responsibility for developing technology solutions and troubleshooting in the Enterprise Data & Analytics domain. Your role will involve applying quality assurance practices, designing and implementing data quality control and automation platforms, and supporting technical analytics products that benefit Emirates Airlines and associated businesses.

Key Responsibilities

• Collaborate closely with product owners, analysts, software engineers, and architects to grasp the technical environment and delivery context, refining both functional and non-functional requirements, and translating them into suitable acceptance criteria.
• Address complex problems involving data analysis by evaluating identifiable factors and selecting appropriate methods for validation and automation.
• Develop comprehensive test strategies and plans that verify core business challenges via functional, data quality, performance, and security tests.
• Create and maintain test data, including generating or mocking data for exploratory analysis and test execution.
• Design and implement automated tests or jobs at moderate to high complexity across various stages of the data pipeline, ensuring adherence to coding standards and best design practices.
• Enhance and maintain data quality rules and observability platforms monitoring the data pipeline health.
• Investigate and resolve complex technical issues independently, following design documentation with minimal supervision.
• Perform data analysis activities including source system evaluation, data modeling, data dictionary creation, data profiling, and source-to-target mapping to ensure alignment with business needs.
• Update data inventories and registries as required, maintaining current metadata and data lineage in accordance with Data Governance standards, principles, and guidelines.

Qualifications and Experience

• A degree in Computer Science, Computational Mathematics, Computer Engineering, Software Engineering, or a related field.
• Electives or specialization in Data & Analytics, such as Data Warehousing, Data Science, or Business Intelligence, are advantageous.
• Minimum of 2 years’ experience in Data Engineering with a focus on data quality and automation.
• At least 2 years’ experience in testing, automation, and support within analytics environments like Data Lakes and Data Warehouses, preferably utilizing Big Data technologies and Microsoft Azure cloud services.
• Proven ability to identify quality issues in complex data pipelines operating on big data platforms and to define validation rules for data health assessment.
• Familiarity with diverse testing methodologies covering functional, performance, and security aspects at job, pipeline, and enterprise levels.
• Experience creating automated validation checks integrated within CI pipelines to verify ETL/ELT job performance.
• Knowledge of both batch and real-time data ingestion and integration technologies handling large volumes of structured and unstructured data.
• Understanding of data architecture concepts such as data modeling, Big Data storage, and dimensional modeling.
• Experience working with data pipeline jobs and defining metrics to ensure data correctness.
• Proficiency in programming languages such as Python or Scala and SQL query writing.
• Exposure to Spark technology and airline industry experience is a plus.

Core Skills & Attributes

• Independently drive improvement in data asset quality.
• Deliver iterative solutions demonstrating tangible value.
• Strong aptitude for data analysis, including source system identification, metadata collection, profiling, and mapping.
• Accountable for the products they develop, following a “You Code It, You Own It” approach.
• Collaborative team player adept at removing obstacles, tackling design challenges, and debugging issues.
• Possesses accountability and a positive work attitude.
• Self-motivated with a passion for exploring new technologies, especially those related to Enterprise Data & Analytics.

Technical Environment & Tools

• Big Data and distributed processing: Spark, Hadoop ecosystem (HDFS, Hive, HBase, Oozie), Airflow, Apache NiFi, Azure Data Lake Storage (ADLS), Azure DataBricks, Azure Data Factory, Elasticsearch, AVRO and PARQUET file formats.
• Data analysis, modeling, and reporting tools: Snowflake, SQL, Data Vault 2.0, MicroStrategy, Power BI.
• Cloud environments: Microsoft Azure, Cloudera stacks.
• Integration and messaging systems: Spark Streaming, SnapLogic, TIBCO, Kafka.
• CI/CD platforms and tools: GIT, Bitbucket, Jenkins, Azure DevOps, Kubernetes, Docker, SonarQube, Gatling.
• Programming languages: Scala, Python.

Additional Information: Salary and Benefits

Based in Dubai, this role offers a competitive tax-free salary package along with exclusive travel benefits, including discounts on global flights and hotel accommodations.