Data Scientist - Data Quality and Pipelines

About LogiX

LogiX empowers organizations to realize the full potential of their data by transforming business processes through comprehensive data strategies and AI-powered solutions. Collaborating with clients across diverse industries, LogiX designs and deploys data platforms that enable smarter decision-making and deliver tangible outcomes. At LogiX, data and AI are fundamental drivers of innovation, operational efficiency, and competitive advantage. The team engages closely with stakeholders to solve complex data challenges through scalable, practical implementations supporting sustainable growth.

Role Overview

This role focuses on developing robust data pipelines and engineering solutions that integrate data from various enterprise sources into a centralized governed platform. You will be responsible for constructing reliable ingestion and transformation pipelines, primarily utilizing Python, SQL, and Azure Data Factory, to process data sourced from SAP and Microsoft 365.

Key Responsibilities

• Develop and maintain dependable data ingestion and transformation pipelines using Python, SQL, and Azure Data Factory to integrate SAP and Microsoft 365 data into a centralized platform.
• Design and implement production-grade data quality measures including validation, reconciliation, anomaly detection, and quarantine processes to ensure end-to-end data accuracy and traceability.
• Conduct analytical investigations to identify trends, anomalies, and critical business drivers through statistical analysis, forecasting, and machine learning techniques for enhanced decision-making.

Required Expertise

• Minimum of seven years' practical experience in Azure Data Engineering, particularly with Azure Data Factory pipeline development, orchestration, monitoring, and troubleshooting.
• Proficient in SAP data integration, encompassing data extraction, integration, reconciliation, and deep understanding of SAP data structures and source-to-target data mapping.
• Experience in ingesting data from Microsoft 365 platforms and APIs, with familiarity ideally in Microsoft Graph API.
• Skilled in data quality engineering, including the creation of automated checks for data validation, reconciliation, completeness, consistency, and integrity.
• Expertise in data observability and exception handling techniques such as data freshness monitoring, anomaly detection, logging, alert systems, quarantine, reprocessing, and data traceability.
• Advanced hands-on abilities in Python and SQL for developing production-grade workflows covering data ingestion, transformation, quality validation, and analytics.
• Comprehensive knowledge of data lineage and auditability ensuring all data can be traced from source through transformations to final reports.
• Strong understanding of scalable ETL/ELT architectures, transformation logic, schema design, and analytical data modelling.
• Competence in applying statistical analysis and anomaly detection to explore patterns, outliers, variances, and underlying business drivers.
• Practical experience integrating forecasting and machine learning models into structured enterprise data scenarios to improve predictive capabilities.
• A production engineering mindset focused on transitioning data pipelines and analytical tools from prototypes to robust, monitored production systems.