Azure Databricks Developer
Kolkata, West Bengal, India · Full Time
Be the first to apply
- Experience
- 5–7 yrs
- Salary
- —
- Openings
- 1
- Posted
- 14 hours ago
- Work mode
- In office
- Education
- Any graduate
- Eligibility
- Applicants must have completed graduation in any discipline to be considered eligible for this role.
- Resume
- Required to apply
Where you'll work
Sign in to tell us what does and doesn't work for you here — it sharpens every match we show you.
Job description
Role Overview
This position requires advanced expertise in Azure data lake architectures and associated Azure cloud services. The successful candidate will contribute to designing and implementing data pipelines, utilizing Azure Data Factory, Databricks, Synapse, and related tools to architect efficient ELT solutions.
Key Responsibilities
- Develop and maintain ELT pipelines by leveraging Azure Data Factory and Azure Databricks Autoloader and Notebook scripting.
- Implement copy activities and data flow tasks within Azure Synapse Analytics.
- Design metadata-driven pipelines with dynamic logic for efficient data processing and management.
- Apply data transformation techniques using Spark and SQL technologies within the Azure ecosystem.
- Maintain code repositories and apply version control processes through Git or equivalent tools.
- Ensure data quality by applying profiling, validation, and cleansing methodologies.
- Engage in debugging and problem-solving activities within ETL workflows and data pipeline operations.
- Implement CI/CD pipelines using Azure DevOps principles to maintain continuous integration and deployment.
Skills and Expertise Required
- Strong understanding of Data Lake architecture, specifically Azure Data Lake Storage (ADLS).
- Hands-on experience with Azure Data Factory (ADF) and Azure Databricks (ADB), including Delta Lake, Delta Live Tables (DLT), and the Lakehouse paradigm.
- Proficiency in languages and frameworks including PySpark and Python for data pipeline development.
- Knowledge of Azure Synapse and Azure Serverless SQL Pool for data warehousing and querying.
- Familiarity with metadata management and pipeline orchestration.
- Experience with Git-based version control systems.
- Understanding of Azure DevOps CI/CD integration workflows.
Additional Information
This opportunity is based in Kolkata and requires candidates with 5 to 7 years of relevant experience. Graduates in any discipline are eligible to apply.
Minimum education
Bachelor's Degree