Data Engineer (Python, Agentic AI, AWS) - Chennai

Role Overview

We are looking for an experienced Data Engineer to join our Retail and Wealth Risk Engineering team within the Enterprise Risk Technology platform. Based in Chennai, this role focuses on designing and implementing scalable, secure AI agent deployments and advanced data pipeline solutions leveraging cutting-edge technologies like AI, Big Data, Databricks, and cloud infrastructure.

Key Responsibilities

• Architect, develop, and sustain scalable AI agents and ELT/ETL pipelines using Python, FAST API, Microservices, PySpark, Kafka, and Databricks ecosystems.
• Develop and deploy Generative AI agents utilizing Google ADK and Google Flash 2.5+ LLMs to automate applications and provide intelligent workflow support with human-in-the-loop systems.
• Construct and manage data federation layers for Lambda and Data Mesh architectures employing tools like Starburst, enhancing AI use cases such as ML, deep learning, and NLP to boost efficiency.
• Automate microservice integrations ensuring scalability, robustness, and maintainability through cloud-native platforms, including OpenShift or Kubernetes with CI/CD pipelines.
• Integrate advanced agentic AI tools such as Devin.AI and GitHub Copilot with sophisticated prompt engineering techniques to improve development and operational workflows.
• Maintain data quality, consistency, and security throughout all data lifecycle stages.
• Contribute actively to refining data engineering methods, standards, and best practices within the team.
• Assess risks prudently during business decision-making, ensuring compliance with policies, ethical guidelines, and regulatory frameworks, protecting the organization’s reputation and assets.

Qualifications

• Minimum 8 years of experience in large-scale software application development with a strong focus on AI agent deployment platforms.
• At least 5 years in a lead engineering role specializing in Python and PySpark for enterprise-level ELT/ETL workloads using Databricks.
• Proficient in agentic AI development employing YAML, JSON, FAST API or Spring Boot, Google ADK, LLM integrations, and platforms like MCP with advanced prompt engineering.
• Expertise in building and automating microservice integrations for data-intensive application environments.
• Competent in programming languages essential for data engineering such as Python or Scala.
• Strong SQL and relational database management skills.
• In-depth knowledge of data modeling, warehousing, Data Mesh, and data federation.
• Excellent communication, collaboration, and analytical problem-solving capabilities.
• Preferred experience includes cloud Big Data services (Cloudera, AWS, Azure, GCP), frontend frameworks (Angular, React), AI/ML application in business solutions, containerization tools (Docker, Kubernetes), banking domain exposure, and industry certifications (AWS Big Data Specialty, Azure Data Engineer Associate).
• Educational background requires a Bachelor’s degree in Computer Science, Engineering, or related fields; Masters degree is advantageous.

Additional Details

• Position available: 2 openings
• Location: Chennai, India
• Interview Process: Karate Test plus two technical or managerial rounds