AWS Databricks Spark Scala Developer

About the Opportunity

Tata Consultancy Services (TCS), a leading global IT and business solutions provider, is hiring an AWS Databricks Spark Scala Developer with 7 to 10 years of experience. TCS operates in 55 countries, employing over 612,000 professionals and generating a revenue of USD 29.08 billion (2023-24). This role is based in Bangalore or Hyderabad and requires working from the office.

Key Responsibilities

• Design, develop, and maintain scalable data processing systems using Scala and Apache Spark.
• Create and deploy JAR applications within Databricks and AWS environments.
• Construct ETL/ELT pipelines to process large-scale structured and unstructured datasets.
• Optimize Spark jobs to enhance performance, ensure scalability, and improve reliability.
• Implement effective data engineering solutions leveraging Databricks on AWS.
• Collaborate with business analysts, architects, and other stakeholders to gather requirements and deliver solutions.
• Monitor, diagnose, and resolve production issues related to data pipelines.
• Conduct code reviews to maintain coding standards and promote best practices.

Required Skills & Experience

• Proficiency in Scala programming language.
• Hands-on experience with Apache Spark including Core, SQL, and DataFrames.
• Practical experience working with Databricks platform.
• Strong knowledge of AWS cloud services such as S3, EC2, Lambda, IAM, and Glue.
• Expertise in developing and deploying JAR files.
• Strong SQL capabilities and understanding of database fundamentals.
• Experience with ETL processes and data warehousing concepts.
• Familiarity with Git and Continuous Integration/Continuous Deployment (CI/CD) tools.
• Understanding of distributed computing architectures and performance tuning techniques.
• Excellent debugging and problem-solving skills.

Preferred Skills

• Experience working with Delta Lake technology.
• Knowledge of Apache Kafka for data streaming.
• Exposure to PySpark framework.
• Understanding of workflow orchestration tools like Airflow or AWS Step Functions.
• Background in DevOps practices and Infrastructure as Code (IaC).
• Awareness of data governance and security protocols.

Additional Information

• The role requires physical presence at the office location (Bangalore or Hyderabad).
• Interview process will be conducted virtually.
• Applicants must be able to join within 30 to 45 days.
• Applicants should be graduates from any discipline.