Research Engineer - Agentic Models

About the Role

At JetBrains, a company dedicated to building powerful developer tools since 2000, AI-driven assistance and agent technologies are integral to the evolution of our integrated development environments (IDEs). We are working on creating multi-step coding agents capable of comprehending extensive codebases, planning modifications, utilizing tools, and iterating interactively with users.

As a Research Engineer in the Agentic Models team, you will be responsible for developing and maintaining the models, training procedures, and evaluation frameworks that drive these advanced coding agents. Your work will blend supervised fine-tuning (SFT) and reinforcement learning (RL) post-training approaches, leveraging distributed GPU and MapReduce infrastructures to deploy models into JetBrains products.

Key Responsibilities

• Develop, implement, and sustain SFT and RL post-training pipelines for multi-step agent workflows.
• Train and customize large language models (LLMs) to support agent functionalities such as planning, tool invocation, and multi-step interactions within IDEs.
• Create and evolve evaluation and simulation platforms where coding agents are tested, measured, and benchmarked on realistic developer tasks.
• Design frameworks and metrics to assess agent behaviors, analyze detailed logs and traces, and integrate evaluation feedback into training data, reward systems, and model adjustments.
• Assess training outcomes and evaluation data to identify opportunities for enhancing model structures, training methods, and datasets.
• Manage large-scale infrastructure involving distributed GPU training and MapReduce-style data processing for large pre-training and fine-tuning datasets.
• Collaborate tightly with research, product, and infrastructure teams to translate strategic product objectives into practical models, experiments, and deployed features.

Candidate Qualifications

• Significant hands-on experience training LLMs—covering pre-training, fine-tuning, or post-training—in research or production environments.
• In-depth knowledge of contemporary deep learning frameworks like PyTorch, and familiarity with specialized LLM training systems such as Megatron, NeMo, or verl.
• Strong theoretical and implementation understanding of LLM architectures, tokenization approaches, data pipeline construction, batching strategies, mixed precision computing, distributed training, and debugging complex training processes.
• Proven ability to autonomously manage projects from concept or identified product challenge through iterative design, experimentation, implementation, and refinement.
• Product-focused outlook with a keen understanding of developer workflows, capable of translating product requirements and potential failure points into modeling and evaluation initiatives.
• Minimum of three years of professional Python programming experience, emphasizing clarity and maintainability within modern machine learning codebases.

Preferred Expertise

• Experience with ML orchestration and workflow management tools such as Kubeflow, Dagster, Airflow, ZenML, as well as job scheduling systems including Kubernetes or SLURM.
• Familiarity with managing large-scale data and training pipelines, including MapReduce clusters, multi-node GPU systems, or workloads involving over one million CPU/GPU hours.
• Competence in designing and maintaining evaluation pipelines for LLMs or agents, integrating metrics, dashboards, experiment tracking, and automatic regression testing.
• Development experience with AI agents, including tool-utilizing agents, planning modules, multi-step workflows, and knowledge of agentic frameworks or design patterns.
• Working knowledge of experiment tracking and observability platforms such as Weights & Biases, MLflow, or Langfuse.
• Expertise in optimizing inference performance and deploying streamlined models in production settings.

Additional Information

JetBrains is committed to fostering an inclusive and diverse workplace that embraces individuals irrespective of their background, identity, religion, age, accessibility requirements, or orientation. We believe exceptional ideas originate from anyone and anywhere.

Your application data will be handled following our Recruitment Privacy Policy.