Staff Software Engineer, Inference

About Anthropic

Anthropic is dedicated to developing AI systems that are reliable, interpretable, and controllable with the goal of ensuring AI benefits both users and society broadly. The organization comprises a rapidly expanding group of researchers, engineers, policy specialists, and leaders committed to building safe, beneficial AI.

Role Overview

The Inference team manages the foundational systems that support Claude, Anthropic's AI service used by millions globally. They operate one of the largest compute-agnostic AI inference deployments in the industry, covering everything from intelligent request routing to fleet orchestration across diverse AI hardware.

The team balances maximizing compute efficiency to accommodate rapid user growth and enabling advanced research by providing scientists with high-performance infrastructure for next-gen model development. They work on complex distributed systems spanning multiple hardware accelerators and cloud platforms.

As a Staff Software Engineer in the Inference team, you will identify and resolve infrastructure challenges to deliver Claude to users while facilitating breakthrough AI research. Candidates should have strong backgrounds in distributed systems, performance tuning, service orchestration, and smart request routing. Familiarity with large language model (LLM) inference optimization and multi-accelerator deployment is a plus.

Key Experience and Skills

• Building and scaling high-performance distributed systems
• Implementing large-scale machine learning deployments
• Expertise in load balancing, request routing, and traffic management
• Knowledge of LLM inference optimization, batching, and caching
• Experience with Kubernetes and public cloud platforms such as AWS or GCP
• Programming proficiency in Python or Rust

Candidate Profile

• Extensive software engineering experience with complex distributed systems
• Result-driven mindset emphasizing adaptability and impactful outcomes
• Proactive in taking on additional responsibilities beyond defined roles
• Interest in expanding knowledge of machine learning infrastructure
• Thrives where technical excellence contributes directly to business and research successes
• Committed to considering the ethical and social ramifications of AI work

Sample Projects

• Designing intelligent routing algorithms to distribute requests efficiently across thousands of accelerators
• Developing autoscaling frameworks to dynamically match compute supply with demand in production and research environments
• Constructing reliable deployment pipelines to launch new models to millions of users
• Integrating new AI hardware platforms to sustain a hardware-agnostic edge
• Adding novel inference features such as structured sampling and prompt caching
• Supporting inference for emerging AI model architectures
• Analyzing observability metrics to optimize performance under real workloads
• Overseeing multi-region deployments and geographic request routing worldwide

Additional Information

The role offers an annual salary ranging from €295,000 to €355,000 EUR.

The minimum educational requirement is a bachelor's degree or an equivalent mix of education, training, and experience related to the job field.

Applicants are considered on a rolling basis with no hard deadline.

There is a hybrid work expectation, with staff generally working from offices at least 25% of the time, though some roles may require more office presence.

Visa sponsorship is available for this position, with support from immigration legal counsel.

Anthropic encourages candidates from diverse and underrepresented backgrounds to apply, recognizing that not all ideal candidates will meet every qualification.

Strict security protocols are in place to protect applicants from scams; official recruiters contact applicants exclusively from verified company email domains.

How Anthropic Operates

Anthropic focuses on large-scale, high-impact AI research conducted collaboratively across the team, emphasizing communication and empirical scientific approaches that align AI research more closely with physics and biology rather than conventional computer science alone.

The team values diverse perspectives and researchers are encouraged to explore recent publications related to advanced interpretability, scaling laws, AI safety, and human preference learning.

Benefits include competitive pay, equity donation matching options, generous vacation and parental leaves, flexible hours, and modern office environments.