- Experience
- Any
- Salary
- USD 100 – USD 500 / hour
- Openings
- 1
- Posted
- 2 days ago
- Work mode
- In office
- Resume
- Required to apply
Sign in to tell us what does and doesn't work for you here — it sharpens every match we show you.
Job description
Role Overview
We seek experienced engineers who design, deploy, and manage large language model (LLM) agents in live production environments. The ideal candidate has direct experience with monitoring and understanding agent utilization within organizations.
Key Responsibilities and Experience
- Delivered agents relied upon by real users and managed issues post-deployment.
- Developed methodologies to evaluate whether agent performance improved or declined following changes.
- Operated agents beyond simple requests, including scheduled jobs, long-running processes, and cloud-based sandbox environments.
- Observed organizational adoption patterns of internal assistants, identifying users who embraced or ignored the technology.
Focus Areas
We place particular emphasis on less-discussed components such as internal monoagents integrated with company data, shared organizational memory, modular reusable skills and playbooks, interfaces agents interact with, and transparency into agent activities and associated costs.
Selection Process
The application includes an initial conversational AI interview without coding or take-home tasks. We seek insights into your approach toward reliability, evaluation, and user adoption of agents, including the trade-offs faced during real system operation. Candidates are encouraged to share detailed, candid experiences.
Candidates who pass this stage will be invited to a 30-minute live discussion with our team. This conversation is compensated with $100 to $500 upon completion, with payment based on the depth of expertise demonstrated.
Relevant Skills
- AI evaluation