Job opportunity
Autonomous AI Agent Evaluator
Outlier
Evaluate autonomous AI agents remotely with Outlier, focusing on agent traces, evaluation rubrics, and complex systems integration. Freelance scheduling is flexible.
Outlier provides human feedback that helps leading AI organizations improve large language models. This work centers on proactive, multi-step agents, including systems that coordinate real-world architectural workflows. Your technical judgment will help assess how these systems behave in live environments.
Key Responsibilities:
Develop rubrics for evaluating agent performance
Inspect and debug detailed agent traces
Stress test autonomous agents across scenarios
Assess complex system behaviors with precision
Requirements:
Two or more years of relevant professional experience
Experience building production-grade modular software
Background in backend engineering or AI automation
Practical experience with live, non-mocked systems
Ability to deliver high-density technical feedback
Bring agent evaluation, debugging, and systems integration skills to Outlier projects training more capable autonomous AI systems.
#J-18808-Ljbffr
Apply on source site Outlier provides human feedback that helps leading AI organizations improve large language models. This work centers on proactive, multi-step agents, including systems that coordinate real-world architectural workflows. Your technical judgment will help assess how these systems behave in live environments.
Key Responsibilities:
Develop rubrics for evaluating agent performance
Inspect and debug detailed agent traces
Stress test autonomous agents across scenarios
Assess complex system behaviors with precision
Requirements:
Two or more years of relevant professional experience
Experience building production-grade modular software
Background in backend engineering or AI automation
Practical experience with live, non-mocked systems
Ability to deliver high-density technical feedback
Bring agent evaluation, debugging, and systems integration skills to Outlier projects training more capable autonomous AI systems.
#J-18808-Ljbffr