AI Agent Evaluation Engineer

Evaluate advanced AI agents remotely with Outlier, applying technical feedback, debugging, and complex systems integration expertise.
This freelance opportunity supports projects training large language models to function as proactive, multi-step agents. Contributors help improve autonomous systems by analyzing complex behaviors and real-world architectural workflows. The work includes writing evaluation rubrics, debugging agent traces, and stress testing agent performance.
Key Responsibilities:

Analyze complex autonomous agent system behaviors
Write detailed evaluation rubrics for agents
Debug multi-turn agent traces and interactions
Stress test agents using live environments
Identify subtle failures in agent behavior
Requirements:

Two or more years of relevant experience
Strong attention to technical detail
Experience building production-grade modular software
Experience with live tools and real interactions
Clear, high-density technical feedback skills
Use agent debugging, evaluation, and systems expertise to help Outlier improve proactive AI systems.

#J-18808-Ljbffr