Evaluate autonomous AI agents through rubric writing, trace debugging, and stress testing while working remotely as a freelance contributor for Outlier.
Outlier helps innovative companies improve AI agents by collecting human feedback. This work focuses on training large language models to function as proactive, multi-step systems. You will assess how agents design, coordinate, and optimize complex workflows. Applicants need at least two years of relevant experience.
Key Responsibilities:

Debug agent traces across multi-turn interactions
Write detailed evaluation rubrics
Stress test autonomous agent behavior
Assess complex system behaviors with precision
Support training for generative AI systems
Requirements:

Experience with backend engineering, AI automation, or systems integration
Ability to maintain production-grade modular software
Practical experience with live environments
Clear, high-density technical feedback skills
Experience handling persistent state or session discovery preferred
Use technical feedback, agent testing, and systems integration experience to strengthen autonomous AI evaluation projects with Outlier.

#J-18808-Ljbffr

AI Agent Evaluation Engineer