• Architect Annotation & Measurement Pipelines: Own the design and implementation of human-in-the-loop and auto-annotation workflows. You will set up systems (e.g., Label Studio) to establish tight confidence metrics and rigorous Inter-Annotator Agreement (IAA). • Drive Cost-Efficiency via Evaluation: Create the ground truth, golden sets, and by-model evaluation criteria that allows us to benchmark models and make informed data science decisions. Your work will directly impact margin by enabling us to deploy the right model for the right task. • Enforce Applied MLOps Standards: Implement "model-as-a-judge" frameworks, deterministic PII scrubbing, and other best practices within our production pipelines to ensure our model-driven products are robust and compliant. • Collaborate on Model Strategy: Work closely with our Data Science team to make applied generative decisions that result in stronger product outcomes and cost efficiencies. • Build & Integrate: Collaborate with engineering to integrate measurement loops into our broader infrastructure (AWS/GCP), ensuring our model lifecycle is automated and observable.