Test Engineer (AI Model Evaluation)

Overview

We are seeking an AI Model Evaluation Engineer (Junior–Mid Level) with strong QA automation expertise and hands-on experience in evaluating OCR, LLM (chatbot) models, and RAG dataset preparation (speech-to-text, text-to-speech, video-to-OCR). The role focuses on automated testing, ground truth creation, and workflow validation to ensure accuracy, compliance, and real-world reliability for production-ready AI systems.

Responsibilities

• Evaluate LLM (chatbot), OCR, and RAG datasets for correctness, bias, compliance, and real-world robustness.
• Design and automate test frameworks for RAG pipelines, workflow triggers, and chatbot responses.
• Craft and validate ground-truth datasets for OCR, TTS, and speech-to-text projects.
• Test chatbot responses for accuracy, context relevance, ethical compliance, and edge cases.
• Conduct load testing to ensure system performance under high-traffic and stress scenarios.
• Integrate open-source LLM evaluation frameworks (e.g., DeepEval, HuggingFace evaluation tools) into testing pipelines.
• Automate data processing & reporting workflows using Google Apps Script and Google Sheets for faster insights.
• Document results, define acceptance criteria, and collaborate with ML engineers, data scientists, and QA teams to enhance model reliability.
• Support CI / CD pipeline integration for model evaluation and regression testing.

Qualifications

Education

Experience

Technical Skills

Seniority level

Employment type

Job function

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr