Evaluation Scenario Writer - AI Agent Testing Specialist
MindriftSepang, Sepang, Malaysia3 hari lalu
Penerangan pekerjaanBachelor’s and / or Master’s degree in Computer Science, Software Engineering, Data Science / Analytics, Artificial Intelligence / Machine Learning, Computational Linguistics / NLP, Information Systems or related fields Background in QA, software testing, data analysis, or NLP annotation Good understanding of test design principles (e.g., reproducibility, coverage, edge cases) Strong written communication skills in English Comfortable with structured formats like JSON / YAML for scenario description Can define expected agent behaviors (gold paths) and scoring logic Basic experience with Python and JavaScript Curious and open to working with AI‑generated content, agent logs, and prompt‑based behavior Ready to learn new methods, able to switch between tasks and topics quickly, and sometimes work with challenging, complex guidelines Fully remote freelance role – only requires a laptop, internet connection, available time, and enthusiasm to take on a challenge Experience in writing manual or automated test cases Familiarity with LLM capabilities and typical failure modes Understanding of scoring metrics (precision, recall, coverage, reward functions) Get paid for your expertise, with rates up to $38 / hr depending on your skills, experience, and project needs Participate in a flexible, remote, freelance project that fits around your primary professional or academic commitments Gain valuable experience to enhance your portfolio through an advanced AI project Influence how future AI models understand and communicate in your field of expertise
Mindrift is looking for a freelance Agent Scenarios Designer based in the specified country. The role focuses on designing realistic and structured evaluation scenarios for LLM‑based agents, testing agent outputs, and refining tests. You will work on a flexible schedule and receive pay up to $38 / hr based on experience.
What We Do
The Mindrift platform, launched and powered by Toloka, connects domain experts with cutting‑edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real‑world expertise from across the globe.
About the Role
You will design realistic and structured evaluation scenarios, create test cases that simulate human‑performed tasks, and define gold‑standard behavior to compare agent actions against. Your work will ensure each scenario is clearly defined, well‑scored, and easy to execute and reuse. You need a sharp analytical mindset, attention to detail, and an interest in how AI agents make decisions.
Responsibilities
- Design structured test scenarios based on real‑world tasks
- Define the golden path and acceptable agent behavior
- Annotate task steps, expected outputs, and edge cases
- Work with developers to test scenarios and improve clarity
- Review agent outputs and adapt tests accordingly
How to Get Started
Apply to this posting, qualify, and you’ll have the chance to contribute to projects aligned with your skills on your own schedule. From creating training prompts to refining model responses, you’ll help shape the future of AI while ensuring technology benefits everyone.
Requirements
Nice to Have
Benefits
#J-18808-Ljbffr
Buat amaran kerja untuk carian ini
Evaluation Writer Ai • Sepang, Sepang, Malaysia
Pekerjaan berkaitan
Position : Content Writer ( VLLM).Tenure : 1 year contract ( Renewable basis).Salary : RM 3200- RM 3700 ( Subject to Experience).
Location : Kuala Lumpur / Petaling Jaya.You will analyze images and write...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
This role is to lead initiatives focused on client acquisition and AI-enhanced user experiences in our global mobile trading platform.
This role is pivotal in driving growth by enhancing user onboar...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
Create and present contents via Youtube Platform.Familiar with Financial Services industry (not limited to fintech, loans, BPL).
Work hours 9am-5pm, Monday - Friday.Hiring Immediately - Permanant an...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
Junior Content Writer
EPS ConsultantsPetaling Jaya, Selangor, MYQuick Apply
- Dinaikkan pangkat
Instructor for AI-Assisted Programming Workshops (part-time)
TripleTenShah Alam, Shah Alam, Malaysia TripleTen Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.Instructor – AI Course Trainer.We’re building an online course for experienced software developers on the practical use of Cursor...Tunjukkan lagiKemas kini terakhir: 12 hari yang lalu
- Dinaikkan pangkat
- Baharu!
Technical Writer & AI Community Manager
Stronium Sdn BhdKuala Lumpur, Kuala Lumpur, Malaysia A leading technology firm in Kuala Lumpur seeks a Technical Writer / Community Manager to keep technical documentation up to date and engage with the partner community.
The ideal candidate has stron...Tunjukkan lagiKemas kini terakhir: 4 jam yang lalu
- Dinaikkan pangkat
Data Analytics Specialist - Associate (Assurance - Data Intelligence Delivery)
Ernst & YoungKuala Lumpur, Kuala Lumpur, Malaysia EY's commitment to the quality and integrity of our audits is exemplified by our global audit methodology and our thorough quality controls that are applied to every client engagement.Together with...Tunjukkan lagiKemas kini terakhir: 1 hari yang lalu
Lead Technical Author
Talent SwitchKuala Lumpur, Malaysia Lead the investigation, evaluation and implementation of next generation technologies, especially generative AI, to help drive productivity and quality gains in technical authoring.Define and execu...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
Product Manager - Growth and AI
Flintex Consulting Pte LtdKuala Lumpur, 14, myQuick Apply
Senior AI Research Engineer, Model Inference (100% Remote)
Tether Operations LimitedKuala Lumpur, 14, MY Join Tether and Shape the Future of Digital Finance.At Tether, we’re not just building products, we’re pioneering a global financial revolution.
Our cutting-edge solutions empower businesses—from ex...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
- Baharu!
Remote Content Writer - Biweekly Pay & Growth Opportunities
KimpSepang, Selangor, Malaysia A content creation company is seeking a Kimp Content Writer to enjoy flexible remote work while delivering high-quality content.
Responsibilities include enriching digital platforms and engaging in ...Tunjukkan lagiKemas kini terakhir: 4 jam yang lalu
AI BUSINESS ANALYST
Talent SwitchKuala Lumpur, Malaysia Work with business stakeholders to identify and define use cases for AI solutions (e.Analyze current workflows and systems to determine where AI can enhance productivity, accuracy, or decision-maki...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
Content Specialist
CartrackKuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia We are a world-leading smart mobility SaaS company with over 2,000,000 subscribers across 23 countries and we are looking for a Content Specialist to join our team.
Our teams are collaborative, vibr...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
- Baharu!
Strategic AI Project Lead for Customer Experience
LenovoKuala Lumpur, Kuala Lumpur, Malaysia A global technology company in Kuala Lumpur is seeking a Strategic Project Manager to lead key AI initiatives within their Customer Engagement Center.
This role is crucial for driving successful imp...Tunjukkan lagiKemas kini terakhir: 4 jam yang lalu
- Dinaikkan pangkat
AI Engineer (Multi-agent system)
Hiredly XKuala Lumpur, Kuala Lumpur, Malaysia As an AI Engineer (Entry to Mid level), you’ll play a hands-on role in the development of our client's multi-agent AI systems.
You’ll implement prompt templates, integrate LLMs, build user-facing fe...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
Content Writer
SHUTTERSTUDIOCyberjaya, Selangor, Malaysia Develop, organize, and manage all published content (images, videos, writing) across all social media platforms.Ensure clients' brand voice, messaging, and visual design consistency across all soci...Tunjukkan lagiKemas kini terakhir: 1 hari yang lalu
- Dinaikkan pangkat
Ads Relevance Specialist - AI Data Service and Operations
TikTokKuala Lumpur, Kuala Lumpur, Malaysia Ads Relevance Specialist - AI Data Service and Operations.TikTok, Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.
About the team : Our Search Operations team supports our efforts to addres...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
Senior Research Engineer Multimodal & Video Foundation Model (100% Remote)
Tether Operations LimitedPuchong, 10, MY Join Tether and Shape the Future of Digital Finance.At Tether, we’re not just building products, we’re pioneering a global financial revolution.
Our cutting-edge solutions empower businesses—from ex...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
- Baharu!
Business Intelligence & AI Transformation Lead
StoreHub Sdn Bhd, OneStoreHub Pte LtdKuala Lumpur, Kuala Lumpur, Malaysia Are you driven, results-oriented and a team player?.With 17,000+ retailers and restaurants in over 15 countries, StoreHub is on a mission to enable everyone, big or small, to build successful busin...Tunjukkan lagiKemas kini terakhir: 4 jam yang lalu
Content Writer
Two95 International Inc.Setia Alam, Selangor, MYQuick Apply
- Dinaikkan pangkat
- Baharu!
Remote SEO Content Writer for Growth and Impact
AlphaiotabpoPetaling Jaya, Selangor, Malaysia A dynamic digital marketing firm based in Kuala Lumpur is seeking a content creator with expertise in social media advertising.
The role involves researching, writing, and optimizing content while a...Tunjukkan lagiKemas kini terakhir: 4 jam yang lalu