هذه الوظيفة لم تعد متاحة
انتهت صلاحية هذه الوظيفة في 17/10/2026. لم تعد تقبل الطلبات.
LLM Evaluator (Model Response Analyst)
Odixcity Consulting
وصف الوظيفة
About the role
We are looking for a detail‑oriented LLM Evaluator (Model Response Analyst) to join our remote team. In this position you will assess the quality of large language model outputs, identify failures, and help improve model safety and performance.
Key responsibilities
- Evaluate and rank model‑generated text using complex rubrics covering factuality, coherence, safety, instruction‑following and creativity.
- Compare multiple responses to the same prompt, select the preferred output and provide clear justifications.
- Provide feedback to modeling and training teams on recurring failure patterns.
- Design adversarial prompts to expose biased, harmful or insecure model behavior.
- Collaborate with QA to refine evaluation guidelines and resolve ambiguous cases.
- Participate in cross‑checking sessions to calibrate scoring and ensure inter‑rater reliability.
- Analyze under‑performing outputs, hypothesize root causes and flag novel behaviors for research.
Required profile
- At least 2 years of professional experience in computational linguistics, data analysis, technical writing, NLP quality assurance or related fields.
- Bachelor’s degree in computer science or a related discipline.
- Proven ability to craft prompts that test model limits and to explain why an output is good or bad.
- Experience with Reinforcement Learning from Human Feedback (RLHF) data collection and evaluation.
- Familiarity with IAA scoring, calibration sessions and A/B testing for AI models.
Required skills
- Prompt engineering
- Reinforcement Learning from Human Feedback (RLHF)
- IAA score analysis
- A/B testing concepts
- Dataset sourcing, cleaning and annotation
- Computational linguistics
- Data analysis
Questions fréquentes
لماذا تبلغ عن هذا العرض؟
اكتشف المزيد
الرواتب والأدلة وعمليات البحث في Chad.
لديك سؤال حول هذا العرض؟
اطرحه هنا: ستصلك تفاصيل العرض كاملة عبر البريد الإلكتروني، فوراً.
عزز فرصك
حمّل سيرتك الذاتية وسنقترح عليك الوظائف التي تناسب ملفك.
جاري تحليل سيرتك الذاتية...
Odixcity Consulting
عروض عمل ذات صلة
-
Senior Software Engineer, Native Mobile (International)
Coinbase -
Forward Deployed Engineer - ProcessOS
Camunda -
Programmeur·euse compétitif·ve – Benchmarks IA (Remote)
Vettedlance -
Enregistreur de flux de travail pour outils de développement (éditeurs de code, IDE, clients API)
Vettedlance -
Ingénieur DevOps, Cloud & Linux – Environnement d'Entraînement IA (Remote)
Vettedlance