RLHF and AI rater jobs · 347 open

Pick the better answer.

347 roles grading and comparing AI answers are open today, on 21 platforms.

Typical pay is $57/hr. 129 of them accept you from any country.

231 of these 347 roles were seen open on their platform in the last 24 hours.

Sort by
  1. micro1Seen open 13h ago$25 to $45/hrAnywhereposted 13h ago
  2. micro1Seen open 13h ago$90 to $140/hrAnywhereposted 13h ago
  3. TeracSeen open 13h ago$150/hrUS +6Agentic AI and RL environmentsposted 5d ago
  4. TeracSeen open 13h ago$150/hrUS onlyposted 5d ago
  5. micro1Seen open 13h ago$45 to $70/hrAnywhereposted 7d ago
  6. MercorSeen open 13h ago$60 to $85/hrAnywhereFinanceposted 2d ago
  7. MercorSeen open 13h ago$40 to $75/hrAnywhereposted 2d ago
  8. micro1Seen open 13h ago$30 to $60/hrUS onlyRed teaming and AI safetyposted 9d ago
  9. WeekdaySeen open 13h ago$80 to $120/hrIndia, USposted 40h ago
  10. WeekdaySeen open 13h ago$60 to $120/hrIndia, USposted 40h ago
  11. WeekdaySeen open 13h ago$60 to $120/hrIndia, USSoftware Engineersposted 40h ago
  12. WeekdaySeen open 13h ago$60 to $120/hrIndia, USSoftware EngineersEngineeringposted 40h ago
  13. Welo DataSeen open 13h ago$20.25/hrUS onlyposted 2d ago
  14. Welo DataSeen open 13h ago$20/hrGermany onlyGermanposted 2d ago
  15. Welo DataSeen open 13h ago$18/hrFrance onlyFrenchposted 2d ago
  16. Welo DataSeen open 13h ago$4/hrIndonesia onlyposted 2d ago
  17. micro1Seen open 13h ago$45/hrUS onlyposted 15d ago
  18. Rise Data LabsSeen open 13h ago$50 to $100/hrUS onlyBusiness and Consultingposted 6d ago
  19. LiftedSeen open 13h agoup to $100/hrArgentina +4Software Engineersposted 14d ago
  20. Rise Data LabsSeen open 13h ago$50 to $100/hrAnywhereFinanceposted 17d ago
  21. Rise Data LabsSeen open 13h ago$50 to $150/hrAnywhereLegalposted 19d ago
  22. Rise Data LabsSeen open 13h ago$50 to $150/hrAnywhereHealthcareposted 19d ago
  23. micro1Seen open 13h ago$10 to $30/hrAnywhereposted 24d ago
  24. micro1Seen open 13h ago$30 to $60/hrAnywhereposted 28d ago
  25. AfterQuerySeen open 3d ago$150/hrAnywhereposted 60d ago
  26. Inworld AISeen open 13h ago$20 to $55/hrUS onlyposted 22d ago
  27. AfterQuerySeen open 3d ago$75/hrAnywhereSoftware Engineersposted 163d ago
  28. AfterQuerySeen open 3d ago$43/hrUK onlyWritingposted 163d ago
  29. AfterQuerySeen open 3d ago$15/hrAnywhereWritingPart-timeposted 163d ago
  30. micro1Seen open 13h ago$25 to $45/hrAnywhereAgentic AI and RL environmentsposted 40d ago
  31. WeekdaySeen open 13h ago$90 to $140/hrUS onlyRed teaming and AI safetyposted 2d ago
  32. WeekdaySeen open 13h ago$90 to $140/hrUS onlySoftware Engineersposted 2d ago
  33. WeekdaySeen open 13h ago$80 to $130/hrUS onlyposted 2d ago
  34. Welo DataSeen open 13h ago$10/hrBulgaria +4posted 13d ago
  35. Welo DataSeen open 13h ago$80 to $92.50/hrNetherlands onlySoftware Engineersposted 16d ago
  36. Welo DataSeen open 13h ago$75 to $90/hrJapan onlyJapaneseSoftware Engineersposted 16d ago
  37. Welo DataSeen open 13h ago$24/hrTaiwan onlyposted 21d ago
  38. VolgaSeen open 13h ago$15 to $35/hrAustralia onlyHealthcareNursesposted 30d ago
  39. MercorSeen open 13h ago$60 to $100/hrDemocratic Republic of Congo onlyEngineeringposted 36d ago
  40. VolgaSeen open 13h ago$15 to $35/hrCanada onlyHealthcareposted 31d ago
  41. VolgaSeen open 13h ago$15 to $35/hrAustralia onlyHealthcareposted 31d ago
  42. VolgaSeen open 13h ago$15 to $35/hrUS onlyHealthcareposted 31d ago
  43. WeekdaySeen open 13h ago$30/hrUS onlyBusiness and Consultingposted 31d ago
  44. MercorSeen open 13h ago$170/hrUS onlyHealthcareposted 39d ago
  45. MercorSeen open 13h ago$2,200/taskAnywhereposted 43d ago
  46. MercorSeen open 13h ago$105 to $140/hrSpain, UKposted 43d ago
  47. LiltSeen open 13h ago~$30/hr est.Netherlands onlySoftware Engineersposted 36d ago
  48. TeracSeen open 13h ago$85/hrUS +2posted 49d ago
  49. TeracSeen open 13h ago$60/hrUS +2posted 48d ago
  50. RWSSeen open 3d ago$24/hrSingapore, Indiaposted 23d ago
Showing 50 of 347 · J / K to move, Enter to open

Seen open in the last 48 hours Last seen open 2 or more days ago

Top-paying RLHF and rater roles today

  1. Radiologist (MD) — Medical Imaging AI Annotation & Evaluation$200 to $400/hrMercor
  2. Dermatologist (MD) — Clinical Image Interpretation & AI Evaluation$270/hrMercor
  3. Head of Insights – Executive Report Evaluation Expert$200/hrMercor
  4. Multilingual Outpatient Physician (MD) — Clinical Documentation & AI Evaluation$170 to $190/hrMercor
  5. Multilingual Primary Care Physician (MD) — Clinical Documentation & AI Evaluation$170 to $190/hrMercor

How a rating task goes

  1. Read the prompt and answers

    You see 1 user prompt and 2 or more answers from the model, in English or in the language the role names.

  2. Score them

    You score each answer on the project's rubric, usually for accuracy, safety and following the instructions, or pick the better of 2.

  3. Write the reason

    You explain your score in 2 or 3 sentences, and a reviewer checks a sample of your work against the same rubric.

What rating listings ask for

53 of the 347 open rating roles name a degree in the listing, and 19 ask for a PhD. 29 mention an AI interview.

Search engine evaluator and speech evaluation roles ask for a native language and a home in a named country. Coding, medical and legal evaluator roles ask for the degree of that field.

Who hires raters, and where

The open rating roles come mostly from Mercor (131), RWS (53) and Welo Data (32). The countries their listings name most often are the US (65), India (42) and Canada (26).

18 rating roles went up in the last 7 days, and 11 have been posted more than once, so those platforms keep hiring for them.

Write code? Sell it too

AI labs license private code you own, and each license pays again. It comes on top of hourly work.

See what my repos are worth →

What is an RLHF job?

An RLHF job means reading answers from an AI model, scoring them or picking the better one, and writing down why. RLHF stands for reinforcement learning from human feedback, and your scores and reasons are the feedback the model is trained on. On Sovrano's Danish and Norwegian evaluator roles, for example, each item shows 1 prompt and 2 responses, and you judge both on how well they follow the instructions.

What does an AI rater do?

An AI rater grades what an AI model produced against a rubric, and the kind of output depends on the role. Mercor's expert Evaluator roles, such as Legal contracts or FP&A, have you grade documents, spreadsheets and slide decks an AI made in your field. Its Audio Generalist Evaluator roles cover 1 language or accent each, such as Tagalog or Argentinian Spanish, and RWS posts Search Engine Evaluator and Speech AI Evaluation roles per language and country. Task Auditor roles on Mercor and Meridial check coding benchmark tasks, such as SWE-Bench or Kubernetes.

How much do AI rater jobs pay?

The 347 RLHF and rater roles open today typically pay $57/hr, and half of them pay $26 to $100/hr. Your field moves the rate the most, and Mercor's physician, radiologist and dermatologist evaluator roles are at the top of the range. Mercor's business, legal and finance Evaluator roles sit in the upper half. RWS's search engine and speech evaluation roles and Outlier's Android evaluator roles are near the bottom, and their rate changes with the country.

Are AI rater jobs work from home?

Yes, nearly all AI rater and RLHF jobs are remote, so you grade AI answers from home on your own computer. RWS's speech evaluation listings say work from home, part-time, 10 or more hours a week, at times you choose. A role that needs you in an office says so in the title, such as Mercor's Romanian audio evaluator role for San Francisco or London, in person.

Can I do AI evaluation jobs from outside the US?

Yes, many AI evaluation jobs are for people outside the US, because language and speech rater roles are posted per country, such as Japan, Germany, Mexico or the Philippines. 129 of the 347 accept you from any country, and each row names the countries for the rest. Mercor's expert Evaluator roles name no country and are in English.

How do I become an AI rater?

You apply to the rater role that matches your language or your profession, and most RLHF and rater listings name no degree. Mercor's Applied Benchmark Specialist roles are the exception, and most of them ask for a PhD in the field, such as physics, chemistry or psychology. Mercor's Audio Generalist and Bilingual Generalist evaluator roles mention an AI interview. Sovrano's roles ask for a native speaker, and RWS's roles name the country you must live in.

Languages

German102French103Finnish17Korean67Polish22Spanish150Portuguese76Japanese74Arabic68Italian55Hindi55

Field

Legal268Psychology28Healthcare441Finance445Business and Consulting327Economics49Software Engineers513Music96Design145Physics73Biology78Chemistry76Engineering197Teaching75Writing60Accountants104Nurses26Pharmacists17

Kind of work

Agentic AI and RL environments50Red teaming and AI safety110Transcription and speech data40Robot training53Part-time419

Where to start

Generalist123DataAnnotation alternatives5,660Outlier alternatives5,783Mercor alternatives4,228No experience249

Platforms

Ethos523micro1334SME Careers269Mercor1,566AfterQuery256All 40 platforms →