Log in Sign up free
Apply now

First seen by our source: 20 days ago

Language Model Evaluator - Fully Remote | Upto $20/hr Part-time

@ Mercor

View all jobs Website

Toronto, Ontario

$15–$20/hr
Apply now
SIGNALS

Overview

Mercor is hiring a remote contract Generalist evaluator (English and Punjabi) to review and rate large language model responses for fact accuracy, reasoning, clarity and tone. The work is part-time at $15 to $20 per hour and is done independently and asynchronously. Applicants must be native Punjabi speakers with a bachelor's degree and significant LLM experience.

Pay:

$15–$20 an hour

Responsibilities:

  • Fact-check responses using trusted public sources and external tools
  • Generate human evaluation data identifying strengths, weaknesses and inaccuracies
  • Assess reasoning quality, clarity, tone and completeness
  • Ensure responses match expected conversational behaviour and guidelines
  • Work independently and asynchronously to meet deadlines

Requirements summary:

  • Bachelor's degree
  • Native speaker of Punjabi
  • Significant experience using large language models
  • Excellent English writing skills
  • Strong attention to detail
  • Background in structured analytical thinking

Tools & skills

ResearchWork independentlyMeet deadlinesAttention to detailAnalytical skillsEditing
Source and dates

Source type not verified · manatal

Employer posting date
Not provided
First seen by our source
2026-10-18
Source record updated
2026-10-18

These dates describe the source record, not a live verification that the employer is still hiring. Check the original posting.

Job details

Travelair: none · ground: none
Work settingHome
PositionMostly sitting
Physical demandsphysical low · computer high
LanguagesEnglish, Punjabi
EducationBachelor’s degree (required)

Summary and details written by AI from the posting; the employer’s posting is the authority.

Apply now

Mercor

Saved company profile

Historical company information from a saved The Jobs API profile (LinkedIn/Crunchbase fields). It is not live-verified; figures may have changed. Missing values remain unknown.

We connect human expertise with leading AI labs and enterprises to train frontier models.

Full company description

Mercor is defining the future of work. We connect human expertise with leading AI labs and enterprises to train frontier models.

LinkedIn followers
654,507
People associated on LinkedIn
5,250
Reported office locations
  • San Francisco, California 94105, US
Business categories
  • Employment
  • Software
  • Recruiting
  • Data Collection and Labeling
  • Artificial Intelligence (AI)
  • Machine Learning
Other recorded names
  • Mercor
Profile identifies a recruitment agency
Yes
Reported total investment
483,600,000

Currency and observation date were not supplied; this is not a current valuation or a funding round.

View all jobs Website

Mercor is defining the future of work. We connect human expertise with leading AI labs and enterprises to train frontier models.

FieldValue
IndustrySoftware Development
Company size51-200 employees
HeadquartersSan Francisco, California
Organization typePrivately Held
Year founded2023
Open jobs on Jobsy1,626
LinkedInView page
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Generalist - English & Punjabi
Type: Contract
Compensation: $15–$20/hour
Location: Remote

Role Responsibilities

  • Conduct fact-checking using trusted public sources and external tools.
  • Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies.
  • Assess reasoning quality, clarity, tone, and completeness of responses.
  • Ensure model responses align with expected conversational behavior and system guidelines.
  • Work independently and asynchronously to meet deadlines while improving AI model performance.

Qualifications

Must-Have

  • Bachelor's degree
  • Native speaker in Punjabi
  • Significant experience using large language models (LLMs)
  • Excellent writing skills in English
  • Strong attention to detail
  • Background or experience in domains requiring structured analytical thinking

Preferred

  • Experience with RLHF, model evaluation, or data annotation work
  • Experience writing or editing high-quality written content
  • Experience comparing multiple outputs and making fine-grained qualitative judgments

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
  • For any help or support, reach out to: [email protected]

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Similar jobs

  • Language Model Evaluator - Fully Remote | Upto $20/hr Part-time

    Toronto, ON
    $15–$20/hr

    Mercor

    Candidates need a bachelor's degree, native Bengali, strong English writing and significant LLM experience. Analytical background in research, policy, linguistics or engineering, plus strong attention to detail, are required.

    Research, Work independently, Meet deadlines, Attention to detail, Analytical skills, Editing

  • Dairy Clerk Part Time Day

    Toronto, ON
    Part Time

    Loblaw

    Candidates need no prior experience, since training is provided. They must be a team player with attention to detail, able to work independently in a fast-paced setting, and able to move up to 50 lbs and stay mobile for a full shift.

  • Dairy Clerk Part Time Day

    Toronto, ON
    Part Time

    Loblaws Inc.

    Candidates need to be a team player with attention to detail who can work independently in a fast-paced setting. They must be flexible with days, evenings and weekends, and able to lift up to 50 lbs and stay mobile for a full shift.

    Customer service, Merchandise display, Health and safety, Work independently, Leadership

  • Senior Manager, Licensing & Account Management – NBA Canada

    Toronto, ON
    Full Time

    CAN NBA Canada

    7+ YOECandidates need a bachelor's degree and 7+ years of work experience, ideally in media, entertainment, licensing, marketing or retail, with sports experience an asset. Strong Excel, PowerPoint and Word skills, financial analysis and forecasting, and excellent communication and presentation skills are required.

    Microsoft Excel, Microsoft PowerPoint, Microsoft Word, Flowhaven, NBA business approval portal

    Director, Marketing Partnerships Lead – NBA Canada

    Toronto, ON
    Full Time

    CAN NBA Canada

    8+ YOECandidates need 8+ years of sales, business development or account management experience, a bachelor's degree, and strong negotiation and senior-level communication skills. Knowledge of sponsorship agreements, the Canadian market and fluent English are required.

  • Critical Environment Electrical Engineering

    Toronto, ON
    $86k–$169k/yrOn-siteFull Time

    Microsoft: Every company has a mission.

    2+ YOECandidates need a bachelor's or master's degree in electrical or computer engineering plus relevant engineering experience, and must be registered as a P.Eng. in Ontario. Critical environment and electrical infrastructure experience is preferred.

    Blueprints, Building management systems, Switchgear, UPS systems, Standby diesel generators

    Senior Critical Environment Mechanical Engineer

    Toronto, ON
    $114k–$204k/yrOn-siteFull Time

    Microsoft: Every company has a mission.

    7+ YOEMicrosoft requires 7+ years of related engineering experience, or a relevant degree plus 3 to 4+ years of experience, and a P. Eng. registration in Ontario. Candidates must be able to obtain Reliability Status screening and meet Microsoft Cloud background checks.

    CAD modeling tools, Predictive modeling software

  • Receptionist/Administrative Support

    Toronto, ON
    $20–$22/hrContract, Temporary

    adecco

    1+ YOECandidates need 1 to 3 years of reception, front desk or office administration experience. They should communicate well in writing and speaking, be organized, service-oriented, and comfortable with Microsoft Word, Excel, Outlook and Teams.

    Microsoft Word, Microsoft Excel, Microsoft Outlook, Microsoft Teams, audiovisual equipment

Locations & environments

Locations

All of CanadaRemote · Hybrid · On-site · Field

Pro tip: add remote jobs to widen your search.

Exclude locations

Workplace type

Company jobs in these results

These are this company’s jobs loaded in your current search. The company may have more openings.