First seen by our source: 2 days ago
Software Engineering, Data Science & Systems Design Experts (AI Model Evaluation)
@ Union Rebellion
Remote in Canada
Overview
Union Rebellion is hiring Technical AI Evaluators on an independent contractor basis to assess and improve AI-generated responses to coding and software engineering tasks. The work involves executing and testing code, identifying bugs and logical flaws, annotating outputs, and fact-checking technical claims. The role is fully remote with flexible hours, full-time or part-time, with weekly payments based on services rendered.
Responsibilities:
- Evaluate AI-generated code responses for accuracy and clarity
- Execute and test code to validate correctness
- Identify bugs, inefficiencies and logical flaws
- Annotate responses with detailed feedback
- Fact-check technical claims against authoritative sources
Requirements summary:
- Bachelor's, Master's or PhD in Computer Science or related field
- 3+ years of professional software or data experience
- Strong proficiency in at least two programming languages
- Ability to solve medium to hard algorithmic problems
- Experience working with LLMs in coding workflows
Tools & skills
Source and dates
Source type not verified · dover
- Employer posting date
- Not provided
- First seen by our source
- 2026-11-04
- Source record updated
- 2026-11-04
These dates describe the source record, not a live verification that the employer is still hiring. Check the original posting.
Job details
| Seniority | Mid level |
| Department | Software development |
| Experience | 3+ years |
| Travel | air: none · ground: none |
| Work setting | Home |
| Position | Mostly sitting |
| Physical demands | physical low · computer high |
| Languages | English |
| Education | Bachelor’s degree (required) |
| Fields of study | Computer Science |
| Perks mentioned | Flexible hours |
Summary and details written by AI from the posting; the employer’s posting is the authority.
Union Rebellion
| Field | Value |
|---|---|
| Open jobs on Jobsy | 5 |
Role Overview
As a Technical AI Evaluator, you will assess and improve AI-generated responses to coding and software engineering tasks. Your work will directly impact how accurately and effectively AI systems solve problems, write code, and communicate technical concepts.
This is a hands-on, intellectually rigorous role suited for professionals with strong coding, analytical, and problem-solving skills.
Key Responsibilities
Evaluate AI-generated responses for accuracy, reasoning, clarity, and completeness
Execute and test code to validate correctness and performance
Identify bugs, inefficiencies, and logical flaws in model outputs
Annotate responses with detailed feedback and improvement suggestions
Assess code quality, readability, and adherence to best practices
Fact-check technical claims using reliable and authoritative sources
Apply structured evaluation frameworks and guidelines consistently
What We’re Looking For
Bachelor’s, Master’s, or PhD in Computer Science or a related field
3+ years of professional experience in software engineering, data science, or related roles
Strong proficiency in at least two programming languages (e.g., Python, JavaScript, C++, Java, SQL, Go, Rust, etc.)
Ability to independently solve medium to hard algorithmic problems
Experience working with or alongside LLMs in coding workflows
Strong attention to detail and ability to evaluate complex technical reasoning
Nice to Have
Experience with AI model evaluation, RLHF, or data annotation
Background in competitive programming
Contributions to open-source projects (merged pull requests preferred)
Experience reviewing production-level code
Ability to explain complex technical concepts clearly to non-experts
What Success Looks Like
You consistently identify subtle bugs, flawed logic, and edge cases
Your feedback improves the accuracy and clarity of AI-generated code
You produce structured, reproducible evaluation outputs
AI systems become more reliable for real-world engineering tasks
Compensation & Terms
Independent contractor engagement
Fully remote with flexible working hours (open to US and non-US candidates)
Full-time or part-time contract options available
Weekly payments based on services rendered
Equal Opportunity
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Journal
Only you can see this. Kept on this device, and in your account when you sign in.