Posted 9 days ago
Senior Systems & Hardware Experts - AI Benchmarking
@ Braintrust
Canada
Overview
Braintrust is hiring senior systems and hardware engineers to write difficult, Linux-terminal-based tasks for an AI agent benchmark. The work involves turning real problems from the candidate's field into challenges that current AI agents mostly fail. Requirements include 9+ years of systems or hardware engineering experience, a completed degree, and at least one publication. Pay is up to $1,350 per task, with onboarding of about 90 minutes plus a live webinar.
Responsibilities:
- Identify difficult problems in your area of expertise
- Translate problems into tasks solvable in a Linux terminal
- Design challenges that expose limitations of advanced AI agents
- Test each task against three leading models with three attempts each
Requirements summary:
- At least 9 years of systems or hardware engineering experience
- Completed bachelor's degree or higher, not currently enrolled
- At least one academic or professional publication
- Practical comfort working in a Linux terminal
Tools & skills
Source and dates
Source type not verified · apify-linkedin
- Employer posting date
- 2026-11-05
- First seen by our source
- 2026-11-14
- Source record updated
- 2026-11-14
These dates describe the source record, not a live verification that the employer is still hiring. Check the original posting.
Job details
| Seniority | Senior |
| Department | Software development |
| Experience | 9+ years |
| Hours per week | 40 (estimated) |
| Travel | air: none · ground: none |
| Work setting | Home |
| Position | Mostly sitting |
| Physical demands | physical low · computer high |
| Languages | English |
| Education | Bachelor’s degree (required) |
Summary and details written by AI from the posting; the employer’s posting is the authority.
Braintrust
| Field | Value |
|---|---|
| Open jobs on Jobsy | 9 |
We’re looking for experienced systems engineering or hardware engineering professionals to develop challenging tasks for an AI benchmark. You’ll draw on problems from your own field and turn them into clearly defined challenges that an AI agent must solve from scratch in a Linux terminal.
What you’ll do
- Identify difficult, meaningful problems in your area of expertise.
- Translate those problems into tasks that can be solved in a Linux terminal.
- Design challenges that expose limitations in advanced AI agents. Each task is tested against three leading models, with three attempts per model, and must prove difficult enough that the models mostly fail.
- At least 9 years of relevant professional experience in systems engineering or hardware engineering. Relevant systems work includes DevOps, site reliability engineering (SRE), cloud engineering, platform engineering, infrastructure engineering or distributed systems.
- A completed bachelor’s degree or higher.
- Applicants must have completed their degrees and must not currently be enrolled in a degree program.
- At least one academic or professional publication.
- Practical comfort working in a Linux terminal.
- Current residence in Argentina, Mexico, Puerto Rico, New Zealand, Canada, the United Kingdom, the United States, Australia, Peru, Colombia, Austria, Ireland, Spain, Luxembourg, Italy, France, Germany, Belgium, the Netherlands, Sweden, Finland, Singapore, Denmark, Norway and Switzerland.
- Task-based project expected to run for the next few weeks.
- Onboarding includes two courses and a graded assessment, taking approximately 90 minutes.
- A live onboarding webinar is also required before moving to tasking.
Earn up to $1,350 per task, with opportunities for higher rates based on task quality and submission volume. Tasks take six to seven hours on average, although initial tasks may take longer as you get familiar with the process.
Journal
Only you can see this. Kept on this device, and in your account when you sign in.