Posted 1 hour ago
Software eng, inference
@ turbopuffer
Canada
Overview
turbopuffer is hiring a software engineer to build inference for search over more than a trillion documents. The role works closely with inference providers to keep p99 latency low, improves how the database talks to them, and integrates new embedding and reranking models. It is a full-time position.
Responsibilities:
- Keep p99 inference latency low with inference providers
- Improve database interaction with inference providers
- Integrate new embedding and reranking models
- Automate own role over time
Requirements summary:
- Systems language experience (Rust, C++, Java, Go)
- Production inference deployment experience
- Experience with large dataset processing
- Understanding of latency-throughput tradeoffs
- Familiarity with vLLM and SGLang
- Clear writing and honest communication
Tools & skills
Source and dates
Employer career site / ATS feed · ashby
- Employer posting date
- 2026-11-16
- First seen by our source
- 2026-11-16
- Source record updated
- 2026-11-16
These dates describe the source record, not a live verification that the employer is still hiring. Check the original posting.
Job details
| Department | Software development |
| Hours per week | 40 (estimated) |
| Work setting | Office |
| Position | Mostly sitting |
| Physical demands | physical low · computer high |
Summary and details written by AI from the posting; the employer’s posting is the authority.
turbopuffer
Saved company profile
Historical company information from a saved The Jobs API profile (LinkedIn/Crunchbase fields). It is not live-verified; figures may have changed. Missing values remain unknown.
serverless {vector, full-text} search from first principles on object storage. faster, 10x cheaper, extremely scalable
Full company description
turbopuffer is a fast search engine that combines vector and full-text search using object storage, making all your data easily searchable. Using only object storage for state and NVMe SSD with memory cache for compute, turbopuffer scales horizontally to handle billions of documents. The system caches only actively searched data while keeping the rest in low-cost object storage, offering competitive pricing. Cold queries process 1 million vectors in 500ms (p90), while warm queries take just 20ms (p50). This architecture means it's as fast as in-memory search engines when cached, but far cheaper to run. Storing data in cache and object storage costs less than traditional replicated disk systems, even for frequently accessed data.
- LinkedIn followers
- 7,114
- People associated on LinkedIn
- 37
- Business categories
- Information Technology
- Database
- Other recorded names
- turbopuffer
- Profile identifies a recruitment agency
- No
- Reported total investment
- 0
Currency and observation date were not supplied; this is not a current valuation or a funding round.
turbopuffer is a fast search engine that combines vector and full-text search using object storage, making all your data easily searchable. Using only object storage for state and NVMe SSD with memory cache for compute, turbopuffer scales horizontally to handle billions of documents. The system caches only actively searched data while keeping the rest in low-cost object storage, offering competitive pricing. Cold…
| Field | Value |
|---|---|
| Industry | Technology, Information and Internet |
| Company size | 11-50 employees |
| Organization type | Privately Held |
| Year founded | 2023 |
| Open jobs on Jobsy | 6 |
| View page |
turbopuffer is looking for a software engineer who wants to build inference for searching +1T documents.
what you'll do:
work closely with our inference providers to make sure that the p99 inference latency stays as low as possible
write code to improve how the database interacts with our inference providers
integrate new embedding & reranking models into turbopuffer
automate yourself out of a job
must have:
engineering
you've worked in a systems language (Rust, C++, Java, Go, etc.)
experience in inference engineering, e.g. managing a production deployment of an LLM/embedding/reranking model
worked on handling/processing large datasets
you love shipping things to prod
domain expertise
you understand the tradeoff between latency and throughput for model deployments and you know how to think about improving model performance
keeping up with state-of-the-art embedding & reranker models and their evaluation
you’re familiar with the most common inference engines like vLLM and SGLang
you know how to manage the infra of model deployments and how to reason about GPU autoscaling.
typist - you can write well and can explain complex ideas simply
human - you build trust and admit what you don’t know
nice to have:
an interest in embedding & reranker model training
prior experience in vector and full-text search
knowledge about in-depth areas of inference engines & GPU engineering
tpuf is hardcore & whimsical. we value:
overstep > understep
show > tell
complexity is earned
showing up as part of our customer’s team
making stuff easy to react to
Journal
Only you can see this. Kept on this device, and in your account when you sign in.