Apply now

Posted 1 hour ago

Software eng, inference

@ turbopuffer

Canada

Full-time
Apply now
SIGNALS

Overview

turbopuffer is hiring a software engineer to build inference for search over more than a trillion documents. The role works closely with inference providers to keep p99 latency low, improves how the database talks to them, and integrates new embedding and reranking models. It is a full-time position.

Responsibilities:

  • Keep p99 inference latency low with inference providers
  • Improve database interaction with inference providers
  • Integrate new embedding and reranking models
  • Automate own role over time

Requirements summary:

  • Systems language experience (Rust, C++, Java, Go)
  • Production inference deployment experience
  • Experience with large dataset processing
  • Understanding of latency-throughput tradeoffs
  • Familiarity with vLLM and SGLang
  • Clear writing and honest communication

Tools & skills

Source and dates

Employer career site / ATS feed · ashby

Employer posting date
2026-11-16
First seen by our source
2026-11-16
Source record updated
2026-11-16

These dates describe the source record, not a live verification that the employer is still hiring. Check the original posting.

Job details

DepartmentSoftware development
Hours per week40 (estimated)
Work settingOffice
PositionMostly sitting
Physical demandsphysical low · computer high

Summary and details written by AI from the posting; the employer’s posting is the authority.

turbopuffer

Saved company profile

Historical company information from a saved The Jobs API profile (LinkedIn/Crunchbase fields). It is not live-verified; figures may have changed. Missing values remain unknown.

serverless {vector, full-text} search from first principles on object storage. faster, 10x cheaper, extremely scalable

Full company description

turbopuffer is a fast search engine that combines vector and full-text search using object storage, making all your data easily searchable. Using only object storage for state and NVMe SSD with memory cache for compute, turbopuffer scales horizontally to handle billions of documents. The system caches only actively searched data while keeping the rest in low-cost object storage, offering competitive pricing. Cold queries process 1 million vectors in 500ms (p90), while warm queries take just 20ms (p50). This architecture means it's as fast as in-memory search engines when cached, but far cheaper to run. Storing data in cache and object storage costs less than traditional replicated disk systems, even for frequently accessed data.

LinkedIn followers
7,114
People associated on LinkedIn
37
Business categories
  • Information Technology
  • Database
Other recorded names
  • turbopuffer
Profile identifies a recruitment agency
No
Reported total investment
0

Currency and observation date were not supplied; this is not a current valuation or a funding round.

turbopuffer is a fast search engine that combines vector and full-text search using object storage, making all your data easily searchable. Using only object storage for state and NVMe SSD with memory cache for compute, turbopuffer scales horizontally to handle billions of documents. The system caches only actively searched data while keeping the rest in low-cost object storage, offering competitive pricing. Cold…

FieldValue
IndustryTechnology, Information and Internet
Company size11-50 employees
Organization typePrivately Held
Year founded2023
Open jobs on Jobsy6
LinkedInView page
Job description

turbopuffer is looking for a software engineer who wants to build inference for searching +1T documents.

what you'll do:

  • work closely with our inference providers to make sure that the p99 inference latency stays as low as possible

  • write code to improve how the database interacts with our inference providers

  • integrate new embedding & reranking models into turbopuffer

  • automate yourself out of a job

must have:

  • engineering

    • you've worked in a systems language (Rust, C++, Java, Go, etc.)

    • experience in inference engineering, e.g. managing a production deployment of an LLM/embedding/reranking model

    • worked on handling/processing large datasets

    • you love shipping things to prod

  • domain expertise

    • you understand the tradeoff between latency and throughput for model deployments and you know how to think about improving model performance

    • keeping up with state-of-the-art embedding & reranker models and their evaluation

    • you’re familiar with the most common inference engines like vLLM and SGLang

    • you know how to manage the infra of model deployments and how to reason about GPU autoscaling.

  • typist - you can write well and can explain complex ideas simply

  • human - you build trust and admit what you don’t know

nice to have:

  • an interest in embedding & reranker model training

  • prior experience in vector and full-text search

  • knowledge about in-depth areas of inference engines & GPU engineering

tpuf is hardcore & whimsical. we value:

  • overstep > understep

  • show > tell

  • complexity is earned

  • showing up as part of our customer’s team

  • making stuff easy to react to

Similar jobs

  • Software Eng (Dev) IV

    Moncton, NB

    IGT, a Nevada Corporation: IGT is a leading global provider of gaming, digital and financial technology solutions, formed through the combination of International Game…

    Candidates need proven experience leading complex software projects from planning to release, expert knowledge of C++, C#, Unity and performance optimization, and experience delivering products in regulated gaming environments with certification processes.

    Unity

Locations & environments

Locations

All of CanadaRemote · Hybrid · On-site · Field

Pro tip: add remote jobs to widen your search.

Exclude locations

Workplace type

Company jobs in these results

These are this company’s jobs loaded in your current search. The company may have more openings.