BackOps AI→

AI/ML Engineer at BackOps AI in San Francisco, CA

Mid LevelHybridFull-timeSan Francisco, CA

Skills

pythondistributed systemscloud infrastructurecontainerizationorchestrationmodel servinginference pipelinesfoundation modelsmodel evaluationexperimentation frameworksengineering fundamentalsproblem solvingtechnical communicationadaptabilityjudgmentpytorchhugging facevllmtritonraykubernetesagentic aimulti-step autonomous workflows

Job Description

AI/ML Engineer

About BackOps

BackOps is building the AI resolution layer for companies that make or move physical goods.

The physical economy still runs on a fragmented web of ERPs, WMSs, TMSs, carrier and vendor portals, email, spreadsheets, legacy systems, and the people who know how to hold all of it together. When something breaks, whether a shipment is delayed, an order changes, a claim needs to be filed, a document is missing, someone has to gather context across systems, determine what should happen next, take action, and make sure the problem is actually resolved.

BackOps automates that work.

Our platform, Relay, deploys AI operators that understand operational workflows, work across the systems our customers already use, take action, navigate exceptions, and carry work through to resolution. We aren't building another dashboard or copilot. We're building the execution layer that sits between systems and gets the work done.

We recently raised a $42M Series B led by Insight Partners, just six months after our Series A, and are scaling across some of the largest companies in logistics, manufacturing, retail, and other industries that power the physical world.

The opportunity ahead is much larger than any single workflow or vertical. We're building the infrastructure for AI to operate reliably inside the real world.

The Role

As an AI/ML Engineer, you'll own the infrastructure and systems that make Relay's underlying intelligence more capable, efficient, and reliable.

Building an AI operator that can reason through a workflow is one challenge. Making that intelligence perform consistently across thousands of real-world operational scenarios, fragmented enterprise systems, and unpredictable conditions is another.

That's the problem you'll help solve.

You'll work at the intersection of machine learning, distributed systems, and production AI, designing the infrastructure that supports model execution, inference optimization, evaluation, and continuous improvement. You'll help determine how we select, adapt, deploy, and optimize the models powering Relay, balancing intelligence and quality with the performance, scalability, and economics required for enterprise production.

This isn't a role focused on integrating AI APIs or maintaining an established ML stack. You'll make foundational technical decisions, build capabilities from the ground up, and work closely with our CTO and engineering teams to advance the intelligence behind our products.

This is a senior individual contributor role with extraordinary leverage. You'll help define the AI/ML foundation BackOps scales on.

What You'll Own

Build the infrastructure behind Relay's intelligence

Own the architecture and engineering of the AI/ML systems that power Relay, enabling our models to run reliably, efficiently, and at scale.

That may include:

Model serving, inference pipelines, and execution infrastructure
Model selection, routing, and orchestration
Shared interfaces for accessing foundation models and specialized ML capabilities
Infrastructure for experimentation, evaluation, and production deployment
Model versioning, configuration, and lifecycle management
Compute orchestration and resource optimization where appropriate

You'll make pragmatic decisions about what we build, what we leverage from existing providers, and where custom infrastructure creates a meaningful advantage.

Make intelligence faster, more efficient, and more scalable

Our agents need to do more than produce the right answer. They need to operate within the performance and reliability constraints of real enterprise workflows.

You'll optimize the infrastructure and execution patterns that determine how efficiently our models perform, including latency, throughput, scalability, and inference cost.

You'll identify bottlenecks across the AI/ML stack, evaluate architectural tradeoffs, and implement improvements that allow us to support increasingly sophisticated workloads without sacrificing quality or reliability.

The goal isn't simply to make models run faster. It's to make better intelligence economically viable at scale.

Continuously improve model intelligence and quality

Model capability is evolving rapidly. Our ability to translate those advances into better outcomes for customers needs to evolve just as quickly.

You'll develop the systems and technical approaches that allow us to systematically improve model quality and effectiveness.

That may include:

Model evaluation, benchmarking, and comparative testing
Model selection and routing based on task complexity and performance
Context engineering, retrieval, and other techniques that improve model effectiveness
Fine-tuning, model adaptation, or specialized ML approaches where warranted
Experimentation frameworks that connect model changes to measurable improvements in production

You'll partner closely with engineers building Relay's agent capabilities to understand where model limitations create execution failures and determine the most effective way to address them.

Engineer reliability into production AI

AI systems are probabilistic. Enterprise operations demand consistency, accountability, and predictable behavior.

You'll build the infrastructure required to operate AI/ML workloads reliably, including observability, failure detection, recovery mechanisms, performance monitoring, and appropriate operational controls.

You'll help establish the engineering standards that allow us to safely introduce new models, evaluate changes, manage infrastructure dependencies, and operate increasingly complex AI systems in production.

Reliability can't be something we add after intelligence improves. It has to be built into the systems that deliver it.

Make AI/ML performance measurable

You'll help establish the technical metrics and feedback loops that tell us whether our AI/ML infrastructure is getting better.

That includes measures such as:

Model accuracy and task-level quality
Inference latency and throughput
Cost per inference and cost per successful resolution
Model reliability, availability, and error rates
Evaluation performance and regression detection
Resource utilization and infrastructure efficiency
Time required to evaluate and deploy new model capabilities

You'll work with Product and Engineering to connect these technical improvements to the outcomes that matter most: more accurate decisions, more reliable execution, and greater autonomous resolution.

What We're Looking For

You may be a strong fit if you:

Have 6+ years of relevant software engineering experience, with meaningful ownership of complex technical systems at the Senior or Staff level.
Have built, deployed, or optimized production AI/ML infrastructure, model-serving systems, inference pipelines, or similarly sophisticated AI systems.
Have exceptional engineering fundamentals and strong proficiency in Python, with experience designing and operating reliable distributed systems.
Understand modern foundation models and the tradeoffs involved in model selection, inference architecture, quality, latency, throughput, scalability, and cost.
Have hands-on experience with cloud infrastructure, containerization, orchestration, and the operational challenges of running production AI workloads.
Understand model evaluation and experimentation, and can connect technical changes to measurable improvements in model quality or system performance.
Can move fluidly between foundational architecture, hands-on implementation, performance debugging, and production operations.
Have operated in an environment where technical requirements evolved rapidly and architectural decisions had to balance immediate needs with long-term scalability.
Have strong opinions, loosely held. You can create clarity with incomplete information, explain technical tradeoffs, and adapt quickly as the technology and evidence evolve.
Hold an unusually high bar for engineering quality while still moving with urgency.

Particularly Compelling

We'd be especially interested if you've:

Built foundational AI/ML infrastructure at a breakout AI company or exceptional technology organization where performance, reliability, and scale were critical.
Worked deeply with technologies such as PyTorch, Hugging Face, vLLM, Triton, Ray, Kubernetes, or comparable model-serving and distributed-computing frameworks.
Designed systems for model optimization, fine-tuning, post-training, evaluation, or continuous improvement of production AI capabilities.
Built infrastructure supporting agentic AI systems, complex model orchestration, or multi-step autonomous workflows.
Delivered measurable improvements in model quality, inference efficiency, infrastructure cost, or production reliability.
Built shared technical capabilities from zero, establishing architectures and engineering standards that other teams could build upon.

Experience with supply chain or logistics is useful. However, exceptional engineering judgment, AI/ML systems depth, and a track record of shipping production infrastructure matter more.

This Probably Isn't the Role for You If

Your AI experience is primarily integrating third-party APIs or building lightweight LLM applications without meaningful infrastructure or systems ownership.
Your background is concentrated in theoretical ML research or offline model development, and you haven't owned the engineering required to bring AI capabilities into production.
You're most comfortable operating within an established infrastructure stack rather than defining foundational systems and architectural patterns.
You think optimizing model performance is exclusively about improving benchmark scores, without considering reliability, latency, cost, and real-world outcomes.
You prefer narrowly scoped implementation work over owning difficult technical problems from architecture through production.

Why BackOps

AI has become remarkably good at generating information.

The next frontier is much harder: AI that can be trusted to do the work.

In the physical economy, that means more than producing the right answer. An agent has to understand what's happening, make a decision, interact with real systems, navigate exceptions, communicate with real people, and remain accountable until the problem is actually resolved.

That creates an unusually challenging engineering problem.

You'll help build the AI/ML foundation that makes this possible, determining how models are deployed, evaluated, optimized, and continuously improved as Relay takes on increasingly sophisticated operational work.

Few engineering teams get the opportunity to shape the underlying intelligence and infrastructure of an emerging AI platform while the category itself is still being invented.

That's the opportunity at BackOps.