Software Engineer, Inference AI/ML at CoreWeave in Sunnyvale, CA

Summary: CoreWeave is The Essential Cloud for AI™, delivering a platform that enables innovators to build and scale AI. The role involves joining the Inference team to implement features and improvements for model serving on the GPU platform, with a focus on latency, reliability, and cost. Responsibilities: - Implement well-scoped features and fixes in Python/Go/C++ for model-serving services (e.g., Triton, vLLM, TensorRT-LLM, Ray Serve) - Write tests, code comments, and short design docs; participate in code reviews - Add basic metrics and dashboards; assist with alarms and runbooks - Follow on-call runbooks and learn incident response in a guided rotation - Contribute to performance experiments (e.g., request batching, concurrency, caching) with guidance Required Qualifications: - BS/MS in CS, EE, or related field, or equivalent practical experience - Foundations in data structures, algorithms, and networked services - Experience with Python or Go (C++ a plus) and Linux fundamentals; Git/CI basics - Exposure to containers and Kubernetes (coursework or projects welcome) - Curiosity about GPU inference concepts (micro-batching, KV cache, streaming) Preferred Qualifications: - Internship or project that deployed a microservice or ML inference demo - Coursework/research with PyTorch or TensorFlow; simple CUDA projects a plus - Familiarity with Grafana/Prometheus/OpenTelemetry or similar tooling Required Skills: Python, Go, C++, Data Structures, Algorithms, Networked Services, Linux Fundamentals, Git, Continuous Integration, Containers, Kubernetes, GPU Inference Concepts Benefits: Medical, dental, and vision insurance - 100% paid for by CoreWeave, Company-paid Life Insurance, Voluntary supplemental life insurance, Short and long-term disability insurance, Flexible Spending Account, Health Savings Account, Tuition Reimbursement, Ability to Participate in Employee Stock Purchase Program (ESPP), Mental Wellness Benefits through Spring Health, Family-Forming support provided by Carrot, Paid Parental Leave, Flexible, full-service childcare support with Kinside, 401(k) with a generous employer match, Flexible PTO, Catered lunch each day in our office and data center locations, A casual work environment, A work culture focused on innovative disruption