Senior Software Engineer, Inference

What this role involves

Senior Software Engineer responsible for leading design reviews, driving architecture, and improving latency, throughput, and reliability of a Kubernetes-native inference platform. The role involves implementing advanced optimizations like speculative decoding and KV-cache reuse, mentoring engineers, and partnering with product and hardware teams to meet strict P99 SLAs.

Skills this posting asks for

• python
• go
• c++
• kubernetes
• ci/cd
• prometheus
• grafana
• opentelemetry
• batching
• caching
• mixed precision
• streaming
• vllm
• triton
• tensorrt-llm
• ray serve
• torchserve
• cuda kernels
• nccl
• sharp
• rdma
• numa
• gpu interconnect

Requirements

• 5 years of experience
• Level: senior

From the employer’s posting

CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of…