Senior Software Engineer - ML Infrastructure

What this role involves

This role involves designing and building production-grade inference infrastructure for SambaNova's RDU architecture. Responsibilities include ownership of integration with serving frameworks like vLLM, public API surfaces, and accuracy verification. The engineer will partner with ML, compiler, and product teams to deliver inference features from prototype to production.

Skills this posting asks for

• python
• vllm
• tensorrt-llm
• sglang
• distributed systems
• llm inference
• rdu architecture
• speculative decoding
• constrained decoding
• function calling
• prompt caching
• long-context inference
• api design
• accuracy testing
• compiler
• runtime

Requirements

• 5 years of experience
• Level: senior

From the employer’s posting

The era of pervasive AI has arrived. In this era, organizations will use generative AI to unlock hidden value in their data, accelerate processes, reduce costs, drive efficiency and innovation to…