Principal System Software Engineer, AI Inference Execution

What this role involves

This role involves developing and optimizing the software stack for d-Matrix's AI compute engine, focusing on inference serving and deployment. The candidate will work with a team of system software experts to build scalable deployment infrastructure and optimize hardware-software co-design. Required expertise includes C/C++/Python, machine learning, distributed systems, and experience with inference servers and deep learning frameworks.

Skills this posting asks for

• c
• c++
• python
• linux
• distributed systems
• machine learning
• tensorrt-llm
• vllm
• sglang
• pytorch
• tensorflow
• onnx runtime
• tensorrt
• nccl
• openmpi
• kubernetes
• ray
• mlops
• system software
• data structures
• computer architecture

Requirements

• 6 years of experience
• Level: principal
• Remote policy: hybrid

From the employer’s posting

At d-Matrix, we are focused on unleashing the potential of generative AI to power the transformation of technology. We are at the forefront of software and hardware innovation, pushing the boundaries of what is possible. Our culture is one of respect and collaboration. We value humility and believ…