AI System Solution Architect

Join Cango Inc as a Senior Solutions Architect focusing on LLM and diffusion model inference on large-scale GPU clusters.

Responsibilities

• Design end-to-end technical architecture for LLM and Diffusion model inference on large-scale GPU clusters.
• Develop innovative solutions in KV Cache management, distributed scheduling, pipelining / batching strategies, memory allocation, and P2P / IB communication.
• Architect a multi-tenant serving framework that balances throughput, latency, and cost.
• Define product positioning and differentiation based on industry trends and company strategy.
• Develop technical evolution plans (e.g., token streaming like vLLM, syntax parsing like SGLang, Diffusion acceleration).
• Align closely with internal GPU infrastructure and business teams to ensure timely product delivery.
• Lead performance engineering efforts including NCCL tuning, NUMA binding, CUDA kernel optimization.
• Drive cross-team collaboration (GPU kernel, compiler, distributed system, frontend APIs) to ensure system stability and scalability.
• Organize benchmarking and performance testing against industry leaders (vLLM, SGLang, TensorRT, etc.).
• Guide engineering team on implementation strategies, experimental methodologies, and optimization pathways.
• Engage with open-source communities and contribute core components to enhance technical influence.
• Communicate directly with North America-based clients to understand their needs for AI inference, training, and deployment.
• Translate customer needs into internal implementation plans and coordinate across operations, engineering, and delivery teams.

Qualifications

Preferred

What We Offer

#J-18808-Ljbffr