SGLang Cookbook

🎯 What You’ll Find Here

This cookbook aggregates battle-tested SGLang recipes covering:

• Models: Mainstream LLMs and Vision-Language Models (VLMs)
• Use Cases: Inference serving, deployment strategies, multimodal applications
• Hardware: GPU and CPU configurations, optimization for different accelerators
• Best Practices: Configuration templates, performance tuning, troubleshooting guides

Each recipe provides step-by-step instructions to help you quickly implement SGLang solutions for your specific requirements.

Guides

Autoregressive Models

Qwen

• Qwen3
• Qwen3-Next
• Qwen3-VL
• Qwen3-Coder-480B-A35B
• Qwen2.5-VL

DeepSeek

• DeepSeek-V3.2
• DeepSeek-V3.1
• DeepSeek-V3
• DeepSeek-R1
• DeepSeek-OCR

Llama

• Llama4-Scout
• Llama3.3-70B
• Llama3.1

GLM

• GLM-Glyph
• GLM-4.5
• GLM-4.5V
• GLM-4.6
• GLM-4.6V
• GLM-4.7
• GLM-4.7-Flash NEW

OpenAI

• gpt-oss

Moonshotai

• Kimi-K2
• Kimi-Linear

MiniMax

• MiniMax-M2

NVIDIA

• Nemotron-Nano-3-30B-A3B

Ernie

• Ernie4.5
• Ernie4.5-VL

InternVL

• InternVL3.5

InternLM

• Intern-S1

Jina AI

• Jina-reranker-m0

Mistral

• Mistral-3
• Devstral 2

Xiaomi

• MiMo-V2-Flash

FlashLabs

• Chroma 1.0NEW

Diffusion Models

FLUX

• FLUX

Qwen-Image

• Qwen-Image
• Qwen-Image-Edit

Wan

• Wan2.1
• Wan2.2

Z-Image

• Z-Image

Benchmarks

• Diffusion Model Benchmark
• LLM Benchmark

Reference

• Server arguments - Understanding all the arguments

🚀 Quick Start

1. Browse the recipe index above to find your model
2. Follow the step-by-step instructions in each guide
3. Adapt configurations to your specific hardware and requirements
4. Join our community to share feedback and improvements

🤝 Contributing

We believe the best documentation comes from practitioners. Whether you’ve optimized SGLang for a specific model, solved a tricky deployment challenge, or discovered performance improvements, we encourage you to contribute your recipes!

Ways to contribute:

• Add a new recipe for a model not yet covered
• Improve existing recipes with additional tips or configurations
• Report issues or suggest enhancements
• Share your production deployment experiences

To contribute:

# Fork the repo and clone locally
git clone https://github.com/YOUR_USERNAME/sglang-cookbook.git
cd sglang-cookbook

# Create a new branch
git checkout -b add-my-recipe

# Add your recipe following the template in DeepSeek-V3.2
# Submit a PR!

🛠️ Local Development

Prerequisites

• Node.js >= 18.0
• Mintlify CLI

Setup and Run

Install the Mintlify CLI and start the development server:

# Install Mintlify CLI globally
npm i -g mint

# Start development server (hot reload enabled)
mint dev

The site will automatically open in your browser at http://localhost:3000.

📖 Resources

• SGLang GitHub
• SGLang Documentation
• Community Slack/Discord

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

---

Let’s build this resource together! 🚀 Star the repo and contribute your recipes to help the SGLang community grow.