vLLM Cheatsheet

Q: How do I install vLLM?

Run `pip install vllm`. Requires Python 3.9+ and a CUDA-capable GPU. For the latest features, install from source: `pip install vllm-nightly`. For ROCm (AMD GPUs), use `pip install vllm-rocm`.

Q: How do I choose between tensor parallel and pipeline parallel?

Tensor parallel (`--tensor-parallel-size`) splits each layer across GPUs — use this for single-node multi-GPU (faster, requires high-speed interconnect). Pipeline parallel (`--pipeline-parallel-size`) splits layers across GPU groups — use for models that don't fit in a single node's memory. Combine both for very large models.

Q: How do I use vLLM with LoRA adapters?

Start with `--enable-lora` and `--lora-modules my-adapter=/path/to/lora`. Then send requests with `model` set to the LoRA module name. Multiple LoRA adapters can be loaded simultaneously and selected per-request via the model field.

Q: How do I optimize GPU memory in vLLM?

Key flags: `--gpu-memory-utilization 0.9-0.95` (how much VRAM to use), `--max-model-len` (reduce for smaller contexts to save memory), `--kv-cache-dtype fp8` (reduce KV cache memory by ~50%), `--max-num-seqs` (limit concurrent requests).

Q: How does vLLM's API compare to OpenAI?

vLLM serves a fully OpenAI-compatible API at `/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`, and `/v1/models`. Most OpenAI SDKs and tools can point to vLLM by simply changing the base URL to `http://localhost:8000`.

vLLM is a high-throughput LLM inference serving engine featuring PagedAttention and Continuous Batching for optimal GPU utilization. It supports multi-GPU tensor/pipeline parallelism, hot-loaded LoRA adapters, and an OpenAI-compatible API — the standard choice for production LLM deployment.

Updated: 2026-07-20·22 commands

Quick Start

``bash # Install pip install vllm

# Start serving a model vllm serve meta-llama/Llama-3.1-8b --host 0.0.0.0 --port 8000

# Test with curl curl -s http://localhost:8000/v1/chat/completions \ -d '{"model":"meta-llama/Llama-3.1-8b","messages":[{"role":"user","content":"Hello"}],"max_tokens":100}'

# Check loaded models curl -s http://localhost:8000/v1/models | jq .`

`Model Serving`

vllm serve is the core command. Models can be HuggingFace IDs (e.g., meta-llama/Llama-3.1-8b) or local paths. Key flags: --port, --host, --dtype, --max-model-len, --gpu-memory-utilization.

`Multi-GPU Inference`

For large models, split across GPUs with tensor parallel (--tensor-parallel-size) or pipeline parallel (--pipeline-parallel-size). Both can be combined for very large deployments.

`LoRA Adapter Support`

vLLM supports hot-loading LoRA adapters without restarting. Enable with --enable-lora and register modules with --lora-modules. Multiple adapters coexist; select per-request by specifying the module name as the model.

`Memory Optimization`

--gpu-memory-utilization controls VRAM usage (default 0.9). --kv-cache-dtype fp8 halves KV cache memory. --max-model-len limits context size. --max-num-seqs controls request concurrency.

`OpenAI-Compatible API`

vLLM serves endpoints at /v1/chat/completions, /v1/completions, /v1/embeddings, and /v1/models`. Any OpenAI SDK can target vLLM by changing the base URL.

Startup & Modes(2)

Command	Level	Example
`vllm --help` Show vLLM CLI help	Basic	`vllm --help`
`pip install vllm` Install vLLM	Basic	`pip install vllm`

ai-server(19)

Command	Level	Example
`vllm serve` Start model inference server (core command)	Basic	`vllm serve facebook/opt-125m`
`vllm serve --port` Specify server port (default 8000)	Basic	`vllm serve meta-llama/Llama-3.1-8b --port 8080`
`vllm serve --host` Specify bind host address	Basic	`vllm serve meta-llama/Llama-3.1-8b --host 0.0.0.0`
`vllm serve --tensor-parallel-size` Set tensor parallel size (multi-GPU)	Expert	`vllm serve meta-llama/Llama-3.1-8b --tensor-parallel-size 2`
`vllm serve --pipeline-parallel-size` Set pipeline parallel size	Expert	`vllm serve meta-llama/Llama-3.1-70b --pipeline-parallel-size 4`
`vllm serve --dtype` Specify inference precision	Intermediate	`vllm serve meta-llama/Llama-3.1-8b --dtype bfloat16`
`vllm serve --max-model-len` Set maximum model context length	Intermediate	`vllm serve meta-llama/Llama-3.1-8b --max-model-len 16384`
`vllm serve --gpu-memory-utilization` Set GPU memory utilization (0-1)	Intermediate	`vllm serve phi-4 --gpu-memory-utilization 0.85`
`vllm serve --enable-lora` Enable LoRA adapter support	Expert	`vllm serve meta-llama/Llama-3.1-8b --enable-lora`
`vllm serve --lora-modules` Load specified LoRA modules	Expert	`vllm serve meta-llama/Llama-3.1-8b --enable-lora --lora-modules my-lora=/path/to/lora`
`vllm serve --api-key` Set API key authentication	Intermediate	`vllm serve meta-llama/Llama-3.1-8b --api-key sk-my-key`
`vllm serve --tokenizer-mode` Set tokenizer mode (auto/slow)	Expert	`vllm serve microsoft/phi-4 --tokenizer-mode slow`
`vllm serve --load-format` Specify model load format	Expert	`vllm serve ./local-model --load-format safetensors`
`curl http://localhost:8000/v1/chat/completions` Send OpenAI-compatible chat request to vLLM	Intermediate	`curl -s http://localhost:8000/v1/chat/completions -d '{"model":"meta-llama/Llama-3.1-8b","messages":[{"role":"user","content":"Hello"}]}'`
`curl http://localhost:8000/v1/completions` Send text completion request to vLLM	Intermediate	`curl -s http://localhost:8000/v1/completions -d '{"model":"meta-llama/Llama-3.1-8b","prompt":"Hello","max_tokens":100}'`
`curl http://localhost:8000/v1/models` List models loaded in vLLM server	Basic	`curl -s http://localhost:8000/v1/models \| jq .`
`curl http://localhost:8000/health` Check vLLM server health	Basic	`curl -s http://localhost:8000/health`
`vllm serve --kv-cache-dtype` Specify KV cache dtype (auto/fp8)	Expert	`vllm serve meta-llama/Llama-3.1-8b --kv-cache-dtype fp8`
`vllm serve --max-num-seqs` Set max concurrent sequences	Expert	`vllm serve meta-llama/Llama-3.1-8b --max-num-seqs 32`

ai-eval(1)

Command	Level	Example
`vllm bench` vLLM benchmark	Expert	`vllm bench latency --model meta-llama/Llama-3.1-8b`

FAQ

How do I install vLLM?

How do I choose between tensor parallel and pipeline parallel?

How do I use vLLM with LoRA adapters?

How do I optimize GPU memory in vLLM?

How does vLLM's API compare to OpenAI?

This cheatsheet is compiled from official tool documentation. Last updated: 2026-07-20.