A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
Fix `validate-config` pre-commit check (#25157)
Harry Mellor committed about 1 year ago
Fix forward reference warning in documentation (#25150)
Harry Mellor committed about 1 year ago
[Docs] add the parallel sampling usage in LLMEngine and AsyncLLM (#24222)
William Song committed about 1 year ago
[Docs] Fix pooling-params doc references in openai_compatible_server.md (#24939)
Kay Yan committed about 1 year ago
Move `StructuredOutputsConfig` from `config/__init__.py` to `config/structured_outputs.py` (#25153)
Harry Mellor committed about 1 year ago
[Model] Improve Pooling Model (#25149)
Jee Jee Li committed about 1 year ago
[Frontend] Support setting logprobs to -1 (#25031)
Chauncey committed about 1 year ago
silu-v1: Fix EPS not being used during max-reduction (#25069)
Elvir Crnčević committed about 1 year ago
Fix: Add explicit #include <omp.h> for OpenMP compatibility on certain toolchains (#24951)
ihb2032 committed about 1 year ago
[Chore] Cleanup guided namespace, move to structured outputs config (#22772)
Aaron Pham committed about 1 year ago
[Doc] Fix cross-reference warnings (#25058)
Punitvara committed about 1 year ago
Add 'path' option to ImagePrompt data_format (#25081)
Gerard Finol committed about 1 year ago
[Qwen] Add fp8 checkpoint support for qwen3-next. (#25079)
Tao He committed about 1 year ago
[Bugfix] when use s3 model cannot use default load_format (#24435)
rongfu.leng committed about 1 year ago
Retrieve `sliding_window` from text config in Gemma3 MM (#25085)
Harry Mellor committed about 1 year ago
[Docs] Fix API Reference (#25140)
Harry Mellor committed about 1 year ago
[Kernel] Better inf handling for grouped topk cu (#24886)
Lumina committed about 1 year ago
[CLI] Use streaming in CLI chat and completion commands (#23769)
Simon Mo committed about 1 year ago
[benchmark] add peak throughput metrics and plot (#23867)
Simon Mo committed about 1 year ago
[Spec Decode] Efficient padded speculation (#24539)
Benjamin Chislett committed about 1 year ago