A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 00:45 UTC
main
[Misc][Easy] Remove the space from the file name
Lu Fang committed almost 2 years ago
[Bugfix] Better FP8 supported defaults
Lucas Wilkinson committed almost 2 years ago
[ROCm][Kernel] Using the correct warp_size value
Gregory Shtrasberg committed almost 2 years ago
[VLM] Update compatibility with transformers 4.49
Cyrus Leung committed almost 2 years ago
Roger Wang committed almost 2 years ago
[Docs] Drop duplicate [source] links
Russell Bryant committed almost 2 years ago
[VLM] Use shared field to pass token ids to model
Cyrus Leung committed almost 2 years ago
Add: Support for Sparse24Bitmask Compressed Models
Rahul Tuli committed almost 2 years ago
[Hardware][Intel-Gaudi] Enable FusedSDPA support for Intel Gaudi (HPU)
Sanju C Sudhakaran committed almost 2 years ago
Chen Zhang committed almost 2 years ago
[core][distributed] exact ray placement control (#12732)
youkaichao committed almost 2 years ago
Akash kaothalkar committed almost 2 years ago
[Doc] Remove performance warning for auto_awq.md (#12743)
Michael Goin committed almost 2 years ago
[V1][Misc] Shorten `FinishReason` enum and use constant strings (#12760)
Nick Hill committed almost 2 years ago
[Bugfix] Fix OpenVINO model runner (#12750)
Harry Mellor committed almost 2 years ago
[Doc] Update PR Reminder with link to Developer Slack (#12748)
Michael Goin committed almost 2 years ago
[Model][Quant] Fix GLM, Fix fused module mappings for quantization (#12634)
Kyle Sayers committed almost 2 years ago
[Misc] Bump the compressed-tensors version (#12736)
Dipika Sikka committed almost 2 years ago
[VLM] Add MLA with pure RoPE support for deepseek-vl2 models (#12729)
Isotr0py committed almost 2 years ago
Refactor `Linear` handling in `TransformersModel` (#12727)
Harry Mellor committed almost 2 years ago