A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 12, 21:00 UTC
main
cleanup at::Tag::needs_fixed_stride_order (#28974)
Boyuan Feng committed 12 months ago
[CI/Build][AMD] Fix import errors in tests/kernels/attention (#29032)
rasmith committed 12 months ago
[Bug] Fix torch dynamo warning Dynamo detected a call to a `functools.lru_cache` (#29038)
Wentao Ye committed 12 months ago
[ci][amd] fix EPLB execution test (#28742)
Bradley D committed 12 months ago
Pleaplusone committed 12 months ago
[V0 Deprecation] Remove `num_lookahead_slots` (#29000)
Cyrus Leung committed 12 months ago
[Frontend] Allow parsed tool arguments (#28820)
Quentin Gallouédec committed 12 months ago
[Model][QwenVL] Replace `torch.repeat_interleave` with faster `np.repeat` (#28964)
Lukas Geiger committed 12 months ago
[Bugfix] Revert custom attention mask for gemma3-mm (#28995)
Isotr0py committed 12 months ago
[Bugfix] Handle broken frames in video loading (#29001)
Canlin Guo committed 12 months ago
[Bugfix] Fix precision loss in LoRA-wrapped RowParallelLinear by fusing bias into GEMM (#28972)
prashanth058 committed 12 months ago
Consolidate Nvidia ModelOpt quant config handling for all quantization methods (#28076)
Shengliang Xu committed 12 months ago
[Feat] Iteration-level profiling for Torch and CUDA profiler (#28987)
Benjamin Chislett committed 12 months ago
[cpu][ci] Add initial set of tests for Arm CPUs (#28657)
Fadi Arafeh committed 12 months ago
[AMD] Use Decoupled Kernel Block Size to Support AITER MLA block_size=1 (#27715)
Qiang Zhang committed 12 months ago
Kuntai Du committed 12 months ago
[torchao] fix safetensors for sharding (#28169)
liangel-02 committed 12 months ago
Nick Hill committed 12 months ago
[GC Debugger] Simply and improve GC Debugger Utils (#29029)
Jialin Ouyang committed 12 months ago
[Refactor] Optimize `select_experts` (#28069)
Wentao Ye committed 12 months ago