@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 00:45 UTC

main

Update `pre-commit` hooks (#12475)

Harry Mellor committed almost 2 years ago

[FlashInfer] Upgrade to 0.2.0 (#11194)

Bowen Wang committed almost 2 years ago

[V1][Metrics] Add initial Prometheus logger (#12416)

Mark McLoughlin committed almost 2 years ago

[Bugfix] Fix gpt2 GGUF inference (#12467)

Isotr0py committed almost 2 years ago

[Bugfix] Fix Granite 3.0 MoE model loading (#12446)

Cyrus Leung committed almost 2 years ago

[Frontend] Support scores endpoint in run_batch (#12430)

Pooya Davoodi committed almost 2 years ago

[V1] Avoid list creation in input preparation (#12457)

Woosuk Kwon committed almost 2 years ago

[DOC] Add link to vLLM blog (#12460)

Yuan Tang committed almost 2 years ago

[Build/CI] Fix libcuda.so linkage (#12424)

Tyler Michael Smith committed almost 2 years ago

[Bugfix/CI] Fix broken kernels/test_mha.py (#12450)

Tyler Michael Smith committed almost 2 years ago

[Bugfix] Disable w16a16 2of4 sparse CompressedTensors24 (#12417)

Tyler Michael Smith committed almost 2 years ago

[Frontend] generation_config.json for maximum tokens(#12242)

Matthew Hendrey committed almost 2 years ago

[Misc] Revert FA on ViT #12355 and #12435 (#12445)

Roger Wang committed almost 2 years ago

← Newer Page 904 of 1120 Older →