@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

main

[Minor] Remove unused code in attention (#2384)

Woosuk Kwon committed almost 3 years ago

Fix eager mode performance (#2377)

Woosuk Kwon committed almost 3 years ago

Changed scheduler to use deques instead of lists (#2290)

Nadav Shmayovits committed almost 3 years ago

Ensure metrics are logged regardless of requests (#2347)

Iskren Ivov Chernev committed almost 3 years ago

Miner fix of type hint (#2340)

ljss committed almost 3 years ago

Bump up to v0.2.7 (#2337)

Woosuk Kwon committed almost 3 years ago

Revert the changes in test_cache (#2335)

Woosuk Kwon committed almost 3 years ago

Remove unused const TIMEOUT_TO_PREVENT_DEADLOCK (#2321)

Ronen Schaffer committed almost 3 years ago

Remove deprecated parameter: concurrency_count (#2315)

Ronen Schaffer committed almost 3 years ago

Enable CUDA graph for GPTQ & SqueezeLLM (#2318)

Woosuk Kwon committed almost 3 years ago

Support GPT-NeoX Models without attention biases (#2301)

Jong-hun Shin committed almost 3 years ago

[BUGFIX] Fix communication test (#2285)

Zhuohan Li committed almost 3 years ago

[BUGFIX] Fix the path of test prompts (#2273)

Zhuohan Li committed almost 3 years ago

[BUGFIX] Fix API server test (#2270)

Zhuohan Li committed almost 3 years ago

← Newer Page 1089 of 1120 Older →