@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

main

Add qwen2 (#2495)

Junyang Lin committed almost 3 years ago

Fix https://github.com/vllm-project/vllm/issues/2540 (#2545)

Philipp Moritz committed almost 3 years ago

migrate pydantic from v1 to v2 (#2531)

Jannis Schönleber committed almost 3 years ago

[Fix] Keep `scheduler.running` as deque (#2523)

Nick Hill committed almost 3 years ago

Add `group` as an argument in broadcast ops (#2522)

Junda Chen committed almost 3 years ago

Add benchmark serving to CI (#2505)

Simon Mo committed almost 3 years ago

Simplify broadcast logic for control messages (#2501)

Zhuohan Li committed almost 3 years ago

refactor complemention api for readability (#2499)

Simon Mo committed almost 3 years ago

[BugFix] Fix abort_seq_group (#2463)

ljss committed almost 3 years ago

Don't download both safetensor and bin files. (#2480)

Nikola Borisov committed almost 3 years ago

[Neuron] Add an option to build with neuron (#2065)

Liangfu Chen committed almost 3 years ago

fix: fix some args desc (#2487)

zspo committed almost 3 years ago

Minor fix in prefill cache example (#2494)

Jason Zhu committed almost 3 years ago

fix stablelm.py tensor-parallel-size bug (#2482)

YingchaoX committed almost 3 years ago

[Experimental] Prefix Caching Support (#1669)

shiyi.c_98 committed almost 3 years ago

OpenAI Server refactoring (#2360)

FlorianJoncour committed almost 3 years ago

← Newer Page 1087 of 1120 Older →