A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 04:15 UTC
main
fix `get_ip` error in pure ipv6 environment (#2931)
Jingru committed over 2 years ago
张大成 committed over 2 years ago
Roy committed over 2 years ago
Don't use cupy when `enforce_eager=True` (#3037)
Roy committed over 2 years ago
[Minor] Remove unused config files (#3039)
Roy committed over 2 years ago
[Minor] Remove gather_cached_kv kernel (#3043)
Woosuk Kwon committed over 2 years ago
Optimize Triton MoE Kernel (#2979)
Philipp Moritz committed over 2 years ago
Add LogProbs for Chat Completions in OpenAI (#2918)
Jared Moore committed over 2 years ago
Port metrics from `aioprometheus` to `prometheus_client` (#2730)
Harry Mellor committed over 2 years ago
[Fix] Fissertion on YaRN model len (#2984)
Woosuk Kwon committed over 2 years ago
Fix nvcc not found in vlm-openai image (#2781)
zhaoyang-star committed over 2 years ago
Include tokens from prompt phase in `counter_generation_tokens` (#2802)
Ronen Schaffer committed over 2 years ago
Remove Flash Attention in test env (#2982)
Woosuk Kwon committed over 2 years ago
[FIX] Fix a bug in initializing Yarn RoPE (#2983)
44670 committed over 2 years ago
Optimize GeGLU layer in Gemma (#2975)
Woosuk Kwon committed over 2 years ago
chore(vllm): codespell for spell checking (#2820)
Massimiliano Pronesti committed over 2 years ago
Use Llama RMSNorm custom op for Gemma (#2974)
Woosuk Kwon committed over 2 years ago
Migrate MistralForCausalLM to LlamaForCausalLM (#2868)
Roy committed over 2 years ago
Added early stopping to completion APIs (#2939)
Mustafa Eyceoz committed over 2 years ago
Ronen Schaffer committed over 2 years ago