@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

main

Support Orion model (#2539)

张大成 committed over 2 years ago

Fix stablelm (#3038)

Roy committed over 2 years ago

[Minor] Remove gather_cached_kv kernel (#3043)

Woosuk Kwon committed over 2 years ago

Optimize Triton MoE Kernel (#2979)

Philipp Moritz committed over 2 years ago

Add LogProbs for Chat Completions in OpenAI (#2918)

Jared Moore committed over 2 years ago

[Fix] Fissertion on YaRN model len (#2984)

Woosuk Kwon committed over 2 years ago

Fix nvcc not found in vlm-openai image (#2781)

zhaoyang-star committed over 2 years ago

Remove Flash Attention in test env (#2982)

Woosuk Kwon committed over 2 years ago

Optimize GeGLU layer in Gemma (#2975)

Woosuk Kwon committed over 2 years ago

chore(vllm): codespell for spell checking (#2820)

Massimiliano Pronesti committed over 2 years ago

Use Llama RMSNorm custom op for Gemma (#2974)

Woosuk Kwon committed over 2 years ago

Added early stopping to completion APIs (#2939)

Mustafa Eyceoz committed over 2 years ago

Update comment (#2934)

Ronen Schaffer committed over 2 years ago

← Newer Page 1081 of 1120 Older →