A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 15:45 UTC
main
[VLM] Use `SequenceData.from_token_counts` to create dummy data (#8687)
Cyrus Leung committed about 2 years ago
[Kernel] Build flash-attn from source (#8245)
Luka Govedič committed about 2 years ago
[beam search] add output for manually checking the correctness (#8684)
youkaichao committed about 2 years ago
[Core] Factor out common code in `SequenceData` and `Sequence` (#8675)
Cyrus Leung committed about 2 years ago
[MISC] add support custom_op check (#8557)
Kunshang Ji committed about 2 years ago
[Core] Rename `PromptInputs` and `inputs`(#8673)
Cyrus Leung committed about 2 years ago
[Bugfix] Fix incorrect llava next feature size calculation (#8496)
zyddnys committed about 2 years ago
[Hardware][AWS] update neuron to 2.20 (#8676)
omrishiv committed about 2 years ago
[Doc] neuron documentation update (#8671)
omrishiv committed about 2 years ago
[Bugfix][Core] Fix tekken edge case for mistral tokenizer (#8640)
Patrick von Platen committed about 2 years ago
[Bugfix] Config got an unexpected keyword argument 'engine' (#8556)
Pastel! committed about 2 years ago
[Misc] Show AMD GPU topology in `collect_env.py` (#8649)
Cyrus Leung committed about 2 years ago
[Bugfix] Validate SamplingParam n is an int (#8548)
saumya-saran committed about 2 years ago
[CI/Build] Removing entrypoints/openai/test_embedding.py test from ROCm build (#8670)
Alexey Kondratiev(AMD) committed about 2 years ago
Niklas Muennighoff committed about 2 years ago
[Core] Support Lora lineage and base model metadata management (#6315)
Jiaxin Shan committed about 2 years ago
[bugfix] [AMD] add multi-step advance_step to ROCmFlashAttentionMetadata (#8474)
William Lin committed about 2 years ago
[Bugfix] Fix Phi3.5 mini and MoE LoRA inference (#8571)
Amit Garg committed about 2 years ago
[Misc] guard against change in cuda library name (#8609)
bnellnm committed about 2 years ago
[CI/Build] Re-enabling Entrypoints tests on ROCm, excluding ones that fail (#8551)
Alexey Kondratiev(AMD) committed about 2 years ago