[Quantization] Improve BitsAndBytesModelLoader (#20242)

Jee Jee Li committed over 1 year ago

[Bugfix] Fix processor initialization in transformers 4.53.0 (#20244)

Isotr0py committed over 1 year ago

[Bugfix] Skip loading extra parameters for modelopt Qwen3 MoE model (#19598)

noiji committed over 1 year ago

[doc] Add Slack and Forum to the top navigation (#20208)

Reid committed over 1 year ago

[Misc] Fix import (#20233)

Woosuk Kwon committed over 1 year ago

[BUGFIX][DEEPSEEK][MODEL_LOAD] fix w13, w2 weight not initialized assert (#20202)

Chendi.Xue committed over 1 year ago

[Model] support dots1 (#18254)

redmoe-moutain committed over 1 year ago

Fix cuda_archs_loose_intersection when handling sm_*a (#20207)

Huy Do committed over 1 year ago

[Quantization] Add compressed-tensors NVFP4 MoE Support (#19990)

Dipika Sikka committed over 1 year ago

[CI Fix] Try fixing eagle e2e test OOM by reducing block allocation (#20213)

Michael Goin committed over 1 year ago

[Refactor] Create a function util and cache the results for `has_deepgemm`, `has_deepep`, `has_pplx` (#20187)

Wentao Ye committed over 1 year ago

[Bugfix] Correct behavior of GraniteMoeHybrid for TensorParallel execution (#20137)

Stan Wozniak committed over 1 year ago

[Frontend] Generalize `v1/audio/transcriptions` endpoint (#20179)

Nicolò Lucchesi committed over 1 year ago

[CI/Build] Add new CI job to validate Hybrid Models for every PR (#20147)

Thomas Parnell committed over 1 year ago

[BugFix] Fix the incorrect func name in the comments. (config.py) (#20185)

Jiayi Yan committed over 1 year ago

[Bugfix] Properly reject requests with empty list guided_choice (#20195)

Michael Goin committed over 1 year ago

fix ci issue distributed 4 gpu test (#20204)

Wentao Ye committed over 1 year ago

Fix num_token_padding support for static per-tensor scaled_fp8_quant (#20188)

Michael Goin committed over 1 year ago

[Misc] Add type assertion of request_id for LLMEngine.add_request (#19700)

Chales Xu committed over 1 year ago

[CI Fix] Pin tests/models/registry.py MiniMaxText01ForCausalLM to revision due to model changes (#20199)

Michael Goin committed over 1 year ago