Commit Graph

  • f150107efd
    [ROCm] Fix cu_seqlens_q off-by-one in AITER FA speculative decode path (#39120) main Andrew Barnes 2026-04-19 14:34:33 -04:00
  • d1135a5087
    Fix MoE backend selection for LoRA (unquantized MoE) (#40273) danisereb 2026-04-19 20:18:40 +03:00
  • 982beae809
    Optimize nemotron VL image/video preprocessing (#40283) Netanel Haber 2026-04-19 18:06:20 +03:00
  • 45232a454e
    [FEAT] [Perf] [Gemma4] Fused Gemma4 Routing Function Triton (#39083) TJian 2026-04-19 17:57:39 +08:00
  • 03ce1c6ed9
    [Bugfix] Kimi-K2 tool parser streaming - fix token leakage, argument truncation, and content dropping (#38579) Flora Feng 2026-04-19 04:30:27 -04:00
  • 4353c9cb4a
    [KV Offload] Pass request context (#39185) omerpaz95 2026-04-19 08:54:59 +03:00
  • 4b7f5ea1a0
    [KV Connector] Allow metrics of multiple connectors of same types in multi connector. (#40010) omerpaz95 2026-04-19 07:49:10 +03:00
  • 38907e4391
    [Frontend] Preserve structured output special tokens in offline LLM.chat (#39352) Luciano Martins 2026-04-18 20:46:07 -03:00
  • d0359f3e04
    [Bugfix] Guard mxfp4_experts_quant bindings on ENABLE_NVFP4_SM100 (#40191) mysterious hhhh 2026-04-19 04:58:46 +08:00
  • ed0622e3a8
    [Attention] TurboQuant: remove redundant random signs, add prior art attribution (#40194) Dan Alistarh 2026-04-18 20:31:59 +02:00
  • b5f6c5f834
    Added general ND x ND matmul and unit test for it (#39909) Yusuf Mohammad 2026-04-18 15:05:21 +01:00
  • bfde49e287
    [DOC] Add fuse_minimax_qk_norm (#39782) Jee Jee Li 2026-04-18 15:41:37 +08:00
  • 153ba7f0f3
    [Refactor] Drop direct dependency on librosa (#39079) Nick Cao 2026-04-18 02:55:38 -04:00
  • 87518c3027
    [ZenCPU] AMD Zen CPU Backend with supported dtypes via zentorch weekly (#39967) Chinmay-Kulkarni-AMD 2026-04-18 11:52:37 +05:30
  • aeee7ef939
    [Bugfix] Fix k_proj's bias for GLM-ASR (#40160) Rishapveer Singh 2026-04-18 07:34:33 +02:00
  • cda19ecf4d
    [Doc] Fix outdated source reference comment in anthropic/serving.py (#40189) z1ying 2026-04-17 22:31:13 -07:00
  • 80b18230e0
    [Frontend] Add multimodal support to /inference/v1/generate endpoint (#38405) Nithin Chalapathi 2026-04-17 20:31:56 -07:00
  • d0697cc7b6
    [Doc] Add Realtime Transcription section to supported_models.md (#39845) z1ying 2026-04-17 20:26:14 -07:00
  • b0755523dc
    [Core] Reduce mm scheduler, get_num_embed overhead (#40143) milesial 2026-04-17 20:25:49 -07:00
  • 993859ceb0
    [XPU] fix all_reduce all-zero accuracy issue under torch.compile (#39844) Chaojun Zhang 2026-04-18 10:33:07 +08:00
  • 48a65ccb02
    [CI] Speed up test_fused_marlin_moe (#40178) Michael Goin 2026-04-17 22:26:21 -04:00
  • 55842a8d69
    [XPU]fake impl for xpu fp8_gemm (#39984) Xinyu Chen 2026-04-18 08:53:56 +08:00
  • 1f45e83756
    Remove outdated tests test_mixtral_moe and test_duplicated_ignored_sequence_group (#40175) Michael Goin 2026-04-17 19:49:43 -04:00
  • a8bffaa133
    [Kernel] Add MXFP4 W4A4 CUTLASS MoE kernel for SM100 (#37463) Michael Goin 2026-04-17 19:42:32 -04:00
  • 5cdddddd4a
    [Kernel] [Helion] Force disable HOP path due to performance regression (#40171) Yanan Cao 2026-04-17 14:36:49 -07:00
  • 6ef1efd51f
    [ROCm] Fix TurboQuant on ROCm: backend routing, flash-attn compat, int64 overflow (#39953) aditi-amd 2026-04-17 13:08:37 -07:00
  • 58da4ee047
    [AMD][CI] Update DeepEP branch (#38396) Ryan Rock 2026-04-17 14:30:20 -05:00
  • 1ae11e2bfc
    [ROCm][CI] Build fastsafetensors from source so it links against libamdhip64 (#39978) Andreas Karatzas 2026-04-17 14:30:08 -05:00
  • 251c18d1f8
    skip fp8e4b15 on xpu (#39957) Xinyu Chen 2026-04-18 00:55:08 +08:00
  • 512765d52d
    [Misc][UX] Map mimo reasoning and tooling parsers (#40089) Roger Wang 2026-04-17 09:49:21 -07:00
  • 640cc9dd7d
    feat: Add LoRA support for Gemma4ForConditionalGeneration (#39291) allgather 2026-04-17 09:39:19 -07:00
  • ceade1952c
    [BugFix] Support custom tool parsers when tool_choice is required and named function (#39870) Jared Wen 2026-04-18 00:38:10 +08:00
  • 747256bb5d
    [Bugfix][Core] Fix stuck chunked pipeline parallelism with async scheduling (#38726) Jing Wang 2026-04-18 00:02:50 +08:00
  • 1174723eba
    Fix TURBOQUANT backend selection in cuda.py (#40060) Michael Goin 2026-04-17 10:31:41 -04:00
  • 6b2b7bd0eb
    Add nvfp4 support to reshape_and_cache_flash (#37332) sychen52 2026-04-17 07:28:00 -07:00
  • 70770268c3
    Add @bbrowning to CODEOWNERS (#40141) Ben Browning 2026-04-17 09:51:48 -04:00
  • 7a51b3e415
    [Bugfix] Fix empty delta detection in Qwen3XMLToolParser streaming (#40090) Chauncey 2026-04-17 21:34:55 +08:00
  • d02421a7db
    [CPU] Refactor CPU affinity and memory management (#39781) Li, Jiang 2026-04-17 21:01:08 +08:00
  • b1dc87a098
    [Models][Gemma4] Prevent GPU/CPU sync in embed_input_ids (#39234) Lukas Geiger 2026-04-17 13:37:21 +01:00
  • 79a5b63253
    [kv_offload]: Fix num CPU blocks for UniformTypeKVCacheSpecs (#39617) Or Ozeri 2026-04-17 15:13:55 +03:00
  • c0c98b8b9a
    [Bugfix] Add Marlin kernel in block scaled mm kernel selection. (#40105) Maral 2026-04-17 18:20:32 +08:00
  • 8d2cff8140
    [Examples] Resettle Observability examples. (#40123) wang.yuqi 2026-04-17 18:13:31 +08:00
  • 4f436782af
    [Misc] Improve new PR bot trigger condition (#40114) Cyrus Leung 2026-04-17 16:56:22 +08:00
  • bf45e6d0a5
    [Doc] Add Gemma 4 to supported models list (#39607) z1ying 2026-04-16 22:42:52 -07:00
  • 978a4462bb
    [CI Failure] Fix Plugin Tests (2 GPUs) Failure (#40083) wang.yuqi 2026-04-17 12:17:39 +08:00
  • 1948d0c467
    [UX] Defer some imports on CLI paths to save ~2s (#40056) Michael Goin 2026-04-16 22:48:37 -04:00
  • 4c47710bf7
    [CI/Build] Apply ruff formatter to pass pre-commit (#40078) Shinichi Hemmi 2026-04-17 09:54:32 +09:00
  • bf9a5ddb24
    [MLA] Optimize mla indexer prepare uniform decode for MTP > 1 (#39458) Giancarlo Delfin 2026-04-16 16:27:51 -07:00
  • 79e799ebbd
    [Bugfix] Temporarily disable B200 fp4 MoE layer tests (#40057) bnellnm 2026-04-16 19:26:55 -04:00
  • c4e601c73c
    Bugfix: Parakeet: .conv.pointwise/depthwise_conv1/2.bias weigths can exist even if convolution_bias=False (#40007) Netanel Haber 2026-04-17 02:22:05 +03:00
  • 29057d3bee
    [Compilation] Add Unit Tests for VllmFusionPatternMatcherPass (#39692) BadrBasowid 2026-04-17 06:57:16 +08:00
  • 219bb5b8c0
    [Misc] Update committers.md (#40058) Matthew Bonanni 2026-04-16 16:48:41 -04:00
  • ad2b1277f9
    [Quantization] Consolidate experts_int8 with fp8 online quantization (#38463) Asaf Gardin 2026-04-16 23:12:20 +03:00
  • b897f00c9c
    Gate SSU dispatch setup (#40039) roikoren755 2026-04-16 23:06:01 +03:00
  • adf9bb3c57
    [CI] Add weight transfer tests to CI (#39821) Sumanth R Hegde 2026-04-16 12:51:45 -07:00
  • b16fda62b7
    [Misc] Add @sfeng33 to CODEOWNERS (#40048) Flora Feng 2026-04-16 15:25:29 -04:00
  • de111f3246
    [Bugfix] Fix bench_serve UTF-8 decode crash on split multi-byte chars (#38732) Yufeng He 2026-04-17 03:01:25 +08:00
  • afabb5f45a
    [bugfix] Normalize tool message content from array to string format (#39899) Jared Wen 2026-04-17 02:54:39 +08:00
  • 3abb7560c0
    [Bugfix] Fix audioflamingo test (#40052) Roger Wang 2026-04-16 11:53:58 -07:00
  • 617d1c2ff1
    [Misc] Move pyav and soundfile to common requirements (#39997) Isotr0py 2026-04-16 23:52:37 +08:00
  • 692db29cd4
    [Bugfix] Fix Ray compiled-DAG SHM channel stalls by detaching zero-copy np.ndarray logprobs buffers (#35736) Nikita Shapovalov 2026-04-16 17:49:29 +02:00
  • 82531edbfb
    [Refactor] Remove resampy dependency (#39524) Isotr0py 2026-04-16 23:48:17 +08:00
  • 3daca38e22
    [Misc] toy_proxy_server handle min_tokens (#39706) Nicolò Lucchesi 2026-04-16 17:08:22 +02:00
  • a302a8fd1b
    [Bugfix] Fix LLM priority normalization for single-string prompts (#40011) daiyu1111 2026-04-16 22:56:06 +08:00
  • 4e8c3f1c19
    [Frontend][last/5] Improve pooling entrypoints | clean up. (#39675) wang.yuqi 2026-04-16 22:53:23 +08:00
  • 5e5afafa21
    [Doc] add docs for online quant frontend (#39736) Vasiliy Kuznetsov 2026-04-16 10:52:58 -04:00
  • 324a3d2bd8
    [CI/Build] Improve stability of CPU tests (#39966) Li, Jiang 2026-04-16 21:50:36 +08:00
  • 4269b79409
    [Model] Use mm_features to compute mrope positions for PaddleOCR-VL (#39888) grYe99 2026-04-16 21:14:00 +08:00
  • edc3648966
    [Kernel][Helion] Fix inductor fusion of Helion HOP (#39944) Yanan Cao 2026-04-16 04:41:26 -07:00
  • 9965f501a8
    [Nixl] Bump Nixl version to 0.10.1 (#39922) Nicolò Lucchesi 2026-04-16 12:53:21 +02:00
  • 17d87168d2
    [Model] Use mm_features for Keye-VL and Keye-1.5-VL M-RoPE (#39869) lalit10 2026-04-16 02:16:06 -07:00
  • 98700c6105
    Fix #33773: Replace unconditional pandas import with PlaceholderModule (#39990) Netanel Haber 2026-04-16 12:06:51 +03:00
  • 10e49d2638
    [Docs] Update PR template to remove release notes google docs (#39982) Simon Mo 2026-04-16 00:22:03 -07:00
  • 8d7c962833
    [Bugfix] Accept **kwargs in MiniMaxM2Parser.__init__() (#39861) Tim Messerschmidt 2026-04-16 09:18:32 +02:00
  • f4ddaf8cf7
    [XPU] use spawn multiproc method on xpu (#39671) Xinyu Chen 2026-04-16 14:42:07 +08:00
  • 2cdf86044d
    Add Jina Embeddings v5 model support (fixes #38633) (#39575) Abhijit Roy 2026-04-16 12:07:10 +05:30
  • 7845379230
    [Bugfix] add support for 'num_attention_groups' in ModelArchConfigConvertorBase for Step3p5 (#39796) realliujiaxu 2026-04-16 13:48:00 +08:00
  • 4b7ca37bd4
    [CPU][IBM Z][Dockefile][Docs] Fix s390x builds for torch 2.11 and update docs for s390x (#39910) R3hankhan 2026-04-16 10:56:21 +05:30
  • 445b7093fd
    [perf][cpu] Accelerate BF16 GELU with LUT impl on Arm CPUs (#37469) Fadi Arafeh 2026-04-16 07:26:17 +02:00
  • 18013df6ae
    [Bugfix] Reject empty tools array with HTTP 400 (#39780) jigangz 2026-04-15 21:08:04 -07:00
  • c0722f22de
    [Mistral Grammar] Fix tool and reasoning parsing (#39217) Julien Denize 2026-04-16 06:05:04 +02:00
  • 951dca8019
    [compile] Invoke split FX graph by codegen. (#38657) Zhengxu Chen 2026-04-16 00:03:41 -04:00
  • 5f7fab881a
    [ROCm][FEAT] Integrate aiter gemm w8a8 ptpc (#33773) vllmellm 2026-04-16 09:55:29 +08:00
  • 343f65234b
    [Model Runner V2][BugFix] fix num_sampled dtype for probabilistic rej… (#39951) Giancarlo Delfin 2026-04-15 18:09:11 -07:00
  • 19fa90ed0d
    [Quantization] - Layerwise reloading of Attention/KV quantized models (#38995) Asaf Gardin 2026-04-16 04:03:32 +03:00
  • 03f8d3a548
    Update to transformers v5 (#30566) Harry Mellor 2026-04-16 00:29:15 +01:00
  • 6dc9491406
    [Model] Fix Gemma 4 token repetition by dynamic BOS injection for PT models (#39842) Luciano Martins 2026-04-15 20:13:07 -03:00
  • 27c0ca50a0
    Update registry for Nemotron-v3 VL Nano/Super (#39747) Collin McCarthy 2026-04-15 16:09:11 -07:00
  • 7c636432c6
    [CI Bug] fix flaky test (#39938) Wentao Ye 2026-04-15 17:20:06 -04:00
  • c77e596e2e
    [FlashAttention] Don't overwrite flash_attn_interface.py when installing precompiled (#39932) Matthew Bonanni 2026-04-15 16:43:15 -04:00
  • ac3dac545b
    [Bugfix][Perf] Indexer upcast WK to BF16 for fusion (#38928) Benjamin Chislett 2026-04-15 16:39:32 -04:00
  • 39ac640490
    [Bug] Fix batch invariant test issue, bs=1 with max_seq_num = 1 (#39320) Wentao Ye 2026-04-15 16:28:43 -04:00
  • 0b790a2501
    [Speculative Decoding] Add DFlash speculators config parsing (#38300) zhanqiuhu 2026-04-15 16:22:15 -04:00
  • 41488f2acd
    [Bugfix][NIXL] Fix _logical_to_kernel_block_ids conversion for non-mamba models (#39724) zhanqiuhu 2026-04-15 16:08:58 -04:00
  • 102d51c9f3
    [CI] Only build release Docker images when NIGHTLY=1 (#39882) Kevin H. Luu 2026-04-15 12:01:13 -07:00
  • 55e1a8e103
    [Mooncake] Fix mixed MLA+Eagle block-size validation (#39596) Zhewen Li 2026-04-15 11:36:47 -07:00
  • 21e5a9f48e
    Bug/test eagle dp v2 (#39838) Monishver 2026-04-15 13:48:12 -04:00
  • 8ad6ff0037
    [Test] Fix @create_new_process_for_each_test("fork") in interactive shell pipeline (#29130) Mark McLoughlin 2026-04-15 17:22:20 +01:00
  • f2145efcb6
    [BugFix] KeyError on scope["method"] for realtime api websocket in AuthenticationMiddleware (#36934) daniebrill 2026-04-15 18:15:01 +02:00
  • ed33310552
    [KVConnector][LMCache] Propagate cache_salt through MP connector for per-user cache isolation (#39837) Roy Huang 2026-04-15 09:10:49 -07:00