oss · 2026-05-11

[feat] Add TensorRT-Engine Qwen3 (dense) model support #

#TensorRT-LLM#Qwen3
  • Add Qwen3ForCausalLM mapping in models/init.py
  • Update config.py to support Qwen3 architecture with qwen3/qwen3_moe types
  • Add Qwen3 weight conversion logic in convert.py
  • Implement Qwen3-specific model modifications in model.py
  • Support attention_bias=False and qk_layernorm=True for Qwen3
  • Enable FP16 and FP8 quantization for Qwen3

Tested with Qwen3-1.7B model successfully. (Support for the Qwen3 MoE architecture will be added in a future update)

Dear Maintainers,

I would like to kindly submit a pull request that adds support for building the Qwen3 model as a TensorRT engine. I would be truly grateful if you could take the time to review it at your convenience.

Thank you very much for your consideration.

ref) vllm vs tensorrt-llm latency (Qwen3-1.7b, L4 GPU, 2048 input_length, 3072 max_length, batch 1)

FP16

  • trt-llm (tensorrt): TTFT: 25.7ms & TPS: 60.9
  • vllm: TTFT: 25.0ms & TPS: 59.2

FP8

  • trt-llm: TTFT: 18.3ms & TPS: 104.9
  • vllm: TTFT: 20.6ms & TPS: 80.2

Related to #5673

Link ↗