oss · 2026-05-11
[feat] Add TensorRT-Engine Qwen3 (dense) model support #
- Add Qwen3ForCausalLM mapping in models/init.py
- Update config.py to support Qwen3 architecture with qwen3/qwen3_moe types
- Add Qwen3 weight conversion logic in convert.py
- Implement Qwen3-specific model modifications in model.py
- Support attention_bias=False and qk_layernorm=True for Qwen3
- Enable FP16 and FP8 quantization for Qwen3
Tested with Qwen3-1.7B model successfully. (Support for the Qwen3 MoE architecture will be added in a future update)
Dear Maintainers,
I would like to kindly submit a pull request that adds support for building the Qwen3 model as a TensorRT engine. I would be truly grateful if you could take the time to review it at your convenience.
Thank you very much for your consideration.
ref) vllm vs tensorrt-llm latency (Qwen3-1.7b, L4 GPU, 2048 input_length, 3072 max_length, batch 1)
FP16
- trt-llm (tensorrt): TTFT: 25.7ms & TPS: 60.9
- vllm: TTFT: 25.0ms & TPS: 59.2
FP8
- trt-llm: TTFT: 18.3ms & TPS: 104.9
- vllm: TTFT: 20.6ms & TPS: 80.2
Related to #5673