https://xuquant.com/en/tags/attention/Recent content in Attention on Xu'Bloghttps://xuquant.com/og-default.pnghttps://xuquant.com/og-default.pngHugo -- 0.152.2enSat, 07 Mar 2026 14:00:00 +0800https://xuquant.com/en/posts/foundation-models/qwen3-vs-qwen3-5-architecture/Sat, 07 Mar 2026 14:00:00 +0800https://xuquant.com/en/posts/foundation-models/qwen3-vs-qwen3-5-architecture/A deep architectural comparison of Qwen3.5 versus Qwen3, examining hybrid attention, joint multimodal training, high-sparsity MoE, and partial RoPE across attention, vision, and MoE dimensionshttps://xuquant.com/en/posts/foundation-models/deepseek_series1_mla/Sat, 13 Sep 2025 10:00:00 +0800https://xuquant.com/en/posts/foundation-models/deepseek_series1_mla/MLA from the deployment perspective of DeepSeek V2/V3: KV cache compression ratio, inference throughput, engineering comparison with GQA/MQA, and real-world gains under long context. The mathematical derivation is in the companion article.