本文展示如何设置文章关键词。
frontmatter 关键词设置
论文链接 deepseek-v4.1
在文章的 frontmatter 中添加 keywords 字段即可设置关键词：
---
title: 我的文章
date: 2026-01-01
keywords:
- 示例关键词一
- 示例关键词二
---
这是篇带有关键词的文章。然后生成的页面会包含以下 meta 标签：
<meta name="keywords" content="示例关键词一, 示例关键词二, ... site.keywords">摘要
随着长周期智能体（Long-horizon Agents）的广泛应用，模型工作负载正变得越来越依赖输入处理。尽管此前的研究已经大幅降低了长上下文计算的成本，但 Prefill（预填充）阶段仍然具有较高的计算开销，而庞大的 KV Cache（键值缓存）则持续给 HBM（高带宽内存）和 SSD 的存储容量及数据传输带宽带来巨大压力。计算、存储和带宽方面的综合需求，已成为进一步降低模型部署成本的主要瓶颈。 为解决这一挑战，我们推出了 DeepSeek-V4.1-Flash，这是一款多模态混合专家模型（Mixture-of-Experts，MoE），拥有 5520 亿（552B）骨干网络参数，支持最长 100 万 Token 的上下文窗口。 通过采用因果编码器—解码器（Causal Encoder-Decoder，CED）架构，该模型在 Decode（解码）阶段每个 Token 激活 160 亿（16B）参数，而在 Prefill（预填充）阶段每个 Token 仅激活 80 亿（8B）参数，从而显著提升了面向智能体工作负载的成本效率。 为了进一步突破 KV Cache 压缩的极限，DeepSeek-V4.1-Flash 将压缩稀疏注意力机制 2（Compressed Sparse Attention 2，CSA2）中的跨层 KV Cache 复用技术与 FP4 KV Cache 相结合。这些设计将其全局 KV Cache 占用量（始终存储在 HBM 中）降低至每个 Token 890 字节，约为 DeepSeek-V4-Flash 对应占用量的四分之一。 此外，通过一项专门的部署优化技术——SWA Bounded Replay（滑动窗口注意力有界重放），DeepSeek-V4.1-Flash 将其持久化 KV Cache 的占用量（始终存储在 SSD 或主机内存中）降低至 DeepSeek-V4-Flash 的约八分之一。 尽管 KV Cache 占用量大幅减少，该模型的性能仍然显著优于基线模型。 此外，我们对 DeepSeek-V4 架构进行了精简，并引入了多项高效的架构扩展。在预训练阶段，DeepSeek-V4.1-Flash 使用了包含 45 万亿（45T）Token 的多模态语料库。随后，我们开展了全面的后训练，使模型在多种基于文本和多模态的智能体应用场景中均表现出色。 模型文件（Model Checkpoints）现已开放获取：https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash.
The widespread adoption of long-horizon agents has made model workloads increasingly inputheavy. Although prior work has substantially reduced the cost of long-context computation, prefill remains computationally expensive, and large KV caches continue to strain HBM and SSD capacity and data-transfer bandwidth. Together, these compute, storage, and bandwidth demands constitute the primary bottleneck to further lowering deployment costs. To address this challenge, we introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. With its Causal Encoder-Decoder (CED) architecture, the model activates 16B parameters per token during decode but only 8B parameters during prefill, substantially improving cost efficiency for agentic workloads. To push the limits of KV cache compression, DeepSeek-V4.1- Flash combines cross-layer KV cache reuse in Compressed Sparse Attention 2 (CSA2) with FP4 KV caching. These designs reduce its global KV cache footprint (always in HBM) to 890 bytes per token, roughly 1/4 of the corresponding footprint of DeepSeek-V4-Flash. Further, through a dedicated deployment optimization known as SWA Bounded Replay, DeepSeek-V4.1-Flash reduces its persistent KV cache footprint (always on SSD or in host memory) to roughly 1/8 of that of DeepSeek-V4-Flash. Despite its much smaller KV cache footprint, the model delivers substantially better performance than the baseline. In addition, we streamline the DeepSeek-V4 architecture and introduce several efficient architectural extensions. We pretrain DeepSeek-V4.1- Flash on a multimodal corpus comprising 45T tokens and conduct comprehensive post-training, yielding strong performance across diverse text-based and multimodal agentic scenarios. Model checkpoints are available at https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash.
Introduction
Applications of long-horizon agents have expanded rapidly in recent years, making ultra-longcontext processing an increasingly important model workload. Supporting such workloads requires not only efficient long-sequence processing, but also the persistent storage, reuse, and transfer of large KV caches. KV cache management has therefore become a foundational capability for model deployment, while introducing substantial challenges across computation, storage, and communication. Prior advances in sparse attention (DeepSeek-AI, 2025, 2026b) have significantly reduced the computational cost of long-sequence processing, making persistent storage and data movement increasingly prominent bottlenecks.
Specifically, DeepSeek-V4 (DeepSeek-AI, 2026b) combines a global attention branch spanning the full context with local Sliding-Window Attention (SWA). The global branch maintains global KV, comprising main KV and indexer K, while SWA maintains local KV states. For a fixed window size, SWA KV storage is bounded independently of sequence length. For sufficiently long sequences, global KV therefore dominates the runtime KV footprint, which is constrained by HBM capacity. In addition, certain KV are persisted for prefix reuse, referred to as persistent KV caches, which are constrained by SSD and host memory capacity. I/O and interconnect bandwidth also limit cache migration and loading. Together, these constraints limit serving throughput, increase deployment costs, and ultimately hinder the deployment and adoption of agents over longer task horizons and across broader application scenarios.
基于开源的model.py生成的模型结构
Transformer(
(embed): ParallelEmbedding()
(layers): ModuleList(
(0): Block(
(attn): Attention(
(wq_a): Linear()
(q_norm): RMSNorm()
(wq_b): ColumnParallelLinear()
(wkv): Linear()
(kv_norm): RMSNorm()
(wo_a): ColumnParallelLinear()
(wo_b): RowParallelLinear()
)
(ffn): MoE(
(gate): Gate()
(experts): ModuleList(
(0-7): 8 x Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(shared_experts): Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(attn_norm): RMSNorm()
(ffn_norm): RMSNorm()
)
(1): Block(
(attn): Attention(
(wq_a): Linear()
(q_norm): RMSNorm()
(wq_b): ColumnParallelLinear()
(wkv): Linear()
(kv_norm): RMSNorm()
(wo_a): ColumnParallelLinear()
(wo_b): RowParallelLinear()
(compressor): Compressor(
(norm): RMSNorm()
(wkv): Linear()
(wgate): Linear()
)
(indexer): Indexer(
(wq_b): ColumnParallelLinear()
(weights_proj): ColumnParallelLinear()
(wk): Linear()
(k_norm): RMSNorm()
)
)
(ffn): MoE(
(gate): Gate()
(experts): ModuleList(
(0-7): 8 x Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(shared_experts): Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(attn_norm): RMSNorm()
(ffn_norm): RMSNorm()
)
(2): Block(
(attn): Attention(
(wq_a): Linear()
(q_norm): RMSNorm()
(wq_b): ColumnParallelLinear()
(wkv): Linear()
(kv_norm): RMSNorm()
(wo_a): ColumnParallelLinear()
(wo_b): RowParallelLinear()
)
(ffn): MoE(
(gate): Gate()
(experts): ModuleList(
(0-7): 8 x Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(shared_experts): Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(attn_norm): RMSNorm()
(ffn_norm): RMSNorm()
)
(3): Block(
(attn): Attention(
(wq_a): Linear()
(q_norm): RMSNorm()
(wq_b): ColumnParallelLinear()
(wkv): Linear()
(kv_norm): RMSNorm()
(wo_a): ColumnParallelLinear()
(wo_b): RowParallelLinear()
(compressor): Compressor(
(norm): RMSNorm()
(wkv): Linear()
)
(indexer): Indexer(
(wq_b): ColumnParallelLinear()
(weights_proj): ColumnParallelLinear()
(wk): Linear()
(k_norm): RMSNorm()
)
)
(ffn): MoE(
(gate): Gate()
(experts): ModuleList(
(0-7): 8 x Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(shared_experts): Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(attn_norm): RMSNorm()
(ffn_norm): RMSNorm()
)
(4-19): 16 x Block(
(attn): Attention(
(wq_a): Linear()
(q_norm): RMSNorm()
(wq_b): ColumnParallelLinear()
(wkv): Linear()
(kv_norm): RMSNorm()
(wo_a): ColumnParallelLinear()
(wo_b): RowParallelLinear()
)
(ffn): MoE(
(gate): Gate()
(experts): ModuleList(
(0-7): 8 x Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(shared_experts): Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(attn_norm): RMSNorm()
(ffn_norm): RMSNorm()
)
)
(norm): RMSNorm()
(head): ParallelHead()
(mtp): ModuleList(
(0): DSparkBlock(
(attn): DSparkAttention(
(wq_a): Linear()
(q_norm): RMSNorm()
(wq_b): ColumnParallelLinear()
(wkv): Linear()
(kv_norm): RMSNorm()
(wo_a): ColumnParallelLinear()
(wo_b): RowParallelLinear()
)
(ffn): MoE(
(gate): Gate()
(experts): ModuleList(
(0-7): 8 x Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(shared_experts): Expert(
(w1): Linear()
(w2): Linear()
(w3): Linear()
)
)
(attn_norm): RMSNorm()
(ffn_norm): RMSNorm()
(main_proj): Linear()
(main_norm): RMSNorm()
(norm): RMSNorm()
(markov_head): DSparkMarkovHead(
(embed): ParallelEmbedding()
(head): ParallelHead()
)
(confidence_head): DSparkConfidenceHead(
(proj): Linear()
)
(embed): ParallelEmbedding()
(head): ParallelHead()
)
)
)
喜欢的话，留下你的评论吧～