RedNote RedKnot

• 1. overview (795 chars)

  RedKnot is an open source long text and large model inference acceleration engine open sourced by the AI ​​Infra team of the REDnote engine architecture department at the end of June 2026. The bottom layer is built on the open source inference framework SGLang and is integrated as an attention layer extension. Its core judgment is very simple: not every attention head requires a complete key-value cache (KV), and not every word element (token) needs to go through a complete feed-forward network (FFN). Along this line of thinking, RedKnot reduces the pre-filling computing power of long text reasoning by about 50 to 70%, and the first word delay (TTFT) brings an acceleration of 1.35 to 2.2 times, and the longer the context, the more obvious the benefit. It is not a consumer-level AI application, but a set of underlying reasoning optimization infrastructure for developers and service providers, open source using the Apache-2.0 protocol.

• 2. Product background (1015 chars)

  RedKnot is led by Xiaohongshu, with cooperative support from Peking University and Huawei Cloud. The core research and development work was completed by Xiao Yi, Anz and others from the AI Infra team of Xiaohongshu's engine architecture department. The supporting paper "RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention" was co-signed by multiple authors from Xiaohongshu, Peking University, and Huawei Cloud, and has been simultaneously published on arXiv (No. 2606.06256). It is not accidental that Xiaohongshu does this - it has deep accumulation in recommendation systems, search and multi-round Agent calling scenarios, and its internal needs for long document retrieval enhanced generation (RAG), multi-round Agent scheduling, etc. are extremely urgent. As the context window of large models continues to grow, the memory usage and first word delay of long text reasoning have become the core bottlenecks for large-scale service implementation. The prefill delay of traditional reasoning engines under long contexts is too high, and FFN calculation under short contexts has become a bottleneck. RedKnot was born to solve these two problems.

• 3. Product functions and usage experience (1969 chars)

  RedKnot's technical skeleton is made up of four pieces, which act on the three orthogonal dimensions of "header, storage, and channel". The benefits are multiplied rather than competing with each other for margin. The first block is the attention head classification, which classifies each "layer + head" combination into four categories: global (global), local (local), retrieval (retrieval), and dense (dense), and determines the storage and reuse strategy of KV according to the category. In actual measurements, the proportion of local headers is stable at 83.4% to 96.8%, which means that the vast majority of headers actually only look at the local window and do not require the complete context. The second block is offline KV multiplexing plus RoPE relocation. The KV of reusable fragments is stored offline. Only necessary tokens are selectively recalculated during service, and rotation position encoding relocation is used to ensure numerical alignment. The third block is elastic sparse or sparse FFN, which makes token-level FFN selection based on attention importance and skips the feedforward calculation of low-contribution tokens. This is especially targeted at the 2K to 8K short clips common in Agent scenarios - at that length, FFN is the real bottleneck of pre-filling, accounting for 50% to 60% of the first word delay. The fourth block is when SegPagedAttention is running, an independent page table plus segmented KV storage is established for each header, so that different categories of headers have different visible windows. They stay on the FlashAttention fast path throughout the entire process without constructing attn_mask, thus avoiding the old problem of the traditional prefix cache returning to the slow core function once the mask is passed in, resulting in a penalty of 4.9 to 7.6 times.   Because RedKnot is integrated as an extension of the attention layer of SGLang, it fully retains the native high-performance capabilities such as RadixAttention, zero-overhead scheduling, PD separation, continuous batch processing, and quantification. Service providers can access it without reconstructing the original architecture. The code has been publicly maintained on GitHub (rednote-machine-learning/RedKnot), and all RAG benchmark tests can be reproduced with one line of command.

• 4. Pricing and business model (570 chars)

  RedKnot itself is open source and free, using the Apache-2.0 protocol and has no commercial pricing. Its value is not in direct monetization, but in reducing the computing power cost of Xiaohongshu itself and external developers running long context models. For inference service providers whose computing resources are increasingly scarce, this idea of ​​​​relieving the burden of long text inference through refined disassembly of the underlying architecture is equivalent to using fewer GPUs to do more work. The team stated that DeepSeek-V4 and the complete Qwen 3.5 series will be fully adapted in the next version. The current open source version is the basic version.

• 5. User reviews (787 chars)

  As a newly open source infrastructure, RedKnot's real user feedback mainly comes from the open source community and technical media, and it has not yet formed a large-scale end-user reputation. The community generally recognizes the innovative perspective that it transforms the KV cache from "passive data blocks uniformly managed by token" to "model-aware runtime infrastructure split by attention heads" and believes that it provides a valuable thinking framework for long context services: whether the benefits of reuse can be realized depends on whether the granularity of reuse and recalculation is aligned with the internal structure of model reasoning. Some developers also pointed out that the content of the GitHub repository is not complete during the initialization stage, and the supporting documents and the adaptation of more models are still on the way, and it will take time to verify the maturity of the project.

• 6. Industry evaluation (1054 chars)

  Many Chinese technology media have given a positive interpretation of RedKnot. Industry opinion believes that it lowers the sparsification granularity from the traditional token level to the attention level, and introduces FFN sparsification that is orthogonal to attention, forming multiplicative benefits with existing attention optimization methods. It is a representative system-level innovation in the field of long text reasoning. In the actual measurement of the 8-card H800, RedKnot accelerated TTFT by 1.6 times to 3.54 times, increased the concurrency capacity of a single card by 4.7 times to 7.8 times, and reduced the computing power consumption in the pre-filling stage by 67% to 79.5%; on the 128K ultra-long context task of DeepSeek-V4-Flash, the first word generation speed was increased by 5.16 times, and the KV data transmission efficiency was optimized. 6.3x, and the inference accuracy remains above 95% of the dense model performance. Some reviews believe that RedKnot forms a complementary relationship with vLLM and SGLang native optimization in the direction of long-context reasoning acceleration, rather than a substitute, and is more suitable as an optional extension to a high-performance reasoning stack.

• 7. Controversies and Risks (755 chars)

  The most clear known risk of RedKnot comes from the team's own honest disclosure: in the long text scenario of Llama-3.3-70B, RedKnot's decoding path is degraded, which still needs further investigation. This means that it is not plug-and-play for all models. The adaptation scope is currently concentrated on verified models such as Qwen, Mistral, and Llama. DeepSeek-V4 and the complete Qwen 3.5 series have yet to be adapted in the next version. In addition, as a low-level system optimization, its benefits are highly dependent on the specific model structure, context length and service load. The stability in general scenarios still needs to be tested in more production environments. Sparsification brings "nearly lossless" rather than "absolutely lossless". In financial, medical and other scenarios that are extremely sensitive to accuracy, careful evaluation is still required.

• 8. Applicable groups and usage suggestions (797 chars)

  RedKnot is suitable for two types of people: one is engineers and teams that run long-context large model services, especially service providers that do long document RAG, multi-round Agent orchestration, long session memory systems, multi-Agent collaboration frameworks, and real-time streaming long text generation. It can directly alleviate the pressure on video memory and first word delay; the other is scholars and infrastructure enthusiasts who study inference system optimization, who can do secondary research based on its open source code. It is not suitable for ordinary users who just want to use ready-made AI applications without touching deployment. If the team pursues a mature solution that can be used out of the box, the native capabilities of vLLM and SGLang are still more secure default choices; if the cost of long context has become a pain point, RedKnot is worth evaluating and trialling as a supplementary extension.

• 9. Summarize (401 chars)

  RedKnot is a solid attempt by Xiaohongshu to overflow its internal reasoning system experience into open source infrastructure. It uses the simple insight of "splitting KV by head" to lift the efficiency ceiling of long text reasoning, cutting off most of the pre-filling computing power without sacrificing accuracy. It may not replace the existing inference engine, but it is likely to become a noteworthy puzzle piece in the high-performance long-context service stack.

No in-depth report yet

• 

  CYcoo

  —

  按头拆 KV 这个思路确实比 token 级稀疏更贴合模型结构，点赞。

---

• 

  Eric_SandersX

  —

  Llama-3.3-70B 长文本 decode 会退化，团队自己都写了，态度可以。

---

• 

  zure1yg

  —

  Apache-2.0 协议，商用无忧，准备 fork 一份研究。

---

• 

  Raymond.Rivera_888

  —

  小红书这次开源挺实在，论文和代码一起放，比只发博客强多了。

---

• 

  PhoenixFireColeman

  —

  GitHub 上 star 涨得很快，社区热度起来了。

---

• 

  h22n5dovn

  —

  仓库现在还是初始化阶段，文档不全，想直接上生产的话得自己啃代码。

---

• 

  PeytonKristensen

  —

  我们做长文档 RAG 的，之前 SGLang 跑 32K 上下文首字延迟经常过两秒，接上 RedKnot 的注意力头分类后稳定在 0.9 秒左右，精度几乎没掉，这周的 P99 指标好看了不少，已经准备推到预发环境再观察一周。

---

• 

  Jack220

  —

  单卡并发能从 4 路提到 30 路以上，长会话记忆场景太友好了。

---

• 

  urpzgze1y

  —

  我们主要在短上下文的 Agent 场景用，2K 到 8K 的片段里 FFN 才是瓶颈，RedKnot 的稀疏 FFN 跳过那些低贡献 token 的前馈，TTFT 直接砍掉一截，多轮工具调用顺滑很多，这块收益比长文本还明显。

---

• 

  DDiaz_7

  —

  认真读了论文，位置无关 KV 复用（PIC）以前加速不明显，是因为传入掩码就退回慢核函数、带来四到七倍惩罚，RedKnot 用 RoPE 重定位加 SegPagedAttention 把这个问题绕开了，工程上很聪明，难怪精度能保住。

---

• 

  haSAN

  —

  在 8 卡 L20Y 上复现了 Qwen3-32B 的 HotpotQA，16K 到 32K 下 F1 确实不低于基线，TTFT 从 1.39x 涨到 1.93x，FLOPs 省了七成左右，和官方数据对得上，没有虚标。

---

• 

  琉璃

  —

  论文署名有北大和华为云，产学研结合，质量有保障。

---

• 

  天涯573

  —

  和 vLLM 的实测对比过，RedKnot 不是替代而是补充，它作为 SGLang 的注意力层扩展存在，保留了 RadixAttention、连续批处理、量化这些能力，接入成本不高，适合已经被 SGLang 栈绑定的团队。

---

• 

  nODElINK

  —

  深度用了两周，结论是长上下文越长越划算，我们 128K 的检索任务首字速度翻了五倍多，KV 传输也优化了不少，不过要注意它目前对 Qwen、Mistral、Llama 适配最好，别的模型得自己验证。

---

• 

  Larry.Moore_202218

  —

  踩了个坑提醒一下：DeepSeek-V4 和完整 Qwen 3.5 系列官方说下一版才完整适配，现在基础版跑这两个模型要自己改配置，别直接照着 README 的默认参数上，容易在 decode 阶段退化。

No reviews yet