同族不同尺寸 LLM 的 KV cache 之间存在大量线性结构：用 500 条 FineWeb-Edu 序列、按 target 的每个 (layer, head) 做闭式 ridge，就能让接收方直接从发送方的 KV 解码、完全跳过 prefill。六对中四对保住 target 自身 73–98% 精度、比 re-prefill 快 2.7–25$\times$；另两对崩到 42–44%。成败由残差落在 attention 敏感子空间的位置决定，而非残差大小。 ---
Hybrid (Attention+SSM) LLMs break prefix caching because in-place SSM state updates forbid prefix roll-back — only exact-match hits work, so naive checkpointing floods the cache with huge, rarely-reused entries. Marconi admits SSM states judiciously (≤2 per sequence, chosen by a reuse taxonomy) and
2.8T 参数 / 104B 激活的原生多模态 MoE，1M 上下文。三根支柱：3:1 的 KDA–Gated MLA 混合注意力（序列维）、Attention Residuals（深度维）、16-of-896 的 Stable LatentMoE（宽度维）。宣称对 Kimi K2 有 ~2.5× scaling 效率增益。多处架构改动的直接动机是解锁 kernel 路径而非模型质量。 ---
A sequence-transduction model built *entirely* from attention, discarding recurrence and convolution. Stacked multi-head self-attention plus position-wise FFNs give $O(1)$ path length and full intra-example parallelism, reaching 28.4 BLEU (EN-DE) / 41.8 BLEU (EN-FR) at a fraction of prior training c
A general-purpose fine-tuning recipe that couples a pre-trained DPR dense retriever (non-parametric memory: a fixed 21M-passage Wikipedia index) with a pre-trained BART seq2seq generator (parametric memory), treating the retrieved document as a latent variable marginalized end-to-end. Sets SotA on o
Scaling a plain GPT-2-style autoregressive decoder to 175B parameters makes task-agnostic **in-context learning** work: with no gradient updates, GPT-3 learns tasks purely from prompt demonstrations, sometimes matching fine-tuned SOTA. The load-bearing finding is that the zero→few-shot gap *widens*
Autoregressive generation scales terribly on multi-step grade-school math. Instead of a bigger generator, train a separate *verifier* that scores 100 sampled solutions and return the best. On the new GSM8K benchmark, 6B verification ≈ finetuned 175B — a ~30× parameter saving, scaling better with dat
把"思考"当成一种不改变环境的语言动作，塞进 agent 的动作空间 $\hat{A}=A\cup L$，让冻结的 PaLM-540B 用少样本交替生成 Thought/Action/Observation。推理指导取信息、取信息纠正推理，QA 上消灭幻觉，决策任务上 1–2 shot 反超训了 10³–10⁵ 条轨迹的 IL/RL。
一个 6.7B 的 GPT-J 用自监督方式给普通文本自动插入 API 调用，只保留"能降低后续 token 预测损失"的调用，再在增广数据上微调。学会自己决定何时/调用哪个工具后，零样本超越 25× 更大的 GPT-3，且不损失语言建模能力。
Reflexion turns a sparse reward (binary/scalar) into a *verbal* self-reflection stored in an episodic memory buffer, so an LLM agent improves across trials without any weight update. Policy is $\theta = \{M_a, mem\}$. Hits 91% HumanEval pass@1, +22% AlfWorld, +20% HotPotQA — but only on strong backb
AutoGen 把复杂 LLM 应用统一抽象为「多个可对话 agent 相互聊天」。核心是两件事：可定制的 conversable agent（LLM/人/工具任意组合）+ conversation programming（用自然语言与代码融合来编排 computation 与 control flow）。六个应用（数学、RAG、ALFWorld、多 agent 编码、动态群聊、对话象棋）证明开箱即用即可匹敌/超过商业与 SOTA 基线，同时大幅减少代码量。
自回归 LLM 会把大量注意力"倾倒"到最初几个 token（注意力汇），因为 SoftMax 强制分母求和为 1。窗口注意力一旦驱逐这几个初始 token 就崩溃。StreamingLLM 保留 4 个初始 token 的 KV + 滑动窗口 KV，无需微调即可稳定处理 4M+ token，比重算基线快 22.2×。 ---
A benchmark of 2294 real GitHub issue→PR tasks across 12 Python repos: given an issue and a full codebase, the model must emit a patch that passes the PR's real tests. Even with an oracle retriever handing over the exact files to edit, the best model resolves only 4.8% — localization and long-contex
当被复用的长上下文 KV cache 存在远端存储、需经普通云网络（单位数 Gbps）取回时，取回延迟可与重算 prefill 相当甚至更久。CacheGen 用改造过的视频编码流水线把 KV cache 编码成紧凑比特流（size 降 3.5–4.3×，TTFT 降 3.2–3.7×），并按带宽逐块自适应流式传输以守住 SLO。
LLM inference has two phases with opposite hardware profiles: prompt (compute-bound) and token generation (memory-bound, power-insensitive). Splitwise runs them on separate, phase-matched machine pools, transferring the KV-cache over InfiniBand with layer-wise async overlap. Result: up to 1.4× throu
A co-designed frontend DSL + serving runtime for multi-call LLM programs. The runtime keeps finished KV caches in a radix tree with LRU eviction and schedules requests longest-shared-prefix-first (provably DFS-optimal), and compresses constrained-decoding FSMs so single-path token runs decode in one
DistServe splits LLM inference so prefill and decoding run on separate GPU pools, killing phase interference and letting each phase pick its own parallelism. A goodput-optimal placement search plus NVLINK-affinity KV transfer yields up to 7.4× higher request rate or 12.6× tighter SLO at >90% attainm
在线 LLM serving 中,prefill(计算受限)与 decode(内存受限)交织导致吞吐-延迟二选一。Sarathi-Serve 用 chunked-prefill + stall-free batching:把长 prefill 切成受 token budget 约束的小块,piggyback 到 decode 批次而不打断 decode。Mistral-7B 提升 2.6×、Falcon-180B 提升 6.9× serving capacity。
Multi-turn LLM serving discards each session's KV cache when idle, then recomputes it on the next turn — up to 99% of prefill cost is this wasted recompute. CachedAttention keeps KV in a tiered DRAM+SSD store (AttentionStore), overlaps load/save with compute, prefetches/evicts using the scheduler's
Public LLM APIs are request-level, so the serving cluster is blind to how an app's many LLM calls connect. Parrot's **Semantic Variable** annotates prompt regions (task/input/output) and doubles as an inter-request data pipe, letting the service recover a request DAG + prompt structure and jointly d
A dense 405B Transformer trained on 15.6T tokens at $3.8\times10^{25}$ FLOPs, deliberately choosing a boring architecture + SFT/rejection-sampling/DPO alignment over MoE + RL. The bet: quality comes from data curation, scale, and complexity management, not architectural novelty.
Qwen3 fuses a reasoning model and a chat model into **one** checkpoint via a 4-stage post-training pipeline (Long-CoT cold start → Reasoning RL → Thinking-Mode Fusion → General RL), adds a user-controllable **thinking budget**, and builds its 6 small models by **on-policy distillation** that beats R
LMCache is an out-of-GPU KV cache layer that extracts KV cache from vLLM/SGLang and stores/shares it across a storage hierarchy (CPU, disk, Redis, remote, network) for two use cases: cross-query prefix reuse and prefill–decode disaggregation. Its core trick is moving KV at a large **chunk** granular
Agentic programs alternate between busy phases (bursts of short tool calls) and idle phases (blocked on a long tool call). MORI ("Memory Offloader with Relative Idleness") reconstructs each live program's *idleness* $\iota$ from step timing, ranks all programs on a **continuous** spectrum, keeps the
RAPTOR builds a bottom-up tree over a corpus by recursively embedding, soft-clustering (UMAP+GMM), and LLM-summarizing chunks, so retrieval can pull context at multiple abstraction levels. Coupled with GPT-4 it lifts QuALITY from 62.3% → 82.6% absolute.
StruQ converts a **base** LLM into a defended one by (a) encoding prompt vs data into two channels with **reserved-token delimiters** the user cannot forge, and (b) **structured instruction tuning**: augment SFT data with attacked samples whose target output ignores the injected instruction. Drives
LLM 缺少指令/数据的显式隔离。本文给出隔离度的**形式化定义**（把探针 $x$ 放进指令位 vs 数据位，比较输出分布差异）、一个只靠"惊讶见证词"子串检测、无需 logits 的**黑盒经验代理指标** $\mathrm{sep}(g)$，以及 9160 条评测数据集 **SEP**。9 个主流模型全部隔离度低（13.3%–73.2%），且**模型越大隔离越差**；提示工程/优化/微调都无法同时兼得高隔离与高效用。
给 LLM 建立"指令特权等级"（System > User > Tool/第三方），并用两种合成数据原语（对齐指令做 context synthesis、非对齐指令做 context ignorance）微调 GPT-3.5 Turbo，使其在冲突时服从高特权指令。鲁棒性最高提升 63%，并泛化到训练中未见的越狱/密码提取攻击（+34%）。 ---
LLMs treat every input token equally, so a lower-priority user/data token can override a system instruction. ISE adds a tiny learned $H{\times}D$ segment-embedding table (system/user/data/output = 0/1/2/3), summed onto token embeddings before self-attention, and trained during SFT. Result: +15.75% a
MoE decode is memory-bound at moderate batch sizes, so latency is linear in the number of *unique* activated experts $T$ (the union over the batch). OEA re-routes tokens at inference (no retraining): guarantee each token a top-$n_i$ baseline, then let tokens piggyback extra experts already loaded fo
In memory-bound MoE decode, GPU runtime is set by the number of *activated expert replicas* (weight loads), not tokens. Token-balancing load balancers (EPLB) inadvertently inflate activated experts and hurt decode. METRO instead minimizes max activated experts/GPU via a lock-guarded greedy kernel +
MPK auto-compiles a multi-GPU PyTorch inference program into a single persistent mega-kernel via an SM-level task/event graph (tGraph) plus an in-kernel worker/scheduler runtime, enabling cross-task pipelining and fine-grained compute-comm overlap. Up to 1.7× lower latency vs SGLang/vLLM.
Constrained decoding masks+renormalizes each step; when the base model puts little mass on schema tokens, this is a reverse-KL projection with a large "projection tax" that steers toward valid-but-wrong outputs. DCCD first generates an unconstrained draft, then constrains conditioned on it — raising
Forcing an 8B model (Qwen3-8B) to do structured self-reflection via `Outlines` FSM-constrained decoding does **not** improve self-correction — accuracy drops 50.0%→38.0% and 96/100 diagnoses collapse to `FORMATTING_MISMATCH`, trapping 58 samples in death loops. This "structure snowballing" is the al
Decouples memory-management from reasoning: a small RL-trained ContextCurator (Qwen2.5-7B) rewrites working memory each turn for a *frozen* frontier TaskExecutor. On WebArena, Gemini-3.0-flash goes 36.4%→41.2% SR with −8.8% tokens; on DeepSearch, ~8× fewer tokens at higher SR. A 7B curator ≈ GPT-4o.
一个统一的世界-动作模型，用 UniDiffuser + 三流 MoT 把视频与动作放进同一生成框架，5 种推理模式共享一套权重。靠相对末端执行器动作表示做跨本体迁移（50–100 条轨迹），并以推理栈实现 54.4× 加速 / 11 Hz。RoboTwin 2.0 达 95.8%/96.1%，WorldArena EWMScore 63.77 第一。
Ψ-RAG replaces RAPTOR's $k$-means/GMM cluster tree with an AHC-style "merging and collapse" abstract tree (no distribution prior), adds an R&A agent with query reorganization for multi-hop causality, and fuses a BM25 sparse index to fix coarse abstraction. First Tree-RAG to beat SOTA Graph-RAG: +25.
首个专为**交互式世界模型**设计的基准：用一个"动作生成框架"把文本/one-hot/相机内外参三类异构控制统一编码为 $(D,T,R,V)$ 四元组（81 个基础动作），配 330k 视频语料 + 4,900 任务 + 9 个可微指标，评测 14 个模型，揭示"生成质量 vs 可控性"权衡（Spearman $r_s=0.8053$ 与人类偏好对齐）。
首个系统梳理 World Action Model (WAM) 的综述：把"预测未来状态 + 生成动作"统一为联合分布 $p(o',a\mid o,l)$，将现有方法切成 Cascaded（先预测后解动作）与 Joint（联合预测）两大范式，并系统整理数据生态与评测协议，指出评测严重解耦、缺乏因果一致性指标。 ---
In xPyD MoE serving, decode latency is set by the *union* of distinct experts a batch loads from HBM, not by request count. ELDR reads each request's prefill expert activations into a compact signature, clusters them offline with balanced K-means (one centroid per decoder), and routes online within
Treat an LLM agent's file-system memory as a *trainable skill*, not a fixed module. Two meta-LLM outer loops — one revising the memory scaffold (code/prompts/schema), one LoRA-training a memory specialist on the agent's own good decisions — lift a frozen 32B agent 2×–4× on Crafter/MiniHack/NetHack,
A separate memory agent runs beside an unmodified action agent, maintaining a structured memory bank every N steps and deciding *whether* to inject a grounded reminder or stay silent. Fixes "behavioral state decay". +8.3 pp Terminal-Bench, +6.8 pp τ² (Sonnet 4.5); learnable on Qwen3.5-27B.
Context engineering (offload/summarize/truncate/isolate) keeps agent contexts short but each rewrite invalidates the KV cache and forces a re-prefill, spiking TTFT. SmoothAgent observes these rewrites are *segment-decomposable* — each prefix segment transforms independently of future tokens — so it
用 Mixture-of-Transformers 把「动作→3D世界预测」(前向模型)与「3D世界→动作」(逆动力学)通过双向注意力耦合进同一隐空间,再加上 3D-grounded 的 2D 视觉想象。仅 6k 小时(其中真机 1k 小时)预训练即达 RoboTwin2.0 hard 93% SR、真机较基线 +20%。
Agentic LLM serving shifts the goal to cluster TPS and makes KV$ reuse dominate (>80% vs 54–62% chat). Cache-aware schedulers overload a few instances by pinning whole sessions. SMetric balances only each session's *first* request and routes follow-ups cache-aware, gaining 10–16% TPS (colocation) an
AgentSM makes a Text-to-SQL agent reuse *structured* prior exploration traces (built offline from synthetic questions) and merges frequently co-occurring tool calls into composite tools. This cuts trajectory length ~25% and lifts accuracy to a SOTA 44.8% on Spider 2.0 Lite — without a heavy reasonin
把预训练视频扩散模型 (Cosmos-Predict2-2B) 通过**单阶段微调、零架构改动**变成机器人策略：动作、未来状态、状态价值都编码成"注入的 latent frame"，共享同一套视频扩散去噪目标。既做直接策略 (LIBERO 98.5% / RoboCasa 67.1% / ALOHA 93.6% SOTA)，又能用 rollout 数据精炼 world model + value 做 best-of-N 规划，再涨 +12.5 分。
LingBot-VA 把机器人操控重构成"先预测未来视频、再逆动力学解码动作"的自回归扩散世界模型：视频/动作 token 交织成单一因果序列，用双流 MoT(视频流从 Wan2.2-5B 初始化, 动作流窄 4×)联合训练，靠 KV-cache 维持长程记忆、部分去噪+异步执行做实时闭环。RoboTwin 2.0 92.9%/91.6%，LIBERO 98.5%，50 演示即可适配。 ---
DreamZero is a 14B autoregressive video-diffusion backbone fine-tuned to *jointly* denoise future frames and actions. By treating action learning as inverse dynamics conditioned on a predicted visual future, it beats SOTA VLAs 2× on unseen-task/env generalization, transfers cross-embodiment from vid
用一致性模型 (consistency model) 同时做图像解码与潜空间动力学预测，构建 action-conditioned 视频世界模型：单张 RTX 4090 上以 15 FPS 稳定 rollout >10 分钟，FVD 243 远超扩散基线；纯世界模型生成数据训练 IL 策略达到与等量真实数据相当的成功率，且 sim 评测与真实性能强相关。
World Action Models entangle two things: a video-prediction training objective and test-time future video synthesis. Fast-WAM keeps video co-training but replaces future denoising with a single-pass world encoder for actions. Result: on-par accuracy, 190 ms latency (>4× faster), and the training obj
One shared Wan video-diffusion backbone exposes two interfaces: a Video Action Model (VAM, 5.5B) that jointly predicts future latents + action chunks, and an Action-Conditioned Video Simulator (ACVS) that scores candidate actions. Heterogeneous 27.3K-h data with supervision masks + a test-time propo
综述界定"交互式视频世界模型"= 闭环、逐帧动作条件、多轮交互的视频生成，并把整个领域拆成三个技术瓶颈——动作可控性、长程一致性与记忆、实时响应性——逐一梳理主流做法（动作注入四法、记忆/Forcing/显式 3D 三条一致性路线、蒸馏与缓存加速），最后横向对比四大应用（开放世界、游戏引擎、具身、自动驾驶）的 benchmark 与指标。
LaWAM gives a VLA policy explicit dynamics foresight without generating future pixels: it keeps the *forward decoder* of a latent action model (usually thrown away) as a Latent World Model (LaWM) that expands a policy-predicted latent action into one latent visual subgoal in a single forward pass. M
一套固定超参数的 model-based RL 智能体，通过 RSSM 世界模型 + 想象空间 actor-critic，在 8 大类 150+ 任务上击败各自领域专用算法，并首次无人类数据、无课程从零采到 Minecraft 钻石。核心是一组量纲无关的鲁棒性技巧（symlog、symexp twohot、KL free bits、百分位回报归一化）。
Halo treats a batch of same-template agentic workflows as one consolidated query-plan DAG, then solves query optimization + scheduling as a single GPU-worker-placement problem (beam search over a prefill/decode + cache-reuse cost model). Runtime adds adaptive batching, prefix-cache sharing, and on-t
在多模型 agent 工作流里，同一段共享上下文被每个专用模型各跑一遍 prefill 并各存一份 KV。PrefillShare 把模型拆成「冻结的共享 prefill 模块 + 各任务专用 decode 模块」，只用 cache-conditioned 微调训 decode 去读 base 的 KV，从而一次 prefill、一份 KV 跨模型复用。精度追平全量微调，p95 延迟降 4.5×、吞吐升 3.9×。 ---
Agentic inference stacks glue a stateless LLM engine (vLLM) to a stateless tool orchestrator (Kubernetes), scheduling each LLM/tool call independently — causing KV-cache thrashing, cross-node memory imbalance, and leaked tool resources. ThunderAgent makes the whole multi-turn workflow a first-class
Agent memory turns every generation step into a frequent, interleaved ANN search/insert against a growing vector index; at scale this consumes 82%+ of runtime. Pancake attacks this with three coordinated tiers — locality-aware multi-level cache (FSM-driven), a hybrid graph unifying multi-agent coars
Orla is a serving layer that sits *between* agent-orchestration frameworks and LLM inference engines. It models an agentic task as a DAG of stages and adds three workflow-aware mechanisms — a stage mapper (per-stage model/backend routing), a workflow orchestrator (two-level scheduling), and a memory
Helium serves batch agentic workflows by modeling them as query-plan DAGs with LLM calls as first-class operators, then applies proactive KV/prompt caching + cost-based cache-aware scheduling over a templated radix tree. Up to 1.56× over KVFlow, 0.9% avg gap to MILP optimum, exact semantics. **Agen
Middleware atop vLLM that serves multi-agent workflows on a *mix* of LLM sizes/families. It couples a semantic router (per-model success confidence), a CPU QRF that predicts a workflow's *total remaining* output tokens, and an in-flight-token load monitor to pick the strongest model within a latency
Multi-LoRA agent serving breaks prefix caching: distinct adapters make identical-text KV caches diverge, so each agent keeps a redundant full cache and GPU memory saturates linearly. ForkKV physically splits KV into a shared base cache ($xW$) + tiny per-agent residual ($xA_i$), forks it with OS-styl
Agentic workflows have wildly unpredictable end-to-end latency, but each LLM's *share* of total execution time is stable. Scepsy exploits this: it traces workflows framework-agnostically, folds them into an "Aggregate LLM Pipeline" (a cheap throughput/latency predictor), then jointly searches fracti
Agent tasks fire 10–100 chained LLM calls, but request-level GPU schedulers discard each session's KV cache across tool calls, inflating latency 3–8×. SAGA makes the whole agent workflow the schedulable unit: workflow-aware eviction (within 1.31× of Bélády), session-affinity batching with work steal
Dynamic multi-agent workflows (retry loops, runtime branches) break both LRU (temporal locality) and KVFlow (static DAG) for KV-cache management. PBKV predicts the next $K$ agent invocations with a tiny GraphSAGE model, scores each cache node by cross-workflow expected reuse, then evicts retired-cac
SPECTRE reuses idle tail-model services as remote speculative drafters for busy large models, and adaptively switches between *parallel* (draft overlaps verify) and *ordinary* (draft waits) modes using a throughput-derived rollback threshold $r^*$. Up to 2.28× over AR, +66% over the best SD baseline
Agentic requests are multi-call DAGs revealed online; users feel end-to-end workflow latency, not per-call latency. HexAGenT maintains a per-workflow standalone horizon $H_w(t)$, ranks ready calls by projected scaled-SLO risk, and jointly picks prefill/decode placement + queue priority on heterogene
Agentic serving is irregular at the turn but stable at the conversation: one compute-bound turn-1 prefill + a long memory-bound tail. Scheduling the conversation (not the turn) replaces unobservable decode-cost prediction with two observable signals — turn-1 input length and per-decoder KV occupancy
Multi-turn agents blow up KV memory/bandwidth far more than compute. IntentKV keeps the base LLM frozen and prunes KV by (a) scoring history against a session-level QueryMemory of accumulated intent + a zero-init residual head, and (b) evicting via slot-map redirection to a dead slot instead of comp
Serving LLM multi-agent workflows under tight GPU budgets breaks because each stage's decode length and KV footprint are unknown, model demand is long-tailed, and clusters differ in readiness vs latency. Maestro predicts per-stage output length and KV need (tool-intent classifier + role-aware regres
Coding agents are long-running, closed-loop LLM sessions (median 36 min, ~20× more tool-driven than user-driven turns) that reuse large, growing prefixes. Default FCFS+LRU serving thrashes their KVCache. CacheWise adds prefix-aware scheduling (dispatch min missing-prefix request) + metadata-predicti
多视角机器人操作世界模型缺乏跨视角 3D 一致性。作者诊断出两个缺陷：无显式跨视角通信路径、无 3D 几何先验，并主张二者必须共存。方案 = 通信路径（Geo-RoPE + 跨视角注意力）+ 几何目标（Latent 3D-REPA）。WorldArena 第 1、AgiBot-Challenge2026 第 2；消融显示超可加耦合。 ---
ABot-M0.5 是一个面向移动操作 (mobile manipulation) 的 World Action Model。核心论点：移动操作失败不是模型规模不够，而是三处结构性错位——时间粒度、动作空间、训练-推理一致性。三个对应机制：帧级 latent action 桥接粗视频与细控制；dual-level MoT 解耦 mobility/manipulation 子空间；Dream Forcing 用自梦视频训练逆动力学。RoboCasa365 平均 40.4%（SOTA）。
Kairos is a 4B native world-action model that trains a video-diffusion backbone from scratch on a passive→human→robot data curriculum, replaces quadratic attention with a hybrid SWA+DSWA+GLA design (proven to bound long-horizon error via a contractive gated-delta memory), and co-designs distillation
KVCOMM针对多智能体LLM通信中的重复prefill，提出无需训练的在线KV复用：先做RoPE位置对齐，再用anchor池估计跨上下文偏移并增量更新。论文在MMLU/GSM8K/HumanEval上保持接近原始精度，同时把多agent首token延迟显著压缩（最高约7.8x）。
TokenCake targets LLM multi-agent serving where tool calls create long KV-cache idle windows and graph-critical agents compete for scarce GPU blocks. It co-designs an event-driven Temporal Scheduler (opportunistic offload + predictive upload) and an agent-aware Spatial Scheduler (dynamic reserved/sh
用“KV 缓存编辑指令”把 agent 的消息改写变成服务端可执行的 span splice；在 MLA 下用闭式 RoPE $\delta$-rotation 让编辑后缓存仍位置正确，从而提升 replay cache hit（+11.2pp）并在部分并发下显著降延迟（峰值 -241ms）。
来自 ALIYUN 通义生产集群一周真实 trace 的首个系统性 KV$ 复用画像：理想命中率仅 62%/54%（远低于合成负载的 >80%），单轮请求贡献 to-B 负载 97% 的命中，KV$ 寿命极短（to-B P99=97s，90% 块 0.3s 内不再复用），因此中等容量缓存即够用。据此把 GDFS 改造成按类别复用概率排序的 workload-aware 淘汰策略，命中率 +1.5–3.9%、QTTFT 降 28.3–41.9%。
FlowKV 通过 KV cache tensor 形状变换 + segment-based 连续分配 + 双向段对齐，将 PD 分离推理中 NCCL KV 传输延迟降低 96%（23,469 次 kernel 调用 → 1 次），配合 Load-Aware Scheduler 实现异构 GPU 部署下 15–49% 端到端加速。 ---
KVCache offloading 系统的 CPU 三大瓶颈（细粒度缓存管理、PCIe 带宽浪费、CPU-centric 同步开销）通过算法-系统协同解决：head 粒度近似缓存 + 零拷贝传输引擎 + GPU 中心同步，解码吞吐提升 9.3%–66.6%，精度几乎无损。
推导 $\kappa_{\text{crit}} = \kappa_M \times \kappa_{HW}$ 分析框架，将 KV offloading 下 prefill 从 compute-bound 转 memory-bound 的临界点分解为模型因子和硬件因子；实证表明真实工作负载超过该阈值 1–2 个数量级，GPU 仅消耗 22–28% TDP，99% 延迟花在 PCIe 传输上。
Tutti 是首个将 CPU 从 GPU↔SSD KV cache 的数据路径和 I/O 控制路径中完全移除的开源方案。通过 GPU-native object store + GPU io_uring + slack-aware I/O scheduling 三项设计，SSD-backed KV cache 达到与 DRAM-backed 近乎相同的推理性能，TTFT 降 78.3%、可服务 RPS 翻倍、成本降 27%。 ### System scope - **Stage coverage**: prefill + decode（prefill 是 TTFT 优化主战场，decode
NetKV adds datacenter network topology and congestion awareness to decode-instance selection in disaggregated LLM inference via a lightweight operator-to-scheduler oracle; on a 64-GPU fat-tree, it cuts mean TTFT by up to 21% over round-robin and 18% over cache+load-aware baselines, with the static t
DSpark 把"起草更快更准"与"验证更省"两条线合成一个无损投机解码系统。起草端用半自回归结构：并行 DFlash 主干一次前向出全块 base logits，再挂一个轻量序列头（Markov 或 RNN）注入块内依赖以抑制后缀衰减。验证端用置信度调度：置信头估计每位前缀存活率，硬件感知调度器按实测 $\text{SPS}(B)$ 吞吐曲线动态选每请求验证长度，早停机制保证严格无损。离线接受长度超 Eagle3 约 26.7%–30.9%、超 DFlash 约 16.3%–18.4%；在 DeepSeek-V4 上较 MTP-1 每用户生成加速 60%–85%（Flash）/ 57%–78
SGLang 在 DeepEP/EPLB 之上加了两个 dispatch-time MoE 负载均衡特性：Waterfill 把 dense shared expert 当作可分派的 slot，按各 rank 当前负载"填谷"到轻负载 rank（V3/R1 +1.48%~+4.66%，V4 Flash 最佳 +4.92%）；LPLB 用每层 min–max 线性规划在 redundant 副本间重分流量（+0.84%~+7.34%，无 redundant 时为负）。两者都不改 logical top-k，保持语义。
DeepSpec 是训练+评测投机解码 draft model 的全栈 Python 代码库（~6.1k LOC, MIT）。同一套 `Qwen3DSparkModel` 通过 config 开关切出 DFlash（CE-only）与 DSpark（Markov 头 + L1 分布匹配 + confidence 头），另有独立的 Eagle3(TTT) 路径。三段流水线：数据准备 → FSDP 训练 → rejection-sampling 评测，附 Qwen3/Gemma4 官方 checkpoint。
RLM is an inference-time scaffold that stops feeding a long prompt $P$ into the model's context window and instead loads $P$ as a variable inside a persistent Python REPL; the root LM sees only constant-size metadata and writes code to peek, decompose, and recursively call itself (`llm_query` / `rlm
把"包裹固定 LLM 的 harness 代码"本身当成搜索目标：一个 coding-agent proposer 通过文件系统读取全部历史候选的源码、分数、执行 trace，逐轮提出新 harness。文本分类比 ACE 高 7.7 分且 context token 少 4×；数学检索 5 个 held-out 模型平均 +4.7 分；TerminalBench-2 上超过手工 harness。
For agents that repeatedly query the *same* large external context (a 50k-entry feedback corpus, a code repo), PEEK caches the reusable "orientation knowledge" — what the context contains, how it is organized, key entities/constants/schemas — as a small constant-size **context map** kept resident in
借鉴硬件投机执行思想，将 agent 的推理-行动循环与用户输入流、工具执行流解耦（Asynchronous I/O），并允许在信息不完整时投机发起工具调用、事后修正（Speculative Tool Calling），配合 clock-based SFT 训练将 3B 边缘模型的端到端延迟降低 1.6–2.2×，同时保持接近非流式基线的准确率。
IdleSpec 在 LLM Agent 工具执行等待期间并行运行推测性规划（progressive + recovery 双策略），通过 Thompson 采样自适应选择策略并在 observation 到达后以 reference 方式聚合 draft candidates。GAIA+FRAMES 上 Gemini-2.5-Flash 达 55.6% 平均准确率（+5.1%），MLE-Bench Any Medal 率 +9.1%，延迟开销接近零。
RTP-LLM: Alibaba 生产级推理引擎（100M+ 用户）。集成 PD 解离、4 层 KV cache 层级、cache 感知调度、模块化推测解码、文件序加载，对比 vLLM/SGLang 实现 6.3x 加载加速、37% TTFT P95 降低、2.52x 多模态吞吐提升。
综合认知神经科学与 AI agent 记忆系统的跨学科综述：提出 nature-based（episodic vs semantic）× scope-based（inside-trail vs cross-trail）双维分类体系，系统梳理记忆存储、管理生命周期、评测基准与安全攻防，覆盖 ~400 篇文献。
MCMA decouples agent memory management from task execution via a DPO-trained Memory Copilot (Qwen3-4B) that learns multi-structure abstraction and hierarchical organization of experience. When no stored memory fits, the copilot itself transfers—achieving +27.85% on ALFWorld and cross-model gains on
MemSkill 将 LLM agent 的 memory 操作从固定 pipeline（add/update/delete）升级为可学习、可进化的 skill bank：一个 RL 训练的 controller 选取 Top-K skill，一个 LLM executor 一次性生成 skill-guided memory，一个 designer 从 hard cases 进化 skill bank；在 LoCoMo、LongMemEval、HotpotQA、ALFWorld 上超越 hand-designed baselines 且跨模型 transfer。 ---
首篇系统综述 Agent Skills 范式：将 LLM agent 的过程性知识封装为可组合、可分发的模块化 SKILL.md 包，涵盖三层渐进加载架构、六种技能获取路径、CUA 部署及安全治理（26.1% 社区技能含漏洞），并提出四层信任等级 + 四阶验证门控的治理框架。
In ~80K-skill agent pools, hiding skill body text drops routing accuracy 31–44pp. SkillRouter, a 1.2B full-text retrieve-and-rerank pipeline, reaches 74.0% Hit@1—beating a 16B base at 13× fewer params and 5.8× lower latency—via false-negative filtering and listwise reranking.
SkillReducer is a two-stage skill debloating framework that compresses LLM agent skill descriptions by 48% and bodies by 39% via delta-debugging-based routing optimization and taxonomy-driven progressive disclosure, improving functional quality by 2.8% (less-is-more effect) with 0.965 cross-model re
提出 Experience Compression Spectrum，将 agent memory / skill / rule 统一为经验压缩轴上的三个点（$L_1$ 5–20×, $L_2$ 50–500×, $L_3$ 1000×+）。映射 20+ 系统后发现全部固定在单一压缩层级，无系统支持自适应跨层级压缩（"missing diagonal"），且 memory 与 skill 社区互引率 <1%。
SSL 将文本形式的 agent skill 文档拆解为三层 JSON 图（调度接口 / 场景执行图 / 原子操作图），用受限词表和源文溯源保证可比性。在 6,184 技能库上，Skill Discovery MRR@50 提升 +0.080，Risk Assessment macro F1 提升 +0.100。 ---
Survey formalizes agent skills as reusable procedural artifacts $S = (M, \mathcal{R}, \mathcal{C})$ bridging the "procedural gap" between tool access and reliable execution. Organizes 122 papers around a four-stage lifecycle (representation → acquisition → retrieval/selection → evolution), identifyi
SLIM treats the active external skill set as a dynamic optimization variable during agentic RL, using leave-one-skill-out marginal contribution estimation to retain, retire, or expand skills — yielding non-monotonic skill trajectories that outperform both persistent-accumulation and forced-zero-skil
MOCHA 将 agent skill 优化形式化为多目标问题（correctness vs. compliance），用 Chebyshev scalarization 覆盖 non-convex Pareto front，结合 HVC 探索与指数退火，6 任务上 +7.5% correctness，baselines 4/6 任务零进展。
878 cybersecurity SKILL.md files coded for four user-comprehension anchors: operational basis (92.1%), output contract (63.0%), boundary disclosure (51.4%), example demonstration (19.0%); only 2.3% provide all four. A 6-skill DNS/C2 subset shows examples make first local checks constructible from sp
Self-evolving skill libraries silently degrade via "library drift" — unbounded skill accumulation without outcome-driven lifecycle management causes retrieval dilution and harmful injection. Three governance mechanisms (outcome-driven retirement, bounded active-cap, meta-skill authoring prior) lift
Mem-π models agent memory as a separate generative policy (7B LM) trained in two stages — SFT on an experience bank, then decision-content decoupled GRPO — to learn both *when* and *what* guidance to produce. Achieves 55.4% avg SR across 4 agentic benchmarks (+22% relative over base agent), with nea
MUSE-Autoskill 将 agent skill 从一次性生成产物升级为有完整生命周期（创建→记忆→管理→评估→迭代）的可进化资产；在 SkillsBench 51 任务上以 68.40% 领先同 backbone 三 agent，自生成 skill 在 35 task 上达 87.94% 超越人工 skill，且可零修改迁移至其他 agent。
SaP 将 prose 格式的 markdown skill library 自动转换为 typed pseudocode（typed contract + concrete action template），经 4-check 确定性验证器把关后，在检索时以 substituted bundle 形式一次性交付给 agent。ALFWorld 134 games × 3 seeds：+74% 相对胜率（82 vs 47 wins，McNemar $p=8.2 \times 10^{-5}$），同时 −23% input tokens — 质量与成本同向改善，根因是打破了 prose 引发的
SkillsInjector reframes skill injection as per-task context construction: a preference-trained planner admits an adaptive skill set by execution-grounded utility, then a curriculum-distilled 8B renderer adapts descriptions relative to co-selected neighbors — +5.1 pp avg over best baseline across tau
KRCore virtualizes pre-initialized kernel-space RDMA DCT connections to achieve 5.4μs connection setup (vs 15.7ms verbs), using fixed O(1) memory regardless of cluster scale, while preserving low-level verbs API compatibility for existing RDMA optimizations.
通过 tiling + recomputation 将 attention 融合为单个 GPU kernel，避免 $N \times N$ 中间矩阵的 HBM 读写，IO 复杂度从 $\Theta(Nd + N^2)$ 降至最优的 $\Theta(N^2 d^2 / M)$，实现 2–4× 加速和线性内存。
利用 LLM 推理的 semi information-agnostic 特性（input length 已知、output length 未知），设计 skip-join MLFQ 调度器消除 head-of-line blocking，配合 proactive KV cache swapping 管理内存开销，吞吐比 vLLM 提升高达 31.4×。
在 FlashAttention 基础上通过三项优化——减少 non-matmul FLOPs、序列维度并行化、warp 间 split-Q 分工——将 A100 上 attention 前向吞吐从 25–40% 提升至 50–73% of peak，端到端训练达 225 TFLOPs/s（72% MFU）。
vLLM 将操作系统虚拟内存分页机制引入 LLM KV cache 管理：固定大小 block 替代连续预分配，消除碎片与冗余复制，结合 copy-on-write 实现跨请求共享，使吞吐提升 2–4×。
FastDecode 将 Transformer 分解为 S-Part（GPU 密集线性层）和 R-Part（CPU 侧 attention+KVCache），彻底移除 GPU 内存中的 KVCache 以支持极大 batch size。使用分布式远程 CPU 的聚合带宽执行 attention，配合 load-stabilizing schedule 和性能模型，在 A10 GPU 上达到 vLLM 的 1.88×–5.04× 吞吐。
DeepSeek-V2 是 236B 总参 / 21B 激活的 MoE 模型，核心创新为 Multi-head Latent Attention (MLA，KVCache 压缩 93.3%) 和 DeepSeekMoE（细粒度专家 + 共享专家隔离）。在 8.1T token 上预训练后经 SFT+GRPO 对齐，以 21B 激活参数达到开源 SOTA 水平，推理吞吐 5.76× DeepSeek 67B。
Trains lightweight routers on human preference data (Chatbot Arena) to dynamically route queries between strong (GPT-4) and weak (Mixtral) LLMs. Achieves >2× cost savings while maintaining >90% strong-model quality. Routers generalize across unseen model pairs without retraining. ---
Mooncake 是 Kimi 的生产推理平台，以 KVCache 为调度核心将 prefill 与 decode 分离到独立集群，利用 CPU DRAM/SSD 构建分布式 KVCache 池实现 prefix 复用，并通过预测式 early rejection 应对过载。在真实负载下多处理 75% 请求，模拟长上下文场景吞吐提升达 525%。
针对 Hopper H100 GPU 的三项技术——producer-consumer warp-specialization、2-stage GEMM-softmax pipelining、FP8 block quantization + incoherent processing——将 FP16 attention 从 FA2 的 35% 利用率提升至 75%（740 TFLOPs/s），FP8 接近 1.2 PFLOPs/s。
Neo 将部分 decode attention 计算和 KVCache 从 GPU 卸载到本机 CPU，通过 asymmetric pipelining（非对称子批次重叠）和 load-aware scheduling（贪心回退到 GPU-only）提升在线推理吞吐。在内存受限的 T4 上达 7.5× 吞吐提升，A10G 上 26%，H100 上 14%，且不牺牲延迟。
即使 LLM 能 100% exact-match 检索到所有证据，其推理/QA/编码能力仍随输入长度增加而实质性退化（13.9%–85%），且该退化在最小化甚至完全消除干扰 token 后依然存在——说明纯"长度"本身是独立于检索失败的性能瓶颈。
FlowMesh treats multi-stage LLM workflows (RLHF, agentic pipelines) as a shared elastic service by decomposing them into fine-grained DAG operators with deterministic identities — enabling cross-tenant deduplication via $H_{\mathrm{task}}$, batching via $H_{\mathrm{exec}}$, and heterogeneous GPU sch
Seedance 1.5 pro 基于 dual-branch MMDiT 实现 native 音视频联合生成，通过 multi-stage data pipeline + SFT + RLHF 后训练 + >10× distillation 加速，在中文语境的唇语同步、方言支持和镜头调度上建立竞争优势。
用小模型提前投机预测 agent 的下一次 tool call 并异步执行，使工具执行与主模型推理重叠。提出 client-side（无需改引擎）和 engine-side（修改 vLLM 保持 KV-cache 驻留）两种方案，client-side 节省 6-21% 端到端时延，理论上界 < 2×；engine-side 额外节省 2-3%。
LoongFlow replaces blind LLM-based evolutionary mutation with a Plan-Execute-Summarize cognitive loop + hybrid evolutionary memory (multi-island MAP-Elites + adaptive Boltzmann selection), achieving >60% efficiency gain over OpenEvolve on algorithmic discovery and 14 Gold medals on MLEBench.
OI-MAS introduces a hierarchical conductor that jointly routes agent roles and model scales per reasoning step via confidence-aware RL. Token log-prob confidence modulates cost penalty: high confidence → penalize expensive models, low confidence → allow escalation. Result: +7.68% avg accuracy over b
TTT-Discover 在测试时对 LLM 执行在线 RL（entropic objective + PUCT state reuse），针对单一科学问题持续学习以发现 SOTA 解，在数学/GPU kernel/算法竞赛/生物分析上全面刷新记录，仅用开源模型和 ~$500/题。
vllm-mlx provides native LLM + multimodal inference on Apple Silicon via MLX, combining continuous batching (4.3× throughput at 16 concurrent), 21–87% higher throughput than llama.cpp on text models, and content-based prefix caching that eliminates redundant vision encoding — delivering 28× speedup
Lightweight dual-branch neural router for graph-based multi-agent systems that fuses semantic embeddings with structural meta-features to estimate per-step task difficulty, dispatching sub-tasks to strong or weak LLMs. Trained via cold-start synthetic data + on-policy negative feedback. Achieves up
OnePiece 将 AIGC 多阶段推理流水线拆分为微服务，以 one-sided RDMA 传输中间结果，用 double-ring buffer 解决无 CPU 参与的 RDMA 死锁，配合 Node Manager 动态调度 GPU，在 Wan2.1 I2V 场景声称节约 16× GPU 资源。
GORGO 通过 additive cost model 联合优化 network latency、KV-cache prefix overlap 和 queue depth 来路由跨区域 LLM 请求，其集中式 proxy 变体实现 median TTFT 2.5× 优于 baseline。
SLA2 修正 SLA 的 renormalization mismatch（用可学习 α 替代 projection）并引入 learnable router + QAT，在视频扩散模型上实现 97% attention sparsity / 18.6× attention speedup 且质量超越 full attention。
Agent Behavioral Contracts (ABC) brings Design-by-Contract to AI agents: a six-tuple $(\mathcal{P}, \mathcal{I}_{\text{hard}}, \mathcal{I}_{\text{soft}}, \mathcal{G}_{\text{hard}}, \mathcal{G}_{\text{soft}}, \mathcal{R})$ with probabilistic $(p,\delta,k)$-satisfaction, Ornstein–Uhlenbeck drift bound
SkillFortify provides the first formal analysis framework for agent skill supply chains: Dolev–Yao attacker model adapted to the 5-phase skill lifecycle, sound static analysis via abstract interpretation over a 4-element capability lattice, SAT-based dependency resolution with capability bounds, and
SuperLocalMemory is a local-first memory system for multi-agent AI defending against OWASP ASI06 memory poisoning via architectural isolation, Bayesian trust scoring (trust gap = 0.90, 72% sleeper degradation), and adaptive zero-LLM re-ranking (+104% NDCG@5) — all without cloud dependencies.
AgentAssay replaces binary pass/fail testing with three-valued stochastic verdicts (Pass/Fail/Inconclusive) backed by SPRT, plus behavioral fingerprinting that detects regressions invisible to pass-rate testing (86% power vs 0%), achieving 78% trial reduction and 100% cost savings via trace-first of
FlashAttention-4 针对 Blackwell GPU 的非对称硬件 scaling（MMA 翻倍但 smem/exp 不变）重新设计 attention kernel pipeline：利用 TMEM 和异步 MMA 重叠 softmax 与矩阵乘，用多项式逼近软件模拟 exp 绕过 MUFU 瓶颈，2-CTA MMA 减半 smem 流量和 dQ 原子操作。B200 BF16 达 1613 TFLOPs/s (71%)，超 cuDNN 9.13 达 1.3×。
SLM-V3 replaces cosine similarity with Fisher-information-weighted retrieval (Theorem 6.1), heuristic decay with Riemannian Langevin dynamics on Poincaré ball (Theorem 6.3), and silent inconsistency with sheaf cohomology ($H^1 \neq 0$ = contradiction) — achieving +12.7 pp over engineering baseline o
Multi-GPU LLM 推理中，CPU 资源不足是隐藏的主导瓶颈：tokenization 占 TTFT 高达 50%，shared-memory broadcast dequeue 延迟膨胀 19×，barrier 同步将单核延迟放大为全局 GPU 停顿。增加 CPU 核心可在 ~1.5% 额外成本下获得 1.36–5.40× TTFT 改善。
AgentOpt introduces client-side optimization for AI agents — model combination selection across pipeline roles via bandit-based search (Matrix UCB-E). Cost gap between best and worst combinations reaches 13–32× at matched accuracy; the strongest standalone model (Opus 4.6) is the worst planner in mu
Blink removes host CPU from LLM inference critical path via SmartNIC (DPU) frontend + GPU-resident persistent scheduler, achieving up to 8.47× P99 TTFT reduction and complete interference immunity where baselines degrade 1–2 orders of magnitude.
CodeComp incorporates Code Property Graph (CPG) priors from Joern into KV cache compression for code tasks — span-level structural protection preserves call sites, branch predicates, and return statements that attention-only methods systematically mis-prune, recovering 91% of full-context accuracy a
FaRM is an RDMA-based distributed computing platform that exposes cluster memory as a shared address space with ACID transactions, achieving 10x throughput and 100x lower latency than TCP/IP via one-sided RDMA lock-free reads and RDMA-write messaging.
RackSched 将 ToR 可编程交换机用作 rack 级微秒调度器，通过 power-of-k-choices 做服务器间负载均衡 + 各服务器内集中调度消除队头阻塞，实现近线性吞吐扩展并保持单服务器水平的尾延迟。 ---
APEX replaces NEO's batch-splitting Asymmetric Pipelining with unified-batch Asynchronous Overlap + deferred cross-iteration synchronization for hybrid CPU-GPU LLM inference. A profiling-driven inequality selects strategy per iteration. Up to 96% throughput over vLLM, 72% over NEO on constrained GPU
Qualitative survey comparing four emerging agent communication protocols—MCP (tool invocation), ACP (structured messaging), A2A (enterprise task delegation), ANP (decentralized P2P)—across 11 dimensions, concluding no single protocol suffices and proposing a phased adoption roadmap from MCP → ACP →
MuonClip（Muon + per-head QK-Clip）解决 Muon 在 1T MoE 上的 attention logit 爆炸，15.5T tokens 零 spike 训练；三阶段 agentic 数据合成 + joint RLVR/self-critique RL 产出开源最强 non-thinking 模型（Tau2 66.1, SWE-bench 65.8, AIME'25 49.5）。
Two-phase DP scheduler converts heterogeneous volunteer GPUs into a practical LLM inference platform: Phase 1 allocates model layers region-locally via DP + water-filling; Phase 2 selects per-request pipeline chains via DAG shortest-path over live DHT metrics. Up to 3.6× throughput over HexGen on 7-
Kimi Linear 是首个在短/长/RL 三大场景全面超越 full attention 的混合线性注意力架构。核心模块 KDA 将 channel-wise 细粒度遗忘门与 delta rule 结合，配合 3:1 KDA-to-MLA 混合结构实现 75% KV cache 节约与 6.3× 解码加速（1M context）。48B 总参 / 3B 激活的 MoE 模型在 1.4T tokens 训练下全面超过纯 MLA baseline。
Harli co-locates PEFT finetuning with LLM decode instances on the same GPU, exploiting decode's ~60% idle SM capacity. A unified CUDA VMM memory allocator, two-stage LR latency predictor, and GreenContext-based QoS scheduler yield 46% avg finetune throughput gain with zero SLO violation.
ICaRus decomposes decoder-only Transformers into a frozen logical encoder (KV generation) and task-specific logical decoders (next-token prediction), enabling identical KV cache sharing across multiple specialized models. With 8 agents it achieves 11.1× P95 latency reduction and 3.8× throughput gain
Qualixar OS 是首个应用层级的 AI agent 编排操作系统，通过 12 种 multi-agent 执行拓扑、LLM 驱动的自动团队设计（Forge）、三层 meta-learning 模型路由、8 模块质量保障流水线（含 Goodhart 检测与 JSD 漂移监控），以及 Claw Bridge 框架兼容层，统一了 10 个 LLM provider 和 8+ agent 框架的编排。20 任务定制评测 100% 准确率、\$0.000039/task；但自改进循环未达统计显著（$p=0.578$）。
FaaSMoE decomposes MoE inference into a lightweight orchestrator (attention + gating) and stateless expert blocks deployed as FaaS functions shared across tenants. Configurable expert-block granularity trades invocation overhead against per-expert elasticity. On Qwen1.5-MoE-2.7B with 6 tenants, achi
BalanceRoute 是部署于 vllm-ascend 之上的集中式 stateful proxy 框架，利用分段线性 F-score 捕获 barrier 同步下 safe/overflow 的 $(G{-}1)$ 倍不对称性，以两阶段贪心+子集选择在毫秒级完成 DP decode 路由。BR-0 无需预测即降低不均衡 4.1×/吞吐+11.8%；BR-H 加 binary termination classifier 达 +15.4%，优势随 $G$ 超线性放大至 +34.5%（$G{=}16$）。
MARLIN is a two-phase game-theoretic multi-agent RL framework that schedules LLM inference requests across geo-distributed datacenters, jointly optimizing TTFT, carbon emissions, water usage, and energy costs. Four parallel SAC agents each propose single-objective scheduling plans; a capital-modulat
纯数据驱动方案治愈 lost-in-the-middle：用 GPT-4 合成 1.75M 条"答案藏在 4K–32K 上下文任意位置"的 QA 数据做 instruction tuning，得到 FilM-7B——VaL Probing 均分 85.9 超 GPT-4-Turbo (79.0)，位置鲁棒性 Gap 仅 13.9 (vs 32.1)，不损短上下文能力。
Smooth K（减均值消除 K 的 channel outlier，softmax 平移不变性保证精确）+ INT8 $QK^\top$ / FP16-with-FP16-accumulator $PV$ 混合精度 attention，RTX4090 达 341 TOPS（2.1× FlashAttention2），端到端精度损失 <0.2%，即插即用。
"Lost-in-the-middle"在多跳QA中呈现二维退化：LLM准确率不仅随证据绝对位置下降（U型），还随多个证据间的相对距离单调退化（lost-in-between）。CoT仅对instruction-tuned模型有效；上下文裁剪以准确率换鲁棒性，无已有方法同时解决两个维度。 ---
GPU 上 GEMM 与 collective 并发（C3）仅实现理想加速比的 21%。调度优先级与 CU 分区提升至 42%；将通信 offload 至 DMA 引擎（ConCCL），消除 compute 与 L1/L2 interference，达 72%，最高 $1.67\times$。
首篇系统性覆盖 KV cache 管理全栈的综述：构建 token/model/system 三层 taxonomy，分类梳理 100+ 方法（选择、预算分配、合并、量化、低秩分解、注意力分组、非 Transformer 架构、分页内存、调度、异构硬件），并汇总 23 个评测基准。
RoPE-based LLM 的 Q/K 矩阵在低频维度出现集中大值 (massive values)，破坏实验证明其专职负责上下文知识理解 (GSM8K 76.9%→4.0%，Passkey 100%→0%)，而参数知识检索仅温和退化。保护大值的量化方法 (AWQ/SmoothQuant) 显著优于不保护的方法 (GPTQ)。
NoLiMa 去除问题与目标信息之间的词面重叠(ROUGE-1 仅 0.069 vs NIAH 的 0.905),迫使模型通过潜在联想推理定位 needle；结果 13 个主流 LLM 中 11 个在 32K 时降至基线 50% 以下，揭示注意力机制在缺失表面匹配线索时的根本脆弱性。
现有LLM serving引擎将agentic program的每次LLM call视为独立请求调度，导致program-level head-of-line blocking。Autellix将OS的Least Attained Service调度提升到program级别（PLAS/ATLAS），按program累计服务时间排优先级，配合anti-starvation和locality-aware load balancing，在相同延迟下实现4-15×吞吐提升。 ### Agent scope - **Task class**: 通用agent serving基础设施——覆盖singl
Qwen2.5-Omni 提出 Thinker-Talker 双组件框架实现 text/image/audio/video → text+speech 端到端流式生成，通过 TMRoPE 对齐音视频时间轴、DPO 提升语音稳定性，在 OmniBench 达到 SOTA (56.13%)，语音指令跟随接近纯文本输入水平。
Microscaling FP4 attention with two-level quantization achieves 1038 TOPS (5× FlashAttention2) on RTX 5090; trainable INT8 attention keeps $dO \cdot V^\top$ in FP16, delivers lossless fine-tuning at 1.67× speedup but degrades pretraining convergence.
SageAttention2++ 将 PV matmul 的累加器从 FP32 切换到 FP16（`mma.f16.f8.f8.f16`），通过缩窄 P/V 量化范围（$P_r \times V_r \leq 1023.5$）避免溢出，实现 3.9× over FlashAttention2 的 attention kernel 加速，精度无损。 ---
PCD：训练免费对比解码，对 RoPE 低频分量施加过旋转构造 local-aware logits $L^*$，与标准 logits 对比 $\tilde{L}=(1+\beta)L-\beta L^*$，缓解长上下文 gold token 后验显著度衰减（PSA）。InfiniteBench KV-Retrieval 8K +7.0%，RULER VT 4K +7.78 F1。 ---
DiT 注意力权重天然分解为高秩稀疏（<10%）+ 低秩密集（>90%）两部分。SLA 据此将注意力块三分类为 critical（FlashAttention $O(N^2)$）、marginal（线性注意力 $O(N)$）、negligible（跳过），融合为单一 Triton kernel，在 Wan2.1-1.3B 上实现 95% 注意力计算削减、13.7× kernel 加速、2.2× 端到端加速且无质量损失。 ---
UniVideo 将冻结 MLLM（Qwen2.5VL-7B）与 MMDiT（HunyuanVideo-13B）组成双流架构，通过 MLP 连接器 + self-attention 对齐，以三阶段训练统一视频理解/生成/编辑，并展现图像编辑→视频编辑的零样本迁移能力。ICLR 2026。
美团提出将 GRPO 适配到 flow matching 视频生成，理论证明其等价于随机噪声搜索（$dR/dv_\theta \approx -\frac{3}{2}\hat{A}\epsilon$），由此导出固定 SDE 时间步、loss 重加权、max group std 三项改进；结合多 reward 加权训练防止 reward hacking，加上 coarse-to-fine + 3D Block Sparse Attention 实现 12.3× 推理加速。13.6B 稠密 DiT 模型在 VBench 2.0 开源第一、Commonsense 全场最佳。
MI300X 理论算力 1.5× H100 但端到端 LLM 推理仅达 37–66%；频率降至 boost 的 52–58%（2100→1083–1217 MHz）是首要瓶颈，软件效率 80–85% 为次要因素；memory BW 4.3 TB/s（81% 利用率）是唯一接近 competitive 的维度。
fabric-lib 抽象 ConnectX-7 与 EFA 的共同语义（可靠无序传输 + WriteImm），通过 ImmCounter 原语实现无序完成通知，为 LLM 系统提供可移植的 RDMA P2P 通信库。三个生产系统验证：KvCache 传输（1.9% TTFT 开销）、RL 权重更新（1T 模型 1.3s）、MoE dispatch（CX-7 上匹配/超越 DeepEP decode 延迟）。 ---
Swizzled Head-first Mapping — a ~15-line Triton WG-ID remapping that confines all workgroups of one attention head (the "Attention Compute Cluster") to a single XCD on AMD MI300X, achieving up to 50% forward-pass speedup and sustaining 80–97% L2 cache hit rates versus ~1% for NUMA-unaware baselines.
提出 "Three Taxes" 框架（Kernel Launch / Bulk Synchronous / Inter-Kernel Data Locality）解构 BSP 模型在多 GPU LLM 推理中的性能开销；基于 AMD Iris 库将 collective communication 融合进 Triton compute kernel，以 tile-level producer-consumer pipeline 替代全局 barrier，在 AG+GEMM 和 Flash Decode 上实现 10–20% 端到端延迟加速。 ---
Gemini 2.5 Flash retrieves 26/26 single-needle factoids at 100% accuracy across all context-fill ratios (13%–92% of 1M-token limit), empirically demonstrating the "Lost in the Middle" U-curve has vanished for simple factoid Q&A at frontier model scale.
HipKittens: C++ tile-based DSL for AMD GPUs. Replaces NVIDIA's wave specialization with 8-wave ping-pong; bypasses HIPCC via pinned registers for AGPR→MFMA input; introduces chiplet-aware XCD grid scheduling for joint L2+LLC reuse. Matches AMD hand-tuned assembly on GEMM/attn fwd, 1.8–2.5× faster on
首篇 Blackwell B200 PTX 微基准实测：TMEM 降低 58% 延迟 (420 vs 1000 cycle)；tcgen05 空间阵列设计实现 ~11 cycle 恒定 MMA 延迟 (Hopper wgmma 线性增长 32→128)；FP4 达 7700 TFLOPS (96% 峰值)；端到端推理 1.6–2×、训练 1.55×、能效 +32%。 ---
Qwen2.5-7B 在上下文长度达到最大容量 40–50%（≈55K tokens/128K）时 F1 从 0.556 骤降至 0.302（降幅 45.5%），通过 natural length distribution analysis + 五方法交叉验证精确定位 critical threshold 于 43.2%，并以 RoPE aliasing / attention dispersion / information bottleneck 三视角统一解释。 ---
K2.5 提出四项训练算法创新：(1) 早期融合低比例 (10:90) 联合视觉-文本预训练；(2) 零视觉 SFT——仅用文本 SFT 激活视觉推理；(3) PARL——冻结子 agent + 编排器 RL 的去耦多 agent 训练；(4) Toggle——交替 budget/full-length 的 token 高效 RL。Agent Swarm 实现 3–4.5× 延迟降低。
面向边缘设备（Apple M4 Pro, 24 GB）的多智能体 KV cache 持久化系统。通过将每个 agent 的 KV cache 以 Q4 格式持久化到磁盘并直接在量化张量上做注意力计算，将上下文恢复从 $O(n)$ prefill 降为 I/O-bound 加载。三个架构不同的模型上实现 11–136× TTFT 加速、4× agent 容量、PPL 影响 −0.7% 至 +3.0%。
AttnRes 用 depth-wise softmax attention（每层一个伪 query $\mathbf{w}_l$）替代固定权重 1 的残差累加，解决 PreNorm dilution；Block AttnRes 将层分 N 块把内存从 $O(Ld)$ 降至 $O(Nd)$。1.25× compute equivalence，Kimi Linear 48B 全面提升（GPQA +7.5），训练开销 <4%。 ---
ByteDance 的统一多模态音视频联合生成模型，采用 dual-branch MMDiT + cross-modal joint denoising + RLHF 后训练，在 Arena.AI T2V/I2V 双榜 Elo #1 (1450/1449)，720p 击败 1080p 竞品。论文为评测报告，算法细节全部未披露。 ---
KVDrive 是跨 HBM/DRAM/SSD 三层的 KV cache 管理系统：注意力感知滑动窗口缓存（2D MCKP 优化 per-layer-per-head 窗口）+ 弹性 SFC 流水线（选择/传输/计算解耦微批并行）+ 协调式多层存储（重要性预热 + SSD 顺序布局 + 稀疏同步），在 GPU 显存受限下长上下文推理吞吐量最高提升 $1.74\times$。
当 resident reusable KV（已计算、待复用的前缀 KV）和 active live KV（在飞请求正使用的 KV）无法同时装入一个物理 KV pool 时，现有运行时（vLLM/SGLang/TRT-LLM）只有 retention primitives 而缺少一个 conformance contract 来定义"被接受的 resident claim 被破坏时运行时必须做什么"。本文提出 Resident KV Claims 合约：定义 claim 生命周期、materialization predicate、active/resident feasibility bou
Open-source Python agent harness (~3M chars) reimplementing Claude Code's architecture with multi-provider support (Anthropic/OpenAI/Copilot/Gemini/Ollama), 43 tools, Pydantic-based tool system, plugin/skill/hook extensibility, multi-agent swarm coordination, and an opinionated personal agent (ohmo)
AMD's modular, composable GPU communication framework providing RDMA+GPU building blocks (IBGDA, P2P, SDMA collectives) plus turnkey MoE dispatch/combine (MORI-EP), point-to-point IO, and symmetric memory with device-linkable bitcode — deployed across SGLang, vLLM, DeepSpeed, and RTP-LLM on MI300X/M
Decompiled TypeScript source (~512K LOC) of Anthropic's Claude Code v2.1.88. AsyncGenerator agent loop (QueryEngine→query()→StreamingToolExecutor) with 40+ tools, React/Ink TUI, 4-tier permission engine, 3-strategy context compaction, 5-mode sub-agent spawning, and 108 dead-code-eliminated internal
Rust-based prefix-cache-aware scheduler/router for LLM inference engines (SGLang, vLLM, ATOM). Routes requests via radix-tree or UMBP block-hash matching to maximize KV cache reuse. Supports PD disaggregation with coordinated prefill/decode dual-dispatch over RDMA. Achieves 91.6 req/s vs 23.7 for ro
LLM/multimodal serving engine (28K stars). Three-process ZMQ architecture (Tokenizer→Scheduler→Detokenizer) with RadixAttention prefix tree, speculative decoding zoo (EAGLE/MTP/DFLASH), HiCache offloading, and 7+ hardware backends. Dual identity: frontend DSL + high-perf runtime.
Rust-core (axum+tokio) request router for vLLM with 6 LB policies — headlined by a concurrent radix-tree cache-affinity algorithm — plus prefill-decode disaggregation over three KV connectors (NIXL/Mooncake/MoRI-IO), circuit breakers, K8s service discovery, and Python CLI via PyO3. Forked from SGLan
Dominant open-source LLM serving engine (81K stars). PagedAttention + continuous batching + CUDA graphs deliver SOTA throughput. V1 engine runs scheduler out-of-process via ZMQ; supports 200+ model architectures, disaggregated P/D, 10+ quantization formats, multi-vendor GPU.
SideQuest replaces heuristic KV cache eviction with a parallel auxiliary LRM thread that semantically reasons about stale tool responses and issues structured deletion commands, achieving 56–65% peak token reduction on agentic benchmarks (FRAMES, BrowseComp) with ≤5% accuracy loss, trained on only 2
首次基于源码 (TypeScript v2.1.88) 解剖生产级 coding agent 完整架构——揭示 "1.6% 决策逻辑 + 98.4% 确定性基础设施" 范式, 追踪 5 values → 13 principles → 实现, 通过 OpenClaw 对比展示部署上下文如何塑造 agent 设计空间。 ---
Cursor replaces static context injection with "dynamic context discovery": tool outputs, chat history, MCP tools, skills, and terminal sessions are all materialized as files on disk, letting the agent pull only what it needs via grep/read. A/B testing shows 46.9% token reduction for MCP-heavy sessio
Signal-driven intelligent router for mixture-of-models, deployed as Envoy ExtProc gRPC sidecar. Uses a DecisionEngine that evaluates recursive boolean expression trees over 20+ signal types to route LLM requests across heterogeneous model fleets, balancing cost, privacy, safety, and capability. Kube
TileRT 用 AOT 编译将模型静态展开为单个 Persistent Engine Kernel，以 tile 为调度粒度、warp/block/GPU 三级特化持续重叠 load/compute/communication，消除近 BS=1 decode 下的 inter-kernel idle，弥合 8×H200 NVL 理论 ~1000 tok/s 与实际几十 tok/s 的量级差距。
TileRT 将 LLM 算子分解为 tile 级任务，在 8×B200 上动态调度计算-IO-通信重叠，实现 batch=1 超低延迟推理。DeepSeek-V3.2 达 600 tok/s，GLM-5 达 500 tok/s，MTP 下 590 tok/s。已在 Z.ai 生产部署。
MoE prefill serving 的三重冗余（计算、内存、通信）根源于「expert 放置与 activation 路由的耦合」。ZeRO-Prefill 提出 AsyncEP，将 expert 从 "按 activation 路由" 反转为 "按 weight 流入"——用后台 D2D AllGather 替代每层同步 AllToAll，配合 frontend 强制的饱和阈值 T 保证 o
KVServe 是第一个以 **vLLM V1 external connector** 形式实现的 service-aware KV-cache 压缩框架。核心创新在于：(1) 将 KV 压缩抽象为三阶段模块化 pipeline（Transform → Quantizer → Codec），每阶段可独立配置替换；(2) 引入基于 analytical model + ε-greedy bandi
**一句话总结**: 面向 agentic workloads 的 speed-of-light LLM 推理引擎，通过编译期 SPMD 并行化、FSM 类型安全调度器、分层插件化内核系统和 Blackwell MLA 内核，达到 TensorRT-LLM 级性能和 vLLM 级可用性。
AVO 将 LLM coding agent 提升为进化搜索的 variation operator，在 NVIDIA B200 上经 7 天自主进化产出 attention kernel：causal MHA 超越 cuDNN 至多 3.5%、FA4 至多 10.5%，峰值 1668 TFLOPS (BF16)；优化 30 分钟迁移至 GQA。
μCUTLASS DSL (~170-line EBNF) + Speed-of-Light roofline guidance turns GPT-5-mini from 0.40× regression to 1.56× speedup on 59 KernelBench problems (H100); each model tier matches the next tier's baseline at lower token cost. SOL-guided scheduling saves 19–43% tokens; integrity pipeline prevents up
把 multi-agent All-Gather round 当作 KV Cache 复用的第一类单位，将 N 个智能体在同一轮中的共享内容的 PIC 复用代价从 O(N) 摊销到 O(1)，并用 Master-Mirror 块稀疏 diff 将 per-agent 存储压缩 11–17×，使同一张 A100 可支撑最多 2.7× 的并发智能体数。
TensorHub 提出 Reference-Oriented Storage (ROS)，一种无数据所有权的存储抽象，直接复用 GPU 上已有的模型权重副本通过 RDMA 传输，结合 pipeline replication、拓扑感知调度和容错机制，在 1024 GPU standalone rollout 中减少 GPU stall 6.7×，elastic rollout 加速 4.8×，跨数据中心 stall 减少 19×。
将 LLM agent 的执行系统统一为一个 **scheduler 模型** $\mathcal{E}=(\mathcal{S},\mathcal{U},\mathcal{P},\mathcal{O},\Delta)$，用 **ready-set 基数 $|\mathcal{U}|$** 把 Agent Loop 和 graph-based executor 放到同一条连续谱上，并据此提出一个牺
PrfaaS 把 LLM serving 的 PD 分离从"单集群 RDMA 岛"扩展到"跨 DC commodity Ethernet"：把长上下文 prefill 选择性 offload 到独立 PrfaaS 集群、KV 经 Ethernet 回流到本地 PD 集群 decode。两根支柱：**混合注意力**（KDA:MLA=3:1 等）把单实例 KV 吞吐压到 ≈3 Gbps 让跨 DC 可行；**长度阈值 $t$ 路由 + 双时间尺度调度**让它实用。1T Kimi-Linear 风格模型、32 H200 + 64 H20 vs 96 H20 同构：**+54% 吞吐、−64% P90
现代 chiplet GPU（AMD MI300X/MI350 八颗 XCD 各自 4 MB 私有 L2；NV Blackwell 双 die）物理上把 L2 切碎了，CUDA/HIP 却还把 L2 当整块设备级资源暴露——结果 LLM decode 时 8 颗 XCD 各自独立把同一份权重从 HBM 拉进自己的 L2，把 40 MB L2 当成 4 MB 用。**Fleet** 做了三件事：(1) 在任务抽象里新增 **Chiplet-task** 绑定 "一颗 XCD + 其 L2 working set"；(2) 以 Mirage MPK 为基础实现持久化 megakernel，每颗 X
GPU 上的微操作（element-wise、小 reduction、KV cache 更新等）在 micro-batch inference 中产生的 kernel launch 开销（每次 3–7 μs）可超过计算本身数量级。GPUOS 提出一套 persistent kernel + runtime operator injection 原语：进程启动时发射一个永不退出的持久 kernel（
DeepSeek-V4 系列包含两款原生支持百万 token 上下文的 MoE 语言模型：**V4-Pro（1.6T 总参/49B 激活）** 和 **V4-Flash（284B 总参/13B 激活）**。核心创新在于 **hybrid CSA+HCA attention** 架构——CSA 将 KV cache 沿序列维度 4× 压缩后再做 sparse top-k 选择，HCA 以 128× 更激进的压缩率做 dense attention——配合 **mHC（Manifold-Constrained Hyper-Connections）** 增强残差连接稳定性，以及 **Muon opt
Kimi K2.6 是 **K2.5 的 post-training + 量化 + orchestration 发布**，不是新底模。`config.json` 与 K2.5 的关键维度**逐字相同**（61L · d=7168 · MLA · 384E top-8 + 1 shared · YaRN 64× → 262K context），主要变化全部在：(1) **默认 INT4 compressed-tensors 出版**（仅量化 routed MoE experts，attention + shared + head 保留 FP16）；(2) **长程 agentic coding
开源的自我进化 AI Agent 框架，具备闭环学习能力（技能自动创建与改进、持久记忆、用户建模），支持 20+ 平台网关和 6 种终端后端，从 5 美元 VPS 到 GPU 集群均可运行。
**一句话总结**: 一个轻量但功能强大的多 Agent 工作流编排框架，让开发者用声明式方式定义 AI Agent 的指令、工具、护栏和协作关系，然后自动执行。
提出 Speculative Speculative Decoding (SSD) 框架，通过在 verification 进行的同时预测可能的 verification outcome 并预先为其准备 speculation（speculation cache），将 drafting 与 verification 完全并行化，消除 speculative decoding 的顺序依赖瓶颈。优化实
MFS解决disaggregated MoE serving中三阶段通信（KV-cache复用 + collective comm + P2D传输）的网络争用问题，通过反转经典MLFQ的Reverse Multi-Level Queue实现Defer-and-Promote策略，无需精确laxity即可近似Least-Laxity-First调度，TTFT SLO达标率提升1.2×–2.4×。 ---
PASTE 通过模式感知的推测性工具执行（speculative tool execution）将 agent 任务完成时间缩短 48.5%、工具执行吞吐提升 1.8×。核心思路：agent 的 tool 调用序列虽然语义多变，但在应用层存在稳定的控制流模式（如 search→fetch、edit→test）和可预测的数据依赖（参数来自前序 tool 输出），PASTE 用 Pattern Tuple $(C, T, f, p)$ 形式化这些模式，在 LLM 还在"思考"时就提前投机执行下一个 tool。
DualPath 在 PD 分离推理架构中增加 storage→decode engine→RDMA→prefill engine 的第二条 KV-Cache 加载路径，聚合所有 engine 的 storage NIC 带宽，配合 CNIC-centric 流量隔离和自适应调度，在 agentic workload 下实现最高 1.87× 离线吞吐和 1.96× 在线服务提升，1152 GPU 近线性扩展。
CMV 将 LLM agent 会话历史建模为 DAG（快照为节点、分支为边），配合三遍流式裁剪算法（保留用户/助手消息、删除 tool 输出/base64/元数据），实现均值 20%、峰值 86% 的 token 缩减，并在 prompt caching 场景下 10 轮内收回缓存失效惩罚。
AIConfigurator 是 NVIDIA 开发的统一性能建模系统，能在 **不使用 GPU profiling** 的情况下，在 30 秒内完成跨框架（TRT-LLM、vLLM、SGLang）的 LLM 推理配置空间搜索。核心思路是将推理分解为可解析建模的基本算子（GEMM、attention、communication、memory），结合预先采集的 kernel-level 性能数据库，
**Q1: 这篇论文试图解决什么核心痛点/问题？**
Concur reframes GPU KV-cache as a shared, finite resource (analogous to network bandwidth) and introduces an **AIMD-inspired, agent-level admission controller** that proactively regulates the number o
Proposes a latency-optimal algorithm for expert replication and reallocation during distributed MoE inference that jointly minimizes load imbalance and data movement overhead, formulated first as ILP
首次从 CPU 侧视角系统刻画 agentic AI 的端到端延迟瓶颈：工具执行（检索、摘要、代码执行、分子生成）占 E2E 延迟高达 88%，CPU 并行化效率远低于 GPU 导致吞吐过早饱和。提出 COMB（微批重叠）和 MAS（混合调度）两种调度优化，分别将服务延迟降低 3.9× 和 2.49×。
**GLM** 是第一个把 Graph-CoT (LLM 在知识图谱上逐步推理) 的 **reasoning 架构** 和 **LLM serving 架构 co-design** 的系统：在上层用 C/R/A 三个专门化 agent + Graph-RAG retriever 取代"单 agent monolithic prompt"，在下层用 vertex-centric KV-cache r
Continuum 为多轮 ReAct agent 推理引入 KV cache time-to-live (TTL) 机制：以 cost-benefit 模型计算最优保留时间，兼顾 reload 成本和排队延迟；结合 program-level FCFS 调度，实现 1.12–3.66× 延迟下降和最高 8.18× 真实 SWE-agent 加速。
Justitia 把 **网络包调度里的 Virtual-Time Fair Queuing (WFQ)** 搬到 **LLM 应用级调度**——以 **KV token-time**（显存占用 × 持续时间）作为成本度量，用 **per-app 轻量 MLP** 在应用到达瞬间预测其成本，再以 **GPS (Generalized Processor Sharing) 参考系统下的 virtua
Qwen3-Omni: Thinker–Talker MoE architecture unifying text/image/audio/video perception and generation without modality degradation vs unimodal counterparts; open-source SOTA on 32/36 audio benchmarks; 234 ms first-packet speech latency via multi-codebook streaming. ---
KVFlow replaces SGLang 的 LRU 前缀缓存驱逐策略，改为 **workflow-aware 的 "steps-to-execution" 优先级 + 完全重叠的 CPU→GPU 预取**，在多智能体（multi-agent）工作流场景下相比 SGLang+HiCache 实现 1.83–2.19× 端到端加速。
Pie decomposes the monolithic LLM generation loop into fine-grained handlers (embed / forward / sample), delegating end-to-end control to user-provided Wasm programs called inferlets — achieving only 2.4% overhead on 8B models while delivering 1.3–3.4× throughput on agentic workflows via application
一个专门针对**多阶段 agentic Text-to-SQL 工作流**的两层调度器 HexGen-Flow——顶层做"异构 GPU + 工作负载平衡"的任务派发，底层在每个模型实例内做"基于剩余 SLO 预算"的紧急度优先队列——把已有 vLLM/VTC/QLM 在独立请求上的调度范式升级到"带阶段依赖 + 端到端 SLO"的 agentic workflow 调度。
**Q1: 这篇论文试图解决什么核心痛点/问题？**
**首个面向多类型 SLO 的 LLM 服务调度系统**——用"保守估计 + 在线精化"的方式处理未知的响应长度与依赖图，然后用 Grouped Margin Goodput Maximization (GMAX) 算法把"刚好够用的带宽"just-in-time 分配给每个请求，使 service goodput 相比 SOTA 提升 1.4× –6.3×，并给出常数竞争比（≈1/8.55）的可
基于7份官方白皮书的深度解读，系统性对比AMD CDNA 1/2/3/4与NVIDIA Ampere/Hopper/Blackwell四代数据中心GPU架构的演进路径、技术取舍与竞争格局。
Multi-turn PD disaggregation wastes 99% of prefill compute re-processing cached history. PPD observes append-prefill causes only 2% TPOT degradation (vs. 48% for full prefill) and dynamically routes Turn 2+ requests to decode nodes via an offline-profiled scoring function, cutting Turn 2+ TTFT by 48
AMD CDNA 4架构全面押注AI精度下探，引入MXFP Microscaling格式(MXFP8/6/4)，矩阵资源翻倍，LDS扩容至160KB，在FP8达5 PFLOPS、MXFP4达10 PFLOPS，同时大胆移除TF32硬件并削减FP64 Matrix性能。
DualPipe — a bidirectional pipeline parallelism algorithm achieving full computation-communication overlap for cross-node MoE training, enabling 671B parameter model training at only \$5.576M cost.
SageAttention2 proposes a quantized attention mechanism that uses INT4 for $QK^\top$ and FP8 for $PV$ with three key techniques (Q+K smoothing, per-thread quantization aligned to MMA instruction layou
HybridFlow combines single-controller (for inter-model coordination) and multi-controller (for intra-model distributed computation) paradigms into a hierarchical hybrid programming model, plus a 3D-Hy
双Die统一GPU突破光刻极限(NV-HBI 10TB/s)，FP4 Tensor Core + 第二代Transformer Engine，NVLink5(1.8TB/s) + NVL72机架级GPU域，推理能效比H100提升25x。
FlexRLHF提出两种模型放置策略（Interleaving和Disaggregated），打破RLHF训练中四个模型Co-located在所有设备上的固定范式，通过精细化的设备分配和训练/推理运行时解耦，在大规模场景下实现最高11×的吞吐量提升。
AMD革命性chiplet GPU架构CDNA 3，通过8个XCD(5nm)+4个IOD(6nm)的3D堆叠实现前所未有的计算密度，引入FP8/TF32/2:4稀疏，并以256MB Infinity Cache和192-256GB HBM3/3E显存成为LLM推理的有力竞争者。
第四代Tensor Core引入FP8精度和Transformer Engine自动精度管理，配合TMA硬件加速和Thread Block Clusters编程模型，实现LLM训练30x加速。
AMD首个多芯片模块(MCM)封装GPU架构CDNA 2，通过双GCD封装和CPU-GPU缓存一致性驱动全球首台Exascale超算Frontier。
AMD首个专用计算GPU架构CDNA，引入Matrix Core Engine和MFMA指令族，将GPU从图形通用架构分化为计算专用架构。
第三代Tensor Core支持全数据类型(含FP64 TC和TF32)，引入MIG多实例GPU和2:4结构化稀疏，奠定数据中心GPU标准。
**趋势**: 本周重心明确集中在 router 层，topology 感知路由与 KV transfer 执行框架初步成型。
**趋势**: 静默周，本周无任何代码活动，项目可能处于早期规划或内部开发阶段。
**趋势**: 轻量周，核心新增 symmetric memory 多线程支持，其余为构建修复与文档完善。
**趋势**: 低活跃周，全部 13 commits 均为 bug fix，重心在权限体系加固和多 provider 认证修复。
**趋势**: 低活周，重心在 DeepSeek V4 生态完善与多硬件后端扩展，核心引擎以 bugfix 和小幅优化为主。
**趋势**: DeepSeek V4 生态适配全面铺开，KV connector 可靠性加固，Model Runner V2 持续推进。