Welcome to Loong’s Lens

Read aboout Install and Update instructions and sampled configuration templates

Learn About All Features in PaperMod

On-Policy Distillation 的发展和应用 在线策略蒸馏（On-Policy Distillation, OPD）经历了从基础理论提出到多样化变体涌现这一经典发展轨迹。可以将这一时间线分为四个主要阶段： 1. 理论根基与前哨（2011 年—2023 年初） 在 OPD 正式成为 LLM 领域的热点之前，其核心思想源于强化学习和模仿学习： DAgger (2011): OPD 的理论基石。Ross 等人提出数据集聚合（Dataset Aggregation）算法，通过在学习者访问的状态下查询专家来减少复合误差。 Policy Distillation (2015): Rusu 等人将其作为深层 Q 网络（DQN）的压缩技术，主要应用于 Atari 游戏等视觉领域。 DistillSpec (2023): 探索了将在线 KD 用于改进投机采样（Speculative Decoding）中的草稿模型。 Lion (2023): 较早的黑盒蒸馏尝试，通过教师识别学生弱点并生成更难的指令，形成对抗式闭环。 2. 经典 OPD 框架的确立（2023 年—2024 年） 这一阶段确立了 LLM 在线策略蒸馏的标准范式： GKD (2023/2024): 核心基础论文。Agarwal 等人正式提出了 通用知识蒸馏 (Generalized Knowledge Distillation) 框架，将蒸馏视为模仿学习问题，支持在前向 KL、逆向 KL 和 JSD 之间切换。 MiniLLM (2023/2024): 关键转折点。Gu 等人将逆向 KL（Reverse-KL）用于在线语言模型蒸馏，使学生更集中于教师分布中的主要模式，并在生成式指令遵循任务上取得了优于传统 KD 的结果。 f-Divergence KD (2023): Wen 等人通过 f-散度统一了序列级蒸馏的理论框架。 DistiLLM (2024): 提出了 Skew-KL 与自适应离策略训练方法，通过复用学生生成的样本，在改善蒸馏目标的同时提高训练效率。 Gemma 2 (2024): 在工业级模型训练中展示了大规模预训练知识蒸馏的效果；公开报告没有说明其训练前缀由学生在线生成，因此不宜将它直接归为严格的 OPD。 3. 多元化与混合架构的扩张（2025 年） 随着推理模型（System 2 Thinking）的兴起，OPD 开始与强化学习（RL）深度融合： ...

Evaluation of LLMs Nowadays, LLMs become more and more powerful to tackle many tasks (e.g., math proplems, content generation). However, evaluating LLMs has been still a big challenge. a A statistical approach to model evaluations —view from Anthropic Suppose an artificial intelligent model outperforms another one on an interest benchmark—such as testing whose abilities of common knowledge or solving computer coding questions. Is this difference in capabilities real? Or could one model just have gotten luckly in the choice of questions on the benchmark?

Rewrite-Retrieve-Read This work introduces a new framework, Rewrite-Retrieve-Read1 instead of the previous retrieve-then-read for the retrieval-augmented LLMs from the perspective of the query rewriting. In this framework, a small language model is adopted as a trainable rewriter to cater to the downstream LLM. Figure 1. Overview of proposed pipeline. (a) standard retrieve-then-read method. (b) LLM as a query rewriter. (c) pipeline with a trainable writer. (Image source: (Query Rewriting for Retrieval-Augmented Large Language Models)) ...

why LLM inference runs slowly excellent solutions cerebras Figure 1. The result of LLaMA3.1-70B inference speed with different solutions. (Image source: Artificial Analysis)

After pre-training on vast datasets and supervised fine-tuning with diverse instruction sets, Large Language Models (LLMs) have achieved remarkable capabilities in text generation. However, LLMs can generate seemingly reasonable sequences—-free from grammatical errors and redundant words—-they may still generate content that lacks truthfulness or accuracy. Are there any methods to mitigate these shortcomings? Researchers at OpenAI have framed these issues as the challenge of LLM alignment. Currently, one of the most prominent approaches to address these challenges is Reinforcement Learning from Human Feedback (RLHF). To implement RLHF, OpenAI has adopted the Proximal Policy Optimization (PPO) algorithm. ...

audioLM Based on SoundStream1 and w2v-BERT2, audioLM3 proposes a framework which consist of three components: tokenizer model, decoder-only Transformer language model, detokenizer model. SoundStream, is neural audio codec with strong performance, which converts input waveforms at 16 kHZ into embeddings while w2v-BERT plays the role to compute semantic tokens. Figure 1. (Image source: AudioLM: a Language Modeling Approach to Audio Generation) ...

open-source component memo: A memory module of AI Agent for memorizing personal preferences, previous interactions, and business stages. Agent