Agent Papers
1.Agent Architecture
- Cognitive Architecture
2.Execution Paradigms
📄CoT:
- Paper:
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models - Year:
NeurIPS 2022
- Link: https://arxiv.org/abs/2201.11903
- Description: 提出了
Chain-of-Thought Prompting即思维链提示方法，思维链是指一系列导向最终输出的中间自然语言推理步骤. 这种提示方法能够引导语言模型生成一系列连贯的中间推理步骤，引导出问题的最终答案。实验发现，这种方法对大规模模型更有效，能显著提升其在算术、常识和符号推理任务上的表现。
论文中的CoT触发方式：
这篇论文思维链是通过 few-shot prompting：给模型少量示例，每个示例都带
question -> reasoning process -> answer，让它学会把中间步骤写出来。同时论文实验发现思维链提示是模型规模的涌现能力，它发现这种方式对 更大规模的模型 效果明显更好，小模型不稳定。
📄Zero-shot CoT:
- Paper:
Large Language Models are Zero-Shot Reasoners - Year:
NeurIPS 2022 - Link:https://arxiv.org/pdf/2205.11916
- Description: 提出了
zero-shot CoT, 一种基于零样本的思维链提示方法. 与原始的CoT不同，因为它不需要一步一步的少量示例，因为它本质上是与任务无关的，通过在问题后附加简单触发语(let's think step by step)来诱导模型显式生成中间推理步骤的方法。实验表明，该方法在较大规模模型上效果更明显，尤其适用于算术和推理类任务。
zero-shot CoT 触发方式：
通过在每个答案前简单地添加这句话-----Let’s think step by step.
📄ReAct:
- Paper:
ReAct: Synergizing Reasoning and Acting in Language Models - Year:
ICLR 2023 - Link: https://arxiv.org/abs/2210.03629
- Description: 提出了
ReAct，一种将reasoning和acting交替结合的基于提示的Agent范式。模型通过few-shot轨迹示例学习在任务中生成Thought -> Action -> Observation的交互循环：先推理当前状态，再调用工具或环境获得反馈，并据此修正后续推理与行动。适用于需要检索、规划和执行的复杂任务。
ReAct Prompt 轨迹示例:：
论文是通过 few-shot ReAct prompt 案例来让 frozen LLM 学会
Thought -> Action -> Observation的交替模式，不是主要靠训练。
- Question Musician and satirist Allie Goertz wrote a song about the “The Simpsons” character Milhouse, who Matt Groening named after who?
- Thought 1 The question simplifies to “The Simpsons” character Milhouse is named after who. I only need to search Milhouse and find who it is named after.
- Action 1 Search[Milhouse] Observation 1 Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening.
- Thought 2 The paragraph does not tell who Milhouse is named after, maybe I can look up “named after”.
- Action 2 Lookup[named after] Observation 2 (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous.
- Thought 3 Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon.
- Action 3 Finish[Richard Nixon]
- …
📄Plan-and-Execute:
- Paper:
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models - Year:
ACL 2023 - Link: https://arxiv.org/abs/2305.04091
- Description: 提出了
Plan-and-Solve的零样本提示方法，来引导LLM首先制定一个计划，将整个任务划分为若干较小的子任务，然后按照该计划逐步完成这些子任务。
对比：
| Method | Trigger Sentence |
|---|---|
| CoT | Let’s think step by step. |
| PS | Let’s first understand the problem and devise a plan to solve the problem. Then, let’s carry out the plan to solve the problem step by step. |
| PS+ | Let’s first understand the problem, extract relevant variables and their corresponding numerals, and devise a plan. Then, let’s carry out the plan, calculate intermediate variables (pay attention to correct numeral calculation and commonsense), solve the problem step by step, and show the answer. |
📄Decomposed Prompting:
- Paper:
Decomposed Prompting: A Modular Approach for Solving Complex Tasks - Year:
ICLR 2023
- Link: https://arxiv.org/pdf/2210.02406
- Description:
3. Planning
- Task Decomposition
- Hierarchical Planning
- Long-horizon Planning
- Dynamic Replanning
4. Tool Use
📄SpaceTools:
- Paper:
Tool-Augmented Spatial Reasoning via Double Interactive RL - Year:
CVPR 2026
- Link: https://arxiv.org/abs/2512.04069
- Description: 提出了一个名为
DIRL双重交互强化学习的两阶段训练框架。第一个阶段即教学阶段，为VLM建立基础的工具使用能力；第二个阶段是探索阶段，主要通过持续强化学习进一步完善VLM多工具协调能力。本质上是通过这个训练框架教VLM在空间推理时如何自主协调多种视觉/机器人工具。
5. Memory
- Short-term Memory
- Long-term Memory
- Episodic Memory
- Retrieval-Augmented Memory
- Memory Management
6. Multi-Agent
- Cooperation
- Debate
- Role-playing
- Communication Protocols
- Swarm / Society
7. Learning and Improvement /Reflection
- Self-Refinement
- Preference Learning
- RL for Agents
- Experience Replay
8. Evaluation
- Benchmarks
- Metrics
- Failure Analysis
- Agent Robustness