Large Language Model based Multi-Agents: A Survey of Progress and Challenges — L2 #

1. TL;DR #

A taxonomic survey that dissects LLM-based multi-agent (LLM-MA) systems along four axes — agents-environment interface, agent profiling, communication, and capability acquisition — then maps the field into two application streams (problem-solving vs world-simulation) and five open challenges. No equations, purely organizing schema.

2. Q1 / Q2 / Q3 #

Q1 — 痛点 (the gap). Single-LLM agents were being extended into systems of many collaborating agents (software teams, robot swarms, 25→17,945-agent societies), but this growth happened as scattered independent efforts. There was no systematic review to position, differentiate, and connect the design choices, so a practitioner had no shared vocabulary for "how do these agents perceive their world / get their roles / talk / improve?" The survey's stated reader questions are verbatim: "What domains and environments do LLM-based multi-agents simulate? How are these agents profiled and how do they communicate? What mechanisms contribute to the growth of agents' capacities?"

Q2 — 方法 (the schema). The paper's contribution is a four-aspect decomposition applied uniformly to ~30 surveyed systems (see §3):

1. Agents-Environment Interface — Sandbox / Physical / None.
2. Agent Profiling — Pre-defined / Model-Generated / Data-Derived.
3. Agent Communication — three paradigms (Cooperative / Debate / Competitive) × four structures (Layered / Decentralized / Centralized / Shared Message Pool) × content (mostly text).
4. Agent Capability Acquisition — feedback source (Environment / Agent-interaction / Human / None) + adjustment strategy (Memory / Self-Evolution / Dynamic Generation).

Two application streams are then classified against this schema: Problem Solving (software dev, embodied agents, science experiments, science debate) and World Simulation (society, gaming, psychology, economy, recommender systems, policy, disease).

The 核心技术壁垒 of this survey is not an algorithm but the self-consistent classification schema itself — specifically the discovery that the same four axes cleanly separate two philosophically opposite goals (converging to a correct answer vs. faithfully diverging into realistic emergent behavior), which is why Table 1's World-Simulation rows can legitimately carry "-" for communication paradigm while Problem-Solving rows never do. See §7.

Q3 — 结果 (what it delivers). A row-by-row Table 1 categorization of the surveyed works, a Table 2 dataset/benchmark index, a review of three frameworks (MetaGPT, CAMEL, AutoGen), and six named challenges (multi-modal, hallucination cascade, collective intelligence, scaling/orchestration, evaluation, applications-and-beyond). No quantitative metric is proposed or measured; there are zero equations.

3. 架构 / 方法图 #

The survey's own Figure 2 is the load-bearing architecture diagram — it organizes an LLM-MA system around the four §3 aspects.



Paper's Figure 2, verbatim (caption: "The Architecture of LLM-MA Systems."). This is the spine of the entire survey: interface (how agents perceive the world), profiling (how each agent gets a role), communication (how agents exchange messages), and capability acquisition (how agents improve). Every later table column and every application subsection is an instantiation of one of these four boxes.

Figure 3 zooms into the communication box, showing the four canonical topologies.



Paper's Figure 3, verbatim (caption: "The Agent Communication Structure."). Left-to-right the reader should notice the increasing centralization: Decentralized (peer-to-peer mesh, favored in world simulation) → Layered (hierarchical, favored in SOP-driven software dev) → Centralized (a coordinator node) → Shared Message Pool (MetaGPT's publish/subscribe blackboard). The structure choice is tightly coupled to the application: layered mirrors the software waterfall, decentralized mirrors real social networks.

The agent loop (redrawn). The survey does not draw a per-turn loop, so the following Mermaid captures the recurring turn structure the paper describes verbally (Werewolf example in §3.1: perceive game state → act → receive environment feedback):

stateDiagram-v2 [*] --> Perceive Perceive --> Act: agent reads environment / messages Act --> Feedback: performs role action (kill, check, post, vote) Feedback --> Adjust: environment / agent / human feedback Adjust --> Perceive: Memory / Self-Evolution / Dynamic Generation Adjust --> [*]: task done or no-feedback simulation

The memory model in surveyed systems is: short-term = the LLM context window (in-context learning), long-term = external vector DB, episodic = communication logs (used by LTC-style self-evolution to generate fine-tuning data). Error recovery is per-system and mostly implicit — the survey notes hallucination can propagate across agents rather than being locally recovered (§6.2). Planning style spans the full range: predetermined workflow (MetaGPT SOP / layered), debate loops (science debate), and dynamic role generation (AutoAgents / AgentVerse). Decomposition is top-down for problem-solving (task → subtask roles) and bottom-up/emergent for world-simulation. No paper-wide step/token/tool budget is defined; scaling cost is deferred to §6.4 as an open problem.

4. 作者证明 #

无形式化作者证明 — 仅实证 (且本文为综述，无自有实验). This is a survey with zero equations and no analytical model; it presents no theorem, convergence guarantee, or success-rate bound of its own. The template's 6-check formal audit is therefore reported as "not applicable — no formalism to verify," with the agent-specific checks answered as evidence-of-absence:

1. Success-rate model / sweep matrix — none proposed. The survey catalogs others' results but runs no sweep over (task difficulty, planning depth, tool set, backbone); the closest structured axis is the Table 1 categorization itself, which is qualitative.
2. Latency budget per turn — not modeled. The survey never claims interactive latency; §6.4 explicitly flags per-agent compute (each agent ≈ a GPT-4 call) and total resource growth as unquantified scaling challenges.
3. Failure-mode classification — the survey names classes qualitatively: hallucination cascade (§6.2), collective-intelligence shortfall from adjusting agents in isolation (§6.3), and cited per-system failures ([Aher et al., 2023] "hyper-accuracy distortion"; [Fan et al., 2023] agents overlook/modify refined beliefs). No dominant class is measured; the dominant systemic risk is argued to be cascade propagation.
4. What could have been bounded — the survey itself flags "scaling laws of multi-agents" (§6.4) as a formalization that should exist but does not; a success-rate-vs-#agents or accuracy-vs-communication-rounds curve is the missing quantity.
5. Monotonicity claim — the one directional finding it repeats is that larger backbones simulate humans more faithfully ([Aher et al., 2023]) yet distort more on knowledge tasks — a non-monotone capability/faithfulness relationship, notable precisely because it breaks the expected "bigger = better" trend.
6. Internal consistency — the schema is self-consistent (see §6 論證鏈): every Table 1 row is expressible in the four-axis vocabulary, and the two application streams partition without overlap.

5. 实验与数据 #

The survey has no self-run experiments; its "data" is the categorization tables and the field-growth plot.



Paper's Figure 1, verbatim (caption: "The rising trend in the research field of LLM-based Multi-Agents ... we count the number of papers of different types at 3-month intervals."). The two top branches (Problem Solving, World Simulation) foreshadow the §4 split; the reader should notice the sharply rising leaf counts binned at 3-month intervals — the motivation-by-volume argument that justifies writing the survey.



Paper's Table 1, verbatim (caption: "Summary of the LLM-MA studies ... Agents-Environment Interface, Agents Profiling, Agents Communication and Agents Capability Acquisition. '-' denotes ... not specifically mentioned"). This is the survey's central artifact. The load-bearing pattern: Problem-Solving rows always carry a communication paradigm (Cooperative/Debate/Competitive), while several influential World-Simulation rows (e.g. [Park et al., 2023] generative agents, [Aher et al., 2023]) carry "-" for both Paradigm and Structure — some simulations have no explicit inter-agent protocol at all. Note also that "Memory" is the near-universal adjustment strategy, with Self-Evolution and Dynamic Generation appearing rarely.



Paper's Table 2, verbatim (caption: "Datasets and Benchmarks commonly used in LLM-MA studies. ' / ' denotes the unavailability of data link."). The reader should notice the asymmetry that seeds §6.5: Problem-Solving domains reuse mature benchmarks (HumanEval, MBPP, MMLU, GSM8K, StrategyQA) with live data links, whereas World-Simulation domains rely on bespoke, often link-less ("/") benchmarks (SOTOPIA, Werewolf, Gender-Discrimination) — evidence for the claimed benchmark shortfall in science-team, economic, and disease-simulation domains.

Concrete quantitative anchors that appear in the prose: societal-simulation scale escalates 25 → 1,000 → 8,563 / 17,945 agents within a single subsection (§4.2.1); recommender simulation Agent4Rec initializes 1,000 agents from MovieLens-1M; three frameworks are singled out (MetaGPT/CAMEL/AutoGen); 68 references; 3 figs / 2 tables / 0 eqs.

6. 論證鏈 #

#	Claim (paper-internal)	Support within the paper
1	Single LLM agents have a capability trio: decision-making thought, tool-use, memory (§2.1).	Definitions drawn from [Weng, 2023] structure; grounds what a "single agent" can already do.
2	Multi-agent adds value via (a) specialization into distinct agents and (b) interaction to simulate complex environments (§1).	Verbatim two-advantage claim; supported by the application breadth list (software, robots, society, policy, games).
3	Therefore LLM-MA systems can be uniformly described by four aspects: interface, profiling, communication, capability acquisition (§3, Fig.2).	The four boxes of Fig.2; each is given a closed sub-taxonomy (3+3+[3×4]+[4+3] options).
4	The four-aspect schema is expressive enough to categorize every surveyed work (§4, Table 1).	Table 1 fills all four aspect-columns for ~30 works with only "-" for genuinely absent elements.
5	Applications partition into Problem-Solving (goal = correct answer) vs World-Simulation (goal = realistic emergent behavior) (§4).	§4.1 vs §4.2 subsections; contrast statement that world-sim "focus[es] on ... diverse methods of agent management" unlike cooperative problem-solving.
6	The field's central promise — collective intelligence — is not yet realized because current Memory/Self-Evolution methods "adjust agents in isolation" (§6.3).	Candid limitation statement; motivates joint multi-agent adaptation as the key open challenge alongside scaling/orchestration (§6.4).

7. 实现 cross-reference #

[实现未公开] — this is a survey; it ships no code of its own. It points to an open-source paper-tracking repository (https://github.com/taichengguo/LLM_MultiAgents_Survey_Papers) and reviews three external frameworks whose implementations are public: MetaGPT ([Hong et al., 2023], arXiv:2308.00352), CAMEL ([Li et al., 2023b], arXiv:2303.17760), AutoGen ([Wu et al., 2023a], arXiv:2308.08155). Backbone requirement noted throughout: each agent is assumed to be a strong instruction-following LLM (GPT-4-class in §6.4), and the survey does not report a minimum-model-size ablation — that sensitivity is left implicit.

核心技术壁垒 (hardest-to-replicate insight). The durable contribution is the schema's discriminative power: choosing exactly these four axes (and no more) so that a single vocabulary describes two goal-opposite regimes. The subtle move is treating "None" as a first-class value for both the environment interface and the feedback source — this is what lets pure debate systems (no environment) and pure observational simulations (no feedback loop) sit in the same table as robotics stacks without breaking the taxonomy. Reproducing the survey's clarity requires reproducing that decision, not just re-listing papers.

关键实现细节 (easy-to-miss tricks).

1. The Shared Message Pool structure is not a generic topology but a specific MetaGPT invention (publish/subscribe by agent profile) folded into the four-structure taxonomy — miss it and you collapse it into "centralized," losing the efficiency argument.
2. The "-" convention in Table 1 is load-bearing meaning, not missing data: a "-" under Paradigm for a world-simulation row is an assertion that the system has no explicit inter-agent communication protocol, which is exactly the observation the survey uses to distinguish observational simulations from interactive ones.