MearcAll papers

arXiv 2210.03629 · cs.CL cs.AI cs.LG

ReAct: Synergizing Reasoning and Acting in Language Models

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and 1 more

Claims were extracted automatically by reducer-1 on 2026-11-12. Claims are quoted in full under the paper's license.

No reproduction recordedclm_ff2323bf
We apply our approach, named ReAct, to a diverse set of language and decision making tasks and demonstrate its effectiveness over state-of-the-art baselines in addition to improved human interpretability and trustworthiness.

ABSTRACT · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_e0b59b8c
Furthermore, on two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples.

ABSTRACT · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_8f818b4c
2 Published as a conference paper at ICLR 2023 We conduct empirical evaluations ofReAct and state-of-the-art baselines on four diverse benchmarks: question answering (HotPotQA, Yang et al., 2018), fact verification (Fever, Thorne et al., 2018), text-based game (ALFWorld, Shridhar et al., 2020b), and webpage navigation (WebShop, Yao et al., 2022).

I NTRODUCTION · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_cf8e82e8
On ALFWorld and WebShop, two or even one-shot ReAct prompting is able to outperform imitation or reinforcement learning methods trained with 103∼ 105 task instances, with an absolute improvement of 34% and 10% in success rates respectively.

I NTRODUCTION · page 3 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_d17b052d
C) Performant and robust: ReAct shows strong generalization to new task instances while learning solely from one to six in-context examples, consistently outperforming baselines with only reasoning or acting across different domains.

I NTRODUCTION · page 4 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_0b5c410a
Specifically, we use a combination of thoughts that decompose questions (“I need to search x, find y, then find z”), extract information from Wikipedia observations (“x was started in 1844”, “The paragraph does not tell x”), perform commonsense (“x is not y, so z must instead be...”) or arithmetic reasoning (“1844 < 1989”), guide 2We find more examples do not improve performance.

M ETHODS · page 4 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_5ccc0ad0
We set 7 and 5 steps for HotpotQA and FEVER respectively as we find more steps will not improve ReAct performance3.

Method · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_aa3ea002
ReAct outperforms Act consistently Table 1 shows HotpotQA and Fever results using PaLM- 540B as the base model with different prompting methods.

R ESULTS AND OBSERVATIONS · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_25f5ef89
ReAct vs. CoT On the other hand, ReAct outperforms CoT on Fever (60.9 vs. 56.3) and slightly lags behind CoT on HotpotQA (27.4 vs. 29.4).

R ESULTS AND OBSERVATIONS · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_5fa59f13
While two ReAct + CoT-SC methods are advantageous at one task each, they both significantly and consistently outperform CoT-SC across different number of samples, reaching CoT-SC performance with 21 samples using merely 3-5 samples.

R ESULTS AND OBSERVATIONS · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_2dd546e1
However, when finetuned with just 3,000 examples, ReAct becomes the best method among the four, with PaLM-8B finetuned ReAct outperforming all PaLM-62B prompting methods, and PaLM-62B finetuned ReAct outperforming all 540B prompting methods.

R ESULTS AND OBSERVATIONS · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_6aa46412
As all prompting methods are still significantly far from domain-specific state-of-the-art approaches (Table 1), we believe finetuning with more human-written data might be a better way to unleash the power of ReAct.

R ESULTS AND OBSERVATIONS · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_4ba3b4a1
It includes 6 types of tasks in which an agent needs to achieve a high-level goal (e.g. examine paper under desklamp) by navigating and interacting with a simulated household via text actions (e.g. go to coffeetable 1, take paper 2, use desklamp 1).

D ECISION MAKING TASKS · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_7ba2dc8b
We compare to an imitation learning (IL) method 5Micheli & Fleuret (2021) finetuned a GPT-2 model on 3553 task instances and achieved a much improved performance than BUTLER, but it is trained on all task types, thus not included as a baseline.

D ECISION MAKING TASKS · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_14eb17b0
Results ReAct outperforms Act on both ALFWorld (Table 3) and Webshop (Table 4).

D ECISION MAKING TASKS · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_957cce3d
On ALFWorld, the bestReAct trial achieves an average success rate of 71%, significantly outperforming the best Act (45%) and BUTLER (37%) trials.

D ECISION MAKING TASKS · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_a43861ba
In fact, even the worse ReAct trial (48%) beats the best trial of both methods.

D ECISION MAKING TASKS · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_b413cc3b
With additional sparse reasoning, ReAct achieves significantly better performance, with an absolute 10% improvement over the previous best success rate.

D ECISION MAKING TASKS · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_c95b1794
As can be seen in Table 3,ReAct substantially outperforms IM-style prompting ( ReAct-IM) (71 vs. 53 overall success rate), with consistent advantages on five out of six tasks.

D ECISION MAKING TASKS · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_63bc06e0
However, we argue that Inner Monologue does not truly comprise of inner thoughts — this is elaborated in Section 4.

R ELATED WORK · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_eb28c981
Through a diverse set of experiments on multi-hop question-answering, fact checking, and interactive decision-making tasks, we show that ReAct leads to superior performance with interpretable decision traces.

C ONCLUSION · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_dc341fce
9 Published as a conference paper at ICLR 2023 with initial promising results, but learning from more high-quality human annotations will be the desiderata to further improve the performance.

C ONCLUSION · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_c1a7e70c
As shown in Table 5, GPT-3 (text-davinci-002, greedy decoding) consistently outperforms PaLM-540B on HotpotQA and ALFWorld, possibly because it is finetuned with human instruction following.

GPT-3 E XPERIMENTS · page 14 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_d405b106
Answer REFUTES Claim Beautiful reached number two on the Billboard Hot 100 in 2003.

F EVER · page 20 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_0a4ef9d8
Action 2 Finish[REFUTES] Claim Beautiful reached number two on the Billboard Hot 100 in 2003.

F EVER · page 20 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_56d08c8c
Answer REFUTES Continued on next page 20 Published as a conference paper at ICLR 2023 Fever Prompts – Continued from previous page Claim Beautiful reached number two on the Billboard Hot 100 in 2003.

F EVER · page 20 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_9b8aa946
Thought 1 I need to search the song Beautiful and find if it reached number two on the Billboard Hot 100 in 2003.

F EVER · page 21 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_3b0ef8f2
Thought 3 It does not mention Billboard, so I need to look up "Billboard Hot 100" to find if it reached number two on it in 2003.

F EVER · page 21 · Automated extraction, not reviewed · 0 reviewed reproductions

Reproduced one of these claims?

Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (2210.03629) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.