clm_ff2323bfWe apply our approach, named ReAct, to a diverse set of language and decision making tasks and demonstrate its effectiveness over state-of-the-art baselines in addition to improved human interpretability and trustworthiness.
arXiv 2210.03629 · cs.CL cs.AI cs.LG
Read on arXiv · CC BY 4.0
Claims were extracted automatically by reducer-1 on 2026-11-12. Claims are quoted in full under the paper's license.
clm_ff2323bfWe apply our approach, named ReAct, to a diverse set of language and decision making tasks and demonstrate its effectiveness over state-of-the-art baselines in addition to improved human interpretability and trustworthiness.
res_e0b59b8cFurthermore, on two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples.
clm_8f818b4c2 Published as a conference paper at ICLR 2023 We conduct empirical evaluations ofReAct and state-of-the-art baselines on four diverse benchmarks: question answering (HotPotQA, Yang et al., 2018), fact verification (Fever, Thorne et al., 2018), text-based game (ALFWorld, Shridhar et al., 2020b), and webpage navigation (WebShop, Yao et al., 2022).
res_cf8e82e8On ALFWorld and WebShop, two or even one-shot ReAct prompting is able to outperform imitation or reinforcement learning methods trained with 103∼ 105 task instances, with an absolute improvement of 34% and 10% in success rates respectively.
clm_d17b052dC) Performant and robust: ReAct shows strong generalization to new task instances while learning solely from one to six in-context examples, consistently outperforming baselines with only reasoning or acting across different domains.
res_0b5c410aSpecifically, we use a combination of thoughts that decompose questions (“I need to search x, find y, then find z”), extract information from Wikipedia observations (“x was started in 1844”, “The paragraph does not tell x”), perform commonsense (“x is not y, so z must instead be...”) or arithmetic reasoning (“1844 < 1989”), guide 2We find more examples do not improve performance.
res_5ccc0ad0We set 7 and 5 steps for HotpotQA and FEVER respectively as we find more steps will not improve ReAct performance3.
res_aa3ea002ReAct outperforms Act consistently Table 1 shows HotpotQA and Fever results using PaLM- 540B as the base model with different prompting methods.
res_25f5ef89ReAct vs. CoT On the other hand, ReAct outperforms CoT on Fever (60.9 vs. 56.3) and slightly lags behind CoT on HotpotQA (27.4 vs. 29.4).
res_5fa59f13While two ReAct + CoT-SC methods are advantageous at one task each, they both significantly and consistently outperform CoT-SC across different number of samples, reaching CoT-SC performance with 21 samples using merely 3-5 samples.
res_2dd546e1However, when finetuned with just 3,000 examples, ReAct becomes the best method among the four, with PaLM-8B finetuned ReAct outperforming all PaLM-62B prompting methods, and PaLM-62B finetuned ReAct outperforming all 540B prompting methods.
clm_6aa46412As all prompting methods are still significantly far from domain-specific state-of-the-art approaches (Table 1), we believe finetuning with more human-written data might be a better way to unleash the power of ReAct.
res_4ba3b4a1It includes 6 types of tasks in which an agent needs to achieve a high-level goal (e.g. examine paper under desklamp) by navigating and interacting with a simulated household via text actions (e.g. go to coffeetable 1, take paper 2, use desklamp 1).
res_7ba2dc8bWe compare to an imitation learning (IL) method 5Micheli & Fleuret (2021) finetuned a GPT-2 model on 3553 task instances and achieved a much improved performance than BUTLER, but it is trained on all task types, thus not included as a baseline.
res_14eb17b0Results ReAct outperforms Act on both ALFWorld (Table 3) and Webshop (Table 4).
res_957cce3dOn ALFWorld, the bestReAct trial achieves an average success rate of 71%, significantly outperforming the best Act (45%) and BUTLER (37%) trials.
res_a43861baIn fact, even the worse ReAct trial (48%) beats the best trial of both methods.
res_b413cc3bWith additional sparse reasoning, ReAct achieves significantly better performance, with an absolute 10% improvement over the previous best success rate.
res_c95b1794As can be seen in Table 3,ReAct substantially outperforms IM-style prompting ( ReAct-IM) (71 vs. 53 overall success rate), with consistent advantages on five out of six tasks.
clm_63bc06e0However, we argue that Inner Monologue does not truly comprise of inner thoughts — this is elaborated in Section 4.
clm_eb28c981Through a diverse set of experiments on multi-hop question-answering, fact checking, and interactive decision-making tasks, we show that ReAct leads to superior performance with interpretable decision traces.
res_dc341fce9 Published as a conference paper at ICLR 2023 with initial promising results, but learning from more high-quality human annotations will be the desiderata to further improve the performance.
res_c1a7e70cAs shown in Table 5, GPT-3 (text-davinci-002, greedy decoding) consistently outperforms PaLM-540B on HotpotQA and ALFWorld, possibly because it is finetuned with human instruction following.
res_d405b106Answer REFUTES Claim Beautiful reached number two on the Billboard Hot 100 in 2003.
res_0a4ef9d8Action 2 Finish[REFUTES] Claim Beautiful reached number two on the Billboard Hot 100 in 2003.
res_56d08c8cAnswer REFUTES Continued on next page 20 Published as a conference paper at ICLR 2023 Fever Prompts – Continued from previous page Claim Beautiful reached number two on the Billboard Hot 100 in 2003.
res_9b8aa946Thought 1 I need to search the song Beautiful and find if it reached number two on the Billboard Hot 100 in 2003.
res_3b0ef8f2Thought 3 It does not mention Billboard, so I need to look up "Billboard Hot 100" to find if it reached number two on it in 2003.
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (2210.03629) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.