Today, a lot of agentic applications (such as, Microsoft Copilot, ChatGPT plugins, AutoGPT, etc) automate a variety of tasks by LLM reasoning, the ability to split a complex task into simpler subtasks.
Reasoning+Acting (shortly, ReAct) is essential and origin for these task completion, which is achieved by advanced LLM reasoning.

To build such autonomous agents, here I’ll show you the fundamentals of ReAct by giving you insights for the following questions. :

• What is ReAct ?
• How it works ?
• How to build ?

Introduction to ReAct

Integrating LLMs (Large Language Models) with external tools is a frequently used technique in the real business applications.
One of such examples is the search integration. LLM doesn’t always give correct answers, and the method to interact with external search engine (for both internet and intranet) is then often applied in real QA systems (which architecture is called RAG pattern). In the real applications, however, a single source of data is not sufficient to complete user tasks. Sometimes it also needs some tools other than search.
Several flexible architectures for the combination between LLM reasoning and additional tools (experts) are then proposed in several papers – such as, in ReAct (Reasoning + Acting) or MRKL (Modular Reasoning, Knowledge and Language).

Now let’s see the brief outline of ReAct prompting with search examples.

For instance, suppose, the following questions is given by a user, and you can use Wikipedia search as external tools.

This question, however, is a little tricky, because it’s a multi-hop question, which cannot be answered from a single Wikipedia entity, but can be answered by referencing multiple entities.

By applying ReAct prompting, this question is disassembled into multiple search’s actions by LLM as follows.
All these disassembled actions are then performed by external tools (such as, Search API), until the answer is obtained or it finds that the answer cannot be obtained.

From “REACT: SYNERGIZING REASONING AND ACTING IN LANGUAGE MODELS” (Shunyu et al., 2022)

The search action is issued by above “Search [entity]” and the outside tool (such as, program code) returns the sentences from the corresponding Wikipedia entity.
LLM reasons to call external tools, gets halted to fetch the tool’s response as observation, and then decides the next action based on all preceding responses.
Here I’ll show you this flow as a diagram as follows.

Note : Because of token limitation, it’s difficult to get entire document in a single Wikipedia article. Hence, “Lookup [entity]” action is also introduced in original paper, which returns the next sentence in the page containing some string, simulating Ctrl+F functionality on the page in web browser.
These 2 actions (Search, Lookup) is such like simulating human’s behaviors.

This technique is also sometimes referred as Augmented Language Models (ALMs).

How to write actual prompts and processing

Now let’s see how it performs in actual prompts.
Reasoning+Acting (ReAct) approach can be performed by few-shot or zero-shot in prompting.
First, I’ll show you few-shot version of ReAct, and next I’ll discuss about zero-shot approach.

The following is the example of in-context few-shot exemplars.

prompt 1

Question: What is the elevation range for the area that the eastern sector of theColorado orogeny extends into?Thought 1: I need to search Colorado orogeny, find the area that the eastern sectorof the Colorado orogeny extends into, then find the elevation range of thearea.Action 1: Search[Colorado orogeny]Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) inColorado and surrounding areas.Thought 2: It does not mention the eastern sector. So I need to look up easternsector.Action 2: Lookup[eastern sector]Observation 2: (Result 1 / 1) The eastern sector extends into the High Plains and is calledthe Central Plains orogeny.Thought 3: The eastern sector of Colorado orogeny extends into the High Plains. So Ineed to search High Plains and find its elevation range.Action 3: Search[High Plains]Observation 3: High Plains refers to one of two distinct land regionsThought 4: I need to instead search High Plains (United States).Action 4: Search[High Plains (United States)]Observation 4: The High Plains are a subregion of the Great Plains. From east to west, theHigh Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130m).[3]Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answeris 1,800 to 7,000 ft.Action 5: Finish[1,800 to 7,000 ft]Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons"character Milhouse, who Matt Groening named after who?Thought 1: The question simplifies to "The Simpsons" character Milhouse is named afterwho. I only need to search Milhouse and find who it is named after.Action 1: Search[Milhouse]Observation 1: Milhouse Mussolini Van Houten is a recurring character in the Fox animatedtelevision series The Simpsons voiced by Pamela Hayden and created by MattGroening.Thought 2: The paragraph does not tell who Milhouse is named after, maybe I can look up"named after".Action 2: Lookup[named after]Observation 2: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whosemiddle name was Milhous.Thought 3: Milhouse was named after U.S. president Richard Nixon, so the answer isRichard Nixon.Action 3: Finish[Richard Nixon]Question: Which documentary is about Finnish rock groups, Adam Clayton Powell or TheSaimaa Gesture?Thought 1: I need to search Adam Clayton Powell and The Saimaa Gesture, and find whichdocumentary is about Finnish rock groups.Action 1: Search[Adam Clayton Powell]Observation 1 Could not find [Adam Clayton Powell]. Similar: [’Adam Clayton PowellIII’, ’Seventh Avenue (Manhattan)’, ’Adam Clayton Powell Jr. State OfficeBuilding’, ’Isabel Washington Powell’, ’Adam Powell’, ’Adam Clayton Powell(film)’, ’Giancarlo Esposito’].Thought 2: To find the documentary, I can search Adam Clayton Powell (film).Action 2: Search[Adam Clayton Powell (film)]Observation 2: Adam Clayton Powell is a 1989 American documentary film directed byRichard Kilberg. The film is about the rise and fall of influentialAfrican-American politician Adam Clayton Powell Jr.[3][4] It was later airedas part of the PBS series The American Experience.Thought 3: Adam Clayton Powell (film) is a documentary about an African-Americanpolitician, not Finnish rock groups. So the documentary about Finnish rockgroups must instead be The Saimaa Gesture.Action 3: Finish[The Saimaa Gesture]Question: What profession does Nicholas Ray and Elia Kazan have in common?Thought 1: I need to search Nicholas Ray and Elia Kazan, find their professions, thenfind the profession they have in common.Action 1: Search[Nicholas Ray]Observation 1: Nicholas Ray (born Raymond Nicholas Kienzle Jr., August 7, 1911 - June 16,1979) was an American film director, screenwriter, and actor best known forthe 1955 film Rebel Without a Cause.Thought 2: Professions of Nicholas Ray are director, screenwriter, and actor. I needto search Elia Kazan next and find his professions.Action 2: Search[Elia Kazan]Observation 2: Elia Kazan was an American film and theatre director, producer, screenwriterand actor.Thought 3: Professions of Elia Kazan are director, producer, screenwriter, and actor.So profession Nicholas Ray and Elia Kazan have in common is director,screenwriter, and actor.Action 3: Finish[director, screenwriter, actor]Question: Which magazine was started first Arthur’s Magazine or First for Women?Thought 1: I need to search Arthur’s Magazine and First for Women, and find which wasstarted first.Action 1: Search[Arthur’s Magazine]Observation 1: Arthur’s Magazine (1844-1846) was an American literary periodical publishedin Philadelphia in the 19th century.Thought 2: Arthur’s Magazine was started in 1844. I need to search First for Womennext.Action 2: Search[First for Women]Observation 2: First for Women is a woman’s magazine published by Bauer Media Group in theUSA.[1] The magazine was started in 1989.Thought 3: First for Women was started in 1989. 1844 (Arthur’s Magazine) < 1989 (Firstfor Women), so Arthur’s Magazine was started first.Action 3: Finish[Arthur’s Magazine]Question: Were Pavel Urysohn and Leonid Levin known for the same type of work?Thought 1: I need to search Pavel Urysohn and Leonid Levin, find their types of work,then find if they are the same.Action 1: Search[Pavel Urysohn]Observation 1: Pavel Samuilovich Urysohn (February 3, 1898 - August 17, 1924) was a Sovietmathematician who is best known for his contributions in dimension theory.Thought 2: Pavel Urysohn is a mathematician. I need to search Leonid Levin next andfind its type of work.Action 2: Search[Leonid Levin]Observation 2: Leonid Anatolievich Levin is a Soviet-American mathematician and computerscientist.Thought 3: Leonid Levin is a mathematician and computer scientist. So Pavel Urysohnand Leonid Levin have the same type of work.Action 3: Finish[yes]Question: {Your actual question}

When you put your actual question in the last line of above prompt, LLM will reply in the same manner as exemplars.
In this example, I’ll put the following question (which is also a multi-hop question for Wikipedia search).

The following highlighted text is an actual response from LLM. (I have omitted the few-shot’s exemplars, because it’s too long.)
In this example, I have used text-davinci-003 model (OpenAI) with temperature 0.0 and max 256 tokens.

... few-shot exemplars (omitted)Question: Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?Thought 1: I need to search David Chanoff and the U.S. Navy admiral, find the ambassador to the United Kingdom, then find the President they served under.Action 1: Search[David Chanoff]Observation 1: David Chanoff is an American author and journalist. He is the co-author ofseveral books, including Inside the Mirage: America's Fragile Partnershipwith Saudi Arabia (2004) and The Game Player: Confessions of the CIA'sOriginal Political Operative (1989).Thought 2: I need to search the U.S. Navy admiral David Chanoff collaborated with.Action 2: Search[U.S. Navy admiral]Observation 2: Admiral Charles R. Larson was a United States Navy four-star admiral whoserved as the United States Ambassador to the United Kingdom from 1998 to2001.Thought 3: Admiral Charles R. Larson served as the United States Ambassador to theUnited Kingdom from 1998 to 2001. So the President they served under isBill Clinton.Action 3: Finish[Bill Clinton]

In this reply, “Observation 1:” means the result of search action.
This result, however, is not obtained by the actual search actions, but it’s generated by LLM.

This means, this result may not be the correct answer. (LLM doesn’t know the facts after the knowledge cutoff date.)
Hence, you should ignore all text under “Observation 1:“, and pick up only “Thought 1:” and “Action 1:” sections.
By the action’s instruction “Action 1: Search[David Chanoff]” (see above), the following steps are then performed in ReAct prompting. (These steps can be performed by the outside program code.)

1. Search “David Chanoff” in Wikipedia entities with external tools (such as, Search API).
2. Fill the search result (the actual result) in “Observation 1” section, and submit the next prompt to LLM as follows again.

prompt 2

... few-shot exemplars (omitted)Question: Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?Thought 1: I need to search David Chanoff and the U.S. Navy admiral, find theambassador to the United Kingdom, then find the President they served under.Action 1: Search[David Chanoff]Observation 1: David Chanoff is a noted author of non-fiction work. His work has typically involved collaborations with the principal protagonist of the work concerned. His collaborators have included; Augustus A. White, Joycelyn Elders, Đoàn Văn Toại, William J. Crowe, Ariel Sharon, Kenneth Good and Felix Zandman. He has also written about a wide range of subjects including literary history, education and foreign for The Washington Post, The New Republic and The New York Times Magazine. He has published more than twelve books.Thought 2:

As you can see above, this prompt includes the previous all instructions (also, including long exemplars) and “Observation 1” result, and then asks for “Thought 2“. (I note that the prompt is always stateless in LLM, and it should include all historical text.)

Next time, LLM will return the following response. (The highlighted text is also the response from LLM.)

... few-shot exemplars (omitted)Question: Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?Thought 1: I need to search David Chanoff and the U.S. Navy admiral, find theambassador to the United Kingdom, then find the President they served under.Action 1: Search[David Chanoff]Observation 1: David Chanoff is a noted author of non-fiction work. His work has typically involved collaborations with the principal protagonist of the work concerned. His collaborators have included; Augustus A. White, Joycelyn Elders, Đoàn Văn Toại, William J. Crowe, Ariel Sharon, Kenneth Good and Felix Zandman. He has also written about a wide range of subjects including literary history, education and foreign for The Washington Post, The New Republic and The New York Times Magazine. He has published more than twelve books.Thought 2: David Chanoff has collaborated with a U.S. Navy admiral. I need to searchthe U.S. Navy admiral and find the ambassador to the United Kingdom.Action 2: Search[U.S. Navy admiral]Observation 2: William J. Crowe Jr. (January 2, 1925 – October 18, 2007) was an admiral in the United States Navy who served as Chairman of the Joint Chiefs of Staff under Presidents Ronald Reagan and George H. W. Bush and then as the United States Ambassador to the United Kingdom.Thought 3: William J. Crowe Jr. was the ambassador to the United Kingdom underPresidents Ronald Reagan and George H. W. Bush. So the answer is RonaldReagan and George H. W. Bush.Action 3: Finish[Ronald Reagan and George H. W. Bush]

Same as above reason, the text under “Observation 2:” are all ignored, and only “Thought 2:” and “Action 2:” sections are used in the next chain.

The framework (program code) then searches “U.S. Navy admiral” in Wikipedia, fill the search result (the actual result) in “Observation 2” section, and submit the next prompt as follows.
(This time, it asks for “Thought 3“.)

prompt 3

... few-shot exemplars (omitted)Question: Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?Thought 1: I need to search David Chanoff and the U.S. Navy admiral, find theambassador to the United Kingdom, then find the President they served under.Action 1: Search[David Chanoff]Observation 1: David Chanoff is a noted author of non-fiction work. His work has typically involved collaborations with the principal protagonist of the work concerned. His collaborators have included; Augustus A. White, Joycelyn Elders, Đoàn Văn Toại, William J. Crowe, Ariel Sharon, Kenneth Good and Felix Zandman. He has also written about a wide range of subjects including literary history, education and foreign for The Washington Post, The New Republic and The New York Times Magazine. He has published more than twelve books.Thought 2: David Chanoff has collaborated with a U.S. Navy admiral. I need to searchthe U.S. Navy admiral and find the ambassador to the United Kingdom.Action 2: Search[U.S. Navy admiral]Observation 2: Admiral of the Navy was the highest-possible rank in the United States Navy, prior to the creation of fleet admiral in 1944. The rank is considered to be at least equivalent to that of a five-star admiral, with Admiral George Dewey being the only officer to be appointed to the rank.Thought 3:

LLM will then return the following response. (The highlighted text is the response from LLM.)

... few-shot exemplars (omitted)Question: Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?Thought 1: I need to search David Chanoff and the U.S. Navy admiral, find theambassador to the United Kingdom, then find the President they served under.Action 1: Search[David Chanoff]Observation 1: David Chanoff is a noted author of non-fiction work. His work has typically involved collaborations with the principal protagonist of the work concerned. His collaborators have included; Augustus A. White, Joycelyn Elders, Đoàn Văn Toại, William J. Crowe, Ariel Sharon, Kenneth Good and Felix Zandman. He has also written about a wide range of subjects including literary history, education and foreign for The Washington Post, The New Republic and The New York Times Magazine. He has published more than twelve books.Thought 2: David Chanoff has collaborated with a U.S. Navy admiral. I need to searchthe U.S. Navy admiral and find the ambassador to the United Kingdom.Action 2: Search[U.S. Navy admiral]Observation 2: Admiral of the Navy was the highest-possible rank in the United States Navy, prior to the creation of fleet admiral in 1944. The rank is considered to be at least equivalent to that of a five-star admiral, with Admiral George Dewey being the only officer to be appointed to the rank.Thought 3: I need to search Admiral William J. Crowe and find the ambassador to theUnited Kingdom.Action 3: Search[Admiral William J. Crowe]Observation 3: Admiral William J. Crowe Jr., KBE, DSC (January 2, 1925 – October 18, 2007) was a United States Navy admiral who served as Chairman of the Joint Chiefs of Staff under Presidents Ronald Reagan and George H. W. Bush and as the United States Ambassador to the United Kingdom under President Bill Clinton.Thought 4: Admiral William J. Crowe was the ambassador to the United Kingdom underPresident Bill Clinton, so the answer is Bill Clinton.Action 4: Finish[Bill Clinton]

Again, the text under “Observation 3:” are also ignored, and only “Thought 3:” and “Action 3:” sections are used.

The framework (program code) then searches “Admiral William J. Crowe” in Wikipedia, fill the search result (the actual result) in “Observation 3” section, and submit the next prompt as follows.
(This time, it asks for “Thought 4“.)

prompt 4

... few-shot exemplars (omitted)Question: Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?Thought 1: I need to search David Chanoff and the U.S. Navy admiral, find theambassador to the United Kingdom, then find the President they served under.Action 1: Search[David Chanoff]Observation 1: David Chanoff is a noted author of non-fiction work. His work has typically involved collaborations with the principal protagonist of the work concerned. His collaborators have included; Augustus A. White, Joycelyn Elders, Đoàn Văn Toại, William J. Crowe, Ariel Sharon, Kenneth Good and Felix Zandman. He has also written about a wide range of subjects including literary history, education and foreign for The Washington Post, The New Republic and The New York Times Magazine. He has published more than twelve books.Thought 2: David Chanoff has collaborated with a U.S. Navy admiral. I need to searchthe U.S. Navy admiral and find the ambassador to the United Kingdom.Action 2: Search[U.S. Navy admiral]Observation 2: Admiral of the Navy was the highest-possible rank in the United States Navy, prior to the creation of fleet admiral in 1944. The rank is considered to be at least equivalent to that of a five-star admiral, with Admiral George Dewey being the only officer to be appointed to the rank.Thought 3: I need to search Admiral William J. Crowe and find the ambassador to theUnited Kingdom.Action 3: Search[Admiral William J. Crowe]Observation 3: William James Crowe Jr. (January 2, 1925 – October 18, 2007) was a United States Navy admiral and diplomat who served as the 11th chairman of the Joint Chiefs of Staff under Presidents Ronald Reagan and George H. W. Bush, and as the ambassador to the United Kingdom and Chair of the Intelligence Oversight Board under President Bill Clinton.Thought 4:

Now LLM will know that the answer is found in “Observation 3“.
Finally, LLM will return the following final answer. (It’s worth noting that this answer format is also the same manner in few-shot exemplars.)

... few-shot exemplars (omitted)Question: Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?Thought 1: I need to search David Chanoff and the U.S. Navy admiral, find theambassador to the United Kingdom, then find the President they served under.Action 1: Search[David Chanoff]Observation 1: David Chanoff is a noted author of non-fiction work. His work has typically involved collaborations with the principal protagonist of the work concerned. His collaborators have included; Augustus A. White, Joycelyn Elders, Đoàn Văn Toại, William J. Crowe, Ariel Sharon, Kenneth Good and Felix Zandman. He has also written about a wide range of subjects including literary history, education and foreign for The Washington Post, The New Republic and The New York Times Magazine. He has published more than twelve books.Thought 2: David Chanoff has collaborated with a U.S. Navy admiral. I need to searchthe U.S. Navy admiral and find the ambassador to the United Kingdom.Action 2: Search[U.S. Navy admiral]Observation 2: Admiral of the Navy was the highest-possible rank in the United States Navy, prior to the creation of fleet admiral in 1944. The rank is considered to be at least equivalent to that of a five-star admiral, with Admiral George Dewey being the only officer to be appointed to the rank.Thought 3: I need to search Admiral William J. Crowe and find the ambassador to theUnited Kingdom.Action 3: Search[Admiral William J. Crowe]Observation 3: William James Crowe Jr. (January 2, 1925 – October 18, 2007) was a United States Navy admiral and diplomat who served as the 11th chairman of the Joint Chiefs of Staff under Presidents Ronald Reagan and George H. W. Bush, and as the ambassador to the United Kingdom and Chair of the Intelligence Oversight Board under President Bill Clinton.Thought 4: Admiral William J. Crowe was the ambassador to the United Kingdom underPresident Bill Clinton, so the answer is Bill Clinton.Action 4: Finish[Bill Clinton]

The framework will then know the final answer is “Bill Clinton“. (Then the chain is stopped.)

Also I summarize how LLM (in this case, OpenAI GPT-3) and external tools (in this case, Search API) interact with each others as a diagram.

As you saw above, LLM provides the thoughts and next actions, and external tools are performed for each corresponding actions.
This loop is repeated, until the final answer is found in the chain.

Note : In today’s language models (such as, latest GPT-4, Claude, Mistral, Llama, etc), you can use function calling to provide built-in tool usage capability for reliable and performant reasoning.
In this post, I don’t use function calling, but it’s worth to use this capability in real production.

Zero-shot Approach – To Make Tools Pluggable

In above example, I have provided few-shot exemplars in prompt, and I have used only “Search [entity]” action. But what if the available tools are added, deleted, or modified ? (For instance, adding “Lookup [entity]” action.)
Once you have used few-shot exemplars, you should add or modify exemplars every time you change the available tools.

To give the flexibility for these changes, the following prompting without few-shot exemplars is often used in real applications.
This approach is called zero-shot prompting, in contrast of previous few-shot prompting.

You are an assistant to give people the answer of questions.Human may provide new question to you with a description and you should use tools to get the answer, rather than directly reply from the description.You have access to the following tools:> Search: Search for a term in Wikipedia and return the first sentence. The input to this tool should be a string to search.> Lookup: Lookup a term in the Wikipedia article and return sentence which describes about a term. The input to this tool should be a string to lookup.To use a tool, please use the following format:```Question: the input question you must answerThought: your thought about what to doAction: the action to take, should be one of [Search, Lookup]Action Input: the input to the actionObservation: the result of the action... (this Thought/Action/Action Input/Observation can repeat N times)Thought: I now know the final answerFinal Answer: the final answer to the original input question```Begin!Now human has provided the following question.Question: Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?

Note : You can use zero-shot prompting also in LangChain by using langchain.agents.ZeroShotAgent class.

This zero-shot reasoning, however, sometimes becomes more difficult to get exact results, and you should then use more performant and large models – such as, GPT-4 – in the difficult zero-shot reasoning.
Generally speaking, few-shot prompting (compared to zero-shot) can greatly boost model performance on a wide range of tasks. Depending on the cases, please consider to use few-shot approach on purpose.

Implementation : Run Your First ReAct

Now we begin to implement an example (simple autonomous agent) with famous LangChain and OpenAI GPT.
LangChain is an open-source and widely-used tool to help you compose a variety of language chains in LLMs – such as, chat system, QA system, advanced RAG, etc.
To start building an agent, I have prepared Ubuntu Server 20.04 LTS in Microsoft Azure.

In this virtual machine, it already has Python 3.8 and you just install additional packages for running example as follows.

# Update pipsudo apt-get updatesudo apt-get install -y python3-pipsudo -H pip3 install --upgrade pip# Install packagespip3 install openaipip3 install langchain  # see below notepip3 install wikipedia

Note : LangChain frequently updates and I have used LangChain version 0.0.300. (Please specify version to install packages, if it cannot run.)

Before running application, you should set the environment variables for OpenAI integration in LangChain.

# Get a token from https://platform.openai.com/account/api-keysexport OPENAI_API_KEY="..."# Set OpenAI organization if you have multiple organizations# export OPENAI_ORGANIZATION="..."

Now let’s build your first ReAct code with LangChain.

The following code is an example of Wikipedia docstore included in LangChain examples, in which the question is disassembled into multiple search actions.
In this demonstration, I have used text-davinci-003 (OpenAI GPT-3.5) model with temperature 0.0. (See the following note.)

Note (Added on Jan 2024) : Currently text-davinci-003 is deprecated in OpenAI-provided models and then please use other models. (I note that the results of chain written in this post will change in other models.)

In the following code, it finally executes ReAct chain by run() function to get the answer.

from langchain.docstore.wikipedia import Wikipediafrom langchain.llms import OpenAIfrom langchain.agents import initialize_agent, Tool, AgentExecutorfrom langchain.agents.react.base import DocstoreExplorer# build toolsdocstore=DocstoreExplorer(Wikipedia())tools = [  Tool(name="Search",func=docstore.search,description="Search for a term in the docstore.",  ),  Tool(name="Lookup",func=docstore.lookup,description="Lookup a term in the docstore.",  )]# build LLMllm = OpenAI(  model_name="text-davinci-003",  temperature=0,)# initialize ReAct agentreact = initialize_agent(tools, llm, agent="react-docstore", verbose=True)agent_executor = AgentExecutor.from_agent_and_tools(  agent=react.agent,  tools=tools,  verbose=True,)# perform question-answeringquestion = "Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?"agent_executor.run(question)

When you run this application, the following verbose outputs are shown in output’s logs. (See below note.)
As you can find, this question is disassembled into 3 search actions (“Search[David Chanoff]“, “Search[U.S. Navy admiral]“, and “Search[Admiral William J. Crowe]“) in this order, and it then gets the final answer “Bill Clinton“.

Note : The result will become different, especially when you use other language models and versions.
This is the case when you use text-davinci-003.

You can run as follows to see what prompt is issued in the bottom of this chain.
In this prompt template, the former part (which is a long long text) is few-shot’s in-context exemplars mentioned above.
The last part (last 2 lines) in template is a target question, in which {input} and {agent_scratchpad} placeholders are replaced every time by LangChain framework depending on user’s inputs.

# output prompt's templateprint(react.agent.llm_chain.prompt.template)

output

Question: What is the elevation range for the area that the eastern sector of theColorado orogeny extends into?Thought 1: I need to search Colorado orogeny, find the area that the eastern sectorof the Colorado orogeny extends into, then find the elevation range of thearea.Action 1: Search[Colorado orogeny]Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) inColorado and surrounding areas.Thought 2: It does not mention the eastern sector. So I need to look up easternsector.Action 2: Lookup[eastern sector]Observation 2: (Result 1 / 1) The eastern sector extends into the High Plains and is calledthe Central Plains orogeny.Thought 3: The eastern sector of Colorado orogeny extends into the High Plains. So Ineed to search High Plains and find its elevation range.Action 3: Search[High Plains]Observation 3: High Plains refers to one of two distinct land regionsThought 4: I need to instead search High Plains (United States).Action 4: Search[High Plains (United States)]Observation 4: The High Plains are a subregion of the Great Plains. From east to west, theHigh Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130m).[3]Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answeris 1,800 to 7,000 ft.Action 5: Finish[1,800 to 7,000 ft]Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons"character Milhouse, who Matt Groening named after who?Thought 1: The question simplifies to "The Simpsons" character Milhouse is named afterwho. I only need to search Milhouse and find who it is named after.Action 1: Search[Milhouse]Observation 1: Milhouse Mussolini Van Houten is a recurring character in the Fox animatedtelevision series The Simpsons voiced by Pamela Hayden and created by MattGroening.Thought 2: The paragraph does not tell who Milhouse is named after, maybe I can look up"named after".Action 2: Lookup[named after]Observation 2: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whosemiddle name was Milhous.Thought 3: Milhouse was named after U.S. president Richard Nixon, so the answer isRichard Nixon.Action 3: Finish[Richard Nixon]Question: Which documentary is about Finnish rock groups, Adam Clayton Powell or TheSaimaa Gesture?Thought 1: I need to search Adam Clayton Powell and The Saimaa Gesture, and find whichdocumentary is about Finnish rock groups.Action 1: Search[Adam Clayton Powell]Observation 1 Could not find [Adam Clayton Powell]. Similar: [’Adam Clayton PowellIII’, ’Seventh Avenue (Manhattan)’, ’Adam Clayton Powell Jr. State OfficeBuilding’, ’Isabel Washington Powell’, ’Adam Powell’, ’Adam Clayton Powell(film)’, ’Giancarlo Esposito’].Thought 2: To find the documentary, I can search Adam Clayton Powell (film).Action 2: Search[Adam Clayton Powell (film)]Observation 2: Adam Clayton Powell is a 1989 American documentary film directed byRichard Kilberg. The film is about the rise and fall of influentialAfrican-American politician Adam Clayton Powell Jr.[3][4] It was later airedas part of the PBS series The American Experience.Thought 3: Adam Clayton Powell (film) is a documentary about an African-Americanpolitician, not Finnish rock groups. So the documentary about Finnish rockgroups must instead be The Saimaa Gesture.Action 3: Finish[The Saimaa Gesture]Question: What profession does Nicholas Ray and Elia Kazan have in common?Thought 1: I need to search Nicholas Ray and Elia Kazan, find their professions, thenfind the profession they have in common.Action 1: Search[Nicholas Ray]Observation 1: Nicholas Ray (born Raymond Nicholas Kienzle Jr., August 7, 1911 - June 16,1979) was an American film director, screenwriter, and actor best known forthe 1955 film Rebel Without a Cause.Thought 2: Professions of Nicholas Ray are director, screenwriter, and actor. I needto search Elia Kazan next and find his professions.Action 2: Search[Elia Kazan]Observation 2: Elia Kazan was an American film and theatre director, producer, screenwriterand actor.Thought 3: Professions of Elia Kazan are director, producer, screenwriter, and actor.So profession Nicholas Ray and Elia Kazan have in common is director,screenwriter, and actor.Action 3: Finish[director, screenwriter, actor]Question: Which magazine was started first Arthur’s Magazine or First for Women?Thought 1: I need to search Arthur’s Magazine and First for Women, and find which wasstarted first.Action 1: Search[Arthur’s Magazine]Observation 1: Arthur’s Magazine (1844-1846) was an American literary periodical publishedin Philadelphia in the 19th century.Thought 2: Arthur’s Magazine was started in 1844. I need to search First for Womennext.Action 2: Search[First for Women]Observation 2: First for Women is a woman’s magazine published by Bauer Media Group in theUSA.[1] The magazine was started in 1989.Thought 3: First for Women was started in 1989. 1844 (Arthur’s Magazine) < 1989 (Firstfor Women), so Arthur’s Magazine was started first.Action 3: Finish[Arthur’s Magazine]Question: Were Pavel Urysohn and Leonid Levin known for the same type of work?Thought 1: I need to search Pavel Urysohn and Leonid Levin, find their types of work,then find if they are the same.Action 1: Search[Pavel Urysohn]Observation 1: Pavel Samuilovich Urysohn (February 3, 1898 - August 17, 1924) was a Sovietmathematician who is best known for his contributions in dimension theory.Thought 2: Pavel Urysohn is a mathematician. I need to search Leonid Levin next andfind its type of work.Action 2: Search[Leonid Levin]Observation 2: Leonid Anatolievich Levin is a Soviet-American mathematician and computerscientist.Thought 3: Leonid Levin is a mathematician and computer scientist. So Pavel Urysohnand Leonid Levin have the same type of work.Action 3: Finish[yes]Question: {input}{agent_scratchpad}

Implementation : Build Your Own Custom Tools and Agents from Scratch

If built-in agents don’t matter, you can also build your own chains, tools, and your own agents in LangChain framework.

Here I will show you example to ask for the amount of company’s invoices with simple arithmetic operations (calculations).
In this example, I’ll define the following external tools.

• GetInvoice[name_of_company] :
  This returns the invoice amount of name_of_company from company’s database.
• Diff[amount1 amount2] :
  This is a simple calculator to return the difference between amount1 and amount2.
• Total[list_of_quantity] :
  This is also a simple calculator to return the total amount (addition) of list_of_quantity. The placeholder list_of_quantity should also be delimited by a space.

With these tools, I’ll ask for the following question. :

In this example, I expect the following procedures in disassembled subtasks.

• Get invoice amount for company C. (GetInvoice action)
• Get invoice amount for company F. (GetInvoice action)
• Calculate total amount of above C and F. (Total action)
• Get invoice amount for company A. (GetInvoice action)
• Get invoice amount for company E. (GetInvoice action)
• Calculate total amount of above A and E. (Total action)
• Calculate the difference between previous total C, F and total A, E. (Diff action)

Note : Of course, LLMs can have the ability to perform simple arithmetic operations (such as, total, difference, etc) when numbers are simple. However, it struggles to perform simple computation with so large numbers, even when it’s primitive operations. (Please try with so large and complicated numbers.)
Hence it’s meaningful to define external tools for even these primitive operations – such as, addition, subtraction, multiplication, etc. (This mitigates hallucination in LLMs.)

Here, to make work the chain-of-thought (CoT) correctly, I want to provide the following prompt with few-shot exemplars.

Question: How much is the difference between the invoice of company A and company B ?Thought: I need to get invoice amount of company A.Action: GetInvoice["A"]Observation: 2000Thought: I need to get invoice amount of company B.Action: GetInvoice["B"]Observation: 1500Thought: I need to get difference of obtained amount between company A and company B.Action: Diff[2000 1500]Observation: 500Thought: So the answer is 500.Action: Finish[500]Question: How much is the total invoice amount of company B, C, and D ?Thought: I need to get invoice amount of company B.Action: GetInvoice["B"]Observation: 1500Thought: I need to get invoice amount of company C.Action: GetInvoice["C"]Observation: 20000Thought: I need to get invoice amount of company D.Action: GetInvoice["D"]Observation: 6700Thought: I need to get total amount of obtained amount B, C, and D.Action: Total[1500 20000 6700]Observation: 28200Thought: So the answer is 28200.Action: Finish[28200]Question: How much is the difference between company C and the total invoice amount of company A, D ?Thought: I need to get invoice amount of company C.Action: GetInvoice["C"]Observation: 20000Thought: I need to get invoice amount of company A.Action: GetInvoice["A"]Observation: 2000Thought: I need to get invoice amount of company D.Action: GetInvoice["D"]Observation: 6700Thought: I need to get total amount of obtained amount A and D.Action: Total[2000 6700]Observation: 8700Thought: I need to get difference of obtained amount C and the total of A, D.Action: Diff[20000 8700]Observation: 11300Thought: So the answer is 11300.Action: Finish[11300]In each action, you cannot use the nested functions, such as Diff[GetInvoice["A"], GetInvoice["B"]].Instead, you should parse into 3 actions - GetInvoice["A"], GetInvoice["B"], and Diff(2000, 1500).Let's start.Question: How much is the difference between the total of company C, F and the total of company A, E ?Thought:

The following is the source code for this custom ReAct chain example.
In this source code, I have defined custom tools and custom agent, put these all components (tools, agent, and OpenAI LLM) together, and finally run the chain by agent_executor.run().

The custom agent will also be helpful for you, in case you want to handle ReAct prompting with non-English languages.

import refrom typing import Sequence, List, Tuple, Optional, Anyfrom langchain.agents.agent import Agent, AgentOutputParserfrom langchain.prompts.prompt import PromptTemplatefrom langchain.prompts.base import BasePromptTemplatefrom langchain.tools.base import BaseToolfrom langchain.agents import Tool, initialize_agent, AgentExecutorfrom langchain.llms import OpenAIfrom langchain.agents.react.output_parser import ReActOutputParser########### define tools##########company_dic = {  "A": 2000,  "B": 1500,  "C": 20000,  "D": 6700,  "E": 1000,  "F": 4100,}def get_invoice(company_name):  return company_dic[company_name.strip("\"")]def diff(value_str):  int_list = [int(x.strip()) for x in value_str.split(",")]  assert(len(int_list) == 2)  return str(abs(int_list[0] - int_list[1]))def total(value_str):  int_list = [int(x.strip()) for x in value_str.split(",")]  return str(sum(int_list))tools = [  Tool(name="GetInvoice",func=get_invoice,description="Get invoice amount of trading company.",  ),  Tool(name="Diff",func=diff,description="Get diffrence.",  ),  Tool(name="Total",func=total,description="Get total of the list of numbers.",  ),]########### define agent##########EXAMPLES = [  """Question: How much is the difference between the invoice of company A and company B ?Thought: I need to get invoice amount of company A.Action: GetInvoice["A"]Observation: 2000Thought: I need to get invoice amount of company B.Action: GetInvoice["B"]Observation: 1500Thought: I need to get difference of obtained amount between company A and company B.Action: Diff[2000, 1500]Observation: 500Thought: So the answer is 500.Action: Finish[500]""",  """Question: How much is the total invoice amount of company B, C, and D ?Thought: I need to get invoice amount of company B.Action: GetInvoice["B"]Observation 1: 1500Thought: I need to get invoice amount of company C.Action: GetInvoice["C"]Observation: 20000Thought: I need to get invoice amount of company D.Action: GetInvoice["D"]Observation: 6700Thought: I need to get total amount of obtained amount B, C, and D.Action: Total[1500, 20000, 6700]Observation: 28200Thought: So the answer is 28200.Action: Finish[28200]""",  """Question: How much is the difference between company C and the total invoice amount of company A, D ?Thought: I need to get invoice amount of company C.Action: GetInvoice["C"]Observation: 20000Thought: I need to get invoice amount of company A.Action: GetInvoice["A"]Observation: 2000Thought: I need to get invoice amount of company D.Action: GetInvoice["D"]Observation: 6700Thought: I need to get total amount of obtained amount A and D.Action: Total[2000, 6700]Observation: 8700Thought: I need to get difference of obtained amount C and the total of A, D.Action: Diff[20000, 8700]Observation: 11300Thought: So the answer is 11300.Action: Finish[11300]""",]SUFFIX = """\nIn each action, you cannot use the nested functions, such as Diff[GetInvoice["A"], GetInvoice["B"]].Instead, you should parse into 3 actions - GetInvoice["A"], GetInvoice["B"], and Diff(2000, 1500).Let's start.Question: {input}{agent_scratchpad}"""TEST_PROMPT = PromptTemplate.from_examples(  examples=EXAMPLES,  suffix=SUFFIX,  input_variables=["input", "agent_scratchpad"],)class ReActTestAgent(Agent):  @classmethod  def _get_default_output_parser(cls, **kwargs: Any) -> AgentOutputParser:return ReActOutputParser()  @classmethod  def create_prompt(cls, tools: Sequence[BaseTool]) -> BasePromptTemplate:return TEST_PROMPT  @classmethod  def _validate_tools(cls, tools: Sequence[BaseTool]) -> None:if len(tools) != 3:  raise ValueError("The number of tools is invalid.")tool_names = {tool.name for tool in tools}if tool_names != {"GetInvoice", "Diff", "Total"}:  raise ValueError("The name of tools is invalid.")  @property  def _agent_type(self) -> str:return "react-test"  @property  def finish_tool_name(self) -> str:return "Finish"  @property  def observation_prefix(self) -> str:return f"Observation: "  @property  def llm_prefix(self) -> str:return f"Thought: "########### run agent##########llm = OpenAI(  model_name="text-davinci-003",  temperature=0.0,)agent = ReActTestAgent.from_llm_and_tools(  llm,  tools,)agent_executor = AgentExecutor.from_agent_and_tools(  agent=agent,  tools=tools,  verbose=True,)question = "How much is the difference between the total of company C, F and the total of company A, E ?"agent_executor.run(question)

Note : You can also use built-in langchain.agents.react.agent.create_react_agent() method, in which the chain is built by LCEL (LangChain Expression Language). See here for source code.
[Added on Jan 2024] You can also use LangGraph for more complex and arbitrary workflows (such as, ReWOO, LLM Compiler, etc) including cyclic flows.

Note : For ReAct implementation in Semantic Kernel, you can use built-in Stepwise Planner.

When you run this code, it will show the following verbose outputs.

OpenAI GPT never know the invoice amount of company E and company F (because I don’t provide these values in few-shot exemplars), but it can provide the correct answer, because the external functions (in this case, get_invoice(), diff(), total()) are properly used in this chain.

This is a trivial example, but ReAct-style reasoning can be used in a wide variety of practical use-cases. (For instance, QA agent can also gather information from multiple sources – such as, FAQ database, document database, customer’s history, …)

You can also find several other research works to use external knowledge in LLM reasoning, and ReAct is a baseline for these advanced techniques.

Note (Added on Sep 2023) : It’s known that multi-agent (in which, each agent has specific role and skill, and works each other) can get better performance rather than a single agent, and currently multi-agent framework is often used in advanced reasoning and autonomous agents. (See this paper.)
The paper “ReWOO – Reasoning WithOut Observation” (Binfeng et. al., 2023) also proposes dividing into Planner and Worker, decoupling reasoning from observations, in order to save the token consumption in prompt. (Because a long prompt cannot fit to the limit of tokens and will also degrade the quality of response.)
A lot of research works are on-going today, and ReAct is fundamental (base idea) to learn these advanced works.

[Change log]

Sep 2023 :
– Updated LangChain version 0.0.100 to 0.0.300 (latest)