Reasoning is important aspect for advanced LLM in business – such as, in autonomous agents.
Today, a lot of reasoning techniques are introduced in many papers, – such as, ReAct, ReWOO, LLM Compiler, and more.

In this blog post, I’ll show you how to implement custom Planner (which performs custom structured planning) for advanced reasoning in Semantic Kernel.

Note : To implement ReAct (Reasoning+Acting) planning in Semantic Kernel, you can use function calling or built-in Stepwise Planner. (See here.)

Run Planner

Before starting the implementation of custom planner, let’s see the brief outline of components in Semantic Kernel – Plugins, Functions, and Planners.

The core component in Semantic Kernel is plugin.
A plugin in Semantic Kernel defines a set of semantic functions and native functions.
The semantic function performs LLM prompting and the native function runs native code.

When some task is given, the Planner decomposes into subtasks (in which available plugins and functions are formulated) to generate a goal-oriented plan to achieve the given task.

From : Semantic Kernel developer blog

In order to understand Planner, let’s see the simple example, 05-using-the-planner.ipynb, in official GitHub repository.

In this example, the following 3 built-in plugins (SummarizePlugin, WriterPlugin, TextPlugin) are imported into kernel.

from semantic_kernel.core_plugins.text_plugin import TextPluginplugins_directory = "../../../prompt_template_samples/"summarize_plugin = kernel.add_plugin(  plugin_name="SummarizePlugin",  parent_directory=plugins_directory)writer_plugin = kernel.add_plugin(  plugin_name="WriterPlugin",  parent_directory=plugins_directory,)text_plugin = kernel.add_plugin(  plugin=TextPlugin(),  plugin_name="TextPlugin")

Next we build the built-in SequentialPlanner as follows.
This Planner builds a plan, in which all functions in plugins are executed sequentially.

from semantic_kernel.planners import SequentialPlannerservice_id = "default"planner = SequentialPlanner(kernel, service_id)

Now let’s build a plan for the following task by using SequentialPlanner.

“Tomorrow is Valentine’s day. I need to come up with a few short poems.
She likes Shakespeare so write using his style. She speaks French so write it in French.
Convert the text to uppercase.”

ask = """Tomorrow is Valentine's day. I need to come up with a few short poems.She likes Shakespeare so write using his style. She speaks French so write it in French.Convert the text to uppercase."""sequential_plan = await planner.create_plan(goal=ask)

When you run this plan, you can eventually get a poem for Valentine’s ideas in uppercase French as follows.

result = await sequential_plan.invoke(kernel)print(result)

output

ASSUREZ-VOUS D'UTILISER UNIQUEMENT LE FRANÇAIS.Ô BELLES ROSES D'UN ROUGE CRAMOISI,ET VIOLETTES D'UN BLEU VIOLET,LA DOUCEUR COMME DU SUCRE IMBUE,ET DANS TA GRÂCE, JE TE VOIS VRAIE.MAIS OH, AMOUR, COMME TU ES TUMULTUEUX,RAGEANT COMME UNE TEMPÊTE, TUMULTUEUX.MON CŒUR BAT LA CHAMADE ET S'AGITE,DANS TES TOURMENTS, JE SUIS IMPUISSANT.TU ES MON SOLEIL, MA LUNE BRILLANTE,POURTANT PARFOIS, TU ME METS EN DIFFICULTÉ.L'AMOUR, UN LUTIN CAPRICIEUX ET FANTASQUE,ME LAISSE SOUVENT DANS UN TRISTE ÉTAT.

Planner – Deep Dive

What is Planner doing internally ?

Simply put, SequentialPlanner is a simple Python class which invokes kernel functions for reasoning and execution sequentially.

First, it submits the following prompt to LLM (in this case, OpenAI GPT) by running semantic function in create_plan() method.
As you can see the below, the prompt for LLM includes a few-shot exemplars and also what kind of functions are available.

prompt (input)

You are a planner for the Semantic Kernel.Your job is to create a properly formatted JSON plan step by step, to satisfy the goal given.Create a list of subtasks based off the [GOAL] provided.Each subtask must be from within the [AVAILABLE FUNCTIONS] list. Do not use any functions that are not in the list.Base your decisions on which functions to use from the description and the name of the function.Sometimes, a function may take arguments. Provide them if necessary.The plan should be as short as possible.For example:[AVAILABLE FUNCTIONS]EmailConnector.LookupContactEmaildescription: looks up the a contact and retrieves their email addressargs:- name: the name to look upWriterPlugin.EmailTodescription: email the input text to a recipientargs:- input: the text to email- recipient: the recipient's email address. Multiple addresses may be included if separated by ';'.WriterPlugin.Translatedescription: translate the input to another languageargs:- input: the text to translate- language: the language to translate toWriterPlugin.Summarizedescription: summarize input textargs:- input: the text to summarizeFunPlugin.Jokedescription: Generate a funny jokeargs:- input: the input to generate a joke about[GOAL]"Tell a joke about cars. Translate it to Spanish"[OUTPUT]{"input": "cars","subtasks": [{"function": "FunPlugin.Joke"},{"function": "WriterPlugin.Translate", "args": {"language": "Spanish"}}]}[AVAILABLE FUNCTIONS]WriterPlugin.Brainstormdescription: Brainstorm ideasargs:- input: the input to brainstorm aboutEdgarAllenPoePlugin.Poedescription: Write in the style of author Edgar Allen Poeargs:- input: the input to write aboutWriterPlugin.EmailTodescription: Write an email to a recipientargs:- input: the input to write about- recipient: the recipient's email address.WriterPlugin.Translatedescription: translate the input to another languageargs:- input: the text to translate- language: the language to translate to[GOAL]"Tomorrow is Valentine's day. I need to come up with a few date ideas.She likes Edgar Allen Poe so write using his style.E-mail these ideas to my significant other. Translate it to French."[OUTPUT]{"input": "Valentine's Day Date Ideas","subtasks": [{"function": "WriterPlugin.Brainstorm"},{"function": "EdgarAllenPoePlugin.Poe"},{"function": "WriterPlugin.EmailTo", "args": {"recipient": "significant_other"}},{"function": "WriterPlugin.Translate", "args": {"language": "French"}}]}[AVAILABLE FUNCTIONS]SummarizePlugin.Topicsdescription: Analyze given text or document and extract key topics worth rememberingargs:- input: SummarizePlugin.MakeAbstractReadabledescription: Given a scientific white paper abstract, rewrite it to make it more readableargs:- input: SummarizePlugin.Summarizedescription: Summarize given text or any text documentargs:- input: Text to summarizeSummarizePlugin.Notegendescription: Automatically generate compact notes for any text or text document.args:- input: WriterPlugin.Acronymdescription: Generate an acronym for the given concept or phraseargs:- input: WriterPlugin.StoryGendescription: Generate a list of synopsis for a novel or novella with sub-chaptersargs:- input: WriterPlugin.AcronymGeneratordescription: Given a request to generate an acronym from a string, generate an acronym and provide the acronym explanation.args:- INPUT: WriterPlugin.TwoSentenceSummarydescription: Summarize given text in two sentences or lessargs:- input: WriterPlugin.Brainstormdescription: Given a goal or topic description generate a list of ideasargs:- input: A topic description or goal.WriterPlugin.NovelOutlinedescription: Generate a list of chapter synopsis for a novel or novellaargs:- input: What the novel should be about.- chapterCount: The number of chapters to generate.- endMarker: The marker to use to end each chapter.WriterPlugin.EmailTodescription: Turn bullet points into an email to someone, using a polite toneargs:- to: - input: - sender: WriterPlugin.ShortPoemdescription: Turn a scenario into a short and entertaining poem.args:- input: The scenario to turn into a poem.WriterPlugin.Translatedescription: Translate the input into a language of your choiceargs:- input: Text to translate- language: Language to translate toWriterPlugin.NovelChapterWithNotesdescription: Write a chapter of a novel using notes about the chapter to write.args:- input: What the novel should be about.- theme: The theme of this novel.- notes: Notes useful to write this chapter.- previousChapter: The previous chapter synopsis.- chapterIndex: The number of the chapter to write.WriterPlugin.AcronymReversedescription: Given a single word or acronym, generate the expanded form matching the acronym letters.args:- INPUT: WriterPlugin.NovelChapterdescription: Write a chapter of a novel.args:- input: A synopsis of what the chapter should be about.- theme: The theme or topic of this novel.- previousChapter: The synopsis of the previous chapter.- chapterIndex: The number of the chapter to write.WriterPlugin.EmailGendescription: Write an email from the given bullet pointsargs:- input: WriterPlugin.Rewritedescription: Automatically generate compact notes for any text or text documentargs:- style: - input: WriterPlugin.TellMeMoredescription: Summarize given text or any text documentargs:- conversationtype: - input: - focusarea: - previousresults: WriterPlugin.EnglishImproverdescription: Translate text to English and improve itargs:- INPUT: WriterPlugin.Shakespearedescription: Convert input to Shakespeare-style text.args:- input: TextPlugin.lowercasedescription: Convert a string to lowercase.args:- input: TextPlugin.trimdescription: Trim whitespace from the start and end of a string.args:- input: TextPlugin.trim_enddescription: Trim whitespace from the end of a string.args:- input: TextPlugin.trim_startdescription: Trim whitespace from the start of a string.args:- input: TextPlugin.uppercasedescription: Convert a string to uppercase.args:- input: PlannerPlugin.CreatePlanargs:- available_functions: - goal: [GOAL]Tomorrow is Valentine's day. I need to come up with a few short poems.She likes Shakespeare so write using his style. She speaks French so write it in French.Convert the text to uppercase.[OUTPUT]

In accordance with the above exemplars, LLM then replies the following JSON plan.

completion (output)

{  "input": "Valentine's Day Poems",  "subtasks": [{"function": "WriterPlugin.Brainstorm"},{"function": "WriterPlugin.ShortPoem"},{"function": "WriterPlugin.Shakespeare"},{"function": "WriterPlugin.Translate", "args": {"language": "French"}},{"function": "TextPlugin.uppercase"}  ]}

Each function gets the output of previous function as an input, and perform the following task. :

1. WriterPlugin.Brainstorm : This function asks LLM for ideas with bullet’s points. (semantic function)
2. WriterPlugin.ShortPoem : This function asks LLM to create a poem. (semantic function)
3. WriterPlugin.Shakespeare : This function asks LLM to convert text into Shakespeare-style text. (semantic function)
   I note that this is not built-in function, but it’s added to this plugin in 05-using-the-planner.ipynb.
4. WriterPlugin.Translate : This function asks LLM to translate text. (semantic function)
5. TextPlugin.uppercase : This function changes text to uppercase. (native function)

Finally, the invoke() method in SequentialPlanner will parse this plan (JSON) and then invoke each functions sequentially.
As I have mentioned above, each function gets the output of previous function as an input.

output of WriterPlugin.Brainstorm

1. Roses are red, violets are blue, sugar is sweet, and so are you.2. You are the sunshine in my day, the moonlight in my night, and the beat in my heart.3. Love is not just a feeling, it's a choice we make every day.4. You are the missing piece in my puzzle, the melody in my song, and the love in my life.5. I love you more than words can say, more than actions can show, and more than you will ever know.6. You are the reason I wake up with a smile, the reason I go to bed with a happy heart, and the reason I believe in love.7. You are my forever Valentine, my soulmate, and my best friend.8. Love is not about finding someone perfect, it's about finding someone who is perfect for you.9. You make my heart skip a beat, my world brighter, and my life complete.##END##

output of WriterPlugin.ShortPoem

Roses are red, violets are blue,Sugar is sweet, and so are you.But let's be real, we all know,Love is messy, and it can blow.You're my sunshine, my moonlight too,But sometimes you make me feel blue.Love is a

output of WriterPlugin.Shakespeare

Oh, fair roses of crimson hue,And violets of violet blue,Sweetness like sugar doth imbue,And in thy grace, I see thee true.But oh, love, how tumultuous,Raging like storm, tempestuous.Mine heart doth flutter and fuss,In thy throes, I am rendered helpless.Thou art my sun, my moon alight,Yet at times, thou bringeth me plight.Love, a fickle and wayward sprite,Doth oft leave me in sorry plight.

output of WriterPlugin.Translate

Assurez-vous d'utiliser UNIQUEMENT le français.Ô belles roses d'un rouge cramoisi,Et violettes d'un bleu violet,La douceur comme du sucre imbue,Et dans ta grâce, je te vois vraie.Mais oh, amour, comme tu es tumultueux,Rageant comme une tempête, tumultueux.Mon cœur bat la chamade et s'agite,Dans tes tourments, je suis impuissant.Tu es mon soleil, ma lune brillante,Pourtant parfois, tu me mets en difficulté.L'amour, un lutin capricieux et fantasque,Me laisse souvent dans un triste état.

output of TextPlugin.uppercase

ASSUREZ-VOUS D'UTILISER UNIQUEMENT LE FRANÇAIS.Ô BELLES ROSES D'UN ROUGE CRAMOISI,ET VIOLETTES D'UN BLEU VIOLET,LA DOUCEUR COMME DU SUCRE IMBUE,ET DANS TA GRÂCE, JE TE VOIS VRAIE.MAIS OH, AMOUR, COMME TU ES TUMULTUEUX,RAGEANT COMME UNE TEMPÊTE, TUMULTUEUX.MON CŒUR BAT LA CHAMADE ET S'AGITE,DANS TES TOURMENTS, JE SUIS IMPUISSANT.TU ES MON SOLEIL, MA LUNE BRILLANTE,POURTANT PARFOIS, TU ME METS EN DIFFICULTÉ.L'AMOUR, UN LUTIN CAPRICIEUX ET FANTASQUE,ME LAISSE SOUVENT DANS UN TRISTE ÉTAT.

In some cases, a single context will be shared across all plugins and functions in a sequential plan.
For instance, in the restaurant reservations, the information – such as, the datetime, the number of persons, meals, or dietary requirements – may be collected through functions. All these information will then be saved in a single context as key-value pairs, and are finally submitted to reservation’s function.
You can design your own planner as you like.

From : Semantic Kernel Overview

Implement custom Planner – Structured Planning Example

Now I implement a simple Planner example to run simple calculations of company’s invoices. (This example is the same as this post, which is built by LangChain with ReAct prompting pattern.)

In this example, I’ll define the following native functions.

• GetInvoice(name_of_company) :
  This extracts and returns the invoice amount of name_of_company from database.
• Diff(value1, value2) :
  This is a simple calculator and returns the difference between value1 and value2.
• Total(values) :
  This is also a simple calculator and returns the sum of integer’s list values.

When I ask for the following question (task) :

“How much is the difference between the total of company C, F and the total of company A, E ?”

we expect to break down into the following subtasks by LLM’s reasoning :

• Get invoice amount for company C and F. (GetInvoice function)
• Calculate total amount of above C and F. (Total function)
• Get invoice amount for company A and E. (GetInvoice function)
• Calculate total amount of above A and E. (Total function)
• Calculate the difference between previous total C, F and total A, E. (Diff function)

While built-in SequentialPlanner can generate a plan for sequential execution, our custom Planner will create the structured plans.
For instance, when we ask for “the difference between company A and company B“, first it should invoke GetInvoice("A") and GetInvoice("B"), and next these results are fed into the function Diff(value1, value2).
In our custom Planner, each inputs and outputs can be nested in a tree structure with any depth.

Now let’s implement a custom Planner.

First, I’ll configure to use Azure OpenAI chat completion endpoint as follows. (Please prepare .env file in advance.)

import osfrom dotenv import load_dotenvload_dotenv()deployment_name = os.environ["AZURE_OPENAI_DEPLOYMENT_NAME"]endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]api_key = os.environ["AZURE_OPENAI_API_KEY"]import semantic_kernel as skimport semantic_kernel.connectors.ai.open_ai as sk_oaikernel = sk.Kernel()service_id = "default"kernel.add_service(  sk_oai.AzureChatCompletion(service_id=service_id,deployment_name=deployment_name,endpoint=endpoint,api_key=api_key,  ),)

Next I prepare a custom plugin as follows.
As you can see below, our custom plugin consists of 3 native functions – get_invoice(), diff(), and total().

from semantic_kernel.functions.kernel_function_decorator import kernel_functionclass DemoPlugin:  def __init__(self):super().__init__()self.company_dic = {  "A": 2000,  "B": 1500,  "C": 20000,  "D": 6700,  "E": 1000,  "F": 4100,}  @kernel_function(description="Get invoice amount of trading company",name="GetInvoice",  )  def get_invoice(self, company_name: str) -> int:return self.company_dic[company_name]  @kernel_function(description="Get diffrence",name="Diff"  )  def diff(self, value1: int, value2: int) -> int:return abs(value1 - value2)  @kernel_function(description="Get total",name="Total"  )  def total(self, values: list) -> int:return sum(values)

Note : You can use built-in semantic_kernel.core_plugins.MathPlugin for primitive arithmetic operations.
For the purpose of your learning, here I have built a custom plugin from scratch.

Now let’s implement a custom Planner as follows.
With LLM reasoning, the following create_plan() method generates a structured JSON plan.
In execute_plan() method, a generated tree structure (plan) is then parsed and executed.

import jsonfrom typing import Dict, Anyfrom semantic_kernel.kernel import Kernelfrom semantic_kernel.functions.kernel_arguments import KernelArgumentsfrom semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettingsfrom semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfigPROMPT = """[GOAL]How much is the difference between the invoice of company A and company B ?[OUTPUT]  {"input": "How much is the difference between the invoice of company A and company B ?","subtasks":{  "function": "DemoPlugin.Diff",  "args":  {"value1":{  "function": "DemoPlugin.GetInvoice",  "args": {"company_name": "A"}},"value2":{  "function": "DemoPlugin.GetInvoice",  "args": {"company_name": "B"}}  }}  }[GOAL]How much is the total invoice amount of company B and D ?[OUTPUT]  {"input": "How much is the total invoice amount of company B and D ?","subtasks":{  "function": "DemoPlugin.Total",  "args":  {"values":[  {"function": "DemoPlugin.GetInvoice","args": {"company_name": "B"}  },  {"function": "DemoPlugin.GetInvoice","args": {"company_name": "D"}  }]  }}  }[GOAL]How much is the difference between company C and the total invoice amount of company A, D ?[OUTPUT]  {"input": "How much is the difference between company C and the total invoice amount of company A, D ?","subtasks":{  "function": "DemoPlugin.Diff",  "args":  {"value1":{  "function": "DemoPlugin.GetInvoice",  "args": {"company_name": "C"}},"value2":{  "function": "DemoPlugin.Total",  "args":  {"values":[  {"function": "DemoPlugin.GetInvoice","args": {"company_name": "A"}  },  {"function": "DemoPlugin.GetInvoice","args": {"company_name": "D"}  }]  }}  }}  }[GOAL]{{$goal}}[OUTPUT]"""class DemoPlanner:  def __init__(self, service_id: str) -> None:self.service_id = service_id  async def create_plan(self,goal: str,kernel: Kernel,prompt: str = PROMPT,  ) -> Dict:# Create the kernel function for running promptexec_settings = PromptExecutionSettings(  service_id=self.service_id,  max_tokens=1024,  temperature=0.0,)prompt_template_config = PromptTemplateConfig(  template=prompt,  execution_settings=exec_settings,)function = kernel.add_function(  function_name="CreatePlan",  plugin_name="PlannerPlugin",  description="Create a plan for the given goal",  prompt_template_config=prompt_template_config,)# Invoke and create plangenerated_plan = await function.invoke(  kernel, KernelArguments(goal=goal))return json.loads(str(generated_plan))  async def execute_plan(self, plan: Dict, kernel: Kernel) -> str:task = plan["subtasks"]result = await self.parse_task(task, kernel)return result  async def parse_task(self, task, kernel) -> Any:if isinstance(task, dict):  if "function" in task.keys():## When it's function#plugin_name, function_name = task["function"].split(".")kernel_function = kernel.get_function(plugin_name, function_name)args = task.get("args", None)args = await self.parse_task(args, kernel)func_res = await kernel_function.invoke(kernel, KernelArguments(**args))task = func_res.value  else:## When it's dictionary#for key in task.keys():  task[key] = await self.parse_task(task[key], kernel)elif isinstance(task, list):  #  # When it's list  #  for i, item in enumerate(task):task[i] = await self.parse_task(item, kernel)# Otherwise (not iterable), do nothing ...return task

Note : For the simplicity, I have used few-shot exemplars which includes above 3 functions. But please use zero-shot approach in prompt, in case that available functions are changed in the future.
See here for details.

Now let’s run our custom Planner.

First, I create a structured plan.
As you can see the output, I can get the correct reasoning structure in JSON.

kernel.add_plugin(plugin=DemoPlugin(), plugin_name="DemoPlugin")planner = DemoPlanner(service_id)ask = "How much is the difference between the total of company C, F and the total of company A, E ?"plan = await planner.create_plan(ask, kernel)print(plan)

output

{  'input': 'How much is the difference between the total of company C, F and the total of company A, E ?',  'subtasks': {'function': 'DemoPlugin.Diff','args': {  'value1': {'function': 'DemoPlugin.Total','args': {  'values': [{  'function': 'DemoPlugin.GetInvoice',  'args': {'company_name': 'C'  }},{  'function': 'DemoPlugin.GetInvoice',  'args': {'company_name': 'F'  }}  ]}  },  'value2': {'function': 'DemoPlugin.Total','args': {  'values': [{  'function': 'DemoPlugin.GetInvoice',  'args': {'company_name': 'A'  }},{  'function': 'DemoPlugin.GetInvoice',  'args': {'company_name': 'E'  }}  ]}  }}  }}

When you execute this plan, you can get the right answer by running above custom plugin (DemoPlugin).

result = await planner.execute_plan(plan, kernel)print(result)

output

21100

Here I have implemented a simple custom Planner, but please see source code for built-in Stepwise Planner (see here) for implementing ReAct prompting.

Note (Added on June 2023) : You can now use a new function-enabled model (gpt-4-0613 and gpt-3.5-turbo-0613) for more reliable reasoning by function calling.
See here for details.

Apr 2024 : Updated source code to the latest version (version 0.9.5b1)

May 2024 : Updated source code to the latest version (version 1.0.2)