Uncategorized

Rich messages with Bot Service (Rich text, Image, Card, etc)

In the previous post, I explained the rest style (raw) messaging format in Microsoft Bot Framework.
In this blog post I will show you the various rich and functional chat bot messages using several examples. These capabilities are almost introduced by Microsoft Bot Framework version 3.

Notice : As I explained in the previous post, the required attributes of json format in the REST (HTTP) is different at the channel (Skype, Facebook, etc). In the following examples I’m using the Skype channel.

Rich Text

You can use not only the plain text but the markdown. (The default message setting is markdown.)
For example, when you send the following HTTP request from your bot, the message is displayed in the chat app like following image (screenshot).

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "text": "# Welcome to **Botland**\n\nPlease visit [my blog](https://blogs.msdn.microsoft.com/tsmatsuz).\n\n---\n\nThis is a test."}

Of course, you can use this markdown in other channels (Slack, Facebook, etc), if the channel supports this types of the rich text. For example, the following screenshot is the result of the e-mail channel.
If you’re using channels which doesn’t support the rich text (like Facebook Messenger Bot, etc), this markdown doesn’t work.

If the channel is Skype, you can also use the XML text (i.e, HTML markup) as follows.
Note that the supported tag is only <b/>, <i/>, <u/>, <s/>, and <a/>.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "text": "Welcome to <b>Botland</b>. Please visit <a name="blogs.msdn.microsoft.com/tsmatsuz">my blog</a>."}

You can also use the special <ss/> tag for displaying emoji (emoticons) in Skype.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "text": "<ss type ="wink">;)</ss>"}

For details, please see the official document “Bot Framework SDK – Activities“.

Attachment

You can attach the media (image/audio/video/file) to your activities using a link to the content.

First I show you the very simple example of the attachment. (As I show you later, you can use more rich expression using the attachment.) The following is the image attachment example.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "text": "This is test attachment.",  "attachments": [{  "contentType": "image/jpg",  "contentUrl": "http://mydeploy.azurewebsites.net/matsu.jpg",  "name": "matsu.jpg"}  ]}

If you attach the binary, the attachment is shown as follows. (This is the Skype screenshot.)

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "text": "This is test attachment.",  "attachments": [{  "contentType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",  "contentUrl": "http://myfiles01.blob.core.windows.net/sample/test01.xlsx",  "name": "test01.xlsx"}  ]}

Card

If the channel (Skype, Slack, Facebook, etc) is having the button, template, or some sort of these capabilities, you can use the Card as the attachment in Microsoft Bot Framework.
For example, the Skype and Facebook is having this kind of rich expressions. (see “Facebook Messenger Platform – Button Template“) And you can send the following HTTP request using the Card attachment.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "attachmentLayout": "list",  "text": "",  "attachments": [{  "contentType": "application/vnd.microsoft.card.hero",  "content": {"text": "What kind of sandwich would you like on your sandwich? ","buttons": [  {"type": "imBack","title": "BLT","value": "1"  },  {"type": "imBack","title": "Black Forest Ham","value": "2"  },  {"type": "imBack","title": "Buffalo Chicken","value": "3"  }]  }}  ]}

If you’re using Skype Bot, the message is displayed as follows.

If it’s Facebook Messenger Bot, the message is displayed as follows.

If you send to e-mail channel, the message is displayed as follows.

If the user pushes this button in the Card, the selected value is replied to the bot as the text message. It’s same like the user’s text input.
These capabilities (button, template, etc) is different at each channel (Skype, Facebook, etc). But the Microsoft Bot Framework converts the expression to fit to each channel, and the developer doesn’t need to care about the differences between channels.
For example, if the Facebook channel is used, the Microsoft Bot Framework converts to the button template in Facebook Messenger Bot. (see “Facebook Messenger Platform – Button Template” for details.) If the e-mail channel is used, the Microsoft Bot Framework converts to the “mailto:” alink with several parameters.

More advanced messages by Card

I show you more rich expression using the Card capability.

First, you can use the 3 types of Card: Hero Card, Thumbnail Card, and Receipt Card. (Actually, there’s 4 types of Cards including the SignIn Card. I will show you the SignIn Card in the next post.)
Let’s see how it looks like.

Hero Card

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "text": "Sample with a hero card",  "attachments": [{  "contentType": "application/vnd.microsoft.card.hero",  "content": {"title": "I'm a hero card","subtitle": "Please visit my site.","images": [  {"url": "https://mydeploy.azurewebsites.net/matsu.jpg"  }],"buttons": [  {"type": "openUrl","title": "Go to my site","value": "https://blogs.msdn.microsoft.com/tsmatsuz"  }]  }}  ]}

Thumbnail Card

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "text": "Sample with a thumbnail card",  "attachments": [{  "contentType": "application/vnd.microsoft.card.thumbnail",  "content": {"title": "I'm a thumbnail card","subtitle": "Please visit my site.","images": [  {"url": "https://mydeploy.azurewebsites.net/matsu.jpg"  }],"buttons": [  {"type": "openUrl","title": "Go to my site","value": "https://blogs.msdn.microsoft.com/tsmatsuz"  }]  }}  ]}

Receipt Card

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "text": "Sample with a receipt card",  "attachments": [{  "contentType": "application/vnd.microsoft.card.receipt",  "content": {"title": "I'm a receipt card","items": [  {"title": "Sushi","subtitle": "2 piece","image": {  "url": "https://mydeploy.azurewebsites.net/sushi.png"},"price": "16.25","quantity": "1"  },  {"title": "Tenpura","subtitle": "1 dish","image": {  "url": "https://mydeploy.azurewebsites.net/tenpura.jpg"},"price": "34.50","quantity": "2"  }],"total": "275.25","tax": "27.52","buttons": [  {"type": "openUrl","title": "Go to my site","value": "https://blogs.msdn.microsoft.com/tsmatsuz"  }]  }}  ]}

For the “type” attribute in the Card, you can use the following values for your needs. (see “Bot Framework SDK – Attachments, Cards and Actions“)
Note that the supported values is different at each channel. For example, the Skype supports “imBack”, “openUrl”, “call”, “showImage”, and “signin” now.

imBackText of message which client will sent back to bot as ordinary chat message. All other participants will see that was posted to the bot and who posted this.
postBackText of message which client will post to bot. Client applications will not display this message.
openUrlURL to be opened in the built-in browser.
callDestination for a call in following format: “tel:123123123123”
playAudioplayback audio container referenced by URL
playVideoplayback video container referenced by URL
showImageshow image referenced by URL
downloadFiledownload file referenced by URL
signinOAuth flow URL. This is almost used in the SignIn Card. (Next time I will explain this capability.)

For example, when you send the following HTTP request, the call button (which calls to some Skype user) is displayed. If the user push this button, the Skype call to tsuyoshi.matsuzaki starts.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "text": "",  "attachments": [{  "contentType": "application/vnd.microsoft.card.hero",  "content": {"text": "Welcome to our help desk !","buttons": [  {"type": "call","title": "Call to tsuyoshi.matsuzaki","value": "skype:tsuyoshi.matsuzaki"  }]  }}  ]}

You can set the multiple cards in one message. In such a case, each Card is displayed vertically (as “list”) by default.
If you want to show by horizontal carousel style, you can specify “carousel” style in “attachmentLayout” attribute.
Let’s see how it looks like.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "attachmentLayout": "carousel",  "text": "Please select your food.",  "attachments": [{  "contentType": "application/vnd.microsoft.card.hero",  "content": {"title": "Sushi","subtitle": "Very fresh shrimp or tuna.","images": [  {"url": "https://mydeploy.azurewebsites.net/sushi.png"  }],"buttons": [  {"type": "imBack","title": "1 piece","value": "sushi,1"  },  {"type": "imBack","title": "2 piece","value": "sushi,2"  },  {"type": "imBack","title": "more","value": "sushi,3+"  }]  }},{  "contentType": "application/vnd.microsoft.card.hero",  "content": {"title": "Tenpura","subtitle": "Japanese first-class vegitables.","images": [  {"url": "https://mydeploy.azurewebsites.net/tenpura.jpg"  }],"buttons": [  {"type": "imBack","title": "1 piece","value": "tenpura,1"  },  {"type": "imBack","title": "2 piece","value": "tenpura,2"  },  {"type": "imBack","title": "more","value": "tenpura,3+"  }]  }},{  "contentType": "application/vnd.microsoft.card.hero",  "content": {"title": "Tofu","subtitle": "Super healthy food condensed by soy milk.","images": [  {"url": "https://mydeploy.azurewebsites.net/tofu.jpg"  }],"buttons": [  {"type": "imBack","title": "1 piece","value": "tofu,1"  },  {"type": "imBack","title": "2 piece","value": "tofu,2"  },  {"type": "imBack","title": "more","value": "tofu,3+"  }]  }}  ]}

For details about Cards, please see the official document “Bot Framework SDK – Attachments, Cards and Actions“.

Custom Channel Message – Rich messages for some specific channel

If you want to use the channel specific expression, you can use the “channelData” attribute instead of “attachments” attribute.

For example, if you want to use the flight update template in the Facebook Messenger Bot Platform, you send the following HTTP request.
Of course, this message is supported only in the Facebook channel.

POST https://facebook.botframework.com/v3/conversations/1245513475467461-737400853067333/activitiesAuthorization: Bearer eyJ0eXAiOi...Content-Type: application/json; charset=utf-8{  "type": "message",  "from": {"id": "737400853067333","name": "mytestbot01"  },  "recipient": {"id": "1245513475467461","name": "Tsuyoshi Matsuzaki"  },  "text": "This is test",  "channelData": {"notification_type": "NO_PUSH","attachment": {  "type": "template",  "payload": {"template_type": "airline_update","intro_message": "Your flight is delayed","update_type": "delay","locale": "en_US","pnr_number": "CF23G2","update_flight_info": {  "flight_number": "KL123",  "departure_airport": {"airport_code": "SFO","city": "San Francisco","terminal": "T4","gate": "G8"  },  "arrival_airport": {"airport_code": "AMS","city": "Amsterdam","terminal": "T4","gate": "G8"  },  "flight_schedule": {"boarding_time": "2015-12-26T10:30","departure_time": "2015-12-26T11:30","arrival_time": "2015-12-27T07:30"  }}  }}  }}

For details about the custom channel message, please see the official document “Bot Framework SDK – Configuring Channels“.

 

When you’re using SDK (.NET), you can use the encapsulated object (class) called “Dialog” or “FormFlow” and this object (class) creates the previous messages internally if possible. These are the very high-level object. (You can also use the primitive elements using SDK.)
But, please remember that these essential elements which I explained here are working in the bottom of APIs.

Please enjoy your Bot development.

Categories: Uncategorized

Tagged as: ,

19 replies»

  1. did skype support typing action. I see that it is there in schema but doesn’t seems to be working neither the the image/gif is supported in Skype. If this is true then can you confirm it?

    Like

    • What you are saying is right. The image/gif seems not to be supported in Skype, and fixed my sample. Thanks for sharing, Puneet-san.

      Like

  2. How to differentiate the channel source(messenger or skype) in code(nodejs), so that I can dynamically compose appropriate message?

    Like

  3. Hi, does the embed Webchat support rich cards and buttons? If not, what are some other alternatives to this (without having to use Facebook Messenger or Skype) as I want to be able to customize the look and feel of the bot. thanks

    Like

    • I’m sorry, but you must use the direct line api and create the custom code for rendering cards in such a case.

      Like

  4. How Does video playback work with embedded Web chat. I see there is a “playVideo” action, However this does not seem to work with web chat.

    Like

  5. hi!,I love your writing so so much! share we be in contact more
    about your post on AOL? I require a specialist on this area to resolve my
    problem. Maybe that’s you! Having a look forward to look you.

    Like

  6. It’s amazing to visit this web page and reading the views of all mates about this piece of writing, while I am also eager of getting familiarity.

    Like

  7. Pretty element of content. I just stumbled upon your website and in accession capital to
    assert that I get in fact loved account your blog posts. Any way
    I will be subscribing for your augment and even I fulfillment you get right of entry to constantly rapidly.

    Like

  8. Thank you for another informative website. The place else may I get that type of info written in such a perfect method?
    I have a mission that I am simply now running
    on, and I have been on the look out for such
    information.

    Like

Leave a reply to iptv server Cancel reply