A case study on examining the purpose of entities in conversational AI chatbots: when and why it is applied in the workflows for conversation designers and bot developers, and the appropriate interaction model in our UI. Let's deep dive into the process that helped me understand, design, and implement the entities feature into a user-friendly and simplistic workflow in the [24]7.ai Conversations SAAS platform.
Preface
AI conversational design and AI chatbot development is new to me. I don't consider myself an expert prior to taking on this feature. This allowed me to approach the project and domain with a fresh mind, prompting me to take baby steps in understanding, rationalizing, and making sense of the complex world of conversational AI chatbot development and entities.
In the chat example below, a user has told the virtual chatbot that he would like to purchase a jacket. The chatbot then asks the user to provide it with product details (entity requirements) so that it can return the appropriate product results based on the user's criteria. The user responds with their desired jacket specifications (entity inputs). The bot would need to process each criteria (entity) individually and map it to the correct entities in the system. The bot would then confirm the entities received with a confirmation message or product results (entity outputs).
Mission
To ensure real-time bot advancement using slot-filling, integrate the design-time functionalities to support the construct of entities in the conversation design platform.
Activities and resources
User interviews: conducted user interviews with conversational designers to understand workflow, pain points, positioning of entities in current process.
Competitive analysis: analyzed functional integration, documentation, and UI of entities in competitor products.
UI design: user journey maps, information architecture, and wireframes as a start to organize our micro-interactions and components.
Concept testing: click-through prototype testing with conversational designers and developer/tech experts.
Consultation and review: consultations with executive stakeholders to ensure sign off, collaborations, and alignment with tech, business, and design strategies.
Functional definition and anatomy
An entity is an element of information (characteristic, ie: noun)
Each entity is made up of values, attributes of the information needed (ie: adjective)
Each value is made up of synonyms, other utterances the user can say in replace of each value
A slot(s) is a requirement needed by the bot to proceed with a task. Can be a group and/or a sequence of entities (commonly referred in Speech/Voice)
Example of enumerated entities with fixed/finite values
How entities are used
Entities are used in the automated process of slot-filling: a conversational designer can create a set of criteria the bot should collect from the user to continue along a journey and return a set of results and fulfill their requests in a conversational style.
User scenarios include the customer inquiring about specific or a set of products, services, options, etc. Examples include: flight options, hotel room options, products to purchase/browse, credit card payment, account lookup, etc.
These user scenarios require specific information from the user to provide to the bot, so the bot has sufficient information to return the correct responses/information to the user.
Conversational designers would like to define and list the set of entities the bot needs to extract from the conversation. They also want to specify their sequence/order/required/optional and acceptable data types.
Scenario: Conversational designer is tasked with designing a conversation to help a user purchase a jacket. The bot needs to fulfill the intent by knowing the jacket criteria the user wants. They need to design the slots, flow, and messages.
Problem statements
Conversational designers (internally) don’t create the entities nor have a window to see them since they are stored in the Model Workbench which is only accessed by Data Scientists.
Conversational designers have to depend and co-collaborate with Data Scientists to determine what entities are available and need to be created. This creates a bottleneck to move efficiently and fast.
Conversational designers currently have no standard or consistent design system or visualization for displaying entities in their flow diagrams. This varies across designers, as they do this in many different ways.
Design goals
Enable conversational designers to independently declare, manage, and use entities in their conversational design flows
Enable data scientists to modify/enhance the regular expressions in the more complex custom entities that conversational designers create
Hypothesis
Competitive analysis has determined various ways of integrating this functionality into their tools. Some complex, some simple. These tools vary in degree of difficulty and user base which doesn’t align with our target users. Some don’t have visual flow diagrams, which is another unique consideration for our tool.
Conversational designers need to see and determine entities as a guidance for constructing unique bot message prompts and bot rephrases in a case by case basis per scenario. Allowing the designer the functionally do this workflow will result in an efficient design workflow without dependencies on other co-collaborators. Conversational designers would be happy with any manifestation of this functionality in the conversational flow designer.
Entities are a set of requirements needed in order for the bot to proceed with a conversation (example: shopping criteria, hotel room criteria, etc). Each entity requires a value (requirements to fulfill the bot understanding results to return: price, location, gender, destination, color, etc). Each value can be supported by synonyms for utterance recognition (men = dad, father, grandfather, boyfriend, etc)
Use cases
As a conversational designer, I want to create reusable global custom entity values, so that I can reuse the same ones across different conversations in my bot (ie: item browse/find)
As a conversational designer, I want to see and be provided with standard system entity values by 247.ai, so that I can reuse the same ones across different conversations in my bot (ie: dates, dollar amounts)
As a conversational designer, I want to use entities in a conversation, so that I can design a flow that proceeds to user fulfillment and satisfaction
As a conversational designer, I want to create unique bot messages for each entity value in a conversation, so the bot can sound in the moment and contextual
As a conversational designer, I want to create unique bot messages per entity, for scenarios where the user gives an unrecognized value, changes an entity value, or the bot wants to confirm their value inputs, so that the bot has sufficient info to proceed with fulfilling the intent
As a conversational designer, I want to see which nodes are entities on my conversational flow diagram, so I can easily parse where they exist
User journey map
Interaction paradigm explorations
Mental model 1
User applies an entity to an existing intent node
Pros and cons
Centralized area to manage all entities that span across entire bot app
Not to best mental model of building your pieces first, then making your reference points later (reversed mentality)
Doesn't provide the right flow and visualization when designing a conversation since they manifest in the moment, in context
Mental model 2
User creates INDIVIDUAL entity nodes for each entity value in a conversation flow
Pros and cons
Natural way to help conversational designers envision entities as separate nodes since they are all different dialogs with logic
Consistent way to designate a node per entity to allow configuration of bot messages
Single entry point to a group of entities and from (from connection line can’t derive from any particular entity because they can be fulfilled at any time or not required)
Vertically stacking them prevents horizontally elongated flows, mitigating the need to scroll
Specific ordering of entities from top to bottom
Mental model 3
User creates ONE entity node for all entity values in a conversation flow
Pros and cons
Can’t see entities on the flow diagram since buried into a node detail
Feels more like data entry as opposed to conversation design
Not most natural and optimal for conversational design since not taking advantage of visual flow diagram use to illustrate relationship of entities to initial and subsequent nodes
Inconsistent interaction model: breaks paradigm (isolate each bot message per entity node)
Testing plan
Who: internal [24]7.ai conversational designers, tech expert, and executive stakeholders: VP of conversation design and VP or product
What and why: UI concept testing to collect user feedback and insights on the usability and general experience of the Entities feature integrated into the Conversation Builder, which will help us further optimize the design for efficiency/sufficiency
When: post-foundational research analysis, pre-UI design finalization and implementation
Where and how: remote on Zoom teleconference. Users will access a click through prototype via web browser and screen share as they navigate and speak. We will conduct this by framing questions, recording, and leading them thorough the prototype
Test script
Framing user objective
"You are tasked with designing a conversation to help a user purchase a jacket. The bot needs to fulfill the intent by knowing the jacket criteria the user wants. You need to design the slots, flow, and messages."
Questions
"Where would you start? How and why?"
"What would your next course of action be?"
"What is your thought process as you are designing a conversation with entities?"
"How might you create an entity? Where, when, why? What do you need to create an entity?"
"What are your thoughts on how entities were displayed in the visual flow diagram?"
"How might you design the next possible step after entities have been designed in the tool?"
Test insights
Users did not click on the Entities tab first. They recognize it was there, but still naturally were inclined to build out their conversation first.
Users naturally want to create entities in the moment during the creation of their conversation flow
Users want to create entities that are specific/contextual to the conversation they are designing
Users see 2 types of entities: contextual (specific to journey they are designing) and global (generalized entities that can be reused across multiple conversations)
Users do want to sequence entities in a particular order
Users naturally were compelled to add entities to a bot message node as opposed to creating separated entity type nodes
UI improvements
Enable designers to initiate the creation or application of an entity in real time of creating a conversation flow, in the node details of a bot message or from the Add New Node dialog
Enable designers to create/distinguish between contextual and global nodes (global can be both custom & system entities, and created in the Entities tab), contextual entities can be applied/created during a conversation flow design (it may appear in the entities tab also)
Enable designers to rearrange the order of entity nodes a la reorder mechanism that may be accessed/initiated from the contextual node actions on hover (reorder icon can also appear when a user hovers over a node)
First-time-run tooltips to highlight/educate users on tabs for Flow, web-hooks, and entities
Final solution
We derived at a solution that enabled users to declare entities in context of the conversational visual flow as they are designing journeys. We also enabled users to manage, create, and view all entities in a single screen. The tool has the flexibility or building entities from a global and contextual level. Entities are designated per a single node, allowing the user to visually see them in line of their flow and making it easy to manage. Here is the example 'Track Package' chat exchange with a bot and user.
Entities appear in a conversational flow
The conversation designer adds entity nodes to their conversation flow to handle each user criteria.
Entity node details
Each entity node enables the conversation designer to construct values and synonyms for each entity, bot responses, and other message configurations.
Entities management
All entities in the conversational flow can also be managed in the Entities settings. This gives the conversation designer a holistic view on all existing entities in their system.
Summary
A comprehensive user research process, along with expert consultations, helped me understand what entities are, how they're used, and the best UI to represent them in our conversation design platform product. Using the double diamond approach to design explorations helped to converge to a validated mental model and micro-interactions. I have a great knowledge of entities now, but wouldn't have attained the intelligence without the persistence, consistency, and support of the team, users, and the process.
About the author
Sean Lazo is a Principal UX Designer at [24]7.ai who leads the inception, assembly, and design of [24]7.ai Conversations, an industry-leading omni-channel AI chatbot SAAS platform. His passions are DesignOps, detailed design, research, and human relations.
コメント