Understanding Prompting
Prompting involves the provision of clear and well-crafted instructions or questions to guide large language models (LLMs) in generating valuable responses. It serves as a guide for the model, so the more instructions and context is provided to the prompt, the more Pathlight will be able to understand and provide the desired output.
To be able to edit prompts, you must first go to the Pipeline settings window:
Select the Conversations icon in the left-hand corner, then choose Settings in the upper-right-hand corner.
__________________________________________________________________________________
The Importance of Context
Context plays a crucial role in prompt engineering, encompassing the additional information or data surrounding a situation or problem, which the model lacks by default. Context guides LLMs in generating accurate responses by providing a deeper understanding of the task at hand.
When designing prompts within Pathlight, it is recommended to use simple and concise language while assuming that the model has extensive knowledge but lacks any knowledge about your business and processes. For example for the Questions portion in Pathlight:
Question 1: Did the agent resolve the case correctly?
Answer: Yes, the agent closed the case by thanking the customer
This example looks probable but may be correct or incorrect depending on your definition of a what a resolved case is. To improve this ambiguity, we need to add the correct context.
Question: Did the agent resolve the case correctly?
Additional Context: Resolved cases have the following criteria:
The customer must accept the resolution.
The agent needs to ask if their are any other issues they need help with.
Answer: No
You can further improve this by adding the Answer Format section:
Question: Did the agent resolve the case correctly?
Additional Context: Resolved cases have the following criteria:
Answer Format: reply yes/no and include an additional sentence explaining why
Answer: No, the agent did not meet the criteria of a case resolution because blah
__________________________________________________________________________________
Writing Effective Prompts
Writing effective prompts requires three important guidelines:
- Clear and specific instruction or question: Clearly communicate the desired action or the question you want the model to answer. Use concise and unambiguous language to ensure the model understands your intention accurately.
-
- Use action verbs: Start prompts with action verbs to guide the model's understanding of the intended action. Instead of asking, "What are the features of the product?", use the prompt, "List the key features of the product."
- Avoid compound questions/instructions: Keep prompts focused on a single task or question. Compound instructions can confuse the model and result in less accurate outputs. Break down complex tasks into smaller, simpler prompts for better results.
-
- Providing context with missing information: Supply any relevant information or data that the model may need to complete the task or answer the question precisely. Context guides the model's understanding and facilitates contextually appropriate responses.
- Outline the desired output format: Clearly specify the desired format for the model's response. Whether you require a short paragraph, bullet points, or a specific structure, outlining the desired output format helps the model generate responses that align with your expectations.
__________________________________________________________________________________
What to avoid
Here are a few examples of potential pitfalls or common mistakes users make when creating prompts:
-
Overly Broad or Vague Prompts: One of the most common mistakes is to ask questions that are too broad or vague. For instance, "What happened in the conversation?" might yield a very generic response from the AI model. Instead, specific questions such as "Did the agent handle the customer's complaint efficiently?" can elicit a more specific, focused response.
-
Assuming Prior Knowledge or Context: Users may assume that the AI has specific knowledge about their organization or context. It's important to remember that while the AI has extensive general knowledge, it doesn't know specifics about a user's company or project. Therefore, necessary context should always be included in the prompts.
-
Neglecting the Answer Format: Often, users forget to provide an expected answer format. For instance, if you need a response in a 'yes' or 'no' format with an explanation, it should be stated in the prompt to get the desired output.
-
Overcomplicating Prompts: Sometimes, in an attempt to be thorough, users may overcomplicate the prompt. This might confuse the AI, leading to inaccurate or unhelpful responses. Prompts should be kept clear and concise, focusing on one task or question at a time.
-
Ignoring the Role of Employee Type in Prompt Context: If the context doesn't align with the role of the employee, the AI might not provide a suitable answer. It's important to customize prompts based on the specific roles or departments in the organization.
By addressing these common mistakes and offering solutions, you could better equip yourself and others to craft effective prompts and get the most out of the AI interactions.
__________________________________________________________________________________
Prompting and Context within Pathlight
Now that we have reviewed best practices for writing prompts and adding context, let's see how we can utilize this within the Pipeline Settings in Pathlight:
Custom Summary:
A custom summary serves as a high-level overview of a conversation, capturing its essence and key points. In Pathlight, you can design the prompts to generate these summaries accurately.
To use this feature, you must activate the "Use Custom Summary Instructions" toggle:
Summary Instructions:
"Provide a 3-4 sentence summary of the included transcript. The first sentence should focus on the reason for the call. The middle sentences should focus on detail and key moments. The last sentence should focus on the outcome of the call and, specifically, if the next meeting was successfully booked. Don't use personal names in the summary.
This way, the model can generate a comprehensive summary such as, "The customer was concerned about the lack of a feature in the product. The agent provided the following solution: they guided the customer on how to access this feature and offered further assistance if needed."
__________________________________________________________________________________
Prompt Context and Employee Type:
Context can vary significantly based on the employee's role or department. For instance, the context for a customer service representative differs from that of a sales representative. Customizing prompts based on the employee type ensures more accurate and relevant responses.
For example:
Prompt Context: Customer support agents are having email conversations with customers about issues with the product {your organization}.
Employee Type: Customer Support Agent
__________________________________________________________________________________
Custom Questions:
You can design prompts for specific questions related to your customer and the interaction. Custom Questions should be more focused on understanding what the customer is communicating.
For example:
Prompt: "What product did the customer call about?"
Additional Context: " List the products the customer called about where they mention returns, purchases, and/or feedback."
Answer Format: "Reply with the product names and what did they say about the product."
Here, Pathlight can produce a response like, "The customer asked about renewing a subscription for product X."
__________________________________________________________________________________
Custom Tagging:
They are customized descriptive labels created by the User in a specific Conversation Pipeline. It is designed to detect conversations based on certain predefined criteria. By adding context (details) to the custom tag, we can better attribute the tag to the correct item in the conversation.
For example:
Tag: Subscription
Detail: Renew, new subscription, cancelled, paused, subscription adjustments
__________________________________________________________________________________
Intent:
Intent can detect the customer's main reason for the interaction. By adding an intent description, you will greatly improve the accuracy of the intent tagging.
For example:
Intent Label: Product feedback
Intent Description: The customer provides feedback on the following products: X, Y, Z.
__________________________________________________________________________________
Resolution:
Specify what it means to resolve a conversation.
For example:
Resolution: When the agent has answered all the customer's questions and provided all the issues mentioned.
__________________________________________________________________________________
We hope this was helpful! Please submit a ticket here if you have any questions or need further assistance.