Showing posts with label copilot. Show all posts
Showing posts with label copilot. Show all posts

Sunday, 11 August 2024

The Evil Co-Worker presents Evil Copilot - Your Untrustworthy AI Assistant

Image generated by gpt4o based on a prompt from Bob Buzzard

Introduction

Regular readers of this blog will be familiar with the work of my Evil Co-Worker to worsen the Salesforce user experience wherever possible. The release of Einstein Copilot has opened up a whole raft of possibilities, where the power of Generative AI can be leveraged to make the world a better place .... but only for them! This week we saw the results of their initial experiments when the Evil Copilot was launched on an unsuspecting Sales Team - your untrustworthy AI assistant.

The Evil Actions

Evil Copilot only has a couple of actions, but these are enough to unsettle a Sales team and help the Evil Co-Worker gain control of important opportunities.

What Should I Work On

The first action is What Should I Work On. An unsuspecting Sales rep asks for advice about which accounts and deals they should focus on today, expecting to be pointed at their biggest deals that will close soon, and the high value accounts that should be closely monitored. Instead they are directed to their low value/low probability opportunities and accounts that they haven't dealt with for ages. They are also informed that it doesn't look like Sales is really for them, and advised to look for a different role! Quite the demotivating start to the day:


Opportunity Guidance


Note also that the rep is advised that they have an opportunity that is a bit tricky and they should seek help from E. Coworker. Before they do this, they use Copilot to look up the opportunity:


It turns out this is their biggest opportunity, so the user seeks the sage advice of Copilot, only to hit another evil action and another knock to their confidence - Copilot flat our says that they aren't up to the job and the best bet is to hand it over to, you guessed it, E. Coworker!


With just a couple of actions, Evil Copilot has most of the Sales team focused on trivia, while all the top opportunities end up in the hands of the Evil Co-Worker - not a bad return for a few hours work!

But Seriously Folks!

Now this is all good fun, but in practice my Evil Co-Worker would require administrator access to my Salesforce instance and very little oversight to be able to unleash Evil Copilot on my users. And I've no doubt there are more than a few companies out there where it would be entirely possible to do this, at least until an angry Sales rep called up to ask what was going on!

But a Copilot implementation doesn't have to be intentionally Evil to cause issues. The Large Language Models will follow the instructions given to them in Prompt Templates, even if that wouldn't be reasonable course of action for a human member of staff - if the instruction is to tell a user they don't appear to be suited to their job, they'll do it without question. While us humans can tell that this isn't appropriate, the models will see it as perfectly reasonable. It won't be picked up as toxic either - simple constructive criticism won't raise any flags. 

That's why you always need to test your Prompt Templates, and your Copilot actions, with a variety of data and users - to ensure that your intended request doesn't turn into something completely different in the wild. We've all sent emails that we were convinced had 100% clarity, only to see someone take them entirely the wrong way, and then we we look at them again we realise how ambiguous or subjective they were. And always have a second pair of eyes reviewing the content of a Prompt Template before making it available to users - Evil Co-Workers are constantly on the lookout for weak points in a process.

Related Posts



Saturday, 11 May 2024

Formatted Output from Copilot Custom Actions


Image generated by DALL-E 3 based on a prompt from Bob Buzzard

Introduction

If you've seen any of the demos of Einstein Copilot, you'll notice that sometimes the responses are nicely formatted using the Lightning Design System, while other times they are simply text on a darker background - e.g. from the TrailblazerDX keynote, the pipeline information is text:


While the contacts are shown as a formatted list, with fields specific to the solution:


This isn't particularly well documented, so it's a matter of trial and error to figure out what works and what doesn't.

Text Responses

Text responses came out the same regardless of what I tried - the text that I return appears on a dark background. If I format it as JSON or CSV, it comes out in JSON or CSV format, even if I included instructions to display as a list of label/value pairs. 

The same goes for HTML markup - it's taken as text and shown to the user. Using \n for a line break works, but aside from that what you return from your custom action is what you see on the screen.

Custom Class

Returning custom class instances introduce a little more formatting. You can only return a single "value" from your custom action, but this can contain a list of custom class instances and Copilot will render each of the properties in it's own "text box". In the example below I return a single instance of my custom class Output, but this contains a list of instances another custom class. These in turn which contain the fields from Task records (as using the Tasks directly throws errors) :

public class Output
{
    @InvocableVariable
    public List<CustomRec> recs;
}

public class CustomRec
{
    @InvocableVariable
    public String ident;
        
    @InvocableVariable
    public String subject;
        
    @InvocableVariable
    public String description;
        
    @InvocableVariable
    public String activityDate;

    public CustomRec(Task task)
    {
        this.ident=(null!=task.id?'ID : ' + task.Id:null);
        this.subject=(null!=task.Subject?'Subject: ' + task.Subject:null);
        this.description=(null!=task.Description?'Description :\n' + task.Description:null);
        this.activityDate=(null!=task.activityDate?'Due Date : ' + task.activityDate:null);
    }
}

Copilot displays the properties from each record, although there isn't any separation between records. There's also a wrinkle in there I wasn't expecting - the properties are displayed in alphabetical order : activityDate, description, ident, subject. While unexpected, this does give me a way to order the properties if I need to:


sObject Records

As before, I can only return a single item from a custom action, so if I want to send back a list of records I have to wrap them in an containing class:

public class Output
{
    @InvocableVariable
    public List<Opportunity> opportunities;

    public Output()
    {
        opportunities=new List<Opportunity>();
    }
}

And this comes out very nicely, with a card for each record:


sObject Records and more!

This is the one that I'm most pleased about - it allows me to display records and some commentary about each record, while still retaining the Lightning Design System formatting for the record itself.

Once again I'm returning a custom class containing a list of other custom classes, but this time it's a wrapper class containing an sObject record and some additional information:

public class Output
{
    @InvocableVariable
    public List<OpportunityWrapper> opportunities;

    public Output()
    {
        opportunities=new List<OpportunityWrapper>();
    }
}

public class OpportunityWrapper
{
    @InvocableVariable
    public Opportunity opp;

    @InvocableVariable
    public String message;

    @InvocableVariable
    public String trailer;
        
    @InvocableVariable
    public String zzzSeparator='-----------------------------';

    public OpportunityWrapper(Opportunity opp, String message, String trailer)
    {
        this.opp=opp;
        this.message=message;
        this.trailer=trailer;
    }
}

I really didn't expect this to work, but it did!


I've taken advantage of the fact that the properties are displayed in alphabetical order to display a message about how near the close date is, then the record, then some commission information, followed by a rather ugly separator. I think if I was going to use this in production I'd have all the message above or below the record so that I didn't have to put a clunky separator in, but I wanted to see if I above and below would work.

Conclusion

I don't think this is too bad, given that Copilot is at its heart a text based tool. Hopefully in time we'll be able to apply more formatting to text output, but at least sObject records are styled for the Lightning Experience. Worst case, we all end up polluting our Salesforce org with a bunch of fake records that are just used as carriers for Copilot responses! 

Related Posts



Sunday, 5 May 2024

Einstein Copilot - AI + (most of your) Data + CRM?

Image created by DALL-E 3 based on a prompt from Bob Buzzard

Introduction

Einstein Copilot from Salesforce went GA around a week ago (25th April 2024, for future readers), which changes my attitude to it. When things are in preview/pilot/beta, I'll take a bit of additional effort tracking down issues if it means I get my hands on things earlier. Once a feature goes GA though, the gloves are off - real money is changing hands to use these features so we hold them to a higher standard.

As the title of this post suggests, my first disappointment post-GA was that I can't use all my data across Generative AI - specifically Tasks and Events, part of the Salesforce Activities feature. This won't come as a surprise to old hands in the Salesforce ecosystem - hands up those who remember using text fields to capture IDs because custom lookups weren't available! More recently, they still aren't supported by the User Interface API, which means some of the standard LWC functionality won't work for them, and it's slightly concerning that an idea to add this capability has been open for 3+ years without even a comment from Salesforce. Yes, Activities are really Tasks, Events and Calendars on each others shoulders in a long overcoat, but this was the choice Salesforce made and they need to own it

There are workarounds for most areas of the platform where support for Activities is not as complete as it could be, but that isn't the case for the first issue that I encountered.

Prompt Template Parameters

Activities can't be specified as parameters to Prompt Builder. The docs suggest they should be available, as they are standard objects:

My use case was I wanted to send an Event record to an LLM, which contained details of a meeting that had taken place with a customer. The record includes the intended outcome, notes from the meeting, and any comments from company attendees after they've had a short period of time to reflect.  The LLM would then recommend the best course of action based on the meeting.This would then be used to create a Copilot action, so my users didn't could focus on doing the next step rather than figuring it out.

Searching for Event in the resource picker gave me pause for thought:


Maybe it's under Activity/Activities? Sadly not:


For the sake of completeness I also checked for Task - same result:


Now this doesn't mean that Activities can't be used with Prompt Builder - I can access the Open Activities and Activity History if I specify another object like Opportunity as the resource:



As this is the only way to specify a record to a Prompt Template, there aren't any workarounds to handle this, or none that don't suck anyway.

I could pass the Who and What information to the template and try to figure out the Event - for example, if the Event was with a Contact to talk about an Opportunity, I could pass the IDs for those in and use Apex/Flow grounding to retrieve the Events that match these IDs and hope I can figure out which one the user is interested in. If there are a number of them, or the user doesn't want something simple like the last one, I'm highly likely to pick the wrong one and give them unwanted advice about it.

I could make the users populate a lookup to a custom object and then pass the ID of that into the Prompt Template. The downside to this is my users have to do extra work in order to make the AI assistant useful - the exact opposite of how it's supposed to work. I'd also have to do a data fix to create custom object records for the recent history of Events.

Rendering Lists of Activities

One of the nice features of Copilot is if my custom action returns a list of sobjects in the correct format, it will apply Lightning Design System formatting:


So I decided to create a custom action that retrieves the Tasks that I should be focusing on - this has a bunch of rules in the action to determine which Tasks are actually important, as anyone who creates a Task for me will naturally mark it as Urgent and High Priority.

I'm able to retrieve the list of Tasks, but Copilot can't render them:


and check out that error message - Copilot fails to render the information provided and suggests you talk to your admin about it! I predict frustration all round!

Undeterred, I changed my code to return a list of custom Apex class instances that mirrored a task, so a property for Id, a property for Subject etc. Slightly better, in that it briefly rendered some text and then errored again.

After quite a bit of trial and error I was able to get a list of custom Apex class instances to render - I had to either remove the Id of the task, or put it into a String property that had any name other than Id. It looked awful by comparison, and I had no way to generate a link to any of the Tasks, as I'd had to put a false nose and dark glasses on the Id. The user could copy and paste the Id, but it seemed like the introduction of an AI assistant was once again leading to more work for the User, not less. So here there is a workaround, but it's not a great user experience.

My suspicion regarding this problem is that if Copilot can detect that there is either a Task or the Id of a Task in the data, it tries to pull some information about the Task sObject and then errors. Maybe via the User Interface API, given this is rendering on the screen, which doesn't support Tasks.

Conclusion

I'm really surprised at this gap, as Tasks and Events are pretty fundamental to Customer Relationship Management. It's either quite the oversight or, left out because the APIs don't support them in. the hope that nobody notices. It's all well and good having every web and commerce interaction with your customer in Data Cloud, but if you can't ask questions of your standard CRM objects, then we really aren't getting the AI + Data + CRM that we've been promised will change our lives.

Related Posts



Sunday, 7 April 2024

Einstein Copliot Custom Actions

Image created by DALL-E 3 based on a prompt by Bob Buzzard

Introduction

One of the key features of Einstein Copilot from Salesforce is its extensibility - while there are a bunch of standard actions, and many more coming, there will always be something more you want to give your users. Custom actions allow you to create an AI assistant that targets your specific business challenges, adding capabilities to your copilot that generate real value for your users.

Scenario

In my last post, I introduced the Sales Coach - a custom Lightning Web Component that you add to an Opportunity record page. When the Sales Coach component renders, it executes Apex code to hydrate a prompt template with details of the Opportunity and request some guidance from an LLM. 

While this works well for users that are viewing an Opportunity record, I also want to provide a way tfor users receive guidance on deals while they are in another part of the Salesforce application - their Home Page, for example.

Creating the Custom Copilot Action

Custom Copilot Actions can be created using Apex, Flow or Prompt Templates. I've gone for a Prompt Template, as I already have this in place for the Sales Coach component. Once I choose the Reference Action Type of Prompt Template, I get to select from a list of existing prompts in the Reference Action selector. 


Once I've chosen my action, I provide details of how the Copilot can use it :



The Copilot Action Instructions is pre-populated from my Prompt Template description. As I want to use it in exactly the same way, I can leave that alone. I then provide the instructions around the Candidate Opportunity input, and check the box in the Output section to Show in conversation - without this, the user won't be able to receive the coaching they so richly deserve.

Note that while the Prompt Template pull several fields from the Opportunity record - Name, Amount, Close Date, Stage Name - I don't need to specify any of this information here. I don't even need to specify that it's an Opportunity record - that is picked up automatically from the Prompt Template input. Pretty simple really.

Once my custom action is created, I need to add it to the Copilot. I do this via the Copilot Builder, selecting my new action from the list of available actions and then clicking a button to add it to the Active Copilot :



As an aside, this Copilot isn't Active yet, so the text in the button is slightly misleading, but not a big deal as I can only have one Copilot so I'm hardly likely to get confused.

Putting it Together

Remember earlier when I created the Copilot action I gave it details of how to use it? The Copilot Action Instructions field does what it says on the tin - it instructs the Copilot as to the purpose of the action and when it should be used:


I've defined this as providing advice to progress a specific opportunity, so in order to include the action in the Copilot plan, I need to ask the it to help me do that. As we're in AI world, I don't need to be as obvious as asking for advice to progress an opportunity, instead I used my own words and asked for "help to win a deal".

Interestingly, when I ran this to capture the screenshots, Copilot told me that it was following the instructions that I'd provided for the action, which I hadn't seen before:




Something to remember when using Prompt Template actions is they will lead to a slower response time, as each one involves a round trip to an LLM. There will always be one round trip, to create the plan, but if you manage to generate a request that involves multiple Prompt Template actions, there will be separate requests made for each of those, and you'll be consuming additional tokens. 

Sunday, 10 March 2024

Hands on with Salesforce Copilot

Image generated by DALL-E 3 based on a prompt from Bob Buzzard

Introduction

Copilot from Salesforce entered beta at the end of Feb '24, without too much of a build-up I think it's fair to say. Given how quiet the likes of Marc Benioff were on the socials, I was half-expecting it to be delayed until nearer to the Summer release. I'm very pleased that wasn't the case, but not as pleased as I was to get access to it.

Setup

Much like Prompt Builder, enabling copilot is pretty simple :

  • Go to Setup -> Einstein Generative AI -> Einstein Setup and toggle it on
  • The go to Setup -> Einstein Generative AI -> Einstein Copilot Studio (Beta) -> Einstein Copilots and toggle that on.
Copilot is then enabled in your org. It's not active though - you have to head into Builder and click the Activate button to see it in the user interface. You can play around with it pretty well inside the Builder though, so don't activate until you are happy.

Accessing


Once you've activated your copilot, you'll see a new Enstein icon up at the top right of the user interface - looks a bit Heisenberg-esque to me, which I think is awesome:


Clicking this opens copilot in a new sidebar, moving the current screen contents over to the left rather than overlaying them - it's the little things:


One key point to note - copilot doesn't always have the context you are working in - the screengrab above is copilot opening while I'm viewing an opportunity. It's tempting to think that I can just refer to 'this opportunity' and it will pick up the details from my current context/URL, but that doesn't happen - instead it says 'Sure, which opportunity is this?"


As an aside, it also really wants me to use the name of the record - if I try the Id, it tells me that nothing was found in the query. This might seem limited, but I think it'sactually working exactly as intended. There's an action in this copilot configuration that retrieves a record and it requires the record name. If I want to use Ids instead, I need a different action.

Back to context though, if I click the Summarize Opportunity button, it does know about the opportunity and gives me the details : 


Making a Request


As always with generative AI, the quality of the response has a pretty direct correlation with the quality of the prompt. Just vaguely asking for help doesn't go well:


I then tried a request that I thought had enough detail, but in hindsight lacked a little clarity:


Third time's the charm though, and now I have a few accounts to focus on:



Extending


The standard functionality is all well and good, but I'm always interested in how I can extend and customise. There was a bit of trial and error involved, as the docs aren't perfect yet, but remember this is a beta feature.

I was trying to think of something that wasn't in place already, and after a short while it struck me - there's just not enough poetry in enterprise software nowadays, so an action to write a haiku about an account was the obvious, if not the only, choice.

This will be a Prompt Template action, so the first job is to define the prompt:

You are a world renowned haiku writer, who has a side gig as a CRM administrator.
Write a haiku based for the account {!$Input:Account.Name} which operates in the {!$Input:Account.Industry} industry. 
The haiku should not include any PII.

Then the copilot action: 

The important parts of the action are :

  • Copilot Action Instructions - this is used by the AI to determine whether an action is appropriate for a plan. The more detailed the better, although in this case there probably won't be multiple actions competing for the job of writing the haiku.
  • Require user confirmation - tick this if any data is going to be changed. 
  • Account Instructions - these are the instructions for the input named Account that I've specified in the prompt. I've identified which fields I'm interested in, the type, and that it's required. I'm not asking the user to enter any data and there's no PII to mask as I'm just using the Name and Industry fields, which is public information
  • Prompt Response Instructions - here I've defined what type of response, but most importantly ticked Show in conversation. Without this nothing will be returned to the user.
Once this action is added to my copilot, I can add writing a haiku to a request and get the user experience I've been craving:


More Information


Saturday, 11 November 2023

OpenAI GPTs - Meet Bob Buzzard 2.0



Introduction


During OpenAI DevDay, the concept of custom GPTs was launched - Chat GPT with a bunch of preset instructions to target a specific problem domain, additional capabilities such as browsing the web, and extra knowledge in terms of information that may not be available on the web. 

In order to create and use GPTs, you need to be a ChatGPT Plus subscriber at $20/month, although in the UK there's VAT to be added so it works out around £20/month. This also gives priority access to new features, the latest models and tools, faster response times and access even at peak times. I signed up just to try out GPTs though, as they looked like a world of fun.

The Replicant


My first custom GPT is my replicant - Bob Buzzard 2.0. A GPT that has been pointed at most of my public and some of my private information. Instructed to respond as I would, you can expect irreverent or sarcastic responses as the mood takes (AI) me. Obviously very focused on Salesforce, and keen on Apex code. 

Right now you'll need to be a ChatGPT Plus user to access custom GPTs, but if you are you can find Bob Buzzard 2.0 at : https://chat.openai.com/g/g-DOVc9phwC-bob-buzzard-2-0  Here's a snippet of a response from my digital twin regarding the impact of log messages on CPU - something I've investigated in detail in the past :


Creating GPTs


This is incredibly simple - you just navigate to the create page and tell it in natural language how you want it to behave, define the skills, point it at additional web sites or upload additional information. It's easy and requires no technical knowledge, which does make me wonder why they announced it at developer day given there's no development needed, but lets not tilt at that windmill.

A Couple of Warnings


First, remember that any private information that you upload to a GPT won't necessarily remain private. If you don't instruct your custom GPT to keep instructions and material private, it will happy share them on request. 

Second, I've given the replicant a mischievous side - from time to time it will just gainsay your original decisions when you ask for help with specific problems, maybe suggesting you have picked the wrong Salesforce technology, or telling you to bin it all off and use another vendor. Think of this as your reminder that a human should always be involved in any decision making based on advice from AI.

I'm Going to be Rich?


Something else that was announced at Developer Day was revenue sharing - if people use Bob Buzzard 2.0 I'll get a slice of the pie. So does this mean I'm going to be rich? Like always, almost certainly not. As you just click a button and answer questions to create a GPT, there will be millions of them before too long. They are so easy to create that something a service like Salesforce development advice, with the vast amount of content already in the public domain, will be extremely competitive - an extremely crowded marketplace of similar products means everybody earns nothing.

That said, I think this is something that genuine creatives will be able to earn with. Rather than having their work used to train models that are can then be used to produce highly derivative works for close to free, they can create their own GPT and at least stand a chance of getting paid. Whether the earnings will be worth it we don't yet know, although history suggests the platform providers will keep everything they can.