Bob Buzzard Blog: August 2024

Sunday 11 August 2024

The Evil Co-Worker presents Evil Copilot - Your Untrustworthy AI Assistant

Image generated by gpt4o based on a prompt from Bob Buzzard

Introduction

Regular readers of this blog will be familiar with the work of my Evil Co-Worker to worsen the Salesforce user experience wherever possible. The release of Einstein Copilot has opened up a whole raft of possibilities, where the power of Generative AI can be leveraged to make the world a better place .... but only for them! This week we saw the results of their initial experiments when the Evil Copilot was launched on an unsuspecting Sales Team - your untrustworthy AI assistant.

The Evil Actions

Evil Copilot only has a couple of actions, but these are enough to unsettle a Sales team and help the Evil Co-Worker gain control of important opportunities.

What Should I Work On

The first action is What Should I Work On. An unsuspecting Sales rep asks for advice about which accounts and deals they should focus on today, expecting to be pointed at their biggest deals that will close soon, and the high value accounts that should be closely monitored. Instead they are directed to their low value/low probability opportunities and accounts that they haven't dealt with for ages. They are also informed that it doesn't look like Sales is really for them, and advised to look for a different role! Quite the demotivating start to the day:

Opportunity Guidance

Note also that the rep is advised that they have an opportunity that is a bit tricky and they should seek help from E. Coworker. Before they do this, they use Copilot to look up the opportunity:

It turns out this is their biggest opportunity, so the user seeks the sage advice of Copilot, only to hit another evil action and another knock to their confidence - Copilot flat our says that they aren't up to the job and the best bet is to hand it over to, you guessed it, E. Coworker!

With just a couple of actions, Evil Copilot has most of the Sales team focused on trivia, while all the top opportunities end up in the hands of the Evil Co-Worker - not a bad return for a few hours work!

But Seriously Folks!

Now this is all good fun, but in practice my Evil Co-Worker would require administrator access to my Salesforce instance and very little oversight to be able to unleash Evil Copilot on my users. And I've no doubt there are more than a few companies out there where it would be entirely possible to do this, at least until an angry Sales rep called up to ask what was going on!

But a Copilot implementation doesn't have to be intentionally Evil to cause issues. The Large Language Models will follow the instructions given to them in Prompt Templates, even if that wouldn't be reasonable course of action for a human member of staff - if the instruction is to tell a user they don't appear to be suited to their job, they'll do it without question. While us humans can tell that this isn't appropriate, the models will see it as perfectly reasonable. It won't be picked up as toxic either - simple constructive criticism won't raise any flags.

That's why you always need to test your Prompt Templates, and your Copilot actions, with a variety of data and users - to ensure that your intended request doesn't turn into something completely different in the wild. We've all sent emails that we were convinced had 100% clarity, only to see someone take them entirely the wrong way, and then we we look at them again we realise how ambiguous or subjective they were. And always have a second pair of eyes reviewing the content of a Prompt Template before making it available to users - Evil Co-Workers are constantly on the lookout for weak points in a process.

Follow @bob_buzzard