Bob Buzzard Blog: Einstein Prompt Templates in Apex

Image by DALL-E 3 based on a prompt from Bob Buzzard

Introduction

Thus far in my journey with Generative AI in Salesforce I've been focused on using prompt templates out of the box - sales emails, field generation, or in custom copilot actions. Salesforce isn't the boss of me though, so I'm also keen to figure out how to include prompts into my own applications.

The Use Case

I want a way for sales reps to get advice on what they should do with an opportunity - is it worth proceeding with, what does it make sense to do next, that kind of thing. I don't want this happening every time I load the page, as this will consume tokens and cost money, so I'm going to make it available as an LWC inside a tab that isn't the default on the opportunity page:

The Coach tab contains my LWC, but the user has to select the tab to receive the coaching. I could make this a button, but pressing buttons fulfils an innate desire for control and exploration in humans, so it could still end up consuming a lot of tokens. Opening tabs, for some reason, doesn't trigger the same urges.

The Implementation

The first thing to figure out was how to "execute" a prompt from Apex - how to hydrate the prompt with data about an opportunity, send it to a Large Language Model, and receive a response. This is achieved via the Apex ConnectAPI namespace. It's a bit of a multi-step process, so you should be prepared to write a little more code than you might expect.

Prepare the Input

The prompt template for coaching takes a single parameter - the id of the Opportunity the user is accessing. Prompt templates receive their inputs via a ConnectApi.EinsteinPromptTemplateGenerationsInput instance, which contains a map of ConnectApi.WrappedValue elements keyed by the API name of the parameter from the prompt template. The ConnectApi.WrappedValue is also a map though, in this case with a single entry of the opportunity id with the key value 'id'. As mentioned, a little more code than you might have expected to get a single id parameter passed to a prompt, but nothing too horrific:

ConnectApi.EinsteinPromptTemplateGenerationsInput promptGenerationsInput = 
                           new ConnectApi.EinsteinPromptTemplateGenerationsInput();

Map<String,ConnectApi.WrappedValue> valueMap = new Map<String,ConnectApi.WrappedValue>();

Map<String, String> opportunityRecordIdMap = new Map<String, String>();
opportunityRecordIdMap.put('id', oppId); 

ConnectApi.WrappedValue opportunityWrappedValue = new ConnectApi.WrappedValue();
opportunityWrappedValue.value = opportunityRecordIdMap;

valueMap.put('Input:Candidate_Opportunity', opportunityWrappedValue);

promptGenerationsInput.inputParams = valueMap;

promptGenerationsInput.isPreview = false;

Note that the ConnectApi.EinsteinPromptTemplateGenerationsInput handles more than just the inputs - for example, I've set the isPreview property to false, indicating that I don't just want to preview the prompt hydrated with data, I want to send it to the LLM and receive a response. There's also a ConnectApi.EinsteinLlmAdditionalConfigInput property to send granular configuration information such as temperature to the LLM

Make the Request

Once I have the input ready, I can send the request to the LLM.

ConnectApi.EinsteinPromptTemplateGenerationsRepresentation generationsOutput = 
    ConnectApi.EinsteinLLM.generateMessagesForPromptTemplate('0hfao000000hh9lAAA', 
                                                             promptGenerationsInput);

Note that I've hardcoded the ID of the prompt template in the request. I really didn't want to do this, but I can't find a way to query the prompt templates in the system. The metadata type is GenAiPromptTemplate, but this doesn't appear to be queryable, not by me at least! This is the name that getSObjectType returns, but it doesn't appear in the Salesforce Object Reference and both Apex and the Tooling API error when I try to use it in a query. I'm sure this is on its way, but if I put this into production I'll probably expose a property on the LWC so that whoever is building the record page can supply the Id there. And if it's my Evil Co-Worker, they can supply a completely different prompt template Id to introduce confusion and chaos, so everybody should be happy.

Extracting the Response

The output from the LLM is in the form of a ConnectAPI.EinsteinLLMGenerationItemOutput. This contains a generations property that is a list of responses from the LLM. Right now I'm just picking the first element in the list and pulling the text property containing the actual information that I want to display to the user. I can also access the safety score if I want.

Putting it in front of the User

The Lightning Web Component is pretty vanilla - when the user opens the tab the record id is passed via a property setting decorated with @api. This triggers the request to the server to send the prompt to the LLM:

@api get recordId() { 
    return this._recordId; 
}

set recordId(value) {
    this._recordId=value;
    GetAdvice({oppId : this._recordId})
    .then(data => {
        this.thinking=false;
        this.advice=data;
    })

Note that I have to do this via an imperative Apex call rather than using the wire service, as calling an Apex ConnectAPI method consumes a DML call, which means my controller method can't be cacheable, which means it can't be wired.

The user receives a holding message that the coach is thinking:

Then after a few seconds, the Coach is ready to point them in the right direction:

Follow @bob_buzzard

Bob Buzzard Blog

Sunday 24 March 2024

Einstein Prompt Templates in Apex - the Sales Coach