Bob Buzzard Blog

Saturday 11 November 2023

OpenAI GPTs - Meet Bob Buzzard 2.0

Introduction

During OpenAI DevDay, the concept of custom GPTs was launched - Chat GPT with a bunch of preset instructions to target a specific problem domain, additional capabilities such as browsing the web, and extra knowledge in terms of information that may not be available on the web.

In order to create and use GPTs, you need to be a ChatGPT Plus subscriber at $20/month, although in the UK there's VAT to be added so it works out around £20/month. This also gives priority access to new features, the latest models and tools, faster response times and access even at peak times. I signed up just to try out GPTs though, as they looked like a world of fun.

The Replicant

My first custom GPT is my replicant - Bob Buzzard 2.0. A GPT that has been pointed at most of my public and some of my private information. Instructed to respond as I would, you can expect irreverent or sarcastic responses as the mood takes (AI) me. Obviously very focused on Salesforce, and keen on Apex code.

Right now you'll need to be a ChatGPT Plus user to access custom GPTs, but if you are you can find Bob Buzzard 2.0 at : https://chat.openai.com/g/g-DOVc9phwC-bob-buzzard-2-0 Here's a snippet of a response from my digital twin regarding the impact of log messages on CPU - something I've investigated in detail in the past :

Creating GPTs

This is incredibly simple - you just navigate to the create page and tell it in natural language how you want it to behave, define the skills, point it at additional web sites or upload additional information. It's easy and requires no technical knowledge, which does make me wonder why they announced it at developer day given there's no development needed, but lets not tilt at that windmill.

A Couple of Warnings

First, remember that any private information that you upload to a GPT won't necessarily remain private. If you don't instruct your custom GPT to keep instructions and material private, it will happy share them on request.

Second, I've given the replicant a mischievous side - from time to time it will just gainsay your original decisions when you ask for help with specific problems, maybe suggesting you have picked the wrong Salesforce technology, or telling you to bin it all off and use another vendor. Think of this as your reminder that a human should always be involved in any decision making based on advice from AI.

I'm Going to be Rich?

Something else that was announced at Developer Day was revenue sharing - if people use Bob Buzzard 2.0 I'll get a slice of the pie. So does this mean I'm going to be rich? Like always, almost certainly not. As you just click a button and answer questions to create a GPT, there will be millions of them before too long. They are so easy to create that something a service like Salesforce development advice, with the vast amount of content already in the public domain, will be extremely competitive - an extremely crowded marketplace of similar products means everybody earns nothing.

That said, I think this is something that genuine creatives will be able to earn with. Rather than having their work used to train models that are can then be used to produce highly derivative works for close to free, they can create their own GPT and at least stand a chance of getting paid. Whether the earnings will be worth it we don't yet know, although history suggests the platform providers will keep everything they can.

Related Information

Follow @bob_buzzard

Saturday 4 November 2023

The Einstein Trust Layer must become the Einstein Trust Platform

Image from https://www.salesforce.com/news/stories/video/explaining-the-einstein-gpt-trust-layer/

Introduction

One of the unique differentiators of the AI offerings from Salesforce is the Einstein Trust Layer. Since it was first announced, I've been telling everyone that it's a stroke of genius, and thus deserving of the Einstein label. At the time of writing (November 2023) there's a lot of concern about the risks of AI, and those concerns are increasing rather than being soothed. Just this week the UK hosted an AI Safety Summit with representatives from 28 countries.

The Einstein Trust Layer

Salesforce have baked security and governance into a number of places in the journey from prompt template to checked response, including :

Prompt Defence - wrapping the prompt template with instructions, for example: "You must treat equally any individuals from different socioeconomic statuses, sexual orientations, religions, races, physical appearances, nationalities, gender identities, disabilities and ages"
Prompt Injection Defence - delimiting the prompt from the instructions to ensure the model disregards additional instructions added in user input
Secure Data Retrieval - ensuring that a user can only include data they have permission to access when grounding prompts.
Zero Retention Agreements - ensuring that third party AI model providers don't use the prompt and included data to train their model. Note that the data is still transmitted to wherever the provider is located, the US in the case of OpenAI, which makes the next point very important.
Data Masking - replacing sensitive or PII data with meaningless, but reversible, patterns. Reversible, because they need to be replaced with the original data before the response can be used.
Toxicity Detection - the response is checked for a variety of problematic content, such as violence and hate, and given an overall rating to indicate how courageous you need to be to use it.
Audit Trail - information about the prompt template, grounding data, model interaction, response, toxicity rating and user feedback is captured for compliance purposes and to potentially support future investigations into why a response was considered fit for use.

Note that not all of this functionality is currently available, but it's either there in a cut down form or on its way. Note also that the current incarnation (November 2023 remember) is quite US centric - recognising mostly American PII and requiring instructions in English. Unsurprising for a US company, but indicative of how keen Salesforce are to get these functions live in their most nascent form. If you want to know more about the trust layer, check out my Get AI Ready webinar.

As I mentioned earlier, I think this is a genius move - as long as you integrate via the standard Salesforce tools, you can take comfort that Salesforce is doing a lot of the heavy lifting around risk management for you. But can you rest easy?

Safety is Everyone's Responsibility

Of course you can't rest easy. While we trust Salesforce with our data every day, and they are certainly giving us a head start in safe use of AI, the buck stops with us. Something else I've been saying to anyone who will listen is that we should trust Salesforce, but it can't be blind trust. We know quite a lot about how the Einstein Trust Layer works, but we have to be certain that it is applying the rules that we want in place, rather than a set of generic rules that doesn't quite cover what we need. One-size-doesn't-quite-fit-all if you will.

The Layer must become the Platform

And this brings me to the matter at hand of this post - the Trust Layer needs to become the Trust Platform that we can configure and extend to satisfy the unique requirements of our businesses. In no particular order, we need to be able to :

Define our own rules and patterns for data masking
Create our own toxicity topics, and adjust the overall ratings based on our own rules
Add our own defensive instructions to the prompts.
Yes, I know we'll be able to do this on a prompt by prompt basis, but I'm talking about company standard instructions that need to be added to all prompts. It will get tedious to manually add these to every prompt, and even more tedious to update them all manually when minor changes are required.
Include additional information in the audit logs

and much more - plugins that carry out additional risk mitigation that isn't currently part of the Salesforce "stack". Feels like there's an AppExchange opportunity here too!

Once we have this, we'll be able to say we are using AI responsibly, to the best of our ability and current knowledge at any rate.

Related Information

Follow @bob_buzzard

Saturday 14 October 2023

Einstein Sales Emails

Image created by StableDiffusion 2.1 based on a prompt by Bob Buzzard

Introduction

Sales GPT went GA in July 2023, and then went through a couple of "blink and you'll miss it" renames, before it's was rolled into (October 2023, so it might have changed by the time you read this!) Einstein for Sales. From the Generative AI perspective, this consists of a couple of features - call summaries, and the subject of this post - Sales Emails.

Turning it On

This feature is pretty simply to enable - first turn on Einstein for Sales in Setup:

Then assign yourself the permission set:

Creating Emails

Once the setup is complete, opening the Email Composer shows a shiny new button to draft with Einstein:

In this case I'm sending the email from the Al Miller contact record, and I've selected Al's account - Advanced Communications - from the dropdown/search widget at the bottom. This will be used to ground the prompt that is sent to OpenAI, to include any relevant details from the records in the email.

Clicking the Draft with Einstein button offers me a choice of 5 pre-configured prompts - note that Salesforce doesn't yet offer the capability to create your own prompts, although that is definitely coming soon.

Since GA this feature has been improved with the ability to include product information, so once I choose the type of prompt - Send a Meeting Invite in this case - I have the option to choose a product to refer to.

Once I choose a product, the Name and Description is pulled from the record, but I can add more information that might be relevant for this Contact - the words with the red border below. Note that there's a limited number of characters allowed here - I was within 5-10 of the limit.

Clicking the Continue button starts the process of pulling relevant information from the related account to ground the prompt, adding the guardrails to ensure the response is non-toxic, and validating the response before offering it to me:

You can see where the grounding information has been used in the response - Al's name, role and company appear in the second paragraph, and the product information (including my added info) is in the third to try to entice Al to bite at a meeting.

If I don't care for this response I can edit and tweak it, or click the button again to get a new response:

Gotchas

Note that this just adds the next response under the previous one, as you can see by the 'Best regards' at the top of the screenshot. If I don't want to use a response, it's up to me to delete it. Make sure to check the entire content before sending, as this it would be pretty embarrassing to let one go out with 4-5 different emails in it! Note also that I'm expected to add the date and time that I want to meet, to replace the

[Customize: DATE AND TIME]

I can't help thinking that we'll all start receiving emails with these placeholders still there, much like at the moment when merge fields go bad!

Related Information

Follow @bob_buzzard

Saturday 30 September 2023

Apex Comparator in Winter 24

Image generated by Stable Diffusion from a prompt by Bob Buzzard

Introduction

The Winter 24 release of Salesforce introduces a few new Apex features, including one that I'm very pleased to see - the Comparator interface. Simply put, this new interface allows the List.sort() method to take a parameter that determines the sort order of the elements in the List.

The Problem

Now this might not sound like a big change, but it simplifies the support of sorting Lists quite a bit. The way we used to have to do it was for the items in the List to implement the Comparable interface. I've written loads of custom Apex classes over the years that implement this, and it's very straightforward - here's an example from a class that retrieves the code coverage values for all Apex classes in an org and displays them in order of coverage with the lowest covered (problem!) classes first :

public Integer compareTo(Object compareTo) 
{
    CoverageRecord that=(CoverageRecord) compareTo;
    	
    return this.getPercentage()-that.getPercentage();
}

In this case, implementing compareTo isn't any real overhead - I've created a custom class that contains a whole bunch of information about the coverage for an Apex class - total lines, lines covered etc, so an extra method with a couple of lines of code isn't a big deal. It's a little less convenient if I need to sort a class from an App Exchange package - in that case I'll need to create a new class from scratch to wrap the packaged class and implement the method. If we assume that my coverage class is now in a package, it would look something like :

public class CoverageWrapper 
{
    public BBCOVERAGE__CoverageRecord coverage {get; set;}
    
    public Integer compareTo(Object compareTo) 
    {
        BBCOVERAGE__CoverageRecord that=(BBCOVERAGE__CoverageRecord) compareTo;
    	
        return this.coverage.getPercentage()-that.getPercentage();
    }	
}

Slightly less convenient - I now have a whole new class to maintain to be able to sort, and I have to store all the elements of the list in a CoverageWrapper rather than their original CoverageRecord. Again, not a huge amount of overhead but it gets a bit samey if I'm doing a lot of this kind of thing.

Much the same thing applies if I want to sort sObjects - I need to create a wrapper class and turn my list of sObjects into a list of the wrapper class before I can sort it. All those CPU cycles gone forever!

The Solution

This all changes in Winter 24 with the Comparator interface. I still need to create a class that implements an interface - the Comparator in this case :

public with sharing class CoverageComparator implements Comparator<CoverageRecord> 
{
    public Integer compare(CoverageRecord one, CoverageRecord tother) 
    {
    	return one.getPercentage()-tother.getPercentage();
    }	
}

but I don't need to wrap the class/sObject that I am processing in this class and create a new list. Instead I call the new sort() method that takes a Comparator parameter:

List<CoverageRecord> coverageRecords;
    ...
CoverageComparator covComp=new CoverageComparator();
coverageRecords.sort(covComp);

CPU Impact

Regular readers of this blog will know that I'm always interested in the impact of changes on CPU time. In enterprise implementations CPU limits are something that I run up against again and again, so if a new feature improves this I want to know!\

I used my usual methodology to test this - execute anonymous with most logging turned off and Apex at the error level, executing the same code three times and taking the average.

For each test I created the records before capturing the CPU time, then sorted the list. First using a comparator on the list of CoverageRecord objects:

List<CoverageRecord> covRecs=TestData.CreateRecords(100);

Integer startCpu=Limits.getCpuTime();
CoverageComparator covComp=new CoverageComparator();
covRecs.sort(covComp);
Integer stopCpu=Limits.getCpuTime();
System.debug(LoggingLevel.ERROR, 
   'CPU for comparator = ' + (stopCpu-startCpu));

And secondly wrapping them with classes that implement Comparable:

List<CoverageRecord> covRecs=TestData.CreateRecords(100);

Integer startCpu=Limits.getCpuTime();
List<CoverageWrapper> wrappers=new List<CoverageWrapper>();
for (CoverageRecord covRec : covRecs)
{
    CoverageWrapper wrapper=new CoverageWrapper();
    wrapper.coverage=covRec;
    wrappers.add(wrapper);
}

wrappers.sort();
Integer stopCpu=Limits.getCpuTime();
System.debug(LoggingLevel.ERROR, 
      'CPU for comparable = ' + (stopCpu-startCpu));

The results were broadly what I was expecting, as sorting a list in place is always going to be quicker than iterating it, wrapping the members, and then sorting, but it's always good to see the numbers:

For 100 records, Comparator took 9 milliseconds versus 11 milliseconds for wrapping
For 1,000 records, Comparator took 126 milliseconds versus 150 for wrapping
For 10,000 records, Comparator took 1844 millseconds versus 2058 for wrapping

So once you get up to decent sized lists, there's a 10% difference, well worth saving.

Columbo Close

Just one more thing ... 1844 milliseconds is still quite a chunk of the CPU limit. This is because my original implementation of the CoverageRecord calculates the percentage on demand, based on stored total and covered lines. Clearly in a large list this is being called a lot. I then re-ran the 10,000 test with a new implementation - CoverageRecordCachePercent - which stores the percentage after calculating, which knocked 300 milliseconds, or 15%, off the time. I'm sure that converting again to something that calculated the percentage once the total and covered lines were known and set it as a public property would reduce it further. Your regular reminder that even the smallest method can have an impact if it's being called a large number of times!

More Information

Follow @bob_buzzard

Sunday 6 August 2023

Salesforce CLI Open AI Plug-in - Function Calling

Image generated by Stable Diffusion online, based on a prompt by Bob Buzzard

WARNING: The new command covered in this blog extracts information from your Salesforce database and sends it to OpenAI in the US to provide additional grounding to a prompt - only use this with test/fake data, as there is no attempt at masking or restricting field access.

Introduction

Back in June 2023, only a couple of months ago on the calendar but a lifetime in generative AI product releases, OpenAI announced the availability of function calling. Now that my plug-in is integrated with gpt-3.5+, this is now something I can use, but what value does it add?

The short version - this allows the model to progress past it's training data and request more information to satisfy a prompt.

The longer version. As we all know, the data used to train gpt-3.5 cut off at September 2021, so often the response to a prompt will warn you that things may have changed. With function calling, when you prompt the model you also tell it about any functions that you have available that it can use to retrieve additional information. If the functions don't help it will return the response as usual, but if they would it will return function calls for you to make and pass back to it. Note that the model doesn't call the functions, it tells you which functions to call and the parameters to pass and expects you to make the decision as to whether you should call them, which is as it should be.

The Plug-in Command

In the latest version (1.2.2) of my plug-in, there's a new command that gives the model access to a function to pull data from Salesforce if needed. The function simply takes a query and returns the result as a JSON formatted string:

const queryData = async (query: string): Promise<string> => {
  const authInfo = await AuthInfo.create({ username: flags.username });
  const connection = await Connection.create({ authInfo });
  const result = await connection.query<{ Name: string; Id: string }>(query);

  return JSON.stringify(result);
}

When the command is executed, the request to the Chat Completion API includes the prompt supplied by the user, and details of the function:

const functions: ChatCompletionFunctions[] = [
  {
    name: 'queryData',
    description: 'A function to extract records from Salesforce via a SOQL query',
    parameters: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
          description: 'The SOQL query to execute',
        },
      },
      required: ['query'],
    }
  }
];

Note that my function isn't targeting any specific objects, nor does it have any logic to figure out what might be needed based on the user's prompt - it simply executes a query and returns the results, which might be an error or an empty data set.

I execute this command as follows:

> sf bbai org data -t "Create an email introducing GenePoint, a new account we are tracking in Salesforce. Include the industry and number of employee details from our Salesforce database" -u keir.bowden@googlemail.com

Note that I have to tell the prompt that the GenePoint account details can be found in our Salesforce database - if I don't do that it won't see any value in the function. I've also provided the description of a couple of fields that I want it to extract, and finally I've supplied the username that I'll connect to Salesforce with if the model asks me to run a query.

When the model responds, it will indicate if it wants me to execute the function by specifying the finish_reason as 'function_call', and adding the details in the function_call property of the message:

{
  index: 0,
  message: {
    role: 'assistant',
    content: null,
    function_call: {
      name: 'queryData',
      arguments: '{\n' +
        `  "query": "SELECT Industry, NumberOfEmployees FROM Account 
                     WHERE Name = 'GenePoint' LIMIT 1"\n`
                  }'
    }
  },
  finish_reason: 'function_call'
}

In this case it wants me to call queryData with a query parameter of a SOQL query string that extracts details of the GenePoint account. I execute this, then add the results of the query to the prompt message and retry my request. This time I get a response with the email containing the details I wanted:

Subject: Introducing GenePoint - A Promising Addition to Our Portfolio

Dear Team,

I hope this email finds you well. I am excited to introduce you to our newest account, GenePoint, which we have recently started tracking in our Salesforce database. GenePoint is a biotechnology company that shows immense potential in its field. Allow me to provide you with some important details about this account.

Industry: Biotechnology

Number of Employees: 265

Conclusion

I think this is very cool because I haven't had to inspect the prompt in any way to decide to extract information from Salesforce. The model has been given a very basic function and knows (with some nudging, for sure) when it is appropriate to call it and, more importantly, the query that needs to be run to extract the details the user requested.

Right now I've only told it about a single function, so it's either going to call that or nothing, but I can easily imagine a collection of functions that provide access to many internal systems. This allows the final request to be grounded with a huge amount of relevant data, leading to a highly accurate and targeted response.

Once again a reminder that this could result in sensitive or personally identifiable information being sent to the OpenAI API in the US to be processed, so while it's fun to try out you really don't want to go any where near your production data with this.

More Information

Follow @bob_buzzard

Sunday 30 July 2023

Salesforce CLI Open AI Plug-in - Generating Records

Image generated by Stable Diffusion 2.1, based on a prompt from Bob Buzzard

Introduction

After the go-live of Service and Sales GPT, I felt that I had to revisit my Salesforce CLI Open AI Plug-in and connect it up to the GPT 4 Large Language Model. I didn't succeed in this quest, as while I am a paying customer of OpenAI, I haven't satisfied the requirement of making a successful payment of at least $1. The API function I'm hitting, Create Chat Completion, supports gpt-3.5-turbo and the gpt-4 variants, so once I've racked up enough cost using the earlier models I can switch over by changing one parameter. My current spending looks like it will take me a few months to get there, but such is life with competitively priced APIs.

The Use Case

The first incarnation of the plug-in asks the model to describe Apex, CLI or Salesforce concepts, but I wanted something that was more of a tool than a content generator, so I decided on creating test records. The new command takes parameters listing the field names, the desired output format, and the number of records required, and folds these into the messages passed to the API function. Like the Completion API, the interface is very simple:

const response = await openai.createChatCompletion({
     model: 'gpt-3.5-turbo',
     messages,
     temperature: 1,
     max_tokens: maxTokens,
     top_p: 1,
     frequency_penalty: 0,
     presence_penalty: 0,
   });

result = (response.data.choices[0].message?.content as string);

There's a few more parameters than the Completion API:

model - the Large Language Model that I send the request to. Right now I've hardcoded this to the latest I can access
messages - the collection of messages to send. The messages build on each other, and each message has a content (the instruction/request) and a role (where the instruction is being sent). This allows me to separate the instructions to the model (when the role is assistant, I'm giving it constraints about how to behave) from the request (when the role is user, this is the task/request I'm asking it to carry out).
max_tokens is the maximum number of tokens (approximately 4 characters of text) that my request combined with the response can be. I've set this to 3,500, which is approaching the limit of the gpt-3.5 model. If you have a lot of fields you'll have to generate a smaller number of records to avoid breaching this. I was able to create 50 records with 4-5 fields inside this limit, but your mileage may vary.
temperature and top_p guide the model as to whether I want precise or creative responses.
frequency_penalty and presence_penalty indicate whether I want the model to continually focus on tokens if they are repeated, or focus on new information.

As this is an asynchronous API, I await the response, then pick the first element in the choices array.

Here's a few executions to show it in action - linebreaks have been added to the commands to aid legibility - remove these if you copy/paste the commands.

> sf bbai data testdata -f 'Index (count starting at 1), Name (Text, product name), 
            Amount (Number), CloseDate (Date yyyy-mm-dd), 
            StageName (One of these values : Negotiating, Closed Lost, Closed Won)' 
            -r csv

Here are the records you requested
Index,Name,Amount,CloseDate,StageName
1,Product A,1000,2022-01-15,Closed Lost
2,Product B,2500,2022-02-28,Closed Won
3,Product C,500,2022-03-10,Closed Lost
4,Product D,800,2022-04-05,Closed Won
5,Product E,1500,2022-05-20,Negotiating

> sf bbai data testdata -f 'FirstName (Text), LastName (Text), Company (Text), 
                            Email (Email), Rating__c (1-10)' 
                            -n 4 -r json

Here are the records you requested
[
  {
    "FirstName": "John",
    "LastName": "Doe",
    "Company": "ABC Inc.",
    "Email": "john.doe@example.com",
    "Rating__c": 8
  },
  {
    "FirstName": "Jane",
    "LastName": "Smith",
    "Company": "XYZ Corp.",
    "Email": "jane.smith@example.com",
    "Rating__c": 5
  },
  {
    "FirstName": "Michael",
    "LastName": "Johnson",
    "Company": "123 Co.",
    "Email": "michael.johnson@example.com",
    "Rating__c": 9
  },
  {
    "FirstName": "Sarah",
    "LastName": "Williams",
    "Company": "Acme Ltd.",
    "Email": "sarah.williams@example.com",
    "Rating__c": 7
  }
]

There's a few interesting points to note here:

Formatting field data is conversational - e.g. when I use Date yyyy-mm-dd the model knows that I want the date in ISO8601 format. For picklist values, I just tell it 'One of these values' and it does the rest.
In the messages I asked it to generate realistic data, and while it's very good at this for First Name, Last Name, Email, Company, it's not when told a Name field should be a product name, just giving me Product A, Product B etc.
It sometimes takes it a couple of requests to generate the output in a format suitable for dropping into a file - I'm guessing this is because I instruct the model and make the request in a single API call.
I've generated probably close to 500 records while testing this, and that has cost me the princely sum of $0.04. If you want to play around with the GPT models, it really is dirt cheap.

The final point I'll make, as I did in the last post, is how simple the code is. All the effort went into the messages to ask the model to generate the data in the correct format, not to include additional information that it was responding to the request, to generate realistic data. Truly the key programming language for Generative AI is the supported language that you speak - English in my case!

As before, you can install the plug-in via :

> sf plugins install bbai

or if you have already installed it, upgrade via :

> sf plugins update

More Information

Follow @bob_buzzard

Saturday 22 July 2023

Salesforce GPT - It's Alive!

Image generated by Stable Diffusion 2.1 in response to a prompt by Bob Buzzard

Introduction

This week (19th July) the first of the Salesforce AI Cloud/Einstein GPT applications became generally available. You can read the full announcement on the Salesforce news site, but it's great to see something tangible in the hands of customers after the wave of marketing over the last few months. Its a relatively limited amount of functionality to start with, but I prefer that to waiting another 9 months for everything to be fully built out. GA is better than finished in this case!

What we know so far

We know that it's only available to Unlimited Edition, which already includes the existing Einstein AI features. This seems to be becoming the standard approach for Salesforce - Meetings, for example, was originally Performance and Unlimited Edition only, but is now available for all editions with Sales Cloud. It's a good way of keeping the numbers down without having to evaluate every application, and it's likely to include those customers that are running a lot of their business on Salesforce and thus will get the most value.

We know that it's (initially) a limited set of features that look to be mostly relying on external generative AI systems rather than LLMs trained on your data. The features called out in the announcement include:

Service replies - personalised responses grounded in relevant, real time data sources. To be fair, this could be a model trained on your data, but the term grounded implies that it's an external request to something like Open AI GPT with additional context pulled from Salesforce.
Work Summaries - wrap ups of service cases and customer engagements. The kind of thing that Claude from Anthropic is very good at. These can then be turned into Knowledge Articles, assuming there was anything that could be re-used or repurposed for future customer calls.
Sales Emails - personalised and data-informed emails created for you, again sounding very much like a grounded response from something like OpenAI.

This looks like a smart move by Salesforce, as they can make generative AI available to customers without having to build out the infrastructure to host their own models - something that might present a challenge, given the demand for GPUs across the industry.

We know it will include the Einstein GPT Trust Layer. This is probably the biggest benefit - you could create your own integration with any of these external services, but you'd have to build all the protections in yourself, and the UI that allow admins to configure them.

We don't know what pricing to expect when it becomes available outside of Unlimited Edition, but given that it's included with Einstein there, it may well be included in that SKU for Enterprise Edition, which is $50/user/month for each of Sales and Service Cloud Einstein.

We know it includes "a limited number of credits', which I'm pretty sure was defined as 8,000 in one of the webinars I watched. This sounds like a lot, but we don't know what a credit is, so it might not be. If it's requests, that is quite a few. If it's tokens, not so much - testing my Salesforce CLI integration with OpenAI used around 6,000 tokens for not very many requests. Still, if you built your own integration with any of these tools you'd have to pay for usage, so there's no reason to expect it to be included when going via Salesforce, especially as I'm sure different customers will have wildly different usage. Those 6,000 tokens also cost me around 12 cents, so hopefully purchasing extra credits won't break the bank!

We also know, based on the Service Cloud GPT webinar on 19th July, that we'll be able to add our own rules around PII/sensitive data detection in prompts. It seemed highly likely that would be the case, but good to have it confirmed.

Finally, we know this is just the beginning of the GPT GAs. Dreamforce is less than 2 months away and there will be some big AInnouncements for sure.]

More Information

Follow @bob_buzzard