Showing posts with label ai cloud. Show all posts
Showing posts with label ai cloud. Show all posts

Sunday, 6 August 2023

Salesforce CLI Open AI Plug-in - Function Calling

Image generated by Stable Diffusion online, based on a prompt by Bob Buzzard

WARNING: The new command covered in this blog extracts information from your Salesforce database and sends it to OpenAI in the US to provide additional grounding to a prompt - only use this with test/fake data, as there is no attempt at masking or restricting field access.

Introduction

Back in June 2023, only a couple of months ago on the calendar but a lifetime in generative AI product releases, OpenAI announced the availability of function calling. Now that my plug-in is integrated with gpt-3.5+, this is now something I can use, but what value does it add? 

The short version - this allows the model to progress past it's training data and request more information to satisfy a prompt. 

The longer version. As we all know, the data used to train gpt-3.5 cut off at September 2021, so often the response to a prompt will warn you that things may have changed. With function calling, when you prompt the model you also tell it about any functions that you have available that it can use to retrieve additional information. If the functions don't help it will return the response as usual, but if they would it will return function calls for you to make and pass back to it. Note that the model doesn't call the functions, it tells you which functions to call and the parameters to pass and expects you to make the decision as to whether you should call them, which is as it should be.

The Plug-in Command

In the latest version (1.2.2) of my plug-in, there's a new command that gives the model access to a function to pull data from Salesforce if needed. The function simply takes a query and returns the result as a JSON formatted string:

const queryData = async (query: string): Promise<string> => {
  const authInfo = await AuthInfo.create({ username: flags.username });
  const connection = await Connection.create({ authInfo });
  const result = await connection.query<{ Name: string; Id: string }>(query);

  return JSON.stringify(result);
}

When the command is executed, the request to the Chat Completion API includes the prompt supplied by the user, and details of the function:

const functions: ChatCompletionFunctions[] = [
  {
    name: 'queryData',
    description: 'A function to extract records from Salesforce via a SOQL query',
    parameters: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
          description: 'The SOQL query to execute',
        },
      },
      required: ['query'],
    }
  }
];

Note that my function isn't targeting any specific objects, nor does it have any logic to figure out what might be needed based on the user's prompt - it simply executes a query and returns the results, which might be an error or an empty data set.

I execute this command as follows:

> sf bbai org data -t "Create an email introducing GenePoint, a new account we are tracking in Salesforce. Include the industry and number of employee details from our Salesforce database" -u keir.bowden@googlemail.com

Note that I have to tell the prompt that the GenePoint account details can be found in our Salesforce database - if I don't do that it won't see any value in the function. I've also provided the description of a couple of fields that I want it to extract, and finally I've supplied the username that I'll connect to Salesforce with if the model asks me to run a query.

When the model responds, it will indicate if it wants me to execute the function by specifying the finish_reason as 'function_call', and adding the details in the function_call property of the message:

{
  index: 0,
  message: {
    role: 'assistant',
    content: null,
    function_call: {
      name: 'queryData',
      arguments: '{\n' +
        `  "query": "SELECT Industry, NumberOfEmployees FROM Account 
                     WHERE Name = 'GenePoint' LIMIT 1"\n`
                  }'
    }
  },
  finish_reason: 'function_call'
}

In this case it wants me to call queryData with a query parameter of a SOQL query string that extracts details of the GenePoint account. I execute this, then add the results of the query to the prompt message and retry my request. This time I get a response with the email containing the details I wanted:

Subject: Introducing GenePoint - A Promising Addition to Our Portfolio

Dear Team,

I hope this email finds you well. I am excited to introduce you to our newest account, GenePoint, which we have recently started tracking in our Salesforce database. GenePoint is a biotechnology company that shows immense potential in its field. Allow me to provide you with some important details about this account.

Industry: Biotechnology

Number of Employees: 265

Conclusion

I think this is very cool because I haven't had to inspect the prompt in any way to decide to extract information from Salesforce. The model has been given a very basic function and knows (with some nudging, for sure) when it is appropriate to call it and, more importantly, the query that needs to be run to extract the details the user requested. 

Right now I've only told it about a single function, so it's either going to call that or nothing, but I can easily imagine a collection of functions that provide access to many internal systems. This allows the final request to be grounded with a huge amount of relevant data, leading to a highly accurate and targeted response.

Once again a reminder that this could result in sensitive or personally identifiable information being sent to the OpenAI API in the US to be processed, so while it's fun to try out you really don't want to go any where near your production data with this.

More Information








Saturday, 22 July 2023

Salesforce GPT - It's Alive!

Image generated by Stable Diffusion 2.1 in response to a prompt by Bob Buzzard

Introduction

This week (19th July) the first of the Salesforce AI Cloud/Einstein GPT applications became generally available. You can read the full announcement on the Salesforce news site, but it's great to see something tangible in the hands of customers after the wave of marketing over the last few months. Its a relatively limited amount of functionality to start with, but I prefer that to waiting another 9 months for everything to be fully built out. GA is better than finished in this case!

What we know so far

We know that it's only available to Unlimited Edition, which already includes the existing Einstein AI features. This seems to be becoming the standard approach for Salesforce - Meetings, for example, was originally Performance and Unlimited Edition only, but is now available for all editions with Sales Cloud. It's a good way of keeping the numbers down without having to evaluate every application, and it's likely to include those customers that are running a lot of their business on Salesforce and thus will get the most value. 

We know that it's (initially) a limited set of features that look to be mostly relying on external generative AI systems rather than LLMs trained on your data. The features called out in the announcement include:

  • Service replies - personalised responses grounded in relevant, real time data sources. To be fair, this could be a model trained on your data, but the term grounded implies that it's an external request to something like Open AI GPT with additional context pulled from Salesforce.
  • Work Summaries - wrap ups of service cases and customer engagements. The kind of thing that Claude from Anthropic is very good at. These can then be turned into Knowledge Articles, assuming there was anything that could be re-used or repurposed for future customer calls.
  • Sales Emails - personalised and data-informed emails created for you, again sounding very much like a grounded response from something like OpenAI.
This looks like a smart move by Salesforce, as they can make generative AI available to customers without having to build out the infrastructure to host their own models - something that might present a challenge, given the demand for GPUs across the industry.

We know it will include the Einstein GPT Trust Layer. This is probably the biggest benefit - you could create your own integration with any of these external services, but you'd have to build all the protections in yourself, and the UI that allow admins to configure them.

We don't know what pricing to expect when it becomes available outside of Unlimited Edition, but given that it's included with Einstein there, it may well be included in that SKU for Enterprise Edition, which is $50/user/month for each of Sales and Service Cloud Einstein.

We know it includes "a limited number of credits', which I'm pretty sure was defined as 8,000 in one of the webinars I watched. This sounds like a lot, but we don't know what a credit is, so it might not be. If it's requests, that is quite a few. If it's tokens, not so much - testing my Salesforce CLI integration with OpenAI used around 6,000 tokens for not very many requests. Still, if you built your own integration with any of these tools you'd have to pay for usage, so there's no reason to expect it to be included when going via Salesforce, especially as I'm sure different customers will have wildly different usage. Those 6,000 tokens also cost me around 12 cents, so hopefully purchasing extra credits won't break the bank!

We also know, based on the Service Cloud GPT webinar on 19th July, that we'll be able to add our own rules around PII/sensitive data detection in prompts. It seemed highly likely that would be the case, but good to have it confirmed.

Finally, we know this is just the beginning of the GPT GAs. Dreamforce is less than 2 months away and there will be some big AInnouncements for sure.]

More Information