Bob Buzzard Blog: August 2023

Image generated by Stable Diffusion online, based on a prompt by Bob Buzzard

WARNING: The new command covered in this blog extracts information from your Salesforce database and sends it to OpenAI in the US to provide additional grounding to a prompt - only use this with test/fake data, as there is no attempt at masking or restricting field access.

Introduction

Back in June 2023, only a couple of months ago on the calendar but a lifetime in generative AI product releases, OpenAI announced the availability of function calling. Now that my plug-in is integrated with gpt-3.5+, this is now something I can use, but what value does it add?

The short version - this allows the model to progress past it's training data and request more information to satisfy a prompt.

The longer version. As we all know, the data used to train gpt-3.5 cut off at September 2021, so often the response to a prompt will warn you that things may have changed. With function calling, when you prompt the model you also tell it about any functions that you have available that it can use to retrieve additional information. If the functions don't help it will return the response as usual, but if they would it will return function calls for you to make and pass back to it. Note that the model doesn't call the functions, it tells you which functions to call and the parameters to pass and expects you to make the decision as to whether you should call them, which is as it should be.

The Plug-in Command

In the latest version (1.2.2) of my plug-in, there's a new command that gives the model access to a function to pull data from Salesforce if needed. The function simply takes a query and returns the result as a JSON formatted string:

const queryData = async (query: string): Promise<string> => {
  const authInfo = await AuthInfo.create({ username: flags.username });
  const connection = await Connection.create({ authInfo });
  const result = await connection.query<{ Name: string; Id: string }>(query);

  return JSON.stringify(result);
}

When the command is executed, the request to the Chat Completion API includes the prompt supplied by the user, and details of the function:

const functions: ChatCompletionFunctions[] = [
  {
    name: 'queryData',
    description: 'A function to extract records from Salesforce via a SOQL query',
    parameters: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
          description: 'The SOQL query to execute',
        },
      },
      required: ['query'],
    }
  }
];

Note that my function isn't targeting any specific objects, nor does it have any logic to figure out what might be needed based on the user's prompt - it simply executes a query and returns the results, which might be an error or an empty data set.

I execute this command as follows:

> sf bbai org data -t "Create an email introducing GenePoint, a new account we are tracking in Salesforce. Include the industry and number of employee details from our Salesforce database" -u keir.bowden@googlemail.com

Note that I have to tell the prompt that the GenePoint account details can be found in our Salesforce database - if I don't do that it won't see any value in the function. I've also provided the description of a couple of fields that I want it to extract, and finally I've supplied the username that I'll connect to Salesforce with if the model asks me to run a query.

When the model responds, it will indicate if it wants me to execute the function by specifying the finish_reason as 'function_call', and adding the details in the function_call property of the message:

{
  index: 0,
  message: {
    role: 'assistant',
    content: null,
    function_call: {
      name: 'queryData',
      arguments: '{\n' +
        `  "query": "SELECT Industry, NumberOfEmployees FROM Account 
                     WHERE Name = 'GenePoint' LIMIT 1"\n`
                  }'
    }
  },
  finish_reason: 'function_call'
}

In this case it wants me to call queryData with a query parameter of a SOQL query string that extracts details of the GenePoint account. I execute this, then add the results of the query to the prompt message and retry my request. This time I get a response with the email containing the details I wanted:

Subject: Introducing GenePoint - A Promising Addition to Our Portfolio

Dear Team,

I hope this email finds you well. I am excited to introduce you to our newest account, GenePoint, which we have recently started tracking in our Salesforce database. GenePoint is a biotechnology company that shows immense potential in its field. Allow me to provide you with some important details about this account.

Industry: Biotechnology

Number of Employees: 265

Conclusion

I think this is very cool because I haven't had to inspect the prompt in any way to decide to extract information from Salesforce. The model has been given a very basic function and knows (with some nudging, for sure) when it is appropriate to call it and, more importantly, the query that needs to be run to extract the details the user requested.

Right now I've only told it about a single function, so it's either going to call that or nothing, but I can easily imagine a collection of functions that provide access to many internal systems. This allows the final request to be grounded with a huge amount of relevant data, leading to a highly accurate and targeted response.

Once again a reminder that this could result in sensitive or personally identifiable information being sent to the OpenAI API in the US to be processed, so while it's fun to try out you really don't want to go any where near your production data with this.

More Information

Follow @bob_buzzard

Bob Buzzard Blog

Sunday 6 August 2023

Salesforce CLI Open AI Plug-in - Function Calling

Introduction

The Plug-in Command

Conclusion