Salesforce CLI OpenAI Plug-in

Tweet

Image generated using StableDiffusion 2.1 via https://huggingface.co/spaces/stabilityai/stable-diffusion

Introduction

I've finally found time to have a play around with the OpenAI API, and was delighted to find that it has a NodeJS library, so I can interact with it via Typescript or JavaScript rather than learning Python. Not that I have any issue with learning Python, but it's not something that is that useful for my day to day work and thus not something I want to invest effort in right now.

As I've said many times in the past, anyone who knows me knows that I love a Salesforce CLI Plug-in, and that is where my mind immediately went once I'd cloned their Github repo and been through the (very) quick overview. 

The Plug-In

I wasn't really interested in surfacing a chat bot via the Salesforce CLI - for a start, it wouldn't really add anything over and above the public OpenAI chat page. 

The first area I investigated was asking it to review Apex classes and call out any inefficiencies or divergence from best practice. While this was entertaining, and found some real issues, it was also wildly inaccurate (and probably something my Evil Co-Worker will move forward with to mess with everyone). 

I then built out a couple of commands to generate titles and abstracts for conference sessions based on Salesforce development concepts, but as the Dreamforce call for papers is complete for this year, that didn't seem worth the effort.

I landed on a set of explainer commands that would ask the AI model to explain something to help a more junior developer:

• apex - explain a topic in the context of the Apex programming language
• cli - explain a Salesforce CLI command, along with an example
• salesforce - explain a Salesforce topic using appropriate language for a specific audience - admins, devs, execs etc.

Accessing the OpenAI API is very straightforward:

Install the OpenAI Package

> npm install openai

Create the API instance

    import { Configuration, OpenAIApi } from 'openai';

    const configuration = new Configuration({
        apiKey: process.env.OPENAI_API_KEY,
    });
    const openai = new OpenAIApi(configuration);

Note that the API Key comes from the OPENAI_API_KEY environment variable - if you want to try out the plug-in yourself you'll need an OpenAI account and your own key. 

Create the Completion

This is where the prompt is passed to the model so that it can generate a response:

    const completion = await openai.createCompletion({
        model: 'text-davinci-003',
        max_tokens: maxTokens,
        prompt,
        temperature
      });

      result = (completion.data.choices[0].text as string);

Note that the completions API is legacy and received it's final update in July 2023. I chose this simply because it's cheap. When you are writing a new plug-in connecting to a third party system you get a lot of stuff wrong and I was keen not to rack up too large a bill while I was learning! The Chat Completions API is slightly different, in that it receives a list of messages and can call functions with results. It's not dramatically different though, so I felt anything I learned applied to both.

I moved this into a shared function, hence most of the call is parameterised. The createCompletion function can take many parameters, but here's the explanation of those that I've used:

• model - the large language model to use
• max_tokens - the maximum number of tokens to generate the response. A token is typically thought to be around 4 characters, so will sometimes represent words and sometimes parts of words. Input and output token counts are how you get charged for API use, so you want to keep them down where you can. The function defaults to 16 which in practice is fairly useless for a response to a human, so I've typically upped it to 256
• prompt - the question/task that I've set the model
• temperature - a value between 0 and 2.0 that indicates how accurate a response I want. I find that 0.6 works pretty well. If you change this, do it in small increments, as I asked it to review an Apex class with a temperature of 1.5 and got pretty much gibberish.

And that's about it. The rest of the code is available at the Github repository - it's mostly standard CLI Plug-In stuff of handling flags and displaying the response.

The Plug-In In Action

Once I've installed the plug-in from NPM, using the following command :

> sf plugins install bbai

I can ask for some explanations. First, explain the Salesforce concept of groups. As I don't provide details of the audience, it defaults to programmers:

> sf bbai explain salesforce -t groups

Here are the results

Groups in Salesforce are a way to organize and manage users in the system. They allow you to categorize users based on their roles and responsibilities, allowing for more streamlined access to information. For example, you can create a group for all the programmers within your organization, allowing them to access the same resources, documents, and tasks. Additionally, groups can be used to assign tasks, send messages, and create reports. By creating groups, you can quickly and easily manage permissions, tasks, and data access for your programming team.

Now the same command for an exec audience - notice the difference?

> sf bbai explain salesforce -t groups -s executive

Here are the results

Groups in Salesforce allow organizations to manage and control access to records and data. Groups are used to assign users to roles within the organization and to grant them access to the appropriate records. By using groups, organizations can easily assign the same set of permissions to multiple users, allowing them to quickly and easily manage access rights. This makes it easier to manage user permissions, and also helps to ensure that users only have access to the data and records they need to do their job. Groups also streamline the process of granting and revoking access rights, making it easier for organizations to stay secure and compliant.

And now I want to start working with some of these concepts, but I don't understand how to get my code into the instance via the CLI:

> sf bbai explain cli -c "source push"             

Here are the results

Salesforce CLI command source push is used to deploy source from the local environment (e.g. a developer’s machine) to a Salesforce org. This command is useful in a DevOps process, as it enables developers to quickly and easily deploy their code changes to a Salesforce org for testing and deployment.

Example Execution: 

sfdx force:source:push --targetusername devorg --sourcepath /path/to/source/directory/

The command above will deploy all source code in the specified directory to the org with the username devorg.

Conclusions

Something that really stood out in this was the simplicity of the integration - it's literally a few lines of code. Which is as it should be - with generative AI, English is the programming language so most of the effort should go into the prompt rather than getting set up to send the prompt. Looking ahead a few years, I can see this being a game changer for integrating systems. Rather than starting with a book of API specifications that you need to adhere to, there will be a single entry point where you send conversational requests to a model trained on the specific data. "Return the id and all name and address fields of all the contacts we have that work for financial services customers in JSON format", without needing to develop that specific interface.

The Cost

This was top of mind for me during this experiment. I've seen situations in the past where request based charging really racked up during testing. OpenAI allows you to set a hard and soft limit, so that you can't get into that situation, but it also charges by the token for both input and output. When you aren't that familiar with how many tokens might be in a 2-3 line prompt. After building my plug-in, with the usual number of mistakes, and testing it for several hours, I'd used the princely sum of $0.16. 

While that might sound like there's nothing to worry about, but the latest models are more expensive per token and can handle significantly more tokens both for input and output, so you may find that things stack up quickly. I've seen news stories lauding the ability to send an entire book to a generative AI as context, but no mention of how much you'll be charged to do that!

More Information

• Github repository for the plug-in
• Plug-in landing page on NPMJS
• OpenAI API Documentation
• Salesforce World Tour London : AI Day
• Predictions for Digital Life in 2035
• Einstein GPT - Rise of the Machines?

Follow @bob_buzzard

Adding IP Ranges from a Salesforce CLI Plug-in

Tweet

Introduction

I know, another week, another post about a CLI plug-in. I really didn't intend this to be the case, but I recently submitted a (second generation) managed package for security review and had to open up additional IP addresses to allow the security team access to a test org. I'm likely to submit more managed packages for review in the future, so I wanted to find a scriptable way of doing this. 

It turned out to be a little more interesting than I'd originally expected, for a couple of reasons.

It Requires the Metadata API

To add an IP range, you have to deploy security settings via the metadata API. I have done this before, to enable/disable parallel Apex unit tests, but this was a little different. If I create a security settings file with the new range(s) and deploy them, per the Metadata API Docs, all existing IP ranges will be turned off:

    To add an IP range, deploy all existing IP ranges, including the one you
    want to add. Otherwise, the existing IP ranges are replaced with the ones
    you deploy. 

Definitely not what I want!

It Requires a Retrieve and Deploy

In order to get the existing IP addresses, I have to carry out a metadata retrieve of the security settings from the Salesforce org, add the ranges, then deploy them. No problem here, I can simply use the retrieve method on the metadata class that I'm already using to deploy. Weirdly, unlike the deploy function, the retrieve function doesn't return a promise, instead it expected me to supply a callback. I couldn't face returning to callback hell after the heaven of async/await in my plug-in development, so I used the Node Util.Promisify function that turns it into a function that returns a promise. Very cool.

const asyncRetrieve = promisify(conn.metadata.retrieve);
const retrieveCheck = await asyncRetrieve.call(...);

The other interesting aspect is that I get the settings back in XML format, but I want a JavaScript object to manipulate, which I then need to turn back into XML to deploy.

To turn XML into JavaScript I use fast-xml-parser, as this has stood me in good stead with my Org Documentor. To get at the NetworkAccess element:

import { parse} from 'fast-xml-parser';

  ...

const settings = readFileSync(join(tmpDir, 'settings', 'Security.settings'), 'utf-8');
const md = parse(settings);

let networkAccess = md.SecuritySettings.networkAccess;

Once I've added my new ranges, I convert back to XML using xml2js:

import { Builder } from 'xml2js';

  ...

const builder = new Builder(
   {renderOpts:
      {pretty: true,
       indent: '    ',
       newline: '\n'},
    stringify: {
       attValue(str) {
           return str.replace(/&/g, '&')
                     .replace(/"/g, '"')
                     .replace(/'/g, ''');
       }
    },
    xmldec: {
        version: '1.0', encoding: 'UTF-8'
    }
});

const xml = builder.buildObject(md);

The Plug-In

This is available as a new command on the bbsfdx plug-in - if you have it installed already, just run 

sfdx plugins: update 

to update to version 1.4

If you don't have it installed already (I won't lie, that hurts) you can run:

sfdx plugins:install bbsfdx
 

and to add one or more ranges: 

sfdx bb:iprange:add -r 192.168.2.1,192.168.2.4:192.168.2.255 -u <user>

The -r switch defines the range(s) - comma separate them. For a range, separate the start and end addresses with a colon. For a single address, just skip the colon and the end address.

Related Posts

• The Plug-in on NPMJS
• The Plug-in Source on Github
• New bbsfdx Plug-in Cleanup Commands
• Generating PDFs with a Salesforce CLI Plug-in
• Org Documentor and AuraEnabled Classes
• Offline mobile app template plug-in Dreamforce 18 session
• Salesforce CLI Play-by-Play
• Salesforce CLI Cheat Sheet

Follow @bob_buzzard

Org Documentor and AuraEnabled Classes

Tweet

Introduction

In the Winter 21 release of Salesforce, access to AuraEnabled methods in Apex classes will be restricted to users with profiles or permission sets that grant access to those classes. If you've been working in a sandbox recently you've probably encountered this already, as the critical update for this was enabled on August 8th.

Figuring out who (if anyone) has access to classes isn't straightforward, as you have to trawl through the profiles and permission sets and check each one. Salesforce have created an Aura Enabled Scanner application that can be installed via an unlocked package, which checks packaged and unpackaged code, but it does require that you login to Salesforce each time you need to check things.

This seemed like a good candidate for my Org Documentor Salesforce CLI Plug-In - something that can be run on a schedule, check that any new classes used as controllers for Aura or Lightning Web Components are accessible, and show which profiles/permission sets have access.

Configuration

To add processing for AuraEnabled classes, an additional stanza is required in the configuration file passed to the bbdoc command - here's what it looks like in my example repo:

"auraenabled": {
    "name": "auraenabled",
    "description": "AuraEnabled Class Access", 
    "subdirectory": ".",
    "image": "images/auraenabled.png",
    "suffix":"object",
    "groups": {
        "other": {
            "name":"other", 
            "title":"AuraEnabled Components",
            "description": "All components with Apex controllers"
        }
    }
}

As with other metadata types, you can specify multiple groups to slice up your components into functional areas - I've lumped them all into one group as I only have one component!

Output

The index (home) page for the org report displays a new card for the AuraEnabled metadata:

An error badge is displayed if there are one or more classes used as controllers for components that aren't accessible from any profile or permission set. 

Clicking in to the details shows the groups and errors:

and clicking into the group shows the detail for each component, with classes that aren't accessible highlighted as errors:

Note that if a Lightning Web Component accesses multiple Apex classes, there will be a row for each class with the same component name in the report, as shown above.

Processing

I was quite pleased by how little code I had to write to add support for this:

• A classes map structure is created in memory, profilesAndPermSetsByClassname, where each entry has the Apex class name as a key and a value object containing lists of permission sets and profiles with access to the class. This is generated by loading all of the profile and permission set metadata and iterating the ClassAccess sections. 
  
  
• The component groups are iterated, and for each entry in the group:
  
  
• If it is an Aura Component, the controller attribute from the <aura:component/> tag is extracted
  
  
• If it is a Lighting Web Component (which can access multiple Apex classes), @salesforce/apex/ lines are identified and the Apex class name extracted. 
  
  For each class:
  
  
• The entry for the class in the profilesAndPermSetsByClassname map is extracted. If this is null or both of the profile and permission set lists are empty, the classname is added to the error collection displayed on the group page.
  
  
• A row is added to the JavaScript object backing the group page showing the profiles/permission sets that have access, adding the error highlight colour if there are none.

Plug-In

The updated bbddoc Salesforce CLI plug-in providing AuraEnabled support can be found on NPM, and the source code is available in the Github repository. 

Why not use the AuraEnabledScanner?

You absolutely should - I do, to configure access. This works with the AuraEnabledScanner rather than replacing it. As development continues, new classes and components are added to your codebase. The Org Documentor flags up any that don't have appropriate access, and you can then login to an org that your metadata is deployed to, run the scanner and fix up the access. 

The scanner also handles managed classes which the Org Documentor doesn't, as it works against your metadata on disk rather than everything installed in a specific org instance.

Related Posts

• The plug-in on NPMJS
• Documentation generated from the sample metadata
• Sample metadata repository
• Plug-in source repository
• Documentor Plugin - Triggers (and Bootstrap)
• Documenting from the metadata source with a Salesforce CLI Plug-in - Part 5
• Documenting from the metadata source with a Salesforce CLI Plug-in - Part 4
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 3
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 2
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 1
• Offline mobile app template plug-in Dreamforce 18 session
• Salesforce CLI Play-by-Play
• Salesforce CLI Cheat Sheet

Follow @bob_buzzard

Generating PDFs with a Salesforce CLI Plug-In

Tweet

Introduction

Yeah, I know, another week another plug-in. Unfortunately this is likely to continue for some time, as my  plug-ins are only really limited by my imagination. I think this one is pretty cool though and I hope, dear reader, you agree.

At Dreamforce 2019, Salesforce debuted Evergreen - a technology to allow Heroku microservices to be invoked from Apex (and other places) seamlessly. One of the Evergreen use cases is creating a PDF from Salesforce data, and one of the strap lines around this is that with Evergreen you have access to the entire Node package ecosystem. Every time I heard this I'd think that's true, but CLI plug-ins have the same access, as they are built on node, and like Evergreen there is no additional authentication required.

Essentially this is the opposite of Evergreen - rather than invoking something that lives on another server that returns a PDF file, I invoke a local command that writes a file to the local file system. I toyed with the idea of calling this plug-in Deciduous, but decided that was but of a lengthy name to type, and I'd have to explain it a lot! (and yes I know that Evergreen is a lot more than creating a PDF, but I liked the image so decided to go with it).

The plug-in is available on NPM - you can install it by executing : sfdx plugins:install bbpdf

EJS Templates

There are a couple of ways to create PDFs with node - using something like PDFKit to add the individual elements to the document, or my preferred route - generating the document from an HTML page. And if I'm generating HTML from Salesforce data or metadata, I'm going to be using EJS templates.

I'm not going to go into too much detail in this post about the creation of the HTML via EJS, but if you want more information check out my blog post on how I used it in my bbdoc plug-in to create HTML documents from Salesforce metadata.  

My plug-in requires two flags to specify the template. The name of the template itself, specified via the --template/-t flag, and the templates directory, specified via the --template-dir/-d flag. The reason I need the directory is that any css or images required by the template will be specified relative to the template's location. You can see this working by cloning the samples repo and trying it out.

Salesforce Data

My plug-in provides two mechanisms to retrieve data from Salesforce.

Query Parameter

This is for the simple use case where the template requires a single record. The query to retrieve the record is supplied via the --query/-q flag.

Querying Salesforce data from a plug-in is quite straightforward, I specify that I need a username which guarantees me access to an org:

protected static requiresUsername = true;

then I create a connection:

const conn = this.org.getConnection();

and finally I run the query, based on the supplied parameter,  and check the results:

const result = await conn.query<object>(this.flags.query);

if (!result.records || result.records.length <= 0) {
   ...
}

Once I have the record I need to pass in the Salesforce data in an object to the template, with the property name expected by the template, and this is provided by the user through the --sobject/-s flag.

This is a bit clunky though, so here's the preferred route.

Query File

A query file is specified via the --query-file/-f flag.  This is a JSON format file containing an object with a property per query:

{
    "contact": {
        "single": true,
        "query": "select Title, FirstName, LastName from Contact where id='00380000023TUDeAAO'"
    },
    "account": {
        "single": true,
        "query": "select id, Name from Account where id='0018000000eqTV7AAM'"
    }
}

The property name is the name that will be supplied to the EJS template - in this example there will be two properties passed to the template - contact and account.  The single property specifies whether the query will result in a single record or an array, and the query property specifies the actual SOQL query that will be executed. Using this mechanism, any number of records or arrays of records can be passed through to the template.

Generating the PDF

In a recurring theme in this post, there are a couple of ways to generate the PDF from the HTML. The simplest in terms of setup and code is to use the html-pdf package, which is a wrapper around PhantomJS. While simple, this is sub-optimal - the package hasn't been updated for a while, development on Phantom is suspended and when I added this to my plug-in, npm reported a critical vulnerability, so clearly another way would be required.

The favoured mechanism on the interwebs was to use Puppeteer (headless chromium node api), so I set about this. After some hoop jumping around creating temporary directories and copying the templates and associated files around, it wasn't actually too bad. I loaded the HTML from the file that I'd created and then asked for a PDF version of it:

const page = await browser.newPage();

await page.goto('file:///' + htmlFile, {waitUntil: 'networkidle0'});
const pdf = await page.pdf({ format: 'A4' });

One area I did go off-piste slightly was not using chromium bundled with puppeteer, instead I rely on the user defining an environment variable for their local installation of chrome and I use that. Mainly because I didn't want my plug-in to have to download 2-300 Mb before it could be used. This may cause problems down the line, so caveat emptor.

The environment variable in question is PUPPETEER_EXECUTABLE_PATH. Here are the definitions from my Macbook Pro running MacOS Catalina:

export PUPPETEER_EXECUTABLE_PATH="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"

and my Surface Pro running Windows 10:

setx PUPPETEER_EXECUTABLE_PATH "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe"

The PDF is then written to the location specified by the --output/-o flag.

The next hoop to jump through came up when I installed the plug-in - Puppeteer is currently at 3.3.0 which requires Node 10.18.1 but the Salesforce CLI node version is 10.15.3. Luckily I was able to switch to Puppeteer version 2.1.1 without introducing any further issues.

Samples Repo

There's a lot of information in this post and a number of flags to wrap your head around - if you'd like to jump straight in with working examples, check out the samples repo. This has a couple of example commands to try out - just make sure you've set your chrome location correctly - and the PDFs that were generated when I ran the plug-in against my dev orgs.

Related Posts

• The Plug-In on NPMJS
• Source for the Plug-In
• Samples Repo
• Documentor Plug-In - Triggers (and Bootstrap)
• Documenting from the metadata source with a Salesforce CLI Plug-in - Part 5
• Documenting from the metadata source with a Salesforce CLI Plug-in - Part 4
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 3
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 2
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 1
• Offline mobile app template plug-in Dreamforce 18 session
• Salesforce CLI Play-by-Play
• Salesforce CLI Cheat Sheet

Documentor Plugin - Triggers (and Bootstrap)

Tweet

Introduction

In previous posts about documenting my org with a Salesforce CLI plugin the focus has been on extracting and processing the object metadata. Now that area has been well and truly covered, it's time to move on. The next target in my sights is triggers - not for any particular reason, just that the real projects that I use this on have triggers that I want to show in my docs!

Trigger Metadata

Trigger metadata is split across two files - the .trigger file, which contains the actual Apex trigger code, and the .trigger-meta.xml file, which contains the supporting information - whether it is active, it's API version. The .trigger file itself contains the object that it is acting on and the action(s) that will cause it to fire:

trigger Book_ai on Book__c (before insert) {

}

Luckily the syntax is rigid - the object comes after the on keyword and the actions appear in brackets after that, so just using standard JavaScript string methods I can locate the boundaries and pull out the information that I want, but not just to display it.

Checking for Duplicates

As we all know, when there is more than one trigger for an object and action, the order they will fire in is not guaranteed. If you are a gambling aficionado then you may find this enjoyable, but most of us prefer a dull, predictable life when it comes to our business automation.

Once I've processed the trigger bodies and extracted the object names and actions, one of the really nice side-effects is that I can easily figure out if there are any duplicates and I can call these out on the triggers page (I've updated the sample metadata to add duplicate triggers - they don't actually do anything so wouldn't really be an issue anyway!) :

Bootstrap

Those that have been following this series of posts may notice that the styling looks a little better now than in earlier versions - this is because I've moved over to the Bootstrap library, pulled in via the CDN so I don't need to worry about including it in my plug-in. 

I've also updated the main page to use cards with badges to make it clear if there are any problems with the processed metadata :

Which I think we can all agree looks a lot better. You can see the latest version of the HTML generated from the sample metadata at https://bbdoc-example.herokuapp.com/index.html

I've also moved some duplicated markup out to an EJS include, which is as easy as I hoped it would be. The markup goes into a file with a .ejs suffix like any other - here's the footer from my pages:

<div class="pt-3"></div>
<footer class="footer">
    <div class="container">
        <div class="row justify-content-between">
            <div class="col-6">
                <p class="text-left"><small class="text-muted">Generated <%=footer.generatedDate%></small>
                </p>
            </div>
            <div class="col-6">
                <p class="text-right"><small class="text-muted">Bob Buzzard's Org Documentor
                        v<%=footer.version%></small></p>
            </div>
        </div>
    </div>
</footer>

And I can include this just by specifying the path:

<%- include ('../common/footer') %>

Note that I've put the footer in a common subdirectory that is a sibling of where the template is stored.

Note also that I've used the <%- tag which says to include the raw html output from the include. Finally, note that I don't pass anything to the footer - it has access to exactly the same properties as the page that includes it. 

As usual, I've tested the plug-in against the sample metadata repo on MacOS and Windows 10. You can find the latest version (3.3.1) on NPMJS at the link shown below.

Related Posts

• The plug-in on NPMJS
• Documentation generated from the sample metadata
• Sample metadata repository
• Plug-in source repository
• Documenting from the metadata source with a Salesforce CLI Plug-in - Part 5
• Documenting from the metadata source with a Salesforce CLI Plug-in - Part 4
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 3
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 2
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 1
• Offline mobile app template plug-in Dreamforce 18 session
• Salesforce CLI Play-by-Play
• Salesforce CLI Cheat Sheet

Follow @bob_buzzard

 

Documenting from the metadata source with a Salesforce CLI Plug-In - Part 5

Tweet

Introduction

In Part 1 of this series, I explained how to generate a plugin and clone the example command.
In Part 2 I covered finding and loading the package.json manifest and my custom configuration file.
In Part 3, I explained how to load and process source format metadata files.

In Part 4, I showed how to enrich fields by pulling in information stored outside of the field definition file.

In this week's exciting episode, I'll put it all together and generate the HTML output. This has been through several iterations :

• When I first came up with the idea, I did what I usually do which is hardcode some very basic HTML in the source code, as at this point I'm rarely sure things are going to work.
• Prior to my first presentation of the plug-in, I decided to move the HTML out to files that were easily changed, but still as fragments. This made showing the code easier, but meant I had to write a fair bit of boilerplate code, but was all I really had time for.
• Once I'd done the first round of talks, I had the time to revisit and move over to a template framework, which was always going to be the correct solution. This was the major update to the plug-in I referred to in the last post.

Template Framework

I'd been using EJS in YASP (Yet Another Side Project) and liked how easy it was to integrate. EJS stands for Easy JavaScript templates, and it's well named. You embed plain old JavaScript into HTML markup and call a function passing the template and the data that needs to be processed by the JavaScript. If you've written Visualforce pages or email templates in the past, this pattern is very familiar (although you have to set up all of the data that you need prior to passing it to the generator function - it won't figure out what is needed based on what you've used like Visualforce will!).

A snippet from my HTML page that lists the groups of objects:

<% content.groups.forEach(function(group){ %>
    <tr>
        <td><%= group.title %></td>
        <td><a href="<%= group.link %>">Click to View</a></td>
    </tr>
<% }); %>

and as long as I pass the function that generates the HTML an object with a property of content, that matches the expected structure, I'm golden.

To use EJS in my plug-in, I install the node module,  :

npm install --save ejs

(Note that I'm using the --save option, as otherwise this won't be identified as a dependency and thus won't be added to the plug-in. Exactly what happened when I created the initial version and installed it on my Windows machine!)

I add the following line to my Typescript file to import the function that renders the HTML from the template:

import { renderFile } from 'ejs';

and I can then call renderFile(template, data) to asynchronously generate the HTML.

Preparing the Data

As Salesforce CLI plug-ins are written in Typescript by default, it's straightforward to keep track of the structure of your objects as you can strongly type them. Continuing with the example of my groups, the data for these is stored in an object with the following structure:

interface ObjectsContent {
    counter: number;
    groups: Array<ObjectGroupContent>;
    footer?: object;
}

Rather than pass this directly, I store this as the content of a more generic object that includes some standard information, like the plug-in version and the date the pages were generated:

let data={
             content: content,
             footer: 
             {
                 generatedDate : this.generatedDate,
                 version : this.config.version
             }
         };  

Rendering the HTML

Once I have the data in the format I need, I execute the rendering function passing the template filename - note that as this is asynchronous I have to return a promise to my calling function:

return new Promise((resolve, reject) => {
    renderFile(templateFile, data, (err, html) => {
        if (err) {
            reject(err);
        }
        else {
            resolve(html);
        }
    })
})  

and my calling function loops through the groups and gets the rendered HTML when the promise resolves, which it writes to the report directory:

this.generator.generateHTML(join('objects', 'objects.ejs'), this.content)
.then(html => {
    writeFileSync(this.indexFile, html);    
});

Moar Metadata

The latest version of the plugin generates HTML markup for an object's Validation Rules and Record types - the example repo of Salesforce metadata has been updated to include these and the newly generated HTML is available on Heroku at : https://bbdoc-example.herokuapp.com/index.html

Related Posts

• Documenting from the metadata source with a Salesforce CLI Plug-in - Part 4
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 3
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 2
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 1
• Parallel Apex Unit Tests and Salesforce CLI Plug-Ins
• Going GUI over the Salesforce CLI Part 2
• Going GUI over the Salesforce CLI
• Mentz - the Mentee Workflow
• Offline mobile app template plug-in Dreamforce 18 session
• Salesforce CLI Play-by-Play
• Salesforce CLI Cheat Sheet

 Follow @bob_buzzard

Documenting from the metadata source with a Salesforce CLI Plug-In - Part 3

Tweet

Introduction

In Part 1 of this series, I explained how to generate a plugin and clone the example command. In Part 2 I covered finding and loading the package.json manifest and my custom configuration file

In this instalment, I'll show how to load and process source format metadata files - a key requirement when documenting from the metadata source, I'm sure you'll agree.

Walking Directories

In order to process the metadata files, I have to be able to find and iterate them. As I'm processing object files, I also need to process the fields subdirectory that contains the metadata for the custom fields that I'm including in my document. I'll use a number of standard functions provided by Node in order to achieve this.

I know the top level folder name for the source format data, as it is the parameter of the -s (--source-dir) switch passed to my plugin command. To simplify the sample code, I'm pretending I know that the custom objects are in the objects subfolder, which isn't a bad guess, but in the actual code this information is pulled from the configuration file. To generate the combined full pathname, I use the standard path.join function, as this figures out which operating system I'm running on and uses the correct separator:

import { join } from 'path';
...
let objSourceDir=join(sourceDir, 'objects');

I then check that this path exists and is indeed a directory - this time while I use the standard fs.lstatSync to check the path is a directory, I use a homegrown function to check that the path exists:

import { lstatSync} from 'fs';
import { fileExists } from '../../shared/files';
 ...
if ( (fileExists(objSourceDir)) && 
     (lstatSync(objSourceDir).isDirectory()) ) {

Once I know the path is there and is a directory, I can read the contents, again using a standard function - fs.readdirSync - and iterate them :

let entries=readdirSync(objSourceDir);
import { appendFileSync, mkdirSync, readdirSync } from 'fs';
  ...
for (let idx=0, len=entries.length; idx<len; idx++) {
    let entry=entries[idx];
    let entryPath=join(objSourceDir, entries[idx]);
}

Reading Metadata

The Salesforce metadata is stored as XML format files, which presents a slight challenge when working in JavaScript. I can use the DOM parser, but I find the code that I end up writing looks quite ugly, with lots of chained functions. My preferred solution is to parse the XML into a JavaScript object using a third party tool - there are a few of these around, but my current favourite is fast-xml-parser because

• it's fast (clue is in the name)
• it's synchronous - this isn't a huge deal when writing a CLI plugin, as the commands are async and can thus await asynchronous functions, but when I'm working with the command line I tend towards synchronous
• it's popular - 140k downloads this week
• it's MIT licensed

It's also incredibly easy to use. After installing by running npm install --save, I just need to import the parse function, load the XML format file into a string and parse it!

import { parse} from 'fast-xml-parser';

let objFileBody=readFileSync(
join(entryPath, entry + '.object-meta.xml'), 
                  'utf-8');
let customObject=parse(objFileBody);

My customObject variable now contains a JavaScript object representation of the XML file, so I can access the XML elements as properties:

let label=customObject.CustomObject.label;
let description==customObject.CustomObject.description;

Which allows me to extract the various properties that I am interested in for my document.

In the next exciting episode I'll show how to enrich the objects by pulling in additional information from other files.

Related Posts

• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 2
• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 1
• Parallel Apex Unit Tests and Salesforce CLI Plug-Ins
• Going GUI over the Salesforce CLI Part 2
• Going GUI over the Salesforce CLI
• Mentz - the Mentee Workflow
• Offline mobile app template plug-in Dreamforce 18 session
• Salesforce CLI Play-by-Play
• Salesforce CLI Cheat Sheet
• SFDX and the Metadata API Part 4 - VSCode Integration
• SFDX and the Metadata API Part 3 - Destructive Changes
• SFDX and the Metadata API Part 2 - Scripting
• SFDX and the Metadata API

Documenting from the metadata source with a Salesforce CLI Plug-In - Part 2

Tweet

Introduction

In Part 1 of this series, I explained how to generate a plugin and clone the example command. In this instalment I'll look at customising the cloned command to locate and load the package.json manifest, and to load a configuration file that defines how my objects should be divided up for documentation purposes. All loading, all the time.

Customising the Command

Renaming

The first thing to do around customising the command is to rename it - as I cloned the sample org command, although it is externally known as bbdoc:doc due to the folder structure, in the source code it is still Org:

export default class Org extends SfdxCommand {

so I change the class name to Doc.

Flags

The next thing is to change the flags supported by the command to those that I need for documentation purposes. The flags for the org command are defined as follows:

protected static flagsConfig = {
  // flag with a value (-n, --name=VALUE)
  name: flags.string({char: 'n', description: messages.getMessage('nameFlagDescription')}),
  force: flags.boolean({char: 'f', description: messages.getMessage('forceFlagDescription')})
};

This shows up that there is slightly more to this than changing the flag names - notice the description properties aren't hardcoded strings. Instead they are retrieved by a message object. Here's the setup for the messages object:

// Initialize Messages with the current plugin directory
Messages.importMessagesDirectory(__dirname);

// Load the specific messages for this file. Messages from @salesforce/command, @salesforce/core,
// or any library that is using the messages framework can also be loaded this way.
const messages = Messages.loadMessages('sample', 'org');

The final line shows that the messages are loaded based on the command name - org.  In the generated plugin, messages are stored in the messages folder and the command specific messages are stored in a file named <command>.json. So for the org command, we have the following file structure:

and org.json has the following contents:

{
  "commandDescription": "print a greeting and your org IDs",
  "nameFlagDescription": "name to print",
  "forceFlagDescription": "example boolean flag",
  "errorNoOrgResults": "No results found for the org '%s'."
}

so as part of defining my flags, I also need to define their descriptions and the description of the command itself as follows:

• config       - configuration file
• report-dir  - directory to store the generated report
• source-dir - source directory containing the org metadata

so I create a doc.json file:

{
  "commandDescription": "generate documentation for an org",
  "configFlagDescription": "configuration file",
  "reportDirFlagDescription": "directory to store the generated report",
  "sourceDirFlagDescription": "source directory containing the org metadata"
}

I initialise the messages with this new file:

const messages = Messages.loadMessages('sample', 'doc');

and update the flags definition in the source code to :

protected static flagsConfig = {
  // flag with a value (-n, --name=VALUE)
  "config": flags.string({char: 'c', description: messages.getMessage('configFlagDescription')}),
  "report-dir": flags.string({char: 'r', required: true, description: messages.getMessage('reportDirFlagDescription')}),
  "source-dir": flags.string({char: 's', required: true, description: messages.getMessage('sourceDirFlagDescription')})
};

Reading the package.json Manifest

The version number of my plugin is stored in the package.json manifest file and I want to include this in the generated HTML, so it's time to locate and read that file. I know the package.json is at the root of my plugin directory, but I don't know where the plugin has been installed. Node has a standard File System module, fs, that has been extended by Salesforce to add a handy utility function - traverseForFile. This starts from the folder the command lives in and makes its way up the folder hierarchy until it finds he requested file, or hits the root directory for the disk.  I execute this to find the location of the package.json file, which is in the plugin root:

const pluginRoot=await fs.traverseForFile(__dirname, 'package.json');

Having found the directory, I then read the package.json file. Note that I'm using the join function from the standard Path module - this allows me to build a path without worrying about the operating system or path separator. Note also that as reading a file is an asynchronous operation, I use the await keyword to stop the plugin until the read is complete.

// get the version
const packageJSON=await fs.readFile(join(pluginRoot, 'package.json'), 'utf-8');

Having read the JSON from the file, I parse it into a JavaScript object using the standard JSON.parse function and extract the version property:

const pkg=JSON.parse(packageJSON);
let version=pkg.version;

Loading the Configuration

I find I'm using configuration files more and more often with plugins. While I could add my configuration to sfdx-project.json, and I do if it is only a few items, when I start getting into lengthy configuration it feels like I'm polluting that file with things that don't belong there.

For this plugin and the example Salesforce metadata, I have the following configuration in bbdoc-config.json:

{
    "objects": {
        "name": "objects",
        "description": "Custom Objects", 
        "subdirectory": "objects",
        "suffix":"object",
        "groups": {
            "events": {
                "name":"events", 
                "title":"Event Objects",
                "description": "Objects associated with events",
                "objects": "Book_Signing__c, Author_Reading__c"
            }
            ,
            "other": {
                "name":"other", 
                "title":"Uncategorised Objects",
                "description": "Objects that do not fall into any other category"
            }
        }
    }
}

The objects property contains the configuration for generating the HTML document from the object metadata. It's in its own property as I intend to add more capability in the future. There's some information about the metadata, where it can be found, the suffix for the metadata files, and then a couple of groupings for the objects - the event specific objects and the rest.

The configuration file is optional - if one isn't provided there will be a single object grouping of uncategorised, so I check if he flag was supplied before trying to load any file.

// load the config, using the default if nothing provided via flags
let config;

if (!this.flags.config) {
  this.ux.log('Using default configuration');
  config=defaultConfig;
}
else {
  config=await fs.readJson(this.flags.config);
}

This time I'm using another Salesforce extension to the fs module - readJSON - which reads and parses the file into an object in one go - I should probably refactor the loading of the package.json file to use this!

In the next instalment, I'll show how to load and process the metadata source files. As before, if you can't wait, you can see the plug-in source on Github or install it yourself from npm.

RELATED POSTS

• Documenting from the metadata source with a Salesforce CLI Plug-In - Part 1
• Parallel Apex Unit Tests and Salesforce CLI Plug-Ins
• Going GUI over the Salesforce CLI Part 2
• Going GUI over the Salesforce CLI
• Mentz - the Mentee Workflow
• Offline mobile app template plug-in Dreamforce 18 session
• Salesforce CLI Play-by-Play
• Salesforce CLI Cheat Sheet
• SFDX and the Metadata API Part 4 - VSCode Integration
• SFDX and the Metadata API Part 3 - Destructive Changes
• SFDX and the Metadata API Part 2 - Scripting
• SFDX and the Metadata API