Saturday, 10 July 2021

The CLI GUI Decodes Command Strings

Introduction

Something I've been meaning to add to the CLI GUI for a while is the ability to enter a CLI command string and decode that into a command page with the appropriate parameters set. This would allow someone to understand more about a command without having to run a help command, match to the parameters, and the additional instructions etc defined in the GUI configuration file can be displayed.

I finally got around to adding that capability this week, which wasn't quite as clunky as I was expecting it to be, although to be fair it is still a little clunky.

The Solution

The decoder is offered as part of a new group named 'Built In' - this is so named because it is added to the configured set of commands in the code and will always be present.


Clicking this opens up the command window with a single input to capture the command string:

In this case it's the command I've used in the past to create an org depdendent package, and made sure to write it down for the next time I needed it,

Clicking the Decode Command changes the page to reflect this command, with any additional instructions. And the first clunky bit, in that I always use the short version of the flags so the command generated in this page doesn't match character for character the one pasted in, although it effectively is the same.


The other clunky part is excluding certain functionality from the page when the command is the decoder, but unless I created a whole new page just for that, I was always going to have to do something along those lines. My command configuration file also now knows about the long and short version of every flag, so that it can decode either. 

The Latest Code

I've pushed the latest code to the Github repository - at the moment it's only tested against MacOS as I won't have access to my Windows machine for a day or so. I can't think of any reason why these changes wouldn't work equally well, but if you hit problems before I get a chance to test, raise an issue in the repo.

One thing to bear in mind is that this can only decode the commands and parameters that it knows about. For example, I haven't configured any of the commands to extract data, so there will be no match found when decoding those. 

Related Posts



Saturday, 3 July 2021

Low Code Still Needs Unit Tests

Introduction

First off let me make clear this is not dumping on low code or some King Canute like attempt to push back the advance of clicks not code in favour of having to write every tiny automation in Apex. When I started working with Salesforce back in 2008, I didn't look at workflow rules and think "Oh nose, I wanted to write a bunch of code to update the contents of a rich text area field", I thought 'That's cool, someone else can configure the simple automation and I get to work on the more challenging scenarios". 

On the developer side we've spent years convincing people that unit tests are a good thing, and we even have test driven development that turns the requirements into tests as a first step, after which we write the code that allows the tests to pass. Some people still moan about having to write tests to deploy to production, but most of us recognise this is a good thing and we'll be grateful we had to do it at some point in the future. 

So when I see messaging (from Salesforce and many others) that not being able to write unit tests is a benefit of whatever Low Code platform is being talked about, it strikes me as an attempt to brand a negative as a positive without explaining the downsides. You might be able to get things into production quicker, but realistically you should be swapping unit test effort for extended QA and UAT effort, not just trousering the difference. If you don't, it's quite likely you are getting bugs into production quicker!

Unit Testing is Important

Unit testing is basically writing code that verifies that other code works as expected. Now that low code includes more and more business logic, and the cyclomatic complexity is increasing, unit testing is something which is needed more than ever. As an aside, I felt the same when Aura components were first introduced and moved a bunch of business logic to the front end, so I did a talk at Dreamforce on unit testing those.  

Unit testing brings the following benefits, which apply equally to all types of code:

  • The nearer you find a bug to the developer, the less it costs to rectify.
  • With good test coverage and appropriate assertions, you can make changes with confidence that if you break behaviour that someone else is relying on, one or more tests will fail (this is the main benefit for me).
  • If someone else makes a change that breaks your code (adding a validation rule is the classic situation here), the next time the unit tests run this will be picked up. 
  • It keeps your users happier, as more bugs are found during the development cycle rather than after the system has been released to them.

Testing No and Low Code

No Code tends to be discrete units that can be used to apply automation, like a workflow rule that sent an email and updated some fields if a record transitioned to a specific state, or a validation rule that checks dependencies between fields. There's no point in unit testing that with many different combinations of record fields, for the same reason that in Apex we don't write unit tests to check that the compiler and run time is working as expected. The workflow engine is tested by Salesforce and we can trust it. Manually running a few records through is fine for testing a scenario like this, as the logic that causes the workflow or validation rule to fire is typically very small and rarely changes.

In Low Code, a number of these kind of items are assembled into some more complex automation in a flow, including conditional logic and loops, which is much closer to traditional code. The medium of expression might be different, but we are now in a world of complexity and reuse, especially if it's a subflow that can be dropped into any other flow. In this world it makes sense to run the flow with a variety of inputs, with the underlying database in a variety of states, and as users with a variety of profiles, and many different combinations if these. Doing all this manually every time there is a change isn't going to be practical. Doing it via test automation tools is possible, but tends to be testing an entire transaction rather than the smallest unit, and it can't happen in production as the changes aren't rolled back at the end of the test.

Right now the only way to do this is in a genuine test environment is to write Apex tests. This somewhat defeats a key benefit of flow for smaller organisations - they would still need a developer on staff to write the unit tests for the flows that the admins have created. (Slightly off topic, I also think it would be hard to find a developer who wanted that job for any length of time, so you'd have to be prepared for some significant churn). Apex can't hook directly into the flow engine in all cases though, so for record triggered there's a compromise that the transaction must actually complete, after which verification that the flow aspect had the desired effect can take place. This is more like an integration test, as there are other aspects of automation taking place that could impact the work done by the flow, but it's definitely better than nothing. Once you have Apex tests, you can take advantage of many existing solutions for scheduled execution, making them part of a deployment pipeline etc.

What Might the Future Hold

Ideally what we need is a Low Code test builder that can execute flows/subflows in isolation and generates actual unit tests, with assertions, that run in an isolated environment where the transactions are rolled back open completion. Generating Apex seems like a reasonable approach to me, as does targeting another engine. Given all the effort that has gone into flow over the last few years, it feels like this should be doable. 

Many years ago when Salesforce was seen as shadow IT, my approach to bring the technology departments into the fold was to tell them how their governance and best practice was needed to ensure that things were done properly and were able to scale. Bringing best practice from the Pro Code world feels like a good way to get everyone on board with the Low Code approach.


Saturday, 26 June 2021

Extracting Limits Information from Test Debug Logs

(The follow up to No Limits by 2Unlimited)

Introduction

In large enterprise Salesforce implementations there's often a particular piece of functionality that is limit-poor - it consumes most of one or more limits as a matter of course.  It might have to retrieve information from myriad standard and custom objects, update a whole load of other objects and do a bunch of processing in between that consumes a lot of CPU time, and when it is invoked with a large number of records it flies close to the maximum values like Icarus flying too close to the sun. Often these areas will have unit tests so that unlike Icarus they don't fall into the sea and drown if the code is changed to add a few more SOQL queries.

Often these tests will try to ensure the published maximum number of records can still be handled, but the problem there is that by the time the test fails you have nowhere to go. Another approach is to determine how much of each limit is currently consumed and fail the test if it goes over, which leads to brittle tests that fail for minor changes or, in the case of CPU time, flappy tests that sometimes pass and sometimes fail  even though the code hasn't changed, depending on what else is happening on the Salesforce instance. 

Ideally there would be some way to figure out when the consumed limits change, but without the nuclear option of failing the unit test. In Apex everything that tests do is rolled back when the test completes, regardless of success or failure, sending notifications isn't an option, nor is saving the limit information to an object. This is something I've been wrestling with for years and haven't been able to come up with a solution for. Until now. And in a turn of events that will surprise nobody, it involves a Salesforce CLI plug-in.

The Plug-in

The limitlog plug-in executes the unit test supplied by the user, then relies on the fact that unit tests always generate a debug log - from the Salesforce help on Debug Logs Order of Precedence :

If you don’t have active trace flags, synchronous and asynchronous Apex tests execute with the default logging levels.

The most recent log for the user is then retrieved, and the limits information parsed from it. There's a bit of a wrinkle around this if your org has a namespace, so the plug-in allows you to specify one if you need something other than the default.

The limits information is then returned in JSON format or written to a Salesforce custom object - you can find the metadata for this at the limitlogsalesforce Github repository. 

The metadata also includes a flow that checks the limit information from the previous run for the unit test and sends me an email if anything is different:

So now I can add the plug-in to my continuous integration operations and get notified if the limits consumption has gone up, without having to force unit test failures. 

Installing and Executing

If you want to capture the limit information in Salesforce, the first thing to do is spin up a scratch org or developer edition, clone the limitlogsalesforce Github repository and push/deploy to this org. Make sure to authenticate against it via the CLI.

Next, identify or create a unit test that has some limit impact - this can be in the same developer/scratch org or a different one.

The install the plug-in into your CLI :

    sfdx plugins:install limitlog

Then execute the test command from the plug-in as follows:

sfdx bblimlog:test -n LimitLogSample -r KABDEV -t LimitLogSampleTest.DoWorkTest -s KAB_TUTORIAL -u LIMITLOG -w

where:

  • -n LimitLogSample is the unique name for this test - the flow uses this to find the previous record and figure out if anything has changed.
  • -r KABDEV is the username to run the tests as. If your tests are in the same org as you are writing the data, you can omit this.
  • -t LimitLogSampleTest.DoWorkTest is the unit test method to run
  • -s KAB_TUTORIAL is the namespace of my dev org - if you don't have a namespace, omit this to pick up the default
  • -u LIMITLOG is the username for the org where the test information will be written to
  • -w says to write the limits objects to Salesforce. If you don't supply this, make sure to add the --json switch to see the output.

You will then see output similar to the following:

Executing test LimitLogSampleTest.DoWorkTest
Retrieving test log file
Extracting limits information
Writing limits information to Salesforce
Limits information written to Salesforce

and the limits record will be available in the Salesforce UI:


and here's the list of records in my org that caused the notifications to be sent when the limits increased:


In future blogs I'll go through how this works under the hood, but this feels like enough for now. As this is the first version of the plug-in I'm sure there will be some scenarios it doesn't handle - if you come across one of these, please raise an issue in the plug-in source repo and I'll see what I can do.

Friday, 18 June 2021

Permission Set Group Assignments with Expiration Dates

Introduction

Permission set group assignments with expiration dates are in beta in Summer 21, and do pretty much what it says on the tin - when a permission set group is assigned to a user, an expiration date can be specified after which the assignment will be removed. This might seem like a small change, but it's a powerful one. The release notes mention the obvious use case - someone needing additional permissions for the duration of a project, but there are a fair few more that spring to my mind. 

Use Cases


Pilot Program 

If you are trialling some new functionality and you want everyone to try it out and give feedback by a certain date, provide access to the app/tab/custom objects through a permission set group and set the expiration date to the pilot end date. The artificial scarcity introduced by a hard deadline usually focuses effort.

Scheduled Elevated Permissions

Does your accounts team need access to a bunch of Salesforce data to carry out invoicing, but only on the last day of the month? You can create permission set assignments programatically, so just set up a scheduled apex job that supplies the expiration date, along the lines of :

PermissionSetAssignment psa=new PermissionSetAssignment(
                                  AssigneeId='00580000001ju2CAAQ',
                                  PermissionSetGroupId='0PG1E000000PAv7WAG',
                                  ExpirationDate=Date.newInstance(2021, 06, 20));
    
insert psa;

and there's no need to clean up the access later.

Elevated Permissions for an Event

When attending a trade show, you often want those attending to be able to capture leads even though that isn't their usual role. Assign them the permission and expire it at the end of the day of the event and it's one less thing to worry about afterwards.

Additional Help for New Joiners

When new staff are being onboarded to Salesforce, it can be useful to give them additional information or a way to get help on the pages that they regularly use, but once they have found their feet you can remove the stabilisers (training wheels for our friends across the pond). Create a group with a permission set containing a New Joiners custom permission and assign that to new joiners with an expiration date of two weeks time, and you can conditionally display information based on that. 


 Like an explanation of what a Lead is and how it is worked


Maybe Another


This one I haven't really thought through, but it relates to the sudo command, which in Unix like systems allows you to take on the security privileges of another user, typically the super user.  Using the programmatic assignment from above, users with the appropriate permissions could elect to give themselves additional permissions for the day, thus allowing them to carry out some destructive changes when they need to, but stopping them from accidentally deleting records on other occasions.

The One that Got Away


My favourite use case that I came up with turned out not to be possible. It was going to be punishing bad actors and the example was removing the ability for a user to change the amount or probability on opportunities if they were doing this too often. A muting permission set seemed like it would offer this capability, but sadly these can only mute other permissions granted from the same permission set group rather than remove a permission from a user regardless of where it came from. Shame.



 

Wednesday, 26 May 2021

The Impact of System.debug

TL;DR

Your System.debug statements are consuming resources even when you aren't capturing debug logs. 
If you are passing complex datatype parameters to System.debug then they will consume more resources, as they will be converted to strings (before being discarded as you aren't capturing debug logs!)

Introduction

This week the London Salesforce Developers hosted Paul Battisson and his Turn it up to 11- Improving the Performance of Your Apex code talk, which reminded me of something that has been on my list of things to look at - the impact of System.debug statements on your code.

In Paul's talk he points out that the act of measuring the consumed CPU time and writing the metric to the debug log consumes a small amount of CPU time. And many Apex developers know that turning on debug logging in Salesforce impacts the duration of a transaction, sometimes significantly if the debug log generates multiple megabytes of information - just leave the developer console open while you use your code-heavy application if you haven't experienced this.

What is less well known is that System.debug statements have an impact on your transactions even if you aren't capturing debug logs, as I will endeavour to show in the rest of this blog.

Methodology

I have a simple Apex class with a static method that loops 5,000 times, incrementing an index from 0 to 4999. A string that starts out life empty has the index concatenated with it each time through the loop, using the + operator. 

At the start of the method, the current time in milliseconds and the CPU time consumed are captured, and at the end of the method this is repeated and the difference (the elapsed milliseconds and consumed CPU time for the method) are stored in a custom object.

public class LoggingOn 
{
    public static void LoggingOn()
    {
        Limits_Consumed__c lc=new Limits_Consumed__c();
        Long startMillis=System.now().getTime();
        Long startCPU=Limits.getCpuTime();
        
        String message='';
        for (Integer idx=0; idx<5000; idx++)
        {
            message+=idx;
        }
        
        lc.Total_Millis__c=System.now().getTime() - startMillis;
        lc.CPU_Time__c=Limits.getCpuTime() - startCPU;
        insert lc;
    }
}

The Tests

For each test, the method was executed twice:

  1. Via execute anonymous via the developer console, with debug logging enabled and the log level at the default of SFDC_DevConsole.
  2. As the only code in a trigger before update of a record (this was the quickest way I could execute the Apex outside of execute anonymous - the trigger is essentially a carrier for it) with debug logging disabled

1. No Debug Statements

Scenario 1 took 387 milliseconds elapsed time and consumed 154 milliseconds of CPU

Scenario 2 took 44 milliseconds elapsed time and consumed 45 milliseconds of CPU

I then added debug statements and ran the method via the two scenarios.

2. Adding a Debug Statement

This used System.debug to log the message variable each time through the loop:

System.debug('Message = ' + message);

Scenario 1 took 736 milliseconds elapsed time and consumed 285 milliseconds of CPU - you can clearly see the impact of debug statements when logging is enabled here.

Scenario 2 took 49 milliseconds elapsed time and 45 milliseconds of CPU - very little change here.

3. Extending the Debug Statement

The debug statement was extended to include the name of the current user:
System.debug('Message = ' + message + ', User ' + UserInfo.getName());

Scenario 1 took 1281 milliseconds elapsed time and consumed 328 milliseconds of CPU. An interesting aspect here is that after a very short period of time, the message aspect was being truncated and the user's name wasn't appearing, but the work was still being done to retrieve it and build the full debug string each time through the loop.

Scenario 2 took 120 milliseconds elapsed time and 109 milliseconds of CPU - starting to ramp up.

4. Including an sObject

At the start of the method, prior to the starting CPU and elapsed time being captured, an account record with four fields (Id, Name, AccountNumber, CreatedDate) was retrieved from the database. This was added to the debug statement:

System.debug('Message = ' + message + ', User ' + UserInfo.getName() + ', Account = ' + acc);

Scenario 1 took 1504 milliseconds elapsed time and consumed 494 milliseconds of CPU, but again most of the time the account record details weren't written to the log. 

Scenario 2 took 185 milliseconds elapsed time and 180 milliseconds of CPU - around a 50% increase.

5. Splitting Into Multiple Debug Statements

To ensure that the information could be seen in the debug logs, the single statement was split into three:

System.debug('Message = ' + message);
System.debug('User = ' + UserInfo.getName());
System.debug('Account = ' + acc);

Scenario 1 took 1969 milliseconds elapsed time and consumed 656 milliseconds of CPU, and this time all of the pertinent information was recorded each time through. 

Scenario 2 took 198 milliseconds elapsed time and 192 milliseconds of CPU - a small increase.

6. Adding an Array

The query to retrieve the record was extended to include the Id, Subject and Description of its 2 related cases, and an additional statement added to debug this:
System.debug('Message = ' + message);
System.debug('User = ' + UserInfo.getName());
System.debug('Account = ' + acc);
System.debug('Cases = ' + acc.cases);

Scenario 1 took 2563 milliseconds elapsed time and consumed 1136 milliseconds of CPU - closing in on double the CPU, so clearly logging an array has quite an impact. 

Scenario 2 took 368 milliseconds elapsed time and 353 milliseconds of CPU, again close on double.

Reviewing the Numbers

Here are the numbers for the tests in an easy to consume format, with the percentage increase from the previous test shown in brackets.

Test Elapsed 1 CPU 1 Elapsed 2 CPU 2
1 387 154 44 45
2 736 (47%) 285 (46%) 49 (11%) 45 (0%)
3 1281 (60%) 328 (15%) 120 (144%) 109 (142%)
4 1504 (17%) 494 (50%) 185 (54%) 180 (65%)
5 1969 (31%) 656 (33%) 198 (7%) 192 (7%)
6 2563 (30%) 1136 (73%) 368 (86%) 353 (84%)

Looking at the scenario 1 results, we can see that the time increases with the number of debug statements and complexity of datatypes. This makes perfect sense as I'm doing more work to find out granular details about the state as the method proceeds.

Looking at the scenario 2 results, we can see that by not capturing debug logs, the numbers are quite a lot smaller. However, the time is increasing as the number of debug statements and complexity of datatypes increase, even though I'm not using any of the information

By leaving the debug statements, I'm forcing various datatypes to be converted to strings in order to pass them as parameters to the System.debug() method, which will do nothing with them. In the case of test 6, my transaction is over a third of a second longer due to a bunch of work that adds zero value - my users will be pleased!

Delete All the Debugs?

So is the solution to delete all the debug statements before deploying to production? In many cases, yes. Often System.debug() statements are leftovers from problems encountered during development and can be safely deleted. But not in all cases.

If you are trying to track down an intermittent problem, maybe not. It might be useful to have the debug statements in there so that you can quickly enable logging for the user having the problem and see exactly what is happening. The downside to this is the transaction is slowed down for everyone all the time, not just the user encountering the problem when they are encountering it.

A better solution may be to allow the system administrator to control whether the System.debug() statements are executed through configuration. That way they are there if needed, but only executed when there is a problem to be tracked down.

To carry out a fairly blunt test of this, I created a custom metadata type with a checkbox to indicate if debugging is enabled. The custom metadata type is retrieved after the starting elapsed and CPU time have been captured, and is checked before each of the debug statements is executed, to try to simulate a little of the real world. My method now looks as follows:

public class LoggingOn 
{
	public static void LoggingOn()
    {
        Account acc=[select id, name, accountnumber, createddate,
                     (select id, subject, description from Cases)
                      from account where id='0011E00001ogFnaQAE'];
                      
        Limits_Consumed__c lc=new Limits_Consumed__c();
        Long startMillis=System.now().getTime();
        Long startCPU=Limits.getCpuTime();
        
        Debugging__mdt systemDebugging=Debugging__mdt.getInstance('System');
        String message='';
        for (Integer idx=0; idx<5000; idx++)
        {
            message+=idx;
            if (systemDebugging.Enable_Debugging__c)
            {
	            System.debug('Message = ' + message);
            }
            if (systemDebugging.Enable_Debugging__c)
            {
	            System.debug('User ' + UserInfo.getName());
            }
            if (systemDebugging.Enable_Debugging__c)
            {
	            System.debug('Account = ' + acc);
            }
            if (systemDebugging.Enable_Debugging__c)
            {
	            System.debug('Cases = ' + acc.cases);
            }
        }
        
        lc.Total_Millis__c=System.now().getTime() - startMillis;
        lc.CPU_Time__c=Limits.getCpuTime() - startCPU;
        insert lc;
    }
}

Executing the final tests again with the custom metadata type set to disable debug statements showed a significant improvement:

Scenario 1 took 475 milliseconds elapsed time (-78%), and consumed 213 milliseconds of CPU (-81%)

Scenario 2 took 66 milliseconds elapsed time (-82%), and consumed 62 milliseconds of CPU (-82%)

Conclusion

Now I'm not suggesting that everyone starts building configuration frameworks to allow individual debug statements to be toggled on or off for specific users - for any decent sized Salesforce implementation, this would be a huge amount of work and an ongoing maintenance headache. What I am saying is think about the debug statements that you leave in your code when you deploy your code to production. While it might not feel like you need to worry about a few statements that debug record details, once you start to scale the volume it can slow down the user experience. Studies have shown that when an application is slow, the users also perceive it to be buggy regardless of whether that is actually the case - once a user takes against your app, they go all in!


Saturday, 15 May 2021

AuraEnabled Apex and User Access


  

Since the Winter 21 release of Salesforce, users have to be given explicit access to AuraEnabled classes that are used as controllers for Aura Components. BrightSIGN is no exception to this, but recently I encountered some strange behaviour around this that took a couple of days to get to the bottom of.

The Problem

A subscriber had reported an issue that when saving their signature image, they received an alert containing the text 'Save Failed: Null'. This sounded fairly straightforward - if an error occurs I trap it and return the details to the front end, where it triggers that alert. However, upon checking the controller, there was no code that set the error details as null. They were initialised as a success message and then switched to explicit error messages if something went wrong.

Maybe it was an exception that I wasn't trapping correctly, although I couldn't see how. In order to save a signature for a record the user needed update access, so I tried a number of tests:

  • Using parent record id that didn't exist
  • No permission to access the parent record
  • Read only permission on the parent record
  • Parent record not shared with the user
  • Parent record shared read only with the user
  • Trigger on attachment save that results in a catchable exception
  • Trigger on attachment save that breaches governor limit and generates an uncatchable exception
In every case, I either received a detailed error message that explained the problem, or the server call returned an error and stack trace. 

The subscriber was seeing this in a community, so I re-ran the tests with different licenses, but was still unable to reproduce the problem.

Access to AuraEnabled 


I'd been doing my testing in a new developer org where I'd installed the package for all users, so when I checked I found that everyone had explicit access to the Apex controller with the AuraEnabled method, but it occurred to me that might not be the case for the subscriber. Sadly when I removed access, I got a sensible error that I didn't have access to the class. Just for the sake of completeness, I asked the subscriber to check this. Lo and behold their user didn't have access to the class, and when they added it everything started working.  Looking at my client side code, it meant that when the callback from the server was invoked, the getState() method returned SUCCESS, which means the action executed successfully. However, the getReturnValue() method returned null, which meant that it hadn't hit my method as the return value wasn't ever set to null. 

So it appeared that when the update was enforced to remove their users access to the classes, it didn't completely remove it. Instead it looked like the user could successfully hit the server but not actually execute the method, just receiving a null response instead. A touch of the uncertainty principle, as there was both access and no access!

So if you are getting a null response from a controller from one of your Aura components and you don't understand why, check the class access - it might save you a day or two of head scratching!

Related Posts


My First (No Longer) Deleted Post

(Update: My post was reinstated - see the update at the end of this missive)

This week I experienced a first on Blogger - one of my posts was deleted. Out of the blue I received an email that my post had been flagged for breaching a content policy (Malware and Viruses) and deleted. 

Just like that, the post was gone. Of course on the internet nothing is ever gone that quickly, if ever! There are various sites out there that scrape the contents of blogs and host it themselves, and I cross post most of my content to Medium, but it's gone from my blog.

The post itself was about my first steps into using Substack - could it be that Blogger were worried about their users following me, Pied Piper style, over to an upstart competitor? This seems highly unlikely given the scale of Blogger - even if everyone that has ever read my blog decided to switch on the basis of one post, I'm fairly sure it wouldn't move the needle in any way that would be visible without a microscope.

What's highly likely is that there are automated tools involved that have picked up a false positive. 

Why do I think it's a false positive? Because as I mentioned above, the same post is hosted elsewhere and I've been through it with a fine-tooth comb, and all I can find is links to news sites or some of my pages that host Salesforce tools and content. Nothing that jumps out as a problem, and in every case the reader has to click a link to navigate to the content. 

Now don't get me wrong, I'm not crying about this - I don't pay to use Blogger so they can do what they like with their own platform. I'd really like to know some more details of what is wrong so that I don't do it again, but if they can't or don't want to tell me then there's not a lot I can do. I've been on here 10 years and this is the first incident of this nature, so it's hardly a vendetta.

The upside to this is that it made me realise that I haven't backed up my blog for a while, so I've taken care of that. If the balloon does go up and my blog suddenly disappears, at least I haven't lost all the ramblings I've spent years crafting!

Back up your work people, wherever it may be!

Update: I replied to the email from Blogger about my post being deleted, explaining that I'd checked the content and didn't know which part was causing the issue, and asking for more information so I could avoid the same problem in the future. 6 hours or so later I received another email telling me that it had been looked at again and reinstated. I might not be so lucky next time, so I'll be backing up regularly!

Saturday, 8 May 2021

The CLI GUI takes on Packages


Introduction

Second generation packaging via the Salesforce CLI is awesome, but it's the area I spend most of my time remembering what parameters it supports, as well as figuring out the ids or names of the packages I'm working with.

I've actually had this functionality in the GUI for a while now, but I hadn't been using it in anger across all the various package types, so I didn't publicise it. Since then I've used it to create managed, unlocked and org dependent packages, and base and extension packages, so I think it's good enough to unleash on the world. The examples below are based on the org-dependent package that I created for London's Calling 2021.

Commands

The updated commands configuration file in the repo adds support for five package commands in their own tab:


(I have them in the order that I typically use them on a day to day basis, but they can easily be reordered by updating the commands.js file).

Create Package



Typically only used when I start a project, the Create Package command will default to the configured dev hub if there is one, but I can change it if for some reason I want to use another. Only dev hub orgs appear in the select list. The Path parameter will be used to update the packageDirectories property from the sfdx-project.json file.

Create Version

This is probably the command I use most when developing packages - creating a new version in the hope that I've finally fixed that elusive bug, but expecting further disappointment! 


When I click the Get Packages button, the available packages managed by the dev hub are retrieved and the sfdx-project.json checked to see if there is already a package defined that it can default to. I can choose any other package if I need to.


The other parameter of note is Definition File - when you create a second generation package, there's no developer edition where the code lives, so something like a scratch org is created as part of the upload process, where the code is deployed, tests run etc. Most of the time I just specify my existing scratch org definition file (config/project-scratch-def.json), but very occasionally I'll have a different shape required for the upload and create a dedicated file.

I can specify a key (password) required to install the package, or bypass this. And I always specify a long wait, as if the command doesn't finish synchronously then the details of the version don't get written to the sfdx-project.json.

Promote Version

There's always a bit of Stockholm Syndrome about this command when developing a package. At the start, I'll do pretty much anything rather promote the version to fix the functionality and commit myself to the package contents, but after a few versions I'll be executing it without a second thought.


As before, the Get Packages button retrieves the packages being administered by the dev hub. Get Package Versions retrieves the versions for the selected package, saving me remembering names or ids. It also only shows my versions that haven't been promoted:


List Packages

A simple command - as it's name suggests it lists the packages currently being administered by the dev hub. 


The package information is shown in the log panel, which automatically opens when the command is executed:


List Package Versions

Again, does exactly what it says on the tin. 


Although this time I can reduce the results based on the when the version was created or modified, and limit it to released versions only. These parameters might seem unnecessary when you are starting out with a package, but after a few years and 100+ versions, you'll be glad of them.

Wrap Up

As usual, I've tested these commands on MacOS and Windows 10, but if you come across any issues feel free to raise them at the Github repository.

Related Posts


Friday, 30 April 2021

Remote Work - The End of the Beginning?


As we head into May in the United Kingdom, COVID-19 cases and deaths have settled at around 2-2.5k and 10-40 respectively, so the planned relaxing of lockdown looks set to continue. Vaccine data from around the world makes us cautiously optimistic that the impact of catching the virus is significantly reduced, so it could be that once we re-open then we stay open. While this is very positive news, it does apply pressure to those of us who have been regularly writing about remote work, as the clock is ticking down on the "all remote all the time" approach, and interest in hearing about it will start to wane.

I've read a number of research items suggesting that people feel less productive when they work from home, in part because when they hit a problem they can't just look up and ask someone a question. Instead they either have to self-serve via Google or track someone down in chat and type out their question. While I can see the impact, it isn't the whole story. As the person who was usually on the receiving end of said questions, and answering a large amount every day, I'm often a lot more productive as I can more easily get my head down and concentrate on some deep work. I can also choose to ignore my email/chat/phone much more easily than I can ignore someone who comes and stands next to me and starts talking! Obviously helping people is a huge part of my job, so I don't expect to go off the grid for days (or even hours) at a time, but even breaking off to tell someone I'm busy and can I come back to them later disturbs my train of thought. I'd really like to see the numbers from both sides to determine the real impact!

The "Work from Anywhere" narrative is being dialled back to a hybrid approach of "Work from Anywhere  Half the Time and the Office the Other Half", which kind of ruins the anywhere aspect. If you have to go into the office 2-3 days a week, you'll need to keep living near to that office unless you really fancy commuting from the beach for several hours a day. To be fair, some companies are sticking to the fully remote option, but typically caveating with "where your role allows it", and what's the betting a lot of roles turn out not to allow it after all.

From a pure getting things done perspective, I've feel like I've always embraced working from anywhere. Prior to the pandemic I used to do a fair bit of travelling, including a few trips every year to the US. This would typically involve several hours on buses/trains/tubes, arriving at an airport three or four hours early and then sitting for ten hours on a plane. With a little planning I could churn out work wherever I found myself, writing designs or documentation when I was offline and Apex and Lightning Components when I had connectivity. 

From a leadership, collaboration, and support perspective it's less than ideal - Zoom calls and messaging tools can cover a lot of this, but sometimes there really is no substitute for a face to face meeting where you can scribble on a whiteboard. But you don't need that all day every day either, which is why I think the classic remote work approach is likely to be rolled out more going forward - employees are based at home and don't have a desk in an office, but they spend a day or two a month meeting with their colleagues in person. 

One thing is for sure, remote work isn't settled as we tentatively venture back into different houses, pubs,  and offices. In November 1942 Churchill gave a speech at the Lord Mayor's Luncheon at Mansion House where he said  :

Now this is not the end. It is not even the beginning of the end. but it is, perhaps, the end of the beginning

and that is where I feel we are with remote work. The beginning was the momentous turn of events that saw us pivot to "as remote as possible" with very little planning, and stick with it for over a year.  We are now coming to the end of the beginning, as offices will soon be able to reopen. When that happens we enter the middle as we try out lots of changes and tweak things as we go. The end is still some way off, which is when we all agree that we have nailed it and remote work is as efficient as it can be. And that will be the normal until the next momentous event that derails everything!


Saturday, 24 April 2021

Book Review - Becoming a Salesforce Certified Technical Architect


 (Disclaimer : I didn’t purchase my copy of this book - I was sent a copy to review by Packt Publishing)

Introduction

Salesforce describes Certified Technical architects as follows:

Salesforce Technical Architects serve as executive-level strategic advisors who focus on business transformation with unrivaled domain expertise in functional, platform and integration architecture. They communicate technical solutions and design tradeoffs effectively to business stakeholders, and provide a delivery framework that ensures quality and success.

Based on that description, it's not surprising that there's a huge amount of interest in becoming one. The path isn't straightforward though, with an awful lot of candidates falling at the last hurdle - the review board. This isn't a new development - in the 12 months after I qualified I heard about two week review board events where all 20 candidates failed.  While that's obviously bad for the candidates, it's also not great for the judges - failing people isn't a fun job, and when you have to do it repeatedly the sessions become a chore. 

One reason people fail is that they haven't got experience of presenting a solution and being judged on it. That has improved with community efforts to come up with example scenarios and run practice review boards. While this is good experience for the day of the board, it's not overly helpful in terms of preparation. And that is where this book comes in.

The Book for When You Book

This is the book you want to buy when you are about to book your review board slot.  It won't turn you into a Technical Architect - you still need the experience and learning - but it will get you ready for the board. First off, it looks at what expertise you need to become CTA - if you don't recognise yourself here and spot gaps, maybe hold off for a little while. It then takes you through the structure of the review board session and gives some useful tactics for how to approach it.

Then there are a bunch of chapters around the various areas that you need to shine at in order to pass the board, with mini-scenarios that are worked through with you. Rather than reading straight through, I'd strongly advice having a go yourself and then compare your solution with the exemplar.If you have a different view, that doesn't mean you are wrong, just make sure you can justify it and your justification stacks up - it's highly likely that you'll be quizzed about the benefits of your approach over the other possible solutions.

The book then finishes off with a couple of full mock scenarios, with example solutions and presentation artefacts and script. Again, look to make the most of this by trying it yourself under exam conditions.

Allow Enough Time


I'm already a CTA so I didn't have to take on each of the scenarios and mocks, but it still took me a couple of months to read through the book in my free time and review it. Don't expect to skim this the night before the board, or to rip through it the weekend before. Set aside enough time to work through it properly and you'll reap the benefit.

The Key Advice

The most important piece of advice in the book, which crops up multiple times, is to own your solution. You've considered all the options and have come up with the best possible solution, so be proud of it when you present, and defend it with everything you have. 

In Summary

You can boost your chances of becoming a CTA with this book. It's as simple as that. Tameem Bahri has done a great job with it.




Saturday, 17 April 2021

Unpackaged Metadata for Package Creation Tests in Spring 21

Introduction

The Spring 21 release of Salesforce introduced a useful new feature for those of us using second generation managed or unlocked packages (which really should be all of us now) - the capability to have metadata available at the time of creating a new package version, but which doesn't get added to the package.

I've written before about how I use unpackaged directories for supplementary code, but this isn't code that is needed at creation time, it's typically utilities to help the development process or sample application code to make sure that the package code works as expected when integrated elsewhere. 

The Scenario

To demonstrate the power of the new unpackaged concept, here's my scenario:

I have a package that defines a framework for calculating the discount on an Opportunity through plug and play rule classes. The package provides a global interface that any rule class has to implement:

public interface DiscountRuleIF 
{
    Decimal GetDiscountPercent(Opportunity opp);
}

 The rules are configured through a Discount_Rule__c custom object, which has the following important fields:

  • Rule_Class__c - the name of the class with the rule interface implementation
  • Rule_Namespace__c - the namespace of the class, in case the class lives in another package
And there's the rule manager, which retrieves all of the rules, iterates them, instantiates the implementing class and executes the GetDiscountPercent method on the dynamically created class, eventually calculating the total discount across all rules:
global with sharing class DiscountManager 
{
    global Decimal GetTotalDiscount(Opportunity opp)
    {
        List<Discount_Rule__c> discountRules=[select id, Rule_Class__c, Rule_Class_Namespace__c
                                              from Discount_Rule__c];
        Decimal totalDiscount=0;

        for (Discount_Rule__c discountRule : discountRules)
        {
            Type validationType = Type.forName(discountRule.Rule_Class_Namespace__c,discountRule.Rule_Class__c);        
            DiscountRuleIF discountRuleImpl = (DiscountRuleIF) validationType.newInstance();
    
             totalDiscount+=discountRuleImpl.GetDiscountPercent(opp);

        }

        return totalDiscount;
    }
}

So the idea is that someone installs this package, creates rules locally (or installs another package that implement the rules), and when an opportunity is saved, the discount is calculated and some action taken, probably discounting the price, but they are only limited by their imagination.

Testing the Package

I don't want to include any rules with my package - it purely contains the framework to allow rules to be plugged in. Sadly, this means that I won't be able to test my rule manager class, as I won't have anything that implements the discount rule interface. For the purposes of checking my class, I can define some supplementary code using the technique from my earlier post via the following sfdx-project.json : 

{
    "packageDirectories": [
        {
            "path": "force-app",
            "default": true,
            "package": "UNPACKMETA",
            "versionName": "ver 0.1",
            "versionNumber": "0.1.0.NEXT"
        },
        {
            "path": "example",
            "default": false
        }
    ],
    "namespace": "BGKBTST",
    "sfdcLoginUrl": "https://login.salesforce.com",
    "sourceApiVersion": "51.0"
}

Then I put my sample implementation and associated test classes into example/force-app/main/classes, and when I execute all tests I get 100% coverage.  I can also create a new package version with no problems. Unfortunately, when I promote my package version via the CLI, the wheels come off:

Command failed
The code coverage required to promote this version has not been met.
Please add additional test coverage and ensure the code coverage check
passes during version creation.

My sample implementation and the associated test code has been successfully excluded from the package, but that means I don't have the required test coverage. Before Spring 21 I'd sigh and pollute my package with code that I didn't want, just to satisfy the test coverage requirement. It would be a small sigh, but they add up over the years.

The unpackagedMetadata property

The new Spring 21 feature is activated by adding the unpackagedMetadata property to my sfdx-project.json, inside the default package directory stanza:

{
    "packageDirectories": [
        {
            "path": "force-app",
            "default": true,
            "package": "UNPACKMETA",
            "versionName": "ver 0.1",
            "versionNumber": "0.1.0.NEXT",
            "unpackagedMetadata": {
                "path": "example"
            }
        },
        {
            "path": "example",
            "default": false
        }
    ],
    "namespace": "BGKBTST",
    "sfdcLoginUrl": "https://login.salesforce.com",
    "sourceApiVersion": "51.0",
}

Note that I still have my supplementary directory defined, but I'm also flagging it as metadata that should be used at create time to determine the test coverage, but not included in the package.

Having created a new version with this configuration, I'm able to promote it without any problem. Just to double check I installed the package into a scratch org and the unpackagedMetadata was, as expected, not present.

Reduced Coverage in the Subscriber Org

One side effect of this is that the code coverage across all namespaces will be reduced in the subscriber org. Depending on your point of view, this may cause you as the package author some angst. It may also cause the admin who installed the package into their org some angst. 

Personally I don't think it matters - the coverage of managed package code doesn't have any impact on the subscriber org, and I don't think it's possible to craft tests inside a managed package that are guaranteed to pass regardless of what org they are installed into. I could do it for my sample scenario, but if it's anything more complex than that, the tests are at the tender mercy of the org configuration. I create an in-memory Opportunity to test my discount manager, but if I had to insert it into the database, there's any amount of validation or expected data that could trip me up. While there are techniques that could be used to help ensure my tests can pass, mandating that everyone who installs my package has to have used them across all their automation seems unlikely to be met with a positive reaction.

What it is more likely to mean is that running all tests across all namespaces is more likely to succeed, as the tests which could easily be derailed by the org configuration can be left out of the package, just leaving the ones that are completely isolated to the package code in place, so swings and roundabouts.

What it does mean is that the package author has the choice as to whether the tests are packaged or not, which seems the way it should be to me.

Related Posts






Saturday, 10 April 2021

Bake It Till You Make It

 


Introduction

As we start to come out of the third lockdown in the UK, blinking in the not so bright light of yet another overcast day, it's important to keep occupied. BrightGen have been pretty good at finding fun things to do, and the baking challenge returned for Spring. We all get an allowance for our ingredients, so the only outlay is our time and loving attention.

I'm not that fussed about cakes, and we'd only just finished up the last couple of slices of my Great BrightGen Bake Off entry from November 2020 (it was in the freezer rather than rotting in a tin!) so I decided to go savoury with beef loaf en croute. I've made this many times over the years, but this time was different. First I was making the pastry from scratch. I haven't done this in about 10 years as the supermarket version is just as good, but that didn't seem in the spirit of the challenge. The second difference was that it really mattered if it came out okay - this wasn't just for guests to eat at a sophisticated dinner party, this was going on social media!

The Hashtag

In the previous incarnation of the BrightGen bake off, the hashtag proved a bit of a challenge. That's supposed to say GBGBO - Great BrightGen Bake Off. It's just about readable, but hardly a triumph. I wasn't expecting much better this time, but I was determined to try!


After making the pastry and leaving it to rest in the fridge for a couple of hours, I nicked off the ends and created the hashtag lettering - minus the hash symbol, as that's just dead space. Not too bad, but boy do you have to be quick with puff pastry or it sticks to everything, and there are a lot more letters this time!

Next  it was time to make the meat loaf - minced beef, onions, apples, breadcrumbs, stock and a few herbs, then 45 minutes in a Bain-Marie and it was cooling. This is by far the easiest part of the whole process, but make sure you leave it to cool down completely or you'll leave half of it in the tin - I know this for a fact as I've spent more than one evening frantically trying to piece it back together like the world's worst game of tetris.


Several hours later I returned to assemble, only to experience the revenge of the hashtag (if you saw the outcome on social media, just forget that for a couple of paragraphs and play along with my artificial jeopardy) - it doesn't fit on the loaf. 


Nothing to be done now, as the whole thing was assembled, and the pastry was milk-washed and already starting to sag. Into the oven and hoping for the best.

Even if I say so myself, it came out alright:


Baker's Remorse

What happened? How did it go so wrong? I knew the letters were large as I wanted them to fill the top of the loaf, but did I really think I could fit 12 letters of that size with the surface area of a 1 pound loaf tin.

If only I’d made a smaller one to take the second part of the hashtag.

Oh wait.

That’s right.

I did!


As I'm still shopping for elderly parents, whenever I cook anything like this I make extra to send to them with their weekly groceries, and it turns out the 1/2 pound loaf tin surface area is perfect for the last 4 letters.

Here’s the finished articles and a sneak peek at the official cutting:







Saturday, 3 April 2021

Unbounded Queries [Featuring Spring 21 FIELDS()]


Introduction

FIELDS() is a new SOQL function in the Spring 21 Salesforce release that allows you to query standard, custom or all fields for an sObject without having to know the names. Could it be, are we finally getting the equivalent of SELECT * so beloved of developers working with an SQL database? Can we retire our Apex code that figures out all of the field names using describe calls? Digging into the docs, the answer is no. You can pull back the standard fields, but not all of them.

Disappointing, right? Maybe at first glance, but from a future proofing your Apex code perspective, absolutely not. It's actually saving you from having unbounded queries lurking like a time bomb in your Salesforce instance, waiting to blow the heap limit on an unsuspecting user.

Unbounded Queries

Simply put, an unbounded query doesn't limit the results that may be returned from the database. 

Most of the time, we (and the rest of the developer world) think of unbounded queries as those without a restrictive WHERE clause, or with no WHERE clause at all. These queries start out life doing their best to be helpful - backing custom pages with search capabilities where the user can choose some criteria to restrict the number of records, but can click search straight away if they want to. When there are 20 records in the system, this isn't a problem. Once you've been life for 5 years and you have 15,000 records, it's an accident waiting to happen. I think of this as vertically unbound, as something needs to be done to limit the depth of the record list retrieved. 

So putting in a LIMIT clause to your query solves the problem and future proofs your application? Not quite. Queries can be unbound horizontally as well as vertically - each record can become so wide that the total number that can be retrieved is massively reduced. I view this is a problem that impacts Salesforce implementations more than other technologies, as it's so easy to add fields to an sObject and change the database schema. No need to involve a DBA, just a few clicks in Setup. The odd field here and there racks up over the years, and if your query is retrieving all fields on a record, the size of each record will become a problem, even if you are restricting the number of records that you retrieve.

Some Numbers

To reflect the 'SELECT *' concept, I threw together some Apex that dynamically creates a query by iterating the field map returned by the schema describe for an sObject. I created 500 Contact sObjects with a variety of fields present, then executed some anonymous Apex to query them back, iterate them and debug the records and debug the heap consumed. I took the simplistic view that the entire heap was down to the records retrieved, so all calculations around the maximum records will probably be over optimistic, but I'm sure you get the point.

With no contacts, the heap was a svelte 1968 bytes. 

With 500 contacts with the First and Last Name fields defined as 'Keir Bowden', the heap was 250,520 bytes, so approx 500 bytes per record. This sounds like a lot for a couple of fields, but there are quite a few standard fields that will be retrieved too - Id, IsDeleted, OwnerId, HasOptedOutOfEmail, HasOptedOutOfFax, DoNotCall, CreatedById, CreatedDate, LastModifiedId, LastModifiedDate, SystemModStamp, IsEmailBounced, PhotoUrl. A whole bunch of stuff that I probably don't care about. Not too much to worry about though, as at that rate I can retrieve 12,000 contacts before approximately blowing the heap. This is a good example of why adding a LIMIT clause to a query is a good idea - 12,000 contacts really isn't that many for a decent sized organisation that has been around for a few years!

Adding Salution of 'Mr' to all of the records caused the heap to creep up to 253,864 bytes - 508 per record. A small increase, but one that has a material impact on my max records - I'm down to 11,811 - a drop of 189 records through a single field that maybe I don't need for my processing.

I then added a unique Email Address and a Lead Source of Purchased List - moving the heap needle to 288,454 bytes - an increase of 70 bytes per record over the previous run. I'm now down to 10,380 records, so I'll have to process 1,431 less records in my transaction to avoid the limit. All because of fields that are present on the record, not because of the fields that I want. 

I upped the ante on my next change - I decided my Sales Reps wanted a notes style field on the record where they could record their unstructured thoughts about the customer. So I added a Text Area (Long) field, and add 1,000 characters to each of my records, which isn't a lot for free text - if you are doubting that, consider that at this point in this blog post I am around 5,000 characters. The impact on the heap was much as expected, and my heap use is now at 794,946 bytes at a hefty 1,600 bytes per record. I'm now down to 3,750 records per transaction! I think it's highly unlikely that I have any interest in the Sales Rep's random musings, but I'm down to around 31% of the records in no small part because of it.

Hopefully there's no need for me to labour the point - specifying a LIMIT clause is only part of the solution, and it's unlikely to help if the size of each record is something that you are exerting no control over.

Other Problems With Unbounded Queries

Even when they aren't breaking limits, unbounded queries bring unwanted behaviour:

  • If you are sending the records to the front end, you could end up transmitting way more data than you need to, taking longer and putting more stress on the client who has to manage the bloated records.
  • It makes the code harder to read - if the only query is to retrieve all fields, you need to dig into how the results are used to figure out which fields are actually needed in specific scenarios.
  • You lose static bindings to field names. This is unlikely to be a problem, as you'll almost certainly be referring to the fields you want to process by name elsewhere in your Apex code. But if you are processing fields using dynamic DML, then if an admin deletes one of the fields you are relying on, the first anyone will know about it is when you try to access the field and it's not there any more.
Even if you absolutely, definitely, have to have all of the fields that are currently defined on an sObject, before using an unbounded query ask yourself if you absolutely, definitely, have to have all the fields that could possibly be defined in the future - including those that really don't belong on the contact but it was easier than making some fundamental changes to the data model! Yes, you'll save yourself some typing, but you'll be storing up problems for the future. Maybe not today, maybe not tomorrow, but some day it's going to go pop!

So Never All Fields?


As always, never say never. For example, I'll sometimes use this technique in debug Lightning Web Components where I want to dump everything out about an object that is in use on a page, even the fields that aren't currently being retrieved. In pretty much every case where I do use this approach, the query will return a maximum of 1 record!

Related Posts