Saturday, 17 April 2021

Unpackaged Metadata for Package Creation Tests in Spring 21


The Spring 21 release of Salesforce introduced a useful new feature for those of us using second generation managed or unlocked packages (which really should be all of us now) - the capability to have metadata available at the time of creating a new package version, but which doesn't get added to the package.

I've written before about how I use unpackaged directories for supplementary code, but this isn't code that is needed at creation time, it's typically utilities to help the development process or sample application code to make sure that the package code works as expected when integrated elsewhere. 

The Scenario

To demonstrate the power of the new unpackaged concept, here's my scenario:

I have a package that defines a framework for calculating the discount on an Opportunity through plug and play rule classes. The package provides a global interface that any rule class has to implement:

public interface DiscountRuleIF 
    Decimal GetDiscountPercent(Opportunity opp);

 The rules are configured through a Discount_Rule__c custom object, which has the following important fields:

  • Rule_Class__c - the name of the class with the rule interface implementation
  • Rule_Namespace__c - the namespace of the class, in case the class lives in another package
And there's the rule manager, which retrieves all of the rules, iterates them, instantiates the implementing class and executes the GetDiscountPercent method on the dynamically created class, eventually calculating the total discount across all rules:
global with sharing class DiscountManager 
    global Decimal GetTotalDiscount(Opportunity opp)
        List<Discount_Rule__c> discountRules=[select id, Rule_Class__c, Rule_Class_Namespace__c
                                              from Discount_Rule__c];
        Decimal totalDiscount=0;

        for (Discount_Rule__c discountRule : discountRules)
            Type validationType = Type.forName(discountRule.Rule_Class_Namespace__c,discountRule.Rule_Class__c);        
            DiscountRuleIF discountRuleImpl = (DiscountRuleIF) validationType.newInstance();


        return totalDiscount;

So the idea is that someone installs this package, creates rules locally (or installs another package that implement the rules), and when an opportunity is saved, the discount is calculated and some action taken, probably discounting the price, but they are only limited by their imagination.

Testing the Package

I don't want to include any rules with my package - it purely contains the framework to allow rules to be plugged in. Sadly, this means that I won't be able to test my rule manager class, as I won't have anything that implements the discount rule interface. For the purposes of checking my class, I can define some supplementary code using the technique from my earlier post via the following sfdx-project.json : 

    "packageDirectories": [
            "path": "force-app",
            "default": true,
            "package": "UNPACKMETA",
            "versionName": "ver 0.1",
            "versionNumber": "0.1.0.NEXT"
            "path": "example",
            "default": false
    "namespace": "BGKBTST",
    "sfdcLoginUrl": "",
    "sourceApiVersion": "51.0"

Then I put my sample implementation and associated test classes into example/force-app/main/classes, and when I execute all tests I get 100% coverage.  I can also create a new package version with no problems. Unfortunately, when I promote my package version via the CLI, the wheels come off:

Command failed
The code coverage required to promote this version has not been met.
Please add additional test coverage and ensure the code coverage check
passes during version creation.

My sample implementation and the associated test code has been successfully excluded from the package, but that means I don't have the required test coverage. Before Spring 21 I'd sigh and pollute my package with code that I didn't want, just to satisfy the test coverage requirement. It would be a small sigh, but they add up over the years.

The unpackagedMetadata property

The new Spring 21 feature is activated by adding the unpackagedMetadata property to my sfdx-project.json, inside the default package directory stanza:

    "packageDirectories": [
            "path": "force-app",
            "default": true,
            "package": "UNPACKMETA",
            "versionName": "ver 0.1",
            "versionNumber": "0.1.0.NEXT",
            "unpackagedMetadata": {
                "path": "example"
            "path": "example",
            "default": false
    "namespace": "BGKBTST",
    "sfdcLoginUrl": "",
    "sourceApiVersion": "51.0",

Note that I still have my supplementary directory defined, but I'm also flagging it as metadata that should be used at create time to determine the test coverage, but not included in the package.

Having created a new version with this configuration, I'm able to promote it without any problem. Just to double check I installed the package into a scratch org and the unpackagedMetadata was, as expected, not present.

Reduced Coverage in the Subscriber Org

One side effect of this is that the code coverage across all namespaces will be reduced in the subscriber org. Depending on your point of view, this may cause you as the package author some angst. It may also cause the admin who installed the package into their org some angst. 

Personally I don't think it matters - the coverage of managed package code doesn't have any impact on the subscriber org, and I don't think it's possible to craft tests inside a managed package that are guaranteed to pass regardless of what org they are installed into. I could do it for my sample scenario, but if it's anything more complex than that, the tests are at the tender mercy of the org configuration. I create an in-memory Opportunity to test my discount manager, but if I had to insert it into the database, there's any amount of validation or expected data that could trip me up. While there are techniques that could be used to help ensure my tests can pass, mandating that everyone who installs my package has to have used them across all their automation seems unlikely to be met with a positive reaction.

What it is more likely to mean is that running all tests across all namespaces is more likely to succeed, as the tests which could easily be derailed by the org configuration can be left out of the package, just leaving the ones that are completely isolated to the package code in place, so swings and roundabouts.

What it does mean is that the package author has the choice as to whether the tests are packaged or not, which seems the way it should be to me.

Related Posts

Saturday, 10 April 2021

Bake It Till You Make It



As we start to come out of the third lockdown in the UK, blinking in the not so bright light of yet another overcast day, it's important to keep occupied. BrightGen have been pretty good at finding fun things to do, and the baking challenge returned for Spring. We all get an allowance for our ingredients, so the only outlay is our time and loving attention.

I'm not that fussed about cakes, and we'd only just finished up the last couple of slices of my Great BrightGen Bake Off entry from November 2020 (it was in the freezer rather than rotting in a tin!) so I decided to go savoury with beef loaf en croute. I've made this many times over the years, but this time was different. First I was making the pastry from scratch. I haven't done this in about 10 years as the supermarket version is just as good, but that didn't seem in the spirit of the challenge. The second difference was that it really mattered if it came out okay - this wasn't just for guests to eat at a sophisticated dinner party, this was going on social media!

The Hashtag

In the previous incarnation of the BrightGen bake off, the hashtag proved a bit of a challenge. That's supposed to say GBGBO - Great BrightGen Bake Off. It's just about readable, but hardly a triumph. I wasn't expecting much better this time, but I was determined to try!

After making the pastry and leaving it to rest in the fridge for a couple of hours, I nicked off the ends and created the hashtag lettering - minus the hash symbol, as that's just dead space. Not too bad, but boy do you have to be quick with puff pastry or it sticks to everything, and there are a lot more letters this time!

Next  it was time to make the meat loaf - minced beef, onions, apples, breadcrumbs, stock and a few herbs, then 45 minutes in a Bain-Marie and it was cooling. This is by far the easiest part of the whole process, but make sure you leave it to cool down completely or you'll leave half of it in the tin - I know this for a fact as I've spent more than one evening frantically trying to piece it back together like the world's worst game of tetris.

Several hours later I returned to assemble, only to experience the revenge of the hashtag (if you saw the outcome on social media, just forget that for a couple of paragraphs and play along with my artificial jeopardy) - it doesn't fit on the loaf. 

Nothing to be done now, as the whole thing was assembled, and the pastry was milk-washed and already starting to sag. Into the oven and hoping for the best.

Even if I say so myself, it came out alright:

Baker's Remorse

What happened? How did it go so wrong? I knew the letters were large as I wanted them to fill the top of the loaf, but did I really think I could fit 12 letters of that size with the surface area of a 1 pound loaf tin.

If only I’d made a smaller one to take the second part of the hashtag.

Oh wait.

That’s right.

I did!

As I'm still shopping for elderly parents, whenever I cook anything like this I make extra to send to them with their weekly groceries, and it turns out the 1/2 pound loaf tin surface area is perfect for the last 4 letters.

Here’s the finished articles and a sneak peek at the official cutting:

Saturday, 3 April 2021

Unbounded Queries [Featuring Spring 21 FIELDS()]


FIELDS() is a new SOQL function in the Spring 21 Salesforce release that allows you to query standard, custom or all fields for an sObject without having to know the names. Could it be, are we finally getting the equivalent of SELECT * so beloved of developers working with an SQL database? Can we retire our Apex code that figures out all of the field names using describe calls? Digging into the docs, the answer is no. You can pull back the standard fields, but not all of them.

Disappointing, right? Maybe at first glance, but from a future proofing your Apex code perspective, absolutely not. It's actually saving you from having unbounded queries lurking like a time bomb in your Salesforce instance, waiting to blow the heap limit on an unsuspecting user.

Unbounded Queries

Simply put, an unbounded query doesn't limit the results that may be returned from the database. 

Most of the time, we (and the rest of the developer world) think of unbounded queries as those without a restrictive WHERE clause, or with no WHERE clause at all. These queries start out life doing their best to be helpful - backing custom pages with search capabilities where the user can choose some criteria to restrict the number of records, but can click search straight away if they want to. When there are 20 records in the system, this isn't a problem. Once you've been life for 5 years and you have 15,000 records, it's an accident waiting to happen. I think of this as vertically unbound, as something needs to be done to limit the depth of the record list retrieved. 

So putting in a LIMIT clause to your query solves the problem and future proofs your application? Not quite. Queries can be unbound horizontally as well as vertically - each record can become so wide that the total number that can be retrieved is massively reduced. I view this is a problem that impacts Salesforce implementations more than other technologies, as it's so easy to add fields to an sObject and change the database schema. No need to involve a DBA, just a few clicks in Setup. The odd field here and there racks up over the years, and if your query is retrieving all fields on a record, the size of each record will become a problem, even if you are restricting the number of records that you retrieve.

Some Numbers

To reflect the 'SELECT *' concept, I threw together some Apex that dynamically creates a query by iterating the field map returned by the schema describe for an sObject. I created 500 Contact sObjects with a variety of fields present, then executed some anonymous Apex to query them back, iterate them and debug the records and debug the heap consumed. I took the simplistic view that the entire heap was down to the records retrieved, so all calculations around the maximum records will probably be over optimistic, but I'm sure you get the point.

With no contacts, the heap was a svelte 1968 bytes. 

With 500 contacts with the First and Last Name fields defined as 'Keir Bowden', the heap was 250,520 bytes, so approx 500 bytes per record. This sounds like a lot for a couple of fields, but there are quite a few standard fields that will be retrieved too - Id, IsDeleted, OwnerId, HasOptedOutOfEmail, HasOptedOutOfFax, DoNotCall, CreatedById, CreatedDate, LastModifiedId, LastModifiedDate, SystemModStamp, IsEmailBounced, PhotoUrl. A whole bunch of stuff that I probably don't care about. Not too much to worry about though, as at that rate I can retrieve 12,000 contacts before approximately blowing the heap. This is a good example of why adding a LIMIT clause to a query is a good idea - 12,000 contacts really isn't that many for a decent sized organisation that has been around for a few years!

Adding Salution of 'Mr' to all of the records caused the heap to creep up to 253,864 bytes - 508 per record. A small increase, but one that has a material impact on my max records - I'm down to 11,811 - a drop of 189 records through a single field that maybe I don't need for my processing.

I then added a unique Email Address and a Lead Source of Purchased List - moving the heap needle to 288,454 bytes - an increase of 70 bytes per record over the previous run. I'm now down to 10,380 records, so I'll have to process 1,431 less records in my transaction to avoid the limit. All because of fields that are present on the record, not because of the fields that I want. 

I upped the ante on my next change - I decided my Sales Reps wanted a notes style field on the record where they could record their unstructured thoughts about the customer. So I added a Text Area (Long) field, and add 1,000 characters to each of my records, which isn't a lot for free text - if you are doubting that, consider that at this point in this blog post I am around 5,000 characters. The impact on the heap was much as expected, and my heap use is now at 794,946 bytes at a hefty 1,600 bytes per record. I'm now down to 3,750 records per transaction! I think it's highly unlikely that I have any interest in the Sales Rep's random musings, but I'm down to around 31% of the records in no small part because of it.

Hopefully there's no need for me to labour the point - specifying a LIMIT clause is only part of the solution, and it's unlikely to help if the size of each record is something that you are exerting no control over.

Other Problems With Unbounded Queries

Even when they aren't breaking limits, unbounded queries bring unwanted behaviour:

  • If you are sending the records to the front end, you could end up transmitting way more data than you need to, taking longer and putting more stress on the client who has to manage the bloated records.
  • It makes the code harder to read - if the only query is to retrieve all fields, you need to dig into how the results are used to figure out which fields are actually needed in specific scenarios.
  • You lose static bindings to field names. This is unlikely to be a problem, as you'll almost certainly be referring to the fields you want to process by name elsewhere in your Apex code. But if you are processing fields using dynamic DML, then if an admin deletes one of the fields you are relying on, the first anyone will know about it is when you try to access the field and it's not there any more.
Even if you absolutely, definitely, have to have all of the fields that are currently defined on an sObject, before using an unbounded query ask yourself if you absolutely, definitely, have to have all the fields that could possibly be defined in the future - including those that really don't belong on the contact but it was easier than making some fundamental changes to the data model! Yes, you'll save yourself some typing, but you'll be storing up problems for the future. Maybe not today, maybe not tomorrow, but some day it's going to go pop!

So Never All Fields?

As always, never say never. For example, I'll sometimes use this technique in debug Lightning Web Components where I want to dump everything out about an object that is in use on a page, even the fields that aren't currently being retrieved. In pretty much every case where I do use this approach, the query will return a maximum of 1 record!

Related Posts

Thursday, 1 April 2021

Einstein Not Your Worst Action

Einstein Next Best Action has been a feature of Salesforce for a while now, bringing the right recommendations, to the right people, at the right time. But not everyone wants to provide a stellar service to their customers - perfect is the enemy of good, after all - and sometimes good enough is good enough.

To satisfy this demand, Bob Buzzard Enterprises is pleased to launch Einstein Not Your Worst Action - Artificial Mediocrity for the apathetic masses. Using some, but nothing like all, of the power of Einstein, you can bring tolerable recommendations to people who might be interested within a few hours. 

Rather than Recommendations and Strategies, we favour Suggestions and Tactics. No need to plan long term, just knee-jerk react based on incomplete information and move on, secure in the knowledge that you did an okay job. 

Einstein Not Your Worst Action is available for Salesforce Limited edition for one day only - contact your Unaccountable Executive to find out some of the details.

Saturday, 27 March 2021

Org Documentor - Field Count Badges


With large, mature implementations, it's sometimes difficult to keep track of how close you are getting to the field limits - for example, the total number of fields on an object, which varies from instance to instance, or the total number of relationship fields, which is 40 unless Salesforce are willing to raise it for you. This is compounded when you have multiple development streams working in parallel, as the total number of relationship fields in only known when the various branches are merged ready for cutting a release branch. 

As Peter Drucker said, if you can't measure it you can't improve it, so the Org Documentor is getting into the measurement business.

Field Count Badges

The latest release (3.6.0) of the Org Documentor aims to help with this, by adding Bootstrap badges containing field counts to the object documentation. Right now it's the total number of fields on the object and the total number of relationship fields, as the screen grab below from the sample metadata output shows:

If there are no relationship fields, the badge isn't rendered:

Right now the badges don't change colour to flag up that the count is close to the limit, as there's no way to know if that is the case, as the Documentor is designed to work against object metadata and doesn't have access to any orgs that the metadata is installed into. Flagging the total number of fields as close to the Enterprise Edition limit isn't helpful if it will only be installed into Unlimited Edition, and vice versa is even worse.  It's also possible that there are other fields on the object that aren't version controlled - added by a managed package for example.

So I decided to leave it as the standard Bootstrap secondary badge colour, which will remain the case unless I get feedback from users that they'd like something else. I'm also open to suggestions for additional badges - if you have any strong feelings on either of these topics, please create an issue at the plug-in Github repo.

As always, you can view the latest output for the sample metadata on the public site.

Updated Plug-in

Version 3.6.0 of the plug-in has this new functionality and can be found on NPM

If you already have the plug-in installed, just run sfdx plugins:update to upgrade to 3.6.0 - run sfdx plugins once you have done that to check the version.

The source code for the plug-in can be found in the Github repository.

Related Posts

Thursday, 25 March 2021

The Value of Diminishing

I've been reading recently about diminished reality, where hardware devices and/or software remove items or stimuli from your environment. What drew me in was the diminished aspect, which I originally assumed was a joke about a device that would make things worse, but that isn't the case. While we hear a lot about augmented reality, where your experience is improved by additional information about the objects in view, diminished reality is all about improving your experience by hiding things. Not the things you are interested in, but the things that are a distraction, or simply not relevant.

A lot of us already have diminished reality devices and use them a lot - we call them noise cancelling headphones, which remove the external sounds that would distract us to improve our listening experience, although I've never thought of them in that way.

There are plenty of practical applications for diminished reality, for example seeing what rooms look like without the furniture and clutter when viewing real estate. Handle with care though - even though diminished reality removed the coffee table from your view, you'll still bark your shin when you walk into it. I could even use it to remove my lockdown locks when I'm on video calls, as these appear to be a major distraction for anyone I'm talking to!

This got me thinking about improving the user experience by removing rather than adding when working with Salesforce data. The customer 360 view without the clutter from the individual's entire history with your company, just showing the important engagements and key information. In my experience, most of us focus on pushing more information onto a page rather than less, but that isn't always a good thing as eventually it can end up overwhelming. Dividing information up into sections helps a little, but it's still up to the user to sort wheat from chaff.

What constitutes clutter would be quite subjective though, so ideally everyone would be able to define their own version of what matters. Imagine the ability on a per-object basis to provide different lenses on a record - click a button to change the view of the record based on metadata configuration that you define to effortlessly breeze through the Sales, Marketing, Service view, or combinations. 

We can do this to some degree already, with page layouts by switching applications, but it's not real time and all users don't necessarily have access to all layouts. If less is more, then many versions of less is much more!

Subscribe to my Substack

Saturday, 20 March 2021

London's Calling 2021

London's Calling 2021 was an all remote affair - the 2020 edition started this trend, but there was a small in person presence as the physical facilities were all in place. A sound decision this year, as we are currently under restrictions that mean we can meet one person for exercise/coffee in a public open space. I guess we could all have our own mini-event where two of us get together and present a talk to each other, but quite hard to scale I feel.

It was a very different day for me - typically I'd be up around 05:30 to walk the dogs before catching the bus to the station for the trip up to London. I still did the walk, but with an extra hour or two in bed. At 07:30 rather than wandering around near The Brewery waiting for it to open, I was sat on the sofa refreshing the Attendify app waiting for it to open! The commute was a lot easier though, and I could spend most of the day barefoot which I'd be reluctant to try in the real city. Lunch was a lot easier to come by as well - hardly any queueing in my kitchen, although the menu wasn't as varied.

The sessions were pre-recorded and replayed as live with the speaker on hand in the chat to answer questions. This is definitely my preferred route for these kind of events - while I like the "anything can happen" excitement of presenting live, home broadband connections just aren't solid enough to rely on for 50+ sessions across a whole day. Mine decided to drop just after I joined the BrightGen sponsor room and it took me a couple of minutes to reconnect via my phone - I suspect I'd have lost most of my audience during that time.  You don't get a lot of questions as the schedule is pretty compressed and most people are jumping on to the next session rather than sticking around to chat, but remember that all of us speakers are contactable via the usual channels after the event. The deck from my talk is available on SlideShare.

The keynote was, as always, thought provoking. The key takeaway for me was that I'm Batman! Or all of us attendees are Batman. Or more accurately, the skill sets that the attendees have will be key to the new normal. A talk will always go down well when the audience are told they are super heroes. 

A common thread between physical and virtual events is not being able to make most of the sessions that look interesting, and this was no exception. Typically half a dozen sessions on at any one time meant that I had to make my decisions based on whether I wanted to ask any questions. The organisers have done a sterling job to make the sessions available immediately after the event, so I'll be catching up over the coming week.

The bit I really missed was the networking - there's an element of this through the Attendify activity feed and the sponsor rooms, but nothing like the in person experience of bumping into someone in a corridor that you haven't seen for ages and hearing about what they've been up to. London's Calling brought this into sharp relief as I've known a lot of the attendees for years but don't catch up with them in person that much, even when there isn't a pandemic.

The other part this is very difficult to replicate is the after party, although if we'd had one of those I might not be sitting here writing this post at 07:00 on Saturday morning!

There's definitely an upside to the all remote approach - no travel means the opportunity to speak for those who don't have the spare funds to jet around the world. Likewise no visas means individuals aren't at the mercy of government departments, and I hope that when things are back to "normal" next year(?) some aspect of this will be retained for a hybrid event - maybe a virtual room or two so the physical audience can listen to virtual speakers, and a virtual ticket so that people who can't get over to London can still get access to the content and join in via the app. I'll be watching what happens with great interest.

The event might have been a little different in its location and execution, but once again it was awesome. Kudos to the organisers, all volunteers let's not forget, for showing some of the bigger events how things should be done. One pure virtual is enough for me though, and I can't wait for next year when we'll (hopefully) be able to get together in person.

Subscribe to my Substack