Saturday, 17 April 2021

Unpackaged Metadata for Package Creation Tests in Spring 21

Introduction

The Spring 21 release of Salesforce introduced a useful new feature for those of us using second generation managed or unlocked packages (which really should be all of us now) - the capability to have metadata available at the time of creating a new package version, but which doesn't get added to the package.

I've written before about how I use unpackaged directories for supplementary code, but this isn't code that is needed at creation time, it's typically utilities to help the development process or sample application code to make sure that the package code works as expected when integrated elsewhere. 

The Scenario

To demonstrate the power of the new unpackaged concept, here's my scenario:

I have a package that defines a framework for calculating the discount on an Opportunity through plug and play rule classes. The package provides a global interface that any rule class has to implement:

public interface DiscountRuleIF 
{
    Decimal GetDiscountPercent(Opportunity opp);
}

 The rules are configured through a Discount_Rule__c custom object, which has the following important fields:

  • Rule_Class__c - the name of the class with the rule interface implementation
  • Rule_Namespace__c - the namespace of the class, in case the class lives in another package
And there's the rule manager, which retrieves all of the rules, iterates them, instantiates the implementing class and executes the GetDiscountPercent method on the dynamically created class, eventually calculating the total discount across all rules:
global with sharing class DiscountManager 
{
    global Decimal GetTotalDiscount(Opportunity opp)
    {
        List<Discount_Rule__c> discountRules=[select id, Rule_Class__c, Rule_Class_Namespace__c
                                              from Discount_Rule__c];
        Decimal totalDiscount=0;

        for (Discount_Rule__c discountRule : discountRules)
        {
            Type validationType = Type.forName(discountRule.Rule_Class_Namespace__c,discountRule.Rule_Class__c);        
            DiscountRuleIF discountRuleImpl = (DiscountRuleIF) validationType.newInstance();
    
             totalDiscount+=discountRuleImpl.GetDiscountPercent(opp);

        }

        return totalDiscount;
    }
}

So the idea is that someone installs this package, creates rules locally (or installs another package that implement the rules), and when an opportunity is saved, the discount is calculated and some action taken, probably discounting the price, but they are only limited by their imagination.

Testing the Package

I don't want to include any rules with my package - it purely contains the framework to allow rules to be plugged in. Sadly, this means that I won't be able to test my rule manager class, as I won't have anything that implements the discount rule interface. For the purposes of checking my class, I can define some supplementary code using the technique from my earlier post via the following sfdx-project.json : 

{
    "packageDirectories": [
        {
            "path": "force-app",
            "default": true,
            "package": "UNPACKMETA",
            "versionName": "ver 0.1",
            "versionNumber": "0.1.0.NEXT"
        },
        {
            "path": "example",
            "default": false
        }
    ],
    "namespace": "BGKBTST",
    "sfdcLoginUrl": "https://login.salesforce.com",
    "sourceApiVersion": "51.0"
}

Then I put my sample implementation and associated test classes into example/force-app/main/classes, and when I execute all tests I get 100% coverage.  I can also create a new package version with no problems. Unfortunately, when I promote my package version via the CLI, the wheels come off:

Command failed
The code coverage required to promote this version has not been met.
Please add additional test coverage and ensure the code coverage check
passes during version creation.

My sample implementation and the associated test code has been successfully excluded from the package, but that means I don't have the required test coverage. Before Spring 21 I'd sigh and pollute my package with code that I didn't want, just to satisfy the test coverage requirement. It would be a small sigh, but they add up over the years.

The unpackagedMetadata property

The new Spring 21 feature is activated by adding the unpackagedMetadata property to my sfdx-project.json, inside the default package directory stanza:

{
    "packageDirectories": [
        {
            "path": "force-app",
            "default": true,
            "package": "UNPACKMETA",
            "versionName": "ver 0.1",
            "versionNumber": "0.1.0.NEXT",
            "unpackagedMetadata": {
                "path": "example"
            }
        },
        {
            "path": "example",
            "default": false
        }
    ],
    "namespace": "BGKBTST",
    "sfdcLoginUrl": "https://login.salesforce.com",
    "sourceApiVersion": "51.0",
}

Note that I still have my supplementary directory defined, but I'm also flagging it as metadata that should be used at create time to determine the test coverage, but not included in the package.

Having created a new version with this configuration, I'm able to promote it without any problem. Just to double check I installed the package into a scratch org and the unpackagedMetadata was, as expected, not present.

Reduced Coverage in the Subscriber Org

One side effect of this is that the code coverage across all namespaces will be reduced in the subscriber org. Depending on your point of view, this may cause you as the package author some angst. It may also cause the admin who installed the package into their org some angst. 

Personally I don't think it matters - the coverage of managed package code doesn't have any impact on the subscriber org, and I don't think it's possible to craft tests inside a managed package that are guaranteed to pass regardless of what org they are installed into. I could do it for my sample scenario, but if it's anything more complex than that, the tests are at the tender mercy of the org configuration. I create an in-memory Opportunity to test my discount manager, but if I had to insert it into the database, there's any amount of validation or expected data that could trip me up. While there are techniques that could be used to help ensure my tests can pass, mandating that everyone who installs my package has to have used them across all their automation seems unlikely to be met with a positive reaction.

What it is more likely to mean is that running all tests across all namespaces is more likely to succeed, as the tests which could easily be derailed by the org configuration can be left out of the package, just leaving the ones that are completely isolated to the package code in place, so swings and roundabouts.

What it does mean is that the package author has the choice as to whether the tests are packaged or not, which seems the way it should be to me.

Related Posts






Saturday, 10 April 2021

Bake It Till You Make It

 


Introduction

As we start to come out of the third lockdown in the UK, blinking in the not so bright light of yet another overcast day, it's important to keep occupied. BrightGen have been pretty good at finding fun things to do, and the baking challenge returned for Spring. We all get an allowance for our ingredients, so the only outlay is our time and loving attention.

I'm not that fussed about cakes, and we'd only just finished up the last couple of slices of my Great BrightGen Bake Off entry from November 2020 (it was in the freezer rather than rotting in a tin!) so I decided to go savoury with beef loaf en croute. I've made this many times over the years, but this time was different. First I was making the pastry from scratch. I haven't done this in about 10 years as the supermarket version is just as good, but that didn't seem in the spirit of the challenge. The second difference was that it really mattered if it came out okay - this wasn't just for guests to eat at a sophisticated dinner party, this was going on social media!

The Hashtag

In the previous incarnation of the BrightGen bake off, the hashtag proved a bit of a challenge. That's supposed to say GBGBO - Great BrightGen Bake Off. It's just about readable, but hardly a triumph. I wasn't expecting much better this time, but I was determined to try!


After making the pastry and leaving it to rest in the fridge for a couple of hours, I nicked off the ends and created the hashtag lettering - minus the hash symbol, as that's just dead space. Not too bad, but boy do you have to be quick with puff pastry or it sticks to everything, and there are a lot more letters this time!

Next  it was time to make the meat loaf - minced beef, onions, apples, breadcrumbs, stock and a few herbs, then 45 minutes in a Bain-Marie and it was cooling. This is by far the easiest part of the whole process, but make sure you leave it to cool down completely or you'll leave half of it in the tin - I know this for a fact as I've spent more than one evening frantically trying to piece it back together like the world's worst game of tetris.


Several hours later I returned to assemble, only to experience the revenge of the hashtag (if you saw the outcome on social media, just forget that for a couple of paragraphs and play along with my artificial jeopardy) - it doesn't fit on the loaf. 


Nothing to be done now, as the whole thing was assembled, and the pastry was milk-washed and already starting to sag. Into the oven and hoping for the best.

Even if I say so myself, it came out alright:


Baker's Remorse

What happened? How did it go so wrong? I knew the letters were large as I wanted them to fill the top of the loaf, but did I really think I could fit 12 letters of that size with the surface area of a 1 pound loaf tin.

If only I’d made a smaller one to take the second part of the hashtag.

Oh wait.

That’s right.

I did!


As I'm still shopping for elderly parents, whenever I cook anything like this I make extra to send to them with their weekly groceries, and it turns out the 1/2 pound loaf tin surface area is perfect for the last 4 letters.

Here’s the finished articles and a sneak peek at the official cutting:







Saturday, 3 April 2021

Unbounded Queries [Featuring Spring 21 FIELDS()]


Introduction

FIELDS() is a new SOQL function in the Spring 21 Salesforce release that allows you to query standard, custom or all fields for an sObject without having to know the names. Could it be, are we finally getting the equivalent of SELECT * so beloved of developers working with an SQL database? Can we retire our Apex code that figures out all of the field names using describe calls? Digging into the docs, the answer is no. You can pull back the standard fields, but not all of them.

Disappointing, right? Maybe at first glance, but from a future proofing your Apex code perspective, absolutely not. It's actually saving you from having unbounded queries lurking like a time bomb in your Salesforce instance, waiting to blow the heap limit on an unsuspecting user.

Unbounded Queries

Simply put, an unbounded query doesn't limit the results that may be returned from the database. 

Most of the time, we (and the rest of the developer world) think of unbounded queries as those without a restrictive WHERE clause, or with no WHERE clause at all. These queries start out life doing their best to be helpful - backing custom pages with search capabilities where the user can choose some criteria to restrict the number of records, but can click search straight away if they want to. When there are 20 records in the system, this isn't a problem. Once you've been life for 5 years and you have 15,000 records, it's an accident waiting to happen. I think of this as vertically unbound, as something needs to be done to limit the depth of the record list retrieved. 

So putting in a LIMIT clause to your query solves the problem and future proofs your application? Not quite. Queries can be unbound horizontally as well as vertically - each record can become so wide that the total number that can be retrieved is massively reduced. I view this is a problem that impacts Salesforce implementations more than other technologies, as it's so easy to add fields to an sObject and change the database schema. No need to involve a DBA, just a few clicks in Setup. The odd field here and there racks up over the years, and if your query is retrieving all fields on a record, the size of each record will become a problem, even if you are restricting the number of records that you retrieve.

Some Numbers

To reflect the 'SELECT *' concept, I threw together some Apex that dynamically creates a query by iterating the field map returned by the schema describe for an sObject. I created 500 Contact sObjects with a variety of fields present, then executed some anonymous Apex to query them back, iterate them and debug the records and debug the heap consumed. I took the simplistic view that the entire heap was down to the records retrieved, so all calculations around the maximum records will probably be over optimistic, but I'm sure you get the point.

With no contacts, the heap was a svelte 1968 bytes. 

With 500 contacts with the First and Last Name fields defined as 'Keir Bowden', the heap was 250,520 bytes, so approx 500 bytes per record. This sounds like a lot for a couple of fields, but there are quite a few standard fields that will be retrieved too - Id, IsDeleted, OwnerId, HasOptedOutOfEmail, HasOptedOutOfFax, DoNotCall, CreatedById, CreatedDate, LastModifiedId, LastModifiedDate, SystemModStamp, IsEmailBounced, PhotoUrl. A whole bunch of stuff that I probably don't care about. Not too much to worry about though, as at that rate I can retrieve 12,000 contacts before approximately blowing the heap. This is a good example of why adding a LIMIT clause to a query is a good idea - 12,000 contacts really isn't that many for a decent sized organisation that has been around for a few years!

Adding Salution of 'Mr' to all of the records caused the heap to creep up to 253,864 bytes - 508 per record. A small increase, but one that has a material impact on my max records - I'm down to 11,811 - a drop of 189 records through a single field that maybe I don't need for my processing.

I then added a unique Email Address and a Lead Source of Purchased List - moving the heap needle to 288,454 bytes - an increase of 70 bytes per record over the previous run. I'm now down to 10,380 records, so I'll have to process 1,431 less records in my transaction to avoid the limit. All because of fields that are present on the record, not because of the fields that I want. 

I upped the ante on my next change - I decided my Sales Reps wanted a notes style field on the record where they could record their unstructured thoughts about the customer. So I added a Text Area (Long) field, and add 1,000 characters to each of my records, which isn't a lot for free text - if you are doubting that, consider that at this point in this blog post I am around 5,000 characters. The impact on the heap was much as expected, and my heap use is now at 794,946 bytes at a hefty 1,600 bytes per record. I'm now down to 3,750 records per transaction! I think it's highly unlikely that I have any interest in the Sales Rep's random musings, but I'm down to around 31% of the records in no small part because of it.

Hopefully there's no need for me to labour the point - specifying a LIMIT clause is only part of the solution, and it's unlikely to help if the size of each record is something that you are exerting no control over.

Other Problems With Unbounded Queries

Even when they aren't breaking limits, unbounded queries bring unwanted behaviour:

  • If you are sending the records to the front end, you could end up transmitting way more data than you need to, taking longer and putting more stress on the client who has to manage the bloated records.
  • It makes the code harder to read - if the only query is to retrieve all fields, you need to dig into how the results are used to figure out which fields are actually needed in specific scenarios.
  • You lose static bindings to field names. This is unlikely to be a problem, as you'll almost certainly be referring to the fields you want to process by name elsewhere in your Apex code. But if you are processing fields using dynamic DML, then if an admin deletes one of the fields you are relying on, the first anyone will know about it is when you try to access the field and it's not there any more.
Even if you absolutely, definitely, have to have all of the fields that are currently defined on an sObject, before using an unbounded query ask yourself if you absolutely, definitely, have to have all the fields that could possibly be defined in the future - including those that really don't belong on the contact but it was easier than making some fundamental changes to the data model! Yes, you'll save yourself some typing, but you'll be storing up problems for the future. Maybe not today, maybe not tomorrow, but some day it's going to go pop!

So Never All Fields?


As always, never say never. For example, I'll sometimes use this technique in debug Lightning Web Components where I want to dump everything out about an object that is in use on a page, even the fields that aren't currently being retrieved. In pretty much every case where I do use this approach, the query will return a maximum of 1 record!

Related Posts


Thursday, 1 April 2021

Einstein Not Your Worst Action



Einstein Next Best Action has been a feature of Salesforce for a while now, bringing the right recommendations, to the right people, at the right time. But not everyone wants to provide a stellar service to their customers - perfect is the enemy of good, after all - and sometimes good enough is good enough.

To satisfy this demand, Bob Buzzard Enterprises is pleased to launch Einstein Not Your Worst Action - Artificial Mediocrity for the apathetic masses. Using some, but nothing like all, of the power of Einstein, you can bring tolerable recommendations to people who might be interested within a few hours. 

Rather than Recommendations and Strategies, we favour Suggestions and Tactics. No need to plan long term, just knee-jerk react based on incomplete information and move on, secure in the knowledge that you did an okay job. 

Einstein Not Your Worst Action is available for Salesforce Limited edition for one day only - contact your Unaccountable Executive to find out some of the details.