Not Hotdog - Salesforce Einstein Edition

Introduction

Anyone who is a fan of HBO’s Silicon Valley show will be familiar with Not Hotdog, Jian Yang’s app that determines whether an item of food is a hotdog or not. In a wonderful example of fiction made fact, the show have released iOS and Android applications in real life - you can read about how they did this on their medium post. Around this time I was working through the Build a Cat Rescue App that Recognises Cat Breeds Trailhead Project, which uses Einstein Vision to determine the breed of cat from an image, and it struck me that I could use this technology to develop a Salesforce version of Not Hotdog.

Building blocks

Trailhead Playground

As I’d already set up Einstein Vision and connected it to my Trailhead Playground, I decided to build on top of that rather than create a new developer edition. 

Einstein Vision Apex Wrappers

A key aspect of the project is the salesforce-einstein-vision-apex repository - Apex wrappers for Einstein Vision produced by Developer Evangelist René Winkelmeyer. The project somewhat glosses over these, but they provide a really nice mechanism to create and train an Einstein Vision dataset and then use that for predictions. It takes away pretty much all the heavy lifting, so thanks René. 

Public Access Community

Let’s be honest, there was no way I was going to build a full-fledged app for this. I did consider building an unmanaged package and including the images I used to train the dataset, but it seemed a bit crazy to have everyone creating and training their own dataset for the same purpose. Given my reach in the Salesforce community this could literally result in tens of duplicate datasets :)

I therefore decided to expose this as an unauthenticated page on a Salesforce community. I had the option of using a Site but I also wanted to play around with unauthenticated access to Lightning Components and the docs say to use a community. 

Putting it all together

I had to make one change to the Einstein Vision Apex Wrappers - I couldn’t get the guest user to be able to access the Salesforce File containing the Einstein Vision key, so I just hardcoded it into the EinsteinVision_PredictionService class. Evil I know, but this is hardly going into production any time soon.

I then created a dataset named ‘nothotdog’ and trained it via a zip file of images. The zip file is organised into a directory per label - in my case there were two directories - ‘Hot Dog’ and ‘Not Hot Dog’.

I then added the following method to the EinsteinVision_Admin class, to match a supplied image in base64 form against the dataset.

@AuraEnabled
public static String GetHotDogPredictionKAB(String base64) {
    String hdLabel='Unable to match hotdog';
    Blob fileBlob = EncodingUtil.base64Decode(base64);
    EinsteinVision_PredictionService service = new EinsteinVision_PredictionService();
    EinsteinVision_Dataset[] datasets = service.getDatasets();
    for (EinsteinVision_Dataset dataset : datasets) {
        if (dataset.Name.equals('nothotdog')) {
            EinsteinVision_Model[] models = service.getModels(dataset);
            EinsteinVision_Model model = models.get(0);
            EinsteinVision_PredictionResult result = service.predictBlob(model.modelId, fileBlob, '');
            EinsteinVision_Probability probability = result.probabilities.get(0);
        }
    }
        
    return hdLabel;
}

Next I needed a lightning component that would allow me to upload a file and send it back to the server, to execute the method from above. However, I also wanted this to work from a mobile device as file inputs on the latest Android and iOS allow you to take a picture and use that. The problem with this is that the image files are pretty huge, so I also needed a way to scale them down before submitting them. Luckily this can be achieved by drawing the image to an HTML5 canvas element scaled to the appropriate size.

Unfortunately this threw up another problem, in that when the Locker Service is enabled you don’t have an image element that can be drawn on a canvas, you have a secure element instead. There is no workaround to this so I had to drop the API version of my component down to 39. I guess one day the Locker Service will be finished and everything will work fine.

There’s a fair bit of code in the NotHotdog Lightning Component bundle so rather than making this the world’s longest post you can view it at this gist.

Next, I needed an app to surface the bundle through a Visualforce page. These are pretty simple, the only change to the usual way this is done is to implement the interface ltng:allowGuestAccess:

<aura:application access="GLOBAL" extends="ltng:outApp"
    implements="ltng:allowGuestAccess">

    <aura:dependency resource="c:NotHotDog"/>
</aura:application>

Finally, the Visualforce page that is accessible via the community:

<apex:page docType="html-5.0" sidebar="false" showHeader="false" standardStylesheets="false"
           cache="false" applyHtmlTag="false">
    <html>
        <head>
            <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no;" />
        </head>
        <body>
            <apex:includeLightning />
            <script>
            $Lightning.use("c:NotHotDogApp",
                           function() {
                               $Lightning.createComponent(
                                   "c:NotHotDog",
                                   { },
                                   "lightning",
                                   function(cmp) {
                                   }
                               );
                           }
                          );
            </script>
            <div id="lightning" />
        </body>
    </html>
</apex:page>

Yes we’ve got a video

Here’s a video of the app doing it’s thing - first recognising a hotdog and then correctly determining that the BrightGen head office building is not a hotdog. What a time to be alive.

 

 

It’s not bullet proof

The HBO team trained their app with hundreds of thousands of images, I just did a couple of hundred because this isn’t my day job! It’s pretty good on obvious hotdog images, but not so much when you take photos. Your mileage may vary. Also, take photos on a phone in landscape mode as most of them rotate it.

Try it out yourself

If you’d like to enjoy the majesty of this application on your own machine:

• Navigate to the short link : http://bobbuzz.me.uk/nothotdog or
• Scan the QR code below:

Meetup

if you’re in London on Aug 2nd 2017, we’ll have a talk on Einstein Vision at our developer meetup. Sign up at : 

    https://www.meetup.com/LondonSalesforceDevelopers/events/237321315/

Related Information

• Build a Cat Rescue App that Recognises Cat Breeds (Trailhead Project)
• salesforce-einstein-vision-apex (github repository)
• Einstein API Documentation (well worth a look to understand what the apex wrappers are doing under the hood)
• How HBO’s Silicon Valley built “Not Hotdog” with mobile TensorFlow, Keras & React Native (Medium post)

 

 Follow @bob_buzzard

 

 

Lightning Testing Service Part 1

Introduction

Back at Dreamforce 16 I gave a talk on Unit Testing Lightning Components using Jasmine. During that talk I said that I hoped that Salesforce would come up with their own testing framework for Lightning Components. I wasn’t disappointed as the Lightning Testing Service (LTS) went into Pilot at the end of May and I was lucky enough to be invited in. It’s been a slight challenge to find enough time to try out LTS while still taking SFDX through it’s paces and making sure I give full attention to my day job, but it’s worth the effort.

The LTS

The LTS is available on github for anyone to try out - I’m not sure how much support you’ll get if you aren’t on the pilot, but I’ve found it works as-is. Now that SFDX is in open beta you can easily combine the two - I’ve just done exactly that and it took around 30 minutes including signing up for trial orgs, downloading the latest SFDX CLI etc. 

The Lightning Testing Service is agnostic about the JavaScript testing framework that you use, but all the samples are based on Jasmine. Having used a few of them I think this is a good idea as Jasmine has a great set of features and most importantly an equivalent for most of the features of the Apex testing framework. The one area that Jasmine is lacking, I think, is the documentation. There are plenty of examples but not that much in the way of explanation as to how the examples actually work. While you can dig into the code as it’s all open source (https://jasmine.github.io/), if you are reasonably new to JavaScript and/or front end unit testing it’s a struggle. I found Jasmine JavaScript Testing by Paulo Ragonha to be an excellent introduction. While the latter chapters of the book focus on React, the first 6 chapters cover generic testing with Jasmine and explain the concepts and features really well (I have no affiliation with the book or author).

Apex eye view

Jasmine concepts map to Apex test concepts as follows: 

Apex	Jasmine	Syntax
Suite	Suite	describe('initialise', function () {...})
Test Method	Spec	it('gets the accounts', function () {...})
Assert	Expectation	expect(component.get("v.status")).toBe("true");
Setup	beforeEach/All	beforeEach(function(){...});

Jasmine also has a couple of concepts that Apex doesn’t have:

• afterEach/All - teardown code to be executed after a spec (afterEach) or the last spec (afterAll). You don’t have this in Apex as test transactions are rolled back so there is nothing to teardown. 
• Spies - these allow you to stub out a function, track the number of times it has been called, the parameters it was called with. These are really useful when you don’t have transactions that automatically rollback as you need to make sure you stub out anything that might commit to the Salesforce database.

Running Tests

One of the challenges when unit testing on the front end is figuring out how to execute the tests. The route I went was to make it the responsibility of a component to notify a controlling component that it had unit tests and to schedule those tests. There were a couple of downsides to this:

1. The test code was tightly coupled with the production code so would be deployed to production
2. The controlling component had to know how many components had tests so that it could wait until the appropriate number had notified it that their tests were queued.

When I presented this I made the point that there are a number of ways of doing this, and the LTS takes a somewhat different approach.

There still has to be a component that is responsible for loading Jasmine, setting up the reporter(s) and managing the tests, and LTS examples have one of these. This component also schedules the tests by loading one or more static resource that contains a collection of Jasmine test suites. As these resources are loaded by the <ltng:require /> tag, the JavaScript code is automatically executed by the browser and schedules the test with the Jasmine runner.

This approach has the upside of decoupling the test code from the actual component, allowing you full control over whether you want to deploy them to production, and removing the requirement for the component executing the tests to know anything about how many tests are being executed. It also allows you to easily group tests into functional areas.

The downside is that it decouples the test code from the actual component, which means that if you want to stub out a method it has to be exposed as part of the components API via an <aura:method /> attribute. I’m not mad keen on this as it feels like I’m exposing the internals for pure testing purposes and I can’t stop my Evil Co-Worker from creating components that use these methods for nefarious purposes. That said, I’m pretty sure it would be possible to leave tests that rely on access to a components internals inside the component itself by dynamically creating the component once the Jasmine framework is all set up. This is something I hope to cover in a later blog post assuming I can get it working!

SFDX Integration

This is probably the coolest aspect of the LTS. The SFDC CLI with Force plugin 40 includes a new command to execute lightning component unit tests :

sfdx force:lightning:test:run

This creates a browser session and navigates to a lightning application (default Tests.app), which executes the tests. The CLI is then able to get at the results of the tests and output them. I’m not sure how this last piece works, but it feels like something you’d need to find a way to replicate if using another JavaScript testing framework.  What it means, however, is that you can include these unit tests in a continuous integration process, thus ensuring that your components work as expected.

That’s it for part 1 - there’s a lot of information at the github repo and there’s no point in me replicating that just to get some eyes on my blog.

Related Posts

• Salesforce DX Beta Setup Guide
• Github repo for the LTS
• Jasmine home page
• Recording of my Dreamforce talk
• Slideshare of my Dreamforce talk
• Github repo for my Dreamforce talk

 Follow @bob_buzzard