Flow Tests in Winter 23
Introduction
Low code flow testing became Generally Available in the Winter 23 release of Salesforce. Currently limited to record triggered flows, and excluding record deletes, we now have a mechanism to test flows without having to manually execute them every time.
Of course we've always been able to include flow processing in Apex tests - in fact we had no choice. If a record was saved to the database, then all the configured automation happened whether we liked it or not. What we couldn't accurately test was whether the state of the system after the test completed was down to the flow, or something else that happened as part of the transaction. (Incidentally, this is why you shouldn't put logic in triggers - you can only test that by committing a transaction, which brings in all the other automation that has the potential to break your test). Now we can isolate the flow - although not as much as you might want to it turns out.
In this post I'm mainly focusing on what is missing from flow testing, as I'm comparing it to Apex unit testing which is obviously far more mature. While this might read as negative, it really isn't - I think it's great that flows are getting their own testing mechanism - something I've been demanding for a while!
To try this out, I've created a few tests against the Book Order Count flow from the process automation superset - this runs when a new order is received from a contact, iterating the contents of all of their orders and calculating the total number of books bought over their customer lifetime.
Lack of Isolation
While record triggered flow tests are isolated from the need to write records to the database, they aren't isolated from the contents of the database. In one way this is a good thing - if I want to execute my flow with an order containing line items, I need to use an existing record as the test only allows me to supply fields for a test parent record, not create child records. In every other way, this is not a good thing. If I go this route my test relies on specific records being present in the database, so they'll fail if executed in a brand new org. It also relies on the specific records not changing, which is entirely out of my control and thus makes my test very fragile.
Even when I'm not using line items, I find myself relying on existing data - to create an empty book order I need to identify a contact that exists in the system, instead of creating a contact as part of the test that is then discarded. This also leads to fragility - for example, I created a new contact named Testy McTest and used this in a test to confirm that if there are no orders for the contact the flow correctly identifies this. Here's a screenshot of my test passing - job done!
However, at some point in time another user running a manual test (or possibly my Evil Co-Worker, who has realised that randomly adding new data can mess up lots of tests) creates an order for Testy with a single book:
I'm blissfully ignorant of this, right up until I run my test again - knowing my luck in front of the steering committee to show the investment was worthwhile, and it no longer works:
Nothing in the flow has changed, but the data in the system that it relied on has, and that is entirely out of my control.
For anything other than simple flows, I think right now you'd have to be looking at running flow tests in a dedicated scratch org or sandbox that you control the contents of and access to.
Open the Box
Flow testing has the interesting concept of asserting that a node was visited rather than verifying the outputs are consistent with the node being visited. I can understand where this comes from - in Apex you can unit test the fine detail logic quite easily, as long as you've followed best practice around separation of concerns and functional decomposition, but in flows it's a lot more difficult as the UI doesn't really help you identify the relationships between subflows and parents. Understanding it doesn't mean I like it though - this is an example of Open Box Testing, where the test knows the intimate details of the implementation that it is testing. Open Box Testing does have some advantages, but the big disadvantage is that it tightly couples the test to the item being tested. If the logic needs to change and that involves removing nodes, you are likely to have to revisit your tests and remove your asserts around those nodes, whereas if you've used closed box techniques and are simply asserting the outcome, the entire implementation can change and your tests don't care.
Asserting
Checking the contents of complex variables managed by the flow, collections for example, also seems a little tricky. Right now I think I'd have to add variables to the flow to represent things like an empty collection, a record that I want to look for in the collection, or to store the length so that I can verify how many records I have. This is something that I really don't like to do - change my implementation purely for the benefit of the testing framework.
This is pretty much for the same reason as the lack of isolation I mentioned above - I don't have the ability to create variables in the test, so I have to compare against items that already exist.
Launching from External Tools
I've scoured the docs around flow tests and the APIs, and I can't find any way to launch a flow test from an external tool like the Salesforce CLI. In my view this is something that will hold back adoption - if the only way include this in our CI/CD processes is something like Selenium driving the UI, it all becomes a bit too clunky and we'll likely continue with the Apex testing approach, in spite of the limitations. I'd love to have missed this, so if that is the case please let me know and I'll gladly add a mea culpa update.
Conclusion
As I said earlier, this post has mainly been calling out what I can't do in flow testing, based on what I'm used to doing with test frameworks for programming languages like Apex, Java and JavaScript. I'm sure that flow testing will continue to receive significant investment and will become much more powerful as the releases roll by. Even though it is of limited benefit right now, the benefit is still tangible and you'll be much better off with flow tests and without them.
Related Posts
