Decision tables in the USoft platform – Part 2

What do you do as a USoft practitioner when a client shows you a decision table?

USoft Studio is for specifying rules. USoft Definer is for implementing rules. But you cannot draw a decision table in USoft Studio, and the Rules Engine you build in USoft Definer listens to business rules, not to decision tables.

Here is an an analysis and some suggestions. You must have come across decision tables yourself! Please add any comments, criticism and ideas that you have.

This is a 2-part article. Part 1 explored what decision tables really ARE. This Part 2 talks about how you can apply decision tables in the USoft platform.

Decision tables in USoft

Your client is a local council. They confront you with this decision table for determining the Budget Flexibility of a person applying for housing benefit.

What is best practice for you as a USoft developer? What should you do with this decision table in USoft?

USoft is a low-code platform. In an ideal world, USoft Studio would let you draw this table in the diagrammer, and then when you press a Red Button, USoft would generate a complete working application for you that makes the decision.

In practice, it doesn't quite work that way... This is partly because USoft lacks certain features. But another reason is that USoft capitalises on a different kind of capability: it is a rules engine, not a decision engine.

So you don’t get all the automation. You will need to put in some manual work. What is the best way to proceed?

In Part 1, you saw that decisions are often based on other decisions. The conclusion of one decision often acts as input for the next decision, and so on, until you have made the top-level decision:

No worries at this level. By nature, USoft is very good at this kind of "cascading up" of information. Simply write constraint SQL in USoft Definer as you would normally do. Write constraints like these:

update   applicant
set      housing_benefit_eligibility =
           <some calculation involving person_sustainability
            and housing_situation>

update   applicant
set      housing_situation =
           <some calculation involving budget_flexibility
            and lodging_suitability>

These constraints will activate each other automatically and in the correct order, no matter the order in which you write them.

But the decision tables themselves (the rules they contain, the values they contain) are not so easy. There are 2 typical ways to embed decision tables in USoft. Each has its own pros and cons, so both deserve attention. In practice, you can mix the two, but to keep the discussion simple, I will present them as separate solutions:

• Solution 1: Decision table as specification in USoft Studio
• Solution 2: Decision table as implementation in USoft Definer

Before looking at the details, however, we first need to ask the question: is decision modelling important for USoft developers at all? If USoft has means of its own, why bother with decision tables?

Is decision modelling important for USoft developers?

USoft does not fully support decision modelling.  But even if it did, you probably would not want to base the entire architecture of your USoft application on decision modelling.

Decision modelling is helpful beyond doubt. The Decision Model and DMN help you see the structure and purpose of decisions that you have in your application. They make explicit what is the underlying logical structure. This way, you don't get lost in the many visualisation possibilities that decision tables offer in the area of infographics. You can separate the rules logic from any (good) reasons you may have to (also) present user-friendly, non-normalised visualisations. (For more on these concepts, go to Part 1.)

Decision modelling also has limitations. You have to do a lot of work to detail just 1 top-level decision. If that decision is considered less important later, chances are you have to tear the whole structure down and start over. By contrast, in a USoft rules engine, you just remove the rules that are no longer important and the rest still works.

Decision modelling has a lot to say about how to draw a conclusion, but it does not have much to say about what to do with the result. The idea is to build a decision engine that produces an outcome based on a combination of input parameters, data and rules. From an architecture point of view, this is basically a good old procedure call: a request-response mechanism. A quick comparison shows that the USoft rules engine architecture is richer and more robust. The picture below shows that Decision Model and USoft Rules Engine both use data when they execute decision rules. But in the Decision Model, you make a simple request-response excursion to the data: you "dip into the data" and then come back. In USoft, the result of applying the rules goes right back into the same structured database from where you fetched the input data. This is what gives USoft its high level of data quality.

An interesting use case is to combine USoft with AI algorithms. In that scenario, specialised external AI software does the decision-making, often with the help of scientific data and measurements. The results are delivered to the USoft application. The purpose of the USoft application is to add administrative quality to the AI functionality. It captures and stores the decision results and guarantees a high level of data quality and traceability over time.

A particular added value of USoft is that it is very good at handling changing decision rules by adding timestamp information. This way, you can have decision results that were (or: will be) achieved at different times by applying different versions of the decision rules.

The soft power of decision modelling

A programmer might say: "Where is the added value of decision modelling? I can code the entire logic of the housing benefit example in just a few lines of procedural ("if-then-else") code."

This attitude is cutting too many corners. Logic should be clear not just to coders. Decision tables expose logic in an optimal way to a wide audience. They allow untrained stakeholders to see how decisions are made in the software code.

There is also a more fundamental argument. Decision structures are more than just "if-then-else rules". They are more than a formal calculation machine.

If you like, you can look at the housing benefit rules as a perfect example of technocratic bureaucracy gone too far. The fate of applicants seems to be delegated to an inhuman rules machine. A person who happens to fall into some wrong category along the way, perhaps because of an error, is simply refused housing benefit at the end.

A more interesting perspective is to pay attention to the extra information that the decision model conveys over and above the hard derivation rules. This way, you notice that the organisation at some point tries to get an idea of a person's budget flexibility. Hidden in these words is the concept that a person's entitlement to housing benefit must be linked to his financial possibilities in other areas of life. The decision model makes this policy explicit. You can now design interesting procedures around this idea. For example, a civil servant could be empowered to make allowances if there are individual reasons why a particular applicant has less budget flexibility than it might seem if you applied only the formal rules. Perhaps an exception could be then be made for that person, on condition that the reasons are explained in textual notes in the application and that the civil servant making the exception is tagged to these notes. This is the kind of "soft power" that USoft Studio is ideally suited to explain, and that USoft Definer is ideally suited to build into the application.

OMG expressly states the value of this kind of "soft power" in its SBVR, DMN and CMMN standards. SBVR, for example, is supposed to look at business rules not as formal inference rules, but rather as statements of intended conduct that a human could decide to ignore or violate. It is easy for IT implementors to overlook this value, and build no more than a formal algorithm.

Solution 1: Decision table as specification in USoft Studio

If you go in this direction, you will come across 2 limitations:

• You can't draw a decision table in USoft Studio diagrammer.
• It is not attractive to have hard-coded values in constraint SQL.

Let's analyse this in some more detail, and then find a "way out".

The decision table shown at the top of this Part 2 has the format recommended by the Decision Model. You can re-organise this table slightly, and produce something closer to natural language and easier to read. This kind of re-arrangement is not a USoft quirk. Organisations do this all the time. You end up with something like this. Let's call this a more intensional format:

This format allows you to see that a decision table is really a collection of business rules or, as the Decision Model calls them, a rule family:

In fact, these business rules are the classic USoft specification format for this kind of business logic.

The classic USoft implementation format is the following set of constraints. Each constraint mirrors (implements) one of the business rules:

update  applicant
set     budget_flexibility = 'High'
where   salary_bracket in ( 'A', 'B' )

update  applicant
set     budget_flexibility = 'High'
where   salary_bracket = 'C'
and     total_rent <= 700
and     spendable_income > 1600

update  applicant
set     budget_flexibility = 'Medium'
where   salary_bracket = 'C'
and 
(  
        total_rent > 700
    or  spendable_income <= 1600 
)

update  applicant
set     budget_flexibility = 'Medium'
where   salary_bracket in ( 'D', 'E' )
and     total_rent <= 400

update  applicant
set     budget_flexibility = 'Medium'
where   salary_bracket in ( 'D', 'E' )
and     total_rent > 400 and total_rent <= 900
and     spendable_income > 1000

update  applicant
set     budget_flexibility = 'Low'
where   salary_bracket in ( 'D', 'E' )
and     total_rent > 400 and total_rent <= 900
and     spendable_income <= 1000

update  applicant
set     budget_flexibility = 'Medium'
where   salary_bracket in ( 'D', 'E' )
and     total_rent > 900 and total_rent <= 1300
and     spendable_income > 1600

update  applicant
set     budget_flexibility = 'Low'
where   salary_bracket in ( 'D', 'E' )
and     total_rent > 900 and total_rent <= 1300
and     spendable_income <= 1600

update  applicant
set     budget_flexibility = 'Low'
where   salary_bracket in ( 'D', 'E' )
and     total_rent > 1300

This is a perfectly acceptable solution. You can make this solution very easy-to-find by putting all the rules and constraints in a single Rule Set called "Budget Flexibility".

But it is not a very visual solution. What you want with decision tables is a tabular display (a matrix display, a spreadsheet) that allows you, and all the other stakeholders in the project, to see at a glance how a decision is made. Ideally, people should be able to edit this decision table also.

The "way out" in USoft Studio is to draw the decision table in any tool you like, a specialised decision table tool, or maybe simply MS Excel, to store this drawing as a document, perhaps on your Sharepoint server, and to create a reference formulation in USoft Studio that points to it, like so:

A decision table for determining budget flexibility is located [ here | https://mysharepoint/... ].

The result in USoft Studio will look like this, with a clickable link:

You can now import this formulation in USoft Definer and register the set of (9) UPDATE constraints as being its implementation.

In this solution, values are hard-coded into constraint SQL. This is something of a taboo in the world of database modelling. Certainly it makes the decision table harder to change. But if you enable yourself to release a new version of the application quickly, this is not necessarily a problem. Solution 2 solves this problem, but there, you must do extra work to prevent the application from becoming too adaptable.

Why can't you draw a decision table in USoft Studio diagrammer? The simple answer is that this is on USoft's wishlist but has not been prioritised so far. The capability to draw DRDs (see Part 1 for an explanation) and decision tables and then connect them to each other, and to activities in the existing USoft Studio process flow diagrams, would most certainly have added value. But it's a large investment because, unlike the various diagram types you can draw in the existing USoft Studio tool, decision tables are graphically much closer to text and spreadsheets. The existing diagrammer software cannot simply be re-used. The investment would benefit only Solution 1, not Solution 2.

Solution 2: Decision table as implementation in USoft Definer

USoft Definer is specialised in letting you design and run database tables, and then overlay these tables with rules behaviour. USoft UI tools allow you to realise a variety of grid displays of data defined in USoft Definer.

This makes these tools a good candidate for handling decision tables.  

Shaping a decision table as a (database) table in USoft Definer positions the table as an implementation, not a specification. A problem you need to solve if you normally use USoft Studio or USoft Teamwork for business specifications, is that the decision table suddenly contains part of your business specifications elsewhere: in the runtime implementation accessible to application users.

You can, of course, reference the table from USoft Studio by registering it as the implementation of the business decision. That way, you improve traceability. But colleagues will still have to look in 2 different places to find out the entire working of the system.

In this solution direction, you can re-arrange the decision table slightly, and produce something closer to an ordinary set of relational data. This makes it more convenient for a machine to process. Let's call this a more extensional format:

With this decision table format, all you need is a single constraint:

update  applicant a
set     a.budget_flexibility = 
(
     select  b.budget_flexibility
     from    budget_flexibility b
     where   b.salary_bracket        =    a.salary_bracket
     and     b.total_rent_from       <    a.total_rent
     and     b.total_rent_to         >=   a.total_rent
     and     b.spendable_income_from <    a.spendable_income
     and     b.spendable_income_to   >=   a.spendable_income
)

Best practice: Decision tables in USoft

There is not a single best solution

With decision tables, there is not a single optimal solution. When you compare the Decision Model as a whole with the USoft model, there are too many pros and cons on either side to suit all needs. Within USoft, sometimes the advantages of specifying in USoft Studio take the upper hand, other times it is more attractive to model a decision table in USoft Definer. As for visualisation, extra work to produce a non-normalised table for user-friendliness can sometimes add great value. At other times, this extra work pays off too little, or even leads to excessive maintenance cost.

Make sure you respect or signpoint SPOT

With so many possibilities, it is tempting to specify or implement the same logic in 2 places. As a general rule, avoid this: do not sacrifice Single Point of Truth (SPOT). Giving up SPOT means exposing yourself to the danger that changes occur 3 months later, and colleagues make the change in one place but forget the other place, leaving you with a contradictory situation where the truth (the intended application logic) is lost.

If you absolutely want to have the same information twice, for example a technical implementation in constraints and a non-normalised visualisation for business stakeholders, then carefully signpost which is considered the source and which is the derived, secondary artefact. Always consider visualisation as secondary to underlying decision logic.

Choose Solution 1 or Solution 2 on a per-table basis

Leave the overall structure of decisions-calling-other-decisions to classic USoft constraints.

As for the decision tables themselves, decide on a table-by-table basis whether you apply the classic USoft approach (Solution 1) or implement the decision table by modelling it out in the application (Solution 2). Solution 1 is easier and more robust, but the decision logic is not visible to stakeholders and values must be hard-coded into constraint SQL. Solution 2 allows dynamic decision-making: you could empower end users to change decision parameters between releases of your application, and let the Rules Engine re-calculate results on every change, but you need to do a lot of extra work to guarantee that this "wild" behaviour stays within required boundaries and does not make the application too slow.

If you use Solution 2 to populate a database table with what are essentially design-time data, then you must make sure this data is neatly (re)delivered with every release, and that it is absolutely read-only to end users. Experienced USoft teams often choose this scenario. USoft Delivery Manager supports the delivery of design-time application data.

Use decision tables to model the interaction between USoft and AI

A good use case is to apply USoft for facilitating and embedding AI decisions. In this scenario, decision results that require complex reasoning or statistics are typically produced by external software. These results are consolidated in USoft, where they are typically presented in some kind of table (matrix) format (a database table, a logical view, a component table...), and where they typically take the place of decision table conditions. The purpose of the USoft application is to capture and store the AI results and to further process these results by deriving further decision conclusions. This way, you achieve optimal data integrity and data quality.

Tell us what you think

Do you have special tips for decision tables in USoft?

Do you want to share a successful way of using decision tables in USoft?

What do you need around decision tables to make your work easier?

Please let us know by replying to this blog.

Thank you!