Navigating LabWare’s Message Engine Template
Does your LabWare implementation connect to external applications, such as SAP, another LIMS solution, or an external database? Are there multiple interfaces between LabWare and those external applications? If so, your LabWare implementation will greatly benefit from the Message Engine Template, as it is LabWare’s standardized and validated way to connect with external systems. LabWare’s ME template allows a Lab’s data to flow seamlessly between multiple systems to facilitate a streamlined workflow. Having access to the same data between multiple systems reduces the time needed for users to manually input data as the ME template is an automated process.
In this post, we’ll learn about how LabWare’s Message Engine works, and how your lab could use it to manage interfaces with external applications. We’ll begin by examining the underlying design to understand what Message Engine does. Then we’ll look at how to make changes to, troubleshoot, and improve existing interfaces. Finally, we’ll investigate the use of Message Engine for adding new interfaces.
What Does Labware’s Message Engine Do?
Although the LabWare Message Engine template is a complex mechanism, when described simply it does two things:
1. Validates and pushes LIMS information out to external applications;
AND/OR
2. Processes and validates incoming information from external applications
The manner in which these actions are performed can vary between LabWare implementations, and the template allows for many different kinds of message formats (XML, XLS, TXT, JSON, etc..). Most of Message Engine is not configurable by non-LabWare developers, however there are certain parts (such as field additions) that can be done by non-developers. The Message Engine interface is scalable to accommodate many interfaces at once.
When pushing data to an external system, LabWare will validate the data to ensure that it meets the criteria for data to be pushed. Then it will create a message payload (in XML, JSON, etc.) and post the message to the ME_QUEUE table and wait for an external job to pick up and process the message. This message could be test results that were entered in the LIMS or static data updates that need to be synced between two systems.
The flowchart in example 1 describes how the LabWare Message Engine template handles receiving data:
Once messages have finished processing, the records remain in the ME_QUEUE table with a status of Processed, along with any errors or warnings the message encountered while processing. This way a user can go back and look at any message and see how it is processed.
For example, say there exists an interface between LabWare and SAP. SAP will push information about what tests need to be performed for a certain lot of product. SAP will send a message to LabWare with the product and tests that LabWare needs to perform. LabWare will log the Lot, Samples, and Tests it needs based on its normal static data rules. Then, a user will perform the tests and enter the results into LabWare. Once the results are reviewed, LabWare will automatically check to ensure that they meet the criteria to have the result values sent back to SAP. If they do, Then LIMS will construct a new message and send it back to SAP.
How Can I Make Changes To An Existing Interface?
The bulk of Message Engine processing is done via subroutines, but there are parts of the interface that are configurable, with the right skills and background. For instance, current implementations of the Message Engine template contain a table that holds all of the fields being passed into LIMS for a certain Message type, along with the associated LIMS table field that the incoming data is intended to populate:
Example Message:
Example Configuration:
In this example setup, LabWare LIMS is receiving an external message that is intended to populate Product/Material information. The first row is importing the Product name, and the value being passed into the LIMS from the message will be an exact copy of what was on the message because the translation directives column is NULL, meaning that no translation is required.
For the second row, the date is coming in as “22JUL2022” but the LIMS database may need to store that date in a different format such as “22-JUL-2022” so the translation directive column will indicate to LIMS how the date format is being received and how to translate the value. Then LabWare LIMS will translate the date into the correct format.
For simple interface updates, this type of configuration is easily performed by lab admins, however, more complex translation/validation directive changes will likely need subroutine changes and therefore need a LabWare expert to perform those changes.
How To Troubleshoot And Improve Existing Interfaces?
Although the Message Engine Template is robust, it is very complex—and complex systems aren’t immune to problems. A common issue that we’ve encountered with this solution is seen with high-volume interfaces, where the Message Engine Queue table and the background services that process new messages can get clogged up and create a backlog of messages that aren’t being processed immediately. There are at least three reasons why this could happen:
- The queries that extract or process data from the Message Engine Queue table may be taking too long to run. Consider talking to your database administrator to see if the table is properly indexed.
- The number of messages being processed per day may be too great and either additional computing power may be needed or additional background services need to be added.
- Inefficiencies in the processing method within the interface may be to blame; consider consulting a LabWare expert to identify the root cause.
It is important to note that the ME template was designed for high-volume interfaces, however, changes made to the interfaces over time can result in the issues listed above. If you feel that your system isn’t running as quickly as it used to, a CSols consultant skilled in LabWare can take a look under the hood.
If you’d like more information on how to get the most out of your Message Engine or if you’d like one implemented for your LabWare system, feel free to reach out to us.
