The Awesome New Functionality of ODA Part Two
Part one of this, which covers the background, is here.
Before looking at some code, it’s worth just covering performance. Obviously, on a busy production server, if you’re running code for every save of a document, code needs to be optimised. To give you an idea, here’s a snippet of conversation:
“About” is good, but something more accurate is better. This happened the following morning, early October, with code echoing event messages to the server console. The first outcome was this:
That’s pretty impressive, but why have a thin wrapper for the Message Queue when it can be even slimmer:
I started off by mentioning the Database Listeners that have been in ODA for some time. There’s a reason I did that: the code you write for handling Message Queue events is the same kind of thing.
- A class implementing IEMBridgeSubscriber is passed to the
- The class implements the
getSubscribedEventIds()method, which returns an ArrayList of
- A void
handleMessage()method runs code based on whether the eventId passed in is the relevant
EMEventIdsgetting the content of the message from the
You can get the sample code I used in the OSGi plugin attached at the end of this post. The subscriber is added in
org.openntf.domino.emSample.EmHttpServiceFactory.getServices() – I’ll come back to why.
The code for actually doing everything is in
org.openntf.domino.emSample.EmHttpEventSubscriber. The method to concentrate on for processing the Message Queue is
Lines 2-6 just call a method that increments a HashMap for every event triggered. This was useful for identifying what events are triggered when and how many times. I was hoping in my demo at Connect to use EM_NSFCONFLICTHANDLER, so that’s covered in lines 7-11, but I have been unable to work out when or how that one is triggered. There are two different events for updates, EM_NSFNOTEUPDATE and EM_NSFNOTEUPDATEXTENDED. One was triggered when creating a document in Notes Client, one from XPages, but I’ve since worked out EM_NSFNOTEUPDATE is also sometimes called via server-driven updates. So don’t take those references as actual delineations of the events.
Line 14-15 and 37-38 take the String message from the Message Queue (which is comma-delimited data, like what Nathan echoed in his stress tests) and parses it to a Java object. Each event has its own Java object to parse to, and the parsing is done by
EMBridgeEventFactory.parseEventBuffer. (This is still quite new and not all events have been encountered and tested, so if you’re using an event and it’s not parsing correctly, let the dev team know.)
Line 19-28 are quite interesting. This logs out any update to any field in a document in any database in ExtLib or OpenNTF folders or sub-folders. This was how I demonstrated hacking a document from a view, and still being able to capture what update was made by whom!
Lines 31 and 43 call a
postMessageToWatson() method. This reproduces the Sapho demo from the OGS. As I said in part one, I wrote this (including creating a very basic app with a Form and XPage, and working out I needed to check for two different events) within a couple of hours just before my session. I’ll let you peruse that at your leisure, but the code to write to Watson is just using the Watson Work Services Java SDK I wrote with Christian Guedemann to authenticate an app to Watson Workspace (to have this work, you’ll need to set up a Watson Workspace application, get its app ID and app secret, add it to a space and get the Space ID).
The other interesting class is EmHttpService. This is added in
EmHttpServiceFactory.getServices(). The problem with subscribing in a plugin or extension library (without an extension point) is the subscriber will not get registered until that plugin or extension library is used. HTTP services though get registered as soon as the HTTP task is loaded, because that’s when this
getServices() method of an instance of the com.ibm.xsp.adapter.serviceFactory extension gets called. There are two important aspects of HttpService instances.
First, they appear to get loaded very early on in the HTTP startup process. The
createInstance() method gets called even before ODA starts. It also blocks any other plugin loading until that method has completed. So that method can’t be used to trigger any ODA-related code.
Secondly, it has a
checkTimeout() method, which runs every 30 seconds while the HTTP task is running. This is very useful and would probably be the approach used for scheduling Xots processes. It’s used here to schedule a process running every 5 minutes to dump out the summary of all events triggered and the amount of times they’ve been triggered – the Map updated by line 4 of the
handleMessage() method above.
checkTimeout() methods calls
EventLogPrinter.getInstance().start(), which only runs if it’s not already started and calls the
trySchedule() method in the
EventLogPrinter class. This adds this class as a Xots task and so promptly triggers its
run() method. That runs in a loop as long as
isRunning is not set to false (which happens when the server shuts down or the process is otherwise interrupted). It uses a timeout comparing the next run time to now, which will continually tick down and also checks the tasklet has already run at least once. If it’s time to run (and initially nextRunTime – now will be less than zero, so will run), it calls
EmHttpService.getInstance().dumpLog() to dump out the summary to the server console. Finally it calls
getNextRunTime() to set the next run time to five minutes from now. This is like the Domino agent manager, which runs agents x minutes after completion. (With a Xots scheduler, I’d probably set a tempNextRunTime when it first triggers, then use that at the end, providing it was still in the future. Then it would run every 5 minutes – like LEI – rather than 5 minutes after last completion.)
QueueDispatcher, which kicks off any
IEMBridgeSubscriber waiting for each event, polls the queue every 500 milliseconds, which is why notifications can be processed so promptly.
There’s a lot here, but hopefully this description makes it clearer and I’m sure there are some useful techniques for developers. It’s worth bearing in mind that the
QueueListener tasks in
org.openntf.domino.extmgr.EmBridgeMessageQueue also run as ongoing Xots tasks, so with the EventLogPrinter, that’s three ongoing tasks out of the 10 default Xots threads. That may change in the future. Also, this is very much in its infancy so is still experimental at this stage. Indeed Cameron Gregor has just submitted a pull request for an extension point to add subscribers, which will be a big improvement. My experiences on a test server so far have been very encouraging about its potential.