Blog

The Awesome New Functionality of ODA Part Two

  |   Blog   |   No comment

Part one of this, which covers the background, is here.

Before looking at some code, it’s worth just covering performance. Obviously, on a busy production server, if you’re running code for every save of a document, code needs to be optimised. To give you an idea, here’s a snippet of conversation:

“About” is good, but something more accurate is better. This happened the following morning, early October, with code echoing event messages to the server console. The first outcome was this:

That’s pretty impressive, but why have a thin wrapper for the Message Queue when it can be even slimmer:

Code Walkthrough

I started off by mentioning the Database Listeners that have been in ODA for some time. There’s a reason I did that: the code you write for handling Message Queue events is the same kind of thing.

  • A class implementing IEMBridgeSubscriber is passed to the EMBridgeMessageQueue.addSubscriber() method.
  • The class implements the getSubscribedEventIds() method, which returns an ArrayList of org.openntf.domino.extmgr.events.EMEventIds.
  • A void handleMessage() method runs code based on whether the eventId passed in is the relevant EMEventIds getting the content of the message from the eventMessage String.

You can get the sample code I used in the OSGi plugin attached at the end of this post. The subscriber is added in org.openntf.domino.emSample.EmHttpServiceFactory.getServices() – I’ll come back to why.

EmHttpEventSubscriber

The code for actually doing everything is in org.openntf.domino.emSample.EmHttpEventSubscriber. The method to concentrate on for processing the Message Queue is handleMessage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
	public void handleMessage(EMEventIds eventid, String eventMessage) {
		for (EMEventIds event : EMEventIds.values()) {
			if (event.equals(eventid)) {
				EmHttpService.getInstance().logEvent(event);
			}
		}
		if (EMEventIds.EM_NSFCONFLICTHANDLER.equals(eventid)) {
			// Don't think this ever triggers
			EmHttpServiceFactory.print("Conflict encountered");
			EmHttpServiceFactory.print(eventMessage);
		}
		if (EMEventIds.EM_NSFNOTEUPDATEXTENDED.equals(eventid)) {
			try {
				UpdateExtendedEvent event = new UpdateExtendedEvent();
				EMBridgeEventFactory.parseEventBuffer(eventMessage, event);
				System.out
						.println("Document " + event.getNoteId() + " in " + event.getDbPath() + " updated in Client by "
						+ event.getUsername());
				if (event.getDbPath().contains("ExtLib") | event.getDbPath().contains("OpenNTF")) {
					Session s = Factory.getSession(SessionType.NATIVE);
					Document doc = event.getDocument(s, s.getServerName());
					for (Item itm : doc.getItems()) {
						if (!(itm.getName().startsWith("$"))
								&& itm.getLastModifiedDate().equals(doc.getLastModifiedDate())) {
							System.out.println("Item " + itm + " modified to " + itm.getText());
						}
					}
				}

				if (event.getDbPath().contains("wherespace")) {
					postMessageToWatson(event);
				}
			} catch (Exception e) {
				e.printStackTrace();
			}
		} else if (EMEventIds.EM_NSFNOTEUPDATE.equals(eventid)) {
			UpdateExtendedEvent event = new UpdateExtendedEvent();
			EMBridgeEventFactory.parseEventBuffer(eventMessage, event);
			try {
				System.out.println("Document " + event.getNoteId() + " in " + event.getDbPath()
						+ " updated in XPages by " + event.getUsername());
				if (event.getDbPath().contains("wherespace")) {
					postMessageToWatson(event);
				}
			} catch (Exception e) {
				// TODO: handle exception
			}
		}
	}

Lines 2-6 just call a method that increments a HashMap for every event triggered. This was useful for identifying what events are triggered when and how many times. I was hoping in my demo at Connect to use EM_NSFCONFLICTHANDLER, so that’s covered in lines 7-11, but I have been unable to work out when or how that one is triggered. There are two different events for updates, EM_NSFNOTEUPDATE and EM_NSFNOTEUPDATEXTENDED. One was triggered when creating a document in Notes Client, one from XPages, but I’ve since worked out EM_NSFNOTEUPDATE is also sometimes called via server-driven updates. So don’t take those references as actual delineations of the events.

Line 14-15 and 37-38 take the String message from the Message Queue (which is comma-delimited data, like what Nathan echoed in his stress tests) and parses it to a Java object. Each event has its own Java object to parse to, and the parsing is done by EMBridgeEventFactory.parseEventBuffer. (This is still quite new and not all events have been encountered and tested, so if you’re using an event and it’s not parsing correctly, let the dev team know.)

Line 19-28 are quite interesting. This logs out any update to any field in a document in any database in ExtLib or OpenNTF folders or sub-folders. This was how I demonstrated hacking a document from a view, and still being able to capture what update was made by whom!

Lines 31 and 43 call a postMessageToWatson() method. This reproduces the Sapho demo from the OGS. As I said in part one, I wrote this (including creating a very basic app with a Form and XPage, and working out I needed to check for two different events) within a couple of hours just before my session. I’ll let you peruse that at your leisure, but the code to write to Watson is just using the Watson Work Services Java SDK I wrote with Christian Guedemann to authenticate an app to Watson Workspace (to have this work, you’ll need to set up a Watson Workspace application, get its app ID and app secret, add it to a space and get the Space ID).

 EmHttpService

The other interesting class is EmHttpService. This is added in EmHttpServiceFactory.getServices(). The problem with subscribing in a plugin or extension library (without an extension point) is the subscriber will not get registered until that plugin or extension library is used. HTTP services though get registered as soon as the HTTP task is loaded, because that’s when this getServices() method of an instance of the com.ibm.xsp.adapter.serviceFactory extension gets called. There are two important aspects of HttpService instances.

First, they appear to get loaded very early on in the HTTP startup process. The createInstance() method gets called even before ODA starts. It also blocks any other plugin loading until that method has completed. So that method can’t be used to trigger any ODA-related code.

Secondly, it has a checkTimeout() method, which runs every 30 seconds while the HTTP task is running. This is very useful and would probably be the approach used for scheduling Xots processes. It’s used here to schedule a process running every 5 minutes to dump out the summary of all events triggered and the amount of times they’ve been triggered – the Map updated by line 4 of the handleMessage() method above.

So the checkTimeout() methods calls EventLogPrinter.getInstance().start(), which only runs if it’s not already started and calls the trySchedule() method in the EventLogPrinter class. This adds this class as a Xots task and so promptly triggers its run() method. That runs in a loop as long as isRunning is not set to false (which happens when the server shuts down or the process is otherwise interrupted). It uses a timeout comparing the next run time to now, which will continually tick down and also checks the tasklet has already run at least once. If it’s time to run (and initially nextRunTime – now will be less than zero, so will run), it calls EmHttpService.getInstance().dumpLog() to dump out the summary to the server console. Finally it calls getNextRunTime() to set the next run time to five minutes from now. This is like the Domino agent manager, which runs agents x minutes after completion. (With a Xots scheduler, I’d probably set a tempNextRunTime when it first triggers, then use that at the end, providing it was still in the future. Then it would run every 5 minutes – like LEI – rather than 5 minutes after last completion.)

The QueueDispatcher, which kicks off any IEMBridgeSubscriber waiting for each event, polls the queue every 500 milliseconds, which is why notifications can be processed so promptly.

There’s a lot here, but hopefully this description makes it clearer and I’m sure there are some useful techniques for developers. It’s worth bearing in mind that the QueueDispatcher and QueueListener tasks in org.openntf.domino.extmgr.EmBridgeMessageQueue also run as ongoing Xots tasks, so with the EventLogPrinter, that’s three ongoing tasks out of the 10 default Xots threads. That may change in the future. Also, this is very much in its infancy so is still experimental at this stage. Indeed Cameron Gregor has just submitted a pull request for an extension point to add subscribers, which will be a big improvement. My experiences on a test server so far have been very encouraging about its potential.

org.openntf.domino.emSample

AUTHOR - Paul Withers

Paul Withers has been an IBM Champion since 2011, has been an OpenNTF Board Member since 2013, has worked with XPages since 2009, co-authored XPages Extension Library and was technical editor for Mastering XPages 2nd Edition. He is one of the developers on OpenNTF Domino API as well as contributor to a variety of other OpenNTF projects.

No Comments

Post A Comment