Issue with view indexes being “randomly?” discarded by Domino 11.0.1FP1 server

I recently discovered an issue where a Domino 11.0.1FP1 server appears to be randomly “discarding” view indexes during normal operation. This is resulting in slow application database performance (and of course user complaints!).

Background

Some of our application databases have a “View Update” agent in them that run every night. These agents open and refresh all views that users depend on for application functionality, and are used to avoid the risk of view indexes being discarded due to lack of use (some application databases are very large – in size and number of documents – and so any view build/rebuild can take a long time and impact users).

One of the hosting servers was recently upgraded from Domino 9.0.1FPx to Domino 11.0.1 FP1. After the upgrade we started getting complaints from users that now applications were often very slow to access.

Thankfully, one of the notes.ini settings I apply to servers is “Log_View_Events=1” which allowed me to see that Domino was reporting messages like the following at the time when the databases were slow to access.

Informational, rebuilding view - no container or index (reading view note Title:'|')

However, the views should not need to be rebuilt… this is one of the reasons for the “View Update” agents.

I raised a ticket with HCL, and initially was advised to set the notes.ini setting “Debug_NIF_Update=1”. This provided a lot more information about when views were being accessed, by who, and whether the view indexes were being refreshed or rebuilt, but did not answer the crucial question… “Why is Domino needing to rebuild these view indexes when they were accessed/refreshed the day before?”

I was then advised to set the notes.ini setting “DEBUG_DISCARD_VIEW_OVERRIDE=1” which would apparently print a message to the console when a view index was being discarded.

On the first evening, after applying the debug setting, Domino occasionally reported “discarding view” in the log.

After a number of days, I could see that some days would be fine, whilst other days we would see multiple view indexes getting discarded when the “View Update” agent was running.

Interestingly, the discarding appeared to be happening shortly after the view open/refresh done by the agent, i.e. we would see the following sequence in the console log.

Agent logging to report "<View>" was being refreshed
Domino logging to report the start of refresh for "<View>"
Domino logging to report the finish of update for "<View>"
Domino debug logging to report "discarding view "<View>"

I have now received an update from HCL confirming that there is an issue. I am waiting to get the SPR number and details (and will update the blog when I have the details).

Fingers crossed we can get a fix soon for this issue…