Thoughts on the Problem of XPages

Home » Thoughts on the Problem of XPages

Jesse Gallagher posted a thoughtful blog about the challenges for the way forward with XPages. It's a challenge I've had a number of discussions around, many of which are ongoing.

The reason there's not a simple solution - throw whatever resources are required into XPages - is that it simply doesn't make business sense. There are very emotional arguments, but there are some inescapable points which those emotionally invested into the technology rarely discuss:

  • XPages did not grow the platform as was hoped.
  • XPiNC was a nice idea, but it had problems that could never be overcome. The XulRunner (the web container for the application) got out of date, Mozilla deprecated it, "run on server" had a lot of quirks, and the extensibility (so powerful on the server) was never quite as easy on client.
  • Dojo may have been a good choice at the time, but the rise of jQuery and Bootstrap wasn't anticipated. Integrating Bootstrap was achieved, but is constantly changing and introduces breaking changes (e.g. icons). Whereas jQuery or Bootstrap seem to focus on modules of web development, Dojo seems more monolithic. So replacing parts with something else are more problematic.
  • JavaScript web development matured significantly in the five years after XPages appeared.
  • The lightweight client will run in a browser, competing with XPages and traditional Domino web applications potentially. Authentication raises question marks about whether external users can use the lightweight client, an obvious expected requirement would be multiple browser tabs using multiple Notes IDs, which may be possible, but it's unclear whether the architecture will allow this.
  • Offline on iOS was never a possibility, because of Apple's avoidance of running Java on iPad and iPhone. UPDATE: Jesse has reminded me that solutions like RoboVM can convert Java applications to run on iOS.
  • There may be a lot of XPages applications out there, but as a percentage of all Notes applications, it's got to be small. I would estimate probably 5% at best.
  • The XPages team at IBM reduced significantly as they were moved to other initiatives and do not appear to have been in scope for the acquisition, for whatever reasons. Whether teeth are gnashed or not, it won't change. And wailing about it at those responsible for the platform won't gain sympathy, it will just imply someone focused on resurrecting the past not moving forward.
  • There are various enhancements required for the HTTP server - servlet container, web sockets, HTTP/2, adding LESS/SASS support to the NSF. These are required for modernising XPages, but is the HTTP server as it is a key component for the future? And would a more modern HTTP server be able to support XPages?

All of this means modernising XPages to what's required needs substantial investment, with little potential for growth of the platform as a result. Looking at it objectively, the return on investment is highly questionable at best.

Open Sourcing

One potential route, that has been discussed on occasion by various people, is open sourcing XPages. But the painful truth is that XPages will never appeal to an Apache or Eclipse open source organisation. OpenNTF is far too small. And even with an open source organisation backing it, I don't think there are enough developers willing or able to support it, because of the requirement for a developer base with the requisite technical skills and who are still invested in Domino.

But even if there were, there are still some platform-specific enhancements that would be required and some parts of the runtime that would almost certainly never be open sourced.

SSJS

Java can run as Java anywhere. As Jesse says, there are some "XPages-isms" - the XspContext, the scope maps, etc. If you have the key Java classes for e.g. dominoDocuments and dominoViews, you get a long way to running it agnostically of the XPages runtime. But SSJS requires a lot more. It requires a Language Server Protocol in the IDE to give the content assist and validation. And it also requires something in the runtime to parse the SSJS, which is stored as text strings whether it's inline or in a Script Library, and process the calls.

Unfortunately, SSJS was never positioned as a stepping stone from LotusScript and Formula Language to Java. So many developers never progressed. Admittedly, even if IBM had proclaimed it as a stepping stone, I suspect there would still be a good proportion who would never have followed that advice.

So we have a lot of XPages applications that will always be a lot harder to port to something else, whatever is chosen.

The Challenge of Business Logic

Reusing Java methods or SSJS functions triggered from buttons etc on an XPage as REST service calls from a JavaScript front-end has its own challenges. The code is typically tightly bound to the XPage itself - reference this component, this datasource, that scoped variable. It is also designed to update in-memory state. I doubt many, if any, XPages applications use functional programming approaches. So it's not an easy bit of refactoring to convert the code from onclick or onchange events to REST service calls.

Even if it was, that's just a fraction of XPages code. We don't trigger an onclick and just run the code. We pass back a portion of the web page, or more typically all of the web page. We process a portion or all of the XPage (the "execId"). And this runs SSJS or Java beyond just the onclick code. It runs SSJS or Java that's tied to specific UI components, that manipulates the properties of those components. And it returns HTML for a portion of the web page (the "refreshId").

Converting SSJS or Java to REST services may not cover more than a small percentage of the business logic coded into an XPages application. The bulk of it may be tied to components.

There are two options: move that code to a JavaScript UI, picking up properties passed back from the REST service; or have a compiler that manage and outputs those components, understanding what's coded in the XPage and manipulating the web page as required. It's not easy, and would require quite a bit of investigation.

And this is where the wide variety of XPages applications has a big impact - that XPages is quite accommodating, letting you code how you wish and adapting accordingly. And this is why converting XML to components is the easy part of the problem and why whatever solution arises, it won't be quick.

18 thoughts on “Thoughts on the Problem of XPages”

  1. Call it fortune, call it cynicism, but either way, I never bought the XPages hype, not from day one. Instead, in my work, we got as close as we could to REST, with LotusScript agents, and used Domino as a back-end API for jQuery apps. In recent years, same thing but with React front-ends. You can even embed your React bundle into the .NSF and run it all together.

    With that as my history, it makes far more sense to me to put the efforts into modernizing the HTTP stack to support all HTTP verbs, so we can write true REST APIs, and let Domino be a modern back-end platform. This may sound extreme, but forget the front end. That isn’t where the strengths lie.

    As far as existing XPages apps, and other low-code ideas for the future, I feel that a better path is to instead export a default React App (or Vue, etc) for a .NSF, which has some standard navigation and CRUD. But it needs to be an export because everyone will want to change it. Wire it all up by default, but let people go modify it as needed for their business.

    1. XPages had a lot of strengths, and most frameworks have since adopted the component-driven approach it took from JSF. There is a lot it does for you – both a blessing a curse – but it did provide helpers for things like CSS, regex, HTML (via properties and components, avoiding typos and lots of manual work). LotusScript is okay, but my biggest problems with it were JSON manipulation and debugging. This was one area where OSGi plugins gained a lot, another was the number of Java packages available. But these are all personal preference, there are no major limitations.
      Which HTTP verbs are missing? I’ve enabled PATCH and possibly OPTIONS in an Internet Site document before. They work fine for my OSGi plugin or XAgent REST services. There are other possibilities for an HTTP stack that may offer greater flexibility, we’ll see what happens there. “Project Castle” was originally taking a different approach to HTTP access. Focusing on Domino as a modern back-end platform is a valid argument, some will agree, some will not. Even though HCL are investing a lot, resources are not infinite.
      I agree on low code approach. HCL Leap is a Dojo application, but where modernisation of the HCL portfolio is occurring, React is the framework of choice. If feasible, an option to output as React *or* Vue could give greater value, although Vue may require a separate Node.js app for integration with Domino via gRPC. There are options though for packaging those together, with Kubernetes / k3s. The NSF could be an ODP, considering it would just need views. It depends on demand and ROI.

    2. I know I’m late to this thread, but I agree with Dave, from a strict web development platform that can be customized, adapted, etc to new standards, etc, Domino is great as a back-end without the need to be too involved in serving out more than data. However, from the XPages perspective, I’ve been wholly confused by the lack of a unified development approach. I’ve supported too much “XPages” that were using such a mix of stuff, it actually made things less efficient to run than Dave’s approach. What made Lotus Notes Client great was the ability to develop client applications that did great in client, but needed a lot of development to be web. If you want to keep xpages around and do web stuff for Domino that is out-of-the-box workable, then there needs to be work to do what the client did, but testable and fully web/mobile integrated. If there was an easy way any user, whether they knew much or not of development, could create an application, add forms, views, tables, etc to it, work it over so it looked good in several browsers, then the Developer would “publish” the application, optimized, to the Domino server for web/mobile. That would be the most ideal approach. This way you’d still have a base of users, whether starting out or comfortable with development, who’d take the Developer and make applications because they could ramp up as they gained comfort. If a Drupal/Wordpress approach were used, but with the older “Client” idea to development in mind and where Domino became a subscription based service for developers, with Developer free (as it already is), but based around an easy to use interface for the beginner, with more features for the more knowleagable, I could see Domino revive and beat at Drupal, WordPress, and many more at their own game, if not compete strongly with other platforms. Just my thoughts.

  2. Ferry Kranenburg

    I’m part of a small team of developers currently maintaining a mid-large CRM application. The application is developed entirely on Xpages using Java. We didn’t want to use the ‘out of the box’ components in Xpages. So over the years we have built a very large Java / Xpages based framework around and on top of the ‘Xpages’ part. It allows us to dynamically in runtime code to built all our Xpages / JSF custom components directly into the JSF tree. We just define our pages with Injection and annotations. No Xpages markup needed.

    We use Junit5, Maven, Gradle, Jenkins, Jersey (instead of Apache Wink), Eclipse. We do not use Designer anymore. For authentication our clients do not accept basic authentication anymore. All our clients want two factor authentication (SAML) and for our REST Api we just integrated JWT (json web token) authentication.

    For the front end, are we using the latest Dojo 1.x version. It is currently using Webpack 4 to bundle all Javascript, typescript and SASS modules into one file. Our entire front end contain more than 200+ custom written Dojo widgets using the Dojo Toolkit framework. We do not use any of the built-in Xpages component anymore, all are replaced by our own widgets and our own styling and theming. And they don’t come with unexcepted behaviours, unacceptable styling or functionality we don’t use of just don’t want.

    So… we are already using modern technologies on Domino!

    But then Xpages… It is in the core of our complete product. The biggest part we use of is (the great) ‘partial refresh’ system. It is so sad that Xpages wil not have a future (are we really sure about that btw??) but we will and we need to find a solution for this.

    I really hope HCL find a way to upgrade Xpages to JSF 2.x and convert Xpages to a much thinner layer, just like Jesse says in his blog.

    And that would definitely be the best route for all of us.

    1. It’s great to see the powerful stacks people are building with Domino at its core. The partial refresh of XPages is very powerful, allowing properties to be computed and re-computed on-the-fly. The only down-side, as I said, is it results in distributed business logic. There is no definitive decision about XPages, it’s worth emphasising that. I think the big challenge for the community coming up with innovative solutions is that there’s a lot of how it works that we don’t know. Never say never. Who would have thought there would be a resurgence of Notes Client development before the iPad client was announced? The biggest challenge is finding an approach that is cost-effective, addresses enough use cases and satisfies enough customers.

  3. Before I started at my current employer, I never saw a single Xpages application being used. Most customers I visited had no interest in it, nor were they planning to start using but.

    However, some of the most important apps at the company I work for now are using Xpages. We will have to do some extensive rewriting of applications if Xpages is deprecated. Most likely, we will continue on our strategy of moving things to Electron, and just use the Domino REST API to read to and from traditional nsf files.

    My colleagues already did that for some of the apps, because there were some things Xpages just couldn’t deliver, and it doesn’t seem that will happen either.

    Just like Dave Armstrong above, I never believed Xpages would have much of a future, and now I’m glad I didn’t waste my time learning it. I had the chance a few years back, but I didn’t take it, as I saw that the percentage of Notes applications that were being modernised with it was extremely low.

  4. We tried using XPages, but based on our experience, we decided not to use it and created a SPA JS/REST API framework. It consists of LotusScript agent based RESTFul APIs with our custom Dojo-based framework that combines the latest version of Dojo 1.x with Bootstrap. We create our own widgets and components that integrated and bind directly with our REST APIs. We are looking to upgrade the framework to utilize Dojo 5, but that is a long term project. Our Notes-based LotusScript-based IDE compiles custom widget XMLs and code into these Dojo-based widgets that displays the content and view.

  5. Isn’t there a tendency for high end developers and XPage naysayers to miss one of the main objectives of XPages, – to provide rapid development in the same manner as Notes Forms and Views but with the added advantage of multiple standard controls, multiple data sources per page with far greater flexibility and to make that available via a web browser rather than a Notes Client. To that end surely it has always been extremely successful. I know it has for me and my organisation.

    1. Spot on, and that’s why if a customer wanted a quick development or a cheap application to run for a year or two, I would absolutely recommend XPages. Even for a larger application that needs to run for five years. Pick any JavaScript framework you want, I challenge you to justify a belief that your app will just continue to run for the next five years. But the approach I’d take for something with any longevity would be to keep business logic and UI separate, and the start app I have based on Jesse’s framework allows me to do that, configuring UI component properties based on properties in a Controller class for the page, with property values changed by the business logic where possible.
      Incidentally, an area where I think XPages could get a resurgence is for reporting with the lightweight client – generating dashboards, flexible reports with “joins”, Excel spreadsheets etc.

    2. I agree. While I was stopping XPages development for large applications some years ago, it is still one of the best RAD tools for developing small web applications.

      And as often, now the “I have told you so!”-people are comming up with their solutions and “why it was better to ignore the technology”, 10 years after its release.

      For me, it was a problematic technology after IBM abandoned its development some years ago, without communication to the customers. As a freelancer it is mandatory to have an eye on the needs of the market.

      Now, again, the customers asking themself again: Redevelop an existing XPage application and tie it to the new node.js stack (which is still beta), or migrate to another platform. In my opinion, most customers will choose the last option.

      It was a wise decision to go the XPages path. Otherwise we would not debate about “ways out of the XPages dilemma” – the Domino platform would just not exist anymore.

      1. It’s possible Domino might still exist if XPages had not happened. But the developer and customer base would have been in a much worse state, which raises the big question of whether HCL would have got involved.
        The options going forward are varied, and it’s interesting to see the breadth of options people are using. Domino still has a place in a variety of stacks, which is encouraging.

    3. @Mark Maden – Exactly! The other week I watched the latest HCL webinar on development and went away with no idea what they were recommending the typical Notes or XPages developer should use to create new apps. Okay, new hooks into node.js. Well, that’s great — let’s see how an average Notes developer will quickly build a useful app for their organization!

      Over the life of the product, I’ve prided myself with continually learning new technology as it progressed from @functions, to LotusScript, to Java agents, HTML, CSS, JavaScript, XPages (with some DOJO), XML, JSON, jQuery, jQueryMobile, Bootstrap, ( and dabbling with node/ Vue / Angular / React – enough to know it would be no trivial task to use).

      At this point, I am very unsure what is the best way forward.

        1. Vaadin is nice and has a lot of similarities to XPages. Extending Java classes for components compared to Custom Controls, a full Java framework gives some useful flexibility. The challenge (currently) is interacting with view data. Look at the XPages to Web App series I did, I highlight some of the challenges around view data. Note, scalability was always an aspect I never addressed. Domino-db Java module may (depending on how it’s deployed) overcome that. We’ll have to wait and see. Of course Spring means (typically) not going through Domino’s HTTP layer. Some customers may not like that, but there’s a whole blog post on why I believe that is the best approach for the future.

  6. Pingback: How we use XPages - FoCul

  7. Pingback: How we use XPages | wp2020.focul.net

  8. Pingback: How we use XPages - wp2020.focul.net

  9. Unfortunately, I joined the discussion too late.
    Therefore, I will allow only two or three comments.
    It is easy to create a single page modern application that will load any number of html pages to the master XPage. It’s easy to create a dedicated dispatcher – manager that loads and / or unloads js plugins.
    As Gallagher recommended, you can get rid of an extremely awkward designer and work in any modern familiar framework. What’s left of the dominoes is storage, clusters, replication, excellent security combined with concept author-editor-reader-fields , ftsearch, Pass-thrue mechanisms, and native and / or composite rests. Correct ar create your agent-manager (not necessarily on the domino server), think over the ideology of working with compound forms, composed of various documents and even representations.
    After that, you will consider whether you need to say bye-bye of Domino.
    I highly recommend reading the superb post
    https://www.focul.net/how-we-use-xpages/ by Sean Cull.
    Best regards, Oleg I.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to Top