Why Source Control?

Ray Bilyk posed a very valid question on yesterday’s blog, asking about reasons why to use source control. Although we’ll cover some of this in the session, the main focus is on using source control. So I’ve decided to cover the main reasons I use source control in more detail here.

Architecture

First, it’s important to understand what we mean by source control. Source Control is not about creating an On Disk Project. The On Disk Project is a means to an end. There are a number of elements involved:

• NSF – where you typically do your Domino database development. It doesn’t have to be the only place, it doesn’t have to be the preferred place. If you have a better JavaScript editor or CSS editor than Domino Designer, there’s nothing stopping you using that.
• On Disk Project – a set of flat files corresponding to the contents of the NSF. They are output using DXL and that’s key. If the DXL is not a true reflection of the NSF design element, you’ll have problems. If Domino Designer’s preferences are to export binary DXL, you won’t be able to modify some files outside of Domino Designer. The On Disk Project can reside anywhere. The default is your notes workspace folder, but there are a variety of reasons to choose somewhere else, not least the fact that the workspace folder is the root of most issues with Domino Designer and you may need to replace it. If you want to use an alternative JavaScript editor, CSS editor etc, these are the files you’ll be editing.
• Local Source Control Repository – a folder structure of files that you want to push up to an remote source control repository, plus configuration files. This can include one or more On Disk Projects (you’re probably now seeing another reason to store your On Disk Project outside the workspace folder), as well as any other related files, like documentation, Eclipse projects etc. This source control repository constitutes the current working branch. This only exists for Distributed Source Control solutions like Git and Mercurial. In a centralised source control solution like SVN, there is no local repository and you need to be connected to make changes.
• Remote Source Control Repository – a central copy of the repository, including all commit history. Many Source Control Management systems will also include things like issue tracking, wikis, creating forks etc.

Hopefully this gives a clearer understanding of the moving parts. It also demonstrates why source control does not care about what files are available, as long as they are a bunch of physical files. There’s nothing stopping you putting an actual NSF in a source control repository, though there would be little benefit. It also demonstrates how crucial DXL round-tripping is. If there are areas where source control doesn’t work for an application, it’s usually not a problem with source control per se, it’s a problem with the DXL round-tripping.

Why Source Control?

1. Backup

A couple of years ago at DanNotes I was running a session on XPages. We had a break halfway through the session. When I came back, I couldn’t open my NSF in Domino Designer. It just wouldn’t open. But I had an On Disk Project, so could demo from that. It turned out the .project file used by Eclipse had got corrupt, so if I deleted the NSF, deleted everything related to the NSF from my workspace folder, I was able to recreate the NSF from the On Disk Project. Meanwhile, during the session, I was able to open the XPages and Custom Controls from the On Disk Project and demo the source XML.

Source control is not the only way to handle the problem, but using source control does give you a solution.

2. Sample Mockups

Applications never stay the same. Sometimes you want to do a mockup of a change to a live design element to show users. Sometimes it may have some associated functionality, with bits hard-coded for speed. Creating a new branch can allow you to do that without compromising the existing solution and easily merge the changes in if the development goes ahead. It also keeps your design clean, without having redundant copies of design elements.

3. History

Local History is a great thing, but it’s limited and only pertains to the current developer. Source control gives you a full history. Yes, if you’re moving to source control, you’ve got a cut-off point. But from that point forward you can quickly compare changes of files. Files like Image Resources can’t be compared, but for XPages there are alternatives to using Image Resources. Bear in mind Forms and Views are not as easy to compare outside of Domino Designer, because you’re comparing the DXL.

I’ve recently found it very useful for identifying when I made a change to the OpenNTF Domino API. In the past I’ve also used it to identify and fix regression bugs.

Commits also give you documentation of what changes were made to a group of files on a particular occasion, as well as all the files involved in that change.

4. Integrated Issue Tracking

Many source control management solutions include integrated issue tracking. So by including the issue reference in the commit comment, you automatically track back to issues / feature requests raised. This could be used to better manage QA or UAT of a system, rather than circulating Word / Excel documents. Some issue tracking solutions like JIRA also include time management, if that’s useful. All we cover in the session have REST / Java API access, so you’re not even limited by the out-of-the-box interface.

5. More Than Just A Single NSF

Source control allows you to include documentation in the repository, license files, plugin projects, UML design projects. As long as it’s a physical file, you can include it in a repository and push it up to source control. And in the scenario of a multi-database application, there’s nothing preventing you adding multiple NSFs into a single repository.

Ensure you create each ODP in a separate folder, so you may need multiple folders under notes. That’s because when you initially create an ODP, it deletes anything that’s in that folder. I’ve made that mistake when I was creating ODPs just under a Git Repositories folder, and promptly removed all my local repositories! It wasn’t critical, I just had to pull down the repositories again from the remotes.

6.Develop wherever, whenever

With distributed source control management solutions, developers can work on a system wherever, whenever. If you don’t have connectivity to the remote repository, for whatever reason, commit as normal and push up the changes when you do.

Bear in mind that even if you have full connectivity, syncing from an NSF that’s on a remote server to a local On Disk Project may take time. But does the NSF need to be on a remote server? It’s a good opportunity to consider working locally or on a local Domino server. Your design is backed up to the remote repository, so the worst you can lose is the data.

Indeed quite a few developments I’m involved with have developers working on them who do not even have access to a common Domino server. They are also developments where I specifically need to be able to work disconnected from the internet. Source control gives me a solution for managing that.

7. sessionAsSigner, Java compile, debugging

sessionAsSigner is a great thing with XPages, but if you have multiple signers of related design elements, it causes problems.  If you change a Java class and build the application, any objects using that class held in scopes – sessionScope, applicationScope – are lost. So if another developer is testing something, they may just have lost a key object and get errors they were not expecting. If you start debugging on a server used by multiple developers and someone’s left a breakpoint on an application they’re running, you could get prompted to debug code they’re testing. All annoyances in a multi-developer environment. You may get round this by working on an NTF and refreshing design down, then signing the database. Maybe you need to clean it as well, maybe delete the Java classes to get them to refresh properly. Another solution is using source control and separate databases.

8. Beyond Domino

Because it’s just using flat files, you can use the same source control approach for managing non-Domino projects, such as XPages plugins, DOTS plugins, or developments that will not even be running on Domino like mobile projects using something like Appcelerator or Worklight.

You’re also expanding your skill-set with something that’s not proprietary to one development platform but standard. So if you have PHP developers in-house as well, they can use the same source control solution.

The Downsides

There are a very small number of specific scenarios I’ve encountered where DXL round-tripping doesn’t work, as I’ll mention in the session. I’ve also seen some strange behaviour with inline JavaScript on a traditional Domino web form. But they’re not scenarios that are encountered with XPages applications and I have already identified (better-practice coding) workarounds for all but one, and the other I just haven’t got round to looking for a workaround.

I’ve also heard of people losing development. It’s not something I’ve encountered myself. I’ve always had my NSF in a sub-folder of the repository, I’m not sure if that’s the cause. I also manually sync with the ODP. If I have not synced for a while, I may take a local backup copy of the database before syncing, just in case, but I’ve never needed to use it. The only time I’ve lost code was by creating the ODP into a folder that already has content. Don’t. It deletes everything from the folder before pushing stuff out of the NSF.

Merge conflicts are the big downside. But there are tools, as we’ll show in the session, to make it easier. Forms and views are more challenging, because most developers are not comfortable reading or changing the DXL. But there’s nothing stopping you pasting the DXL into forms in a standalone database and visually comparing, manually making the changes and pasting the resulting DXL into the ODP, if needed. It sounds scary, but it’s not. You can easily access the DXL from the Package Explorer, right-click a design element and select Open With… to open in the DXL editor. From there, it’s just XML, like in the source pane of an XPage or Custom Control.

The terminology of pulling, pushing, branching, rebasing, Git Flow / Hg Flow sounds inhibitive, but we’ll also cover that in the session.

For me, it’s been a learning curve, but one that I’ve found less challenging than XPages or even Java. The biggest headache has been all the moving parts, which is the main reason I wanted to submit the session, to cover everything, end-to-end.