Our doors are always open

I extracted the text:

Portentum plagens tua si caret asse commena
Pergerian numinis resonat celer intus amice
Si tamen esurit luges maxilla calabit
Ostia nam miseris fraler sunt semper aperta

Which google translates as:

If your plagiarism is lacking, please pay attention to it
The pergerian of the god echoes quickly within the friend
But if he is hungry, his jaws will warm
For the doors are always open to the poor brothers

This is what I think it means:

If you are sick please tend it yourself
As the will of god spreads quickly to your friends
But if you are hungry, you will be fed
For our doors are always open to the poor

REMEMBRANCE.md - Tombstones for git projects

There is something wrong with the idea that git gives you confidence to delete.

As so often this is from the perspective of the individual programmer.

Sure, you can delete your code and you can retreive it. Well done you, but you are relying on your memory.

Unlike mere programmers Developers are fully interchangeable members of a multi-decade team.

Tombstones

In the world of online publishing there is a concept of a 'tombstone' which is located at the URL of a retracted paper or any other entity removed for a reason. There are similar usages in programming.

A Book of Remembrance as a collection of Tombstones

My grandfather's ashes were, I believe, scattered at sea, but as a child I was impressed that he still had an entry in the Book of Remembrance at the Crematorium, which gave us a place to visit, and as my unsentimental Grandmother might have said, took up a lot less room.

Now these remebrances are virtual.

Gone but Not Forgotten

Even after using git for a long time Developers still do not delete modules, files or even lines of code. They comment them out, or just leave them in place, holding an enormous amount of context in their heads.

Let's not criticise them, lets take their concern seriously. They want to be able to find things themselves and for future Developers to know what was useful, even if it is not currently required.

REMEMBRANCE.md

At the top of every repository there HAS to be a README.md (or you get nagged by github); similarly we should add a REMEMBRANCE.md which takes the following form:

| File                |    Date    | Description and Reason for Deletion                          |
|---------------------|:----------:|:-------------------------------------------------------------|
| maven-tomcat-plugin | 2013-11-11 | An embedded Tomcat, for testing. The team just walked away   |
| my-perl-script.pl   | 2019-03-23 | A script to remove stones from horses hooves. Moving on up.  |
| tpp-module          | 2024-02-22 | Finished polishing, flushing now.                            |

Lest We Forget

Example REMEMBRANCE.md

File	Date	Description and Reason for Deletion
maven-xxx-plugin	2013-11-11	An embedded Tomcat, for testing. The team just walked away
my-perl-script.pl	2019-03-23	A script to remove stones from horses hooves. Moving on up.
tpp-module	2024-02-22	Finished polishing, flushing now.

This simple practice will give Developers the confidence to delete and unleash the promise of clean projects that git offers.

PS Spelling

It is Remembrance, not Rememberance, I looked it up, though I prefer Rememberance.

Melati Day - Twenty Five Year Anniversary

Happy Melati Day!

On February 16th 2000 WilliamC made the first commit to Melati.

https://github.com/Melati/Melati/commit/7a6fb6762cdbb835b4ff1aeb987bf1671cc598bd

The name ...

  Indonesia's national flowers: the melati (small white sweet smelling
flower) ...
:-)
Why it's a coffee variety (at my favourite shop here in The Hague) I
don't know---I'll ask her next time I'm in.

On 11th July that year we held Melati One.

PS Some five years later git was born!

Five Element Maven Version Numbering System (N5)

The Five Element Maven Version Numbering System (N5) addresses the problem of wack-a-mole Maven dependency management: through the combination of excessive modularisation, microservices fervour, XML Declarative Project Object Models and non-unique (SemVer) versioning enterprise Maven Java projects become unmanageable after about five years.

The solution is a sequential identifier which is unique across the space, and does not require xml parsing to modify.

My first three proposals represent blundering steps towards the N5 Maven Version numbering system.

N5 Unique Versioned Artifact Identifier (Maven version)

\d	Code for project
\d\d\d	Code for repository
\d+	Major
\d+	Minor
\d+	Patch

The previous version, N6, included the Year between the Repository Code and the SemVer Major Version. This was considered too much.

2.130.2.0.6 works well with the Maven tooling and can be updated over the whole space using sed.

Maven Version Versions and a Regex Block Refactoring of Sequential Identifier

In the first post in this series I had thought about the need for a globally unique incrementable artifect id, in the next post I not only designed but tested a proposal for such a globally unique artifact id and how to change it.

A triumph of thought and testing over trial and error.

So I tried using the new scheme with the Maven Release Plugin.

A depressing journey into the rabbit hole that is Apache, Maven and the Release Plugin, a crucial tool in our pipeline.

It turns out that whilst the Maven Artifact Version I had tested against allowed my new scheme the Release Plugin has its own subset of such versions which are gate-kept by a Regular Expression.

          "^((?:\\d+\\.)*\\d+)" // digit(s) and '.' repeated followed by digit (version digits 1.22.0, etc)
                    + "([-_])?" // optional - or _ (annotation separator)
                    + "([a-zA-Z]*)" // alpha characters (looking for annotation - alpha, beta, RC, etc.)
                    + "([-_])?" // optional - or _ (annotation revision separator)
                    + "(\\d*)" // digits (any digits after rc or beta is an annotation revision)
                    + "(?:([-_])?(.*?))?$"); // - or _ followed everything else (build specifier)

This stops any version which does not start with digits and full stops from being recognised as a version.

Current	Incremented	Outcome
VTW360-2023-1.0.3-SNAPSHOT	Unable to parse the version string	F
VTW360.2023.1.0.4-SNAPSHOT	Unable to parse the version string	F
VTW360-2023.1.0.4-SNAPSHOT	Unable to parse the version string	F
VTW-360-2023.1.0.14-SNAPSHOT	Unable to parse the version string	F
VTW-360.2023.1.0.16-SNAPSHOT	Unable to parse the version string	F
2023-VTW360-1.0.3-SNAPSHOT	2023-VTW361-1.0.4-SNAPSHOT	Eh?
2023|VTW-360|1.0.14	Illegal characters in jar name	F
2023#VTW-360#1.0.14-SNAPSHOT	2024#VTW-360#1.0.14-SNAPSHOT	Eh?
2023.VTW-360.1.0.14-SNAPSHOT	2024.VTW-360.1.0.14-SNAPSHOT	Eh?
360.2023-1.0.15-SNAPSHOT	360.2023-2.0.15-SNAPSHOT	Eh?
2023.360.1.0.14-SNAPSHOT	2023.360.1.0.15-SNAPSHOT	Meh
1.0.19-VTW360-2023.SNAPSHOT	hangs	Eh?
1.0.19-VTW360-2023-SNAPSHOT	1.0.19-VTW361-2023-SNAPSHOT	Eh?
1.0.19-2023-VTW360-SNAPSHOT	1.0.19-2024-VTW360-SNAPSHOT	Eh?
1.0.19-YY24-VTW360-SNAPSHOT	1.0.19-YY25-VTW360-SNAPSHOT	F
1-0-19-2025-VTW360-SNAPSHOT	1-1-19-2025-VTW360-SNAPSHOT	Eh?
360.2023.1.0.16-SNAPSHOT	360.2023.1.0.17-SNAPSHOT	MKay

Proposal 3

So we are forced to allocate meaning to digits and place, never a good idea, but it is that or try and change the behaviour of the Release Plugin, a task I do not think I have the time or the energy for.

\d	code for project
\d\d\d	code for repository
\d\d\d\d	year
\d+	Major
\d+	Minor
\d+	Patch

I added the new proposal to the tests, and wrote some more tests based upon the above regex.

We land with 2.130.2024.2.0.6.

However much you think about the problem previous engineers will have constrained your choices.

Maven version schema change

I proposed a new versioning scheme to yield a unique version string for each artifact in our highly modularised Maven java application.

Currently we are running a classic SemVer semantic verioning system. The only shortcoming I can see with SemVer is that it is not globally unique. In the Maven world the version is combined with the artifactId and the groupId to create a globally unique token, but because this is within an XML based Project Object Model (POM) file it is fairly difficult to get that token out without buying into the whole toolchain. As Maven allows ranges (don't use ranges) and inheritance it is practically impossible for mortals to parse a pom correctly.

What problem are we trying to solve?

We want to be able to edit all poms across our collection with a single command to ensure that each version is completely deployed and we have no version clash. We have developed a tool to make the Maven Dependency Tree readable which has allowed us to see how often the team make a change but do not fully roll it out.

The problem we are trying to solve is how to update all usages of an artifact every time it is updated, ensuring we never run with a mixture of versions (almost like a Monolith).

In the wider open source community it is not the responsibility of the library writer to ensure that every usage is updated to the latest version but within an enterprise it is.

Initial proposal

My first proposal, prepending the semantic version number with a unique code for the library, looked good:

SMD110-3.0.8

This system would work if we had started with this or if we were to backfill all previous versions, however we are proposing a change only to new versions and we have to ensure that the new versioing system will play nicely with the existing versions.

By play nicely I mean that SMD110-3.0.8 should be understood to be more recent than 3.0.7.

I had planned to test this by trial and error but then I thought to actually write a test.

Testing the proposal

I found the Maven verson comparison code and wrote some tests.

The tests revealed my assumption to be incorrect (whether this is a bug or not I am not sure), but the tests enabled me to propose a schema change which did play nicely:

    @Test
    public void workableNextYearNewOldComparisonTest() {
        ComparableVersion it = new ComparableVersion("2025-SMD130-2.0.7");
        ComparableVersion previous = new ComparableVersion("2024-SMD130-2.0.6");
        assertTrue(it.compareTo(previous) > 0);
        assertTrue(previous.compareTo(previous) == 0);
        assertTrue(previous.compareTo(it) < 0);
    }

Revised proposal

Prepending the initial proposal with the year is an improvement, the year of publication is useful and it works with the tooling:

<year>-<artefact code>-<semantic version>

2024-SMD110-3.0.8

Test your assumptions, it is easier than trial and error!

Unique versions as pitons out of Maven Dependency Hell

The whole Maven ecosystem, and more, follows the Semantic Versioning proposal from @mojombo who made a success of being right about most things.

Even Oracle use it (no more ojdbc.jar for them).

The problem Semantic Versioning addressed was the opposing forces of

version lock

the inability to upgrade a package without having to release new versions of every dependent package

version promiscuity

assuming compatibility with more future versions than is reasonable

leading to Dependency Hell.

If that is the problem you have then Semantic Versioning will help. Semantic Versioning helps you live within the Maven ecosystem, if you are contributing to that ecosystem then use Semantic Versioning.

That is not the problem that you have if you dug your very own Dependency Hell due to having a modularisation addiction.
If the problem you have given yourself is creating over a hundred inter-dependent modules because of your desire to obey the injunction Do not Repeat Yourself (DRY) within a modular microservice world the problem you have is achieving consistency.

The Semantic Version string MAJOR.MINOR.PATCH is not unique.

What is actually unique is ARTIFACT.MAJOR.MINOR.PATCH.
Or even uniquer (sic) GROUP.ARTIFACT.MAJOR.MINOR.PATCH.

This is given in the Maven Project Object Model (POM) eg

  <groupId>org.melati</groupId>
  <artifactId>melati-parent</artifactId>
  <version>0.8.0-SNAPSHOT</version>

Using Maven and Java and yadayada you can parse this and get the actual unique identifier you want, but life is too short.

We want to use line based unix tools, grep and sed.

Typical Semantic Version strings

3.0.7
6.0.8
2.0.5
2.0.1
5.0.4
2.0.2
1.0.2
1.0.2
2.0.7
1.1.16
3.0.1
1.0.2
1.0.2
5.0.1
5.0.1
5.0.2
5.0.1

These are not all unique and those that are are accidentally unique.

Another way of achieving the same objective would be to duplicate the artifactId in the version. <version>melati-13.0.15-SNAPSHOT</version>

Or to merely ban whitespace: <artifactId>melati</artifactId>
<version>13.0.15-SNAPSHOT</version> becomes <artifactId>melati</artifactId><version>13.0.15-SNAPSHOT</version>

This is fragile: someone will reformat the POM using their IDE and it will fall apart.

To achieve uniqueness we will prepend with an Artifact Code, this code will retain the alphabetic ordering of the artifactId and be gappy so that we can insert new artifacts without breaking that ordering. To make it easy to separate from the Semantic Version the code will be followed by a hyphen not a full stop.

SMD110-3.0.8
SMD120-6.0.9
SMD130-2.0.6
SMD140-2.0.2
SMD150-5.0.5
SMD160-2.0.3
SMD170-1.0.3
SMD180-1.0.3
SMD190-2.0.8
SMD200-1.1.17
SMD210-3.0.2
SMD220-1.0.3
SMD230-1.0.3
SMD240-5.0.2
SMD250-5.0.1
SMD260-5.0.3
SMD270-5.0.2

Now that we have a unique string we have pitons to climb out of the hell we dug for ourselves.

	find * -name pom.xml | \
	grep -v target | \
	xargs sed -b --in-place  's/old-unique-version/new-unique-version/g'