Thursday 30 November 2023

Unique versions as pitons out of Maven Dependency Hell

The whole Maven ecosystem, and more, follows the Semantic Versioning proposal from @mojombo who made a success of being right about most things.

Even Oracle use it (no more ojdbc.jar for them).

The problem Semantic Versioning addressed was the opposing forces of

version lock
the inability to upgrade a package without having to release new versions of every dependent package
version promiscuity
assuming compatibility with more future versions than is reasonable
leading to Dependency Hell.

If that is the problem you have then Semantic Versioning will help. Semantic Versioning helps you live within the Maven ecosystem, if you are contributing to that ecosystem then use Semantic Versioning.

That is not the problem that you have if you dug your very own Dependency Hell due to having a modularisation addiction.
If the problem you have given yourself is creating over a hundred inter-dependent modules because of your desire to obey the injunction Do not Repeat Yourself (DRY) within a modular microservice world the problem you have is achieving consistency.

The Semantic Version string MAJOR.MINOR.PATCH is not unique.

What is actually unique is ARTIFACT.MAJOR.MINOR.PATCH.
Or even uniquer (sic) GROUP.ARTIFACT.MAJOR.MINOR.PATCH.

This is given in the Maven Project Object Model (POM) eg

  <groupId>org.melati</groupId>
  <artifactId>melati-parent</artifactId>
  <version>0.8.0-SNAPSHOT</version>
Using Maven and Java and yadayada you can parse this and get the actual unique identifier you want, but life is too short.

We want to use line based unix tools, grep and sed.

Typical Semantic Version strings

3.0.7
6.0.8
2.0.5
2.0.1
5.0.4
2.0.2
1.0.2
1.0.2
2.0.7
1.1.16
3.0.1
1.0.2
1.0.2
5.0.1
5.0.1
5.0.2
5.0.1

These are not all unique and those that are are accidentally unique.

Another way of achieving the same objective would be to duplicate the artifactId in the version. <version>melati-13.0.15-SNAPSHOT</version>

Or to merely ban whitespace: <artifactId>melati</artifactId>
<version>13.0.15-SNAPSHOT</version>
becomes <artifactId>melati</artifactId><version>13.0.15-SNAPSHOT</version>

This is fragile: someone will reformat the POM using their IDE and it will fall apart.

To achieve uniqueness we will prepend with an Artifact Code, this code will retain the alphabetic ordering of the artifactId and be gappy so that we can insert new artifacts without breaking that ordering. To make it easy to separate from the Semantic Version the code will be followed by a hyphen not a full stop.

SMD110-3.0.8
SMD120-6.0.9
SMD130-2.0.6
SMD140-2.0.2
SMD150-5.0.5
SMD160-2.0.3
SMD170-1.0.3
SMD180-1.0.3
SMD190-2.0.8
SMD200-1.1.17
SMD210-3.0.2
SMD220-1.0.3
SMD230-1.0.3
SMD240-5.0.2
SMD250-5.0.1
SMD260-5.0.3
SMD270-5.0.2

Now that we have a unique string we have pitons to climb out of the hell we dug for ourselves.

	find * -name pom.xml | \
	grep -v target | \
	xargs sed -b --in-place  's/old-unique-version/new-unique-version/g'

Thursday 28 September 2023

Linux Laptop 2023 as a present

My eldest has been using Linux on a 2012 macbook pro with 16GB and an SSD where the optical drive used to be, something she did herself whilst at school, so for her twenty first birthday I wanted to give her a new Linux laptop which she could be proud of and which also had my stamp on it.

I like Linux, OpenSource and plucky engineering, she likes sustainability and being able to run python simulations.

The problem with the macBook is that it feels like the wifi card is beginning to fail, the screen is cracked and it is heavy by today's expectations.

Canvassing the opinion of the remnants of the Oxford Linux Users Group and googling gave me the following candidates.

Candidates

XPS 13 Plus Laptop

https://www.dell.com/en-uk/shop/cty/pdp/spd/xps-13-9320-laptop/

A phenominal choice of screens, but a maximum of 32mb memory. A three year old Dell laptop would be "pocket money prices", as will this be in three years. Work will give you a Dell. Not a twenty first birthday present.

System76 Pangolin

https://system76.com/laptops/pangolin

Not sure about the logo. Does not seem to believe the UK exists. 64GB not available. Shipping costs are high.

Framework Laptop 13

https://frame.work/gb/en/products/laptop-diy-13-gen-amd

Runner up. Nice idea but I am not convinced anyone wants this in a laptop.

Starlabs Starbook

https://starlabs.systems/products/starbook

Blue Cables; 64gb ram; pre-installed Ubuntu; British startup.

Winner


Saturday 16 September 2023

Making maven dependency tree readable

We have a super-POM which ties our POM forest into a tree. It takes 24 minutes just to compile. The dependency tree output is overwhelming, but the below leavs you with a fairly managable todo list of problems.

Assuming org.my-project for your package name


mvn dependency:tree -Dverbose > tree.txt
cat tree.txt | \
grep -v Downloading | \
grep org.my-project | \
grep -v 'omitted for duplicate' | \
grep -v -E 'compile[^ ]' | \
grep -v -E 'test[^ ]' | \
grep -v -E 'runtime[^ ]' | \
grep -v -E 'provided[^ ]' | \
grep -v -E '\-SNAPSHOT[^ ]' | \
grep -v -E ':jar:([0-9]\.[0-9]\.[0-9]):provided \(version managed from \1\)' | \
grep -v -E ':jar:([0-9]\.[0-9]\.[0-9]):compile \(version managed from \1\)' | \
grep -v -E ':jar:([0-9][0-9]\.[0-9]\.[0-9]):compile \(version managed from \1\)' | \
grep -v -E ':jar:([0-9]\.[0-9]\.[0-9][0-9]):compile \(version managed from \1\)' | \
grep -v -E ':jar:([0-9][0-9]\.[0-9]\.[0-9][0-9]):compile \(version managed from \1\)' | \
grep -v -E ':jar:([0-9][0-9]\.[0-9]\.[0-9][0-9][0-9]):compile \(version managed from \1\)' \
> conflicts.txt

Output:
[WARNING] 'parent.relativePath' of POM org.my-project.api-registry:api-registry-parent:3.0.4-SNAPSHOT (api-registry\pom.xml) points at org.my-project:architecture-parent instead of org.my-project.base:base, please verify your project structure @ line 5, column 13
[WARNING] 'parent.relativePath' of POM org.my-project:local-dynamodb-parent:4.2.10-SNAPSHOT (local-dynamodb\pom.xml) points at org.my-project:architecture-parent instead of org.my-project.base:base-java8, please verify your project structure @ line 4, column 11
[WARNING] 'parent.relativePath' of POM org.my-project:local-mongodb-parent:5.0.1-SNAPSHOT (local-mongodb\pom.xml) points at org.my-project:architecture-parent instead of org.my-project.base:base-java11, please verify your project structure @ line 5, column 13
[WARNING] 'parent.relativePath' of POM org.my-project.apps:apps:13.0.8-SNAPSHOT (apps\pom.xml) points at org.my-project:architecture-parent instead of org.my-project.core.check:core-base, please verify your project structure @ line 5, column 13
[WARNING] 'parent.relativePath' of POM org.my-project.events.common:events-common-parent:11.12.22-SNAPSHOT (events-common\pom.xml) points at org.my-project:architecture-parent instead of org.my-project.core.check:core-base, please verify your project structure @ line 5, column 13
[WARNING] 'parent.relativePath' of POM org.my-project.apps:events-processor:12.1.23-SNAPSHOT (events-processor\pom.xml) points at org.my-project:architecture-parent instead of org.my-project.core.check:core-base, please verify your project structure @ line 4, column 11
[WARNING] 'parent.relativePath' of POM org.my-project.kews:kews:1.0.0-SNAPSHOT (kews\pom.xml) points at org.my-project:architecture-parent instead of org.springframework.boot:spring-boot-starter-parent, please verify your project structure @ line 6, column 10
[INFO] ---------< org.my-project.api-registry:api-registry-parent >----------
[INFO] -------------< org.my-project.api-registry:api-registry >-------------
[INFO] ---------< org.my-project.api-registry:api-registry-jersey >----------
[INFO] ---------< org.my-project.api-registry:api-registry-spring >----------
[INFO] ------< org.my-project.audit.logging:audit-logging-parent >-------
[INFO] ----------< org.my-project.audit.logging:audit-logger-dao >-----------
[INFO] --------< org.my-project.audit.logging:audit-logger-stepdefs >--------
[INFO] -----< org.my-project.audit.logging:audit-logger-servlet-filter >-----
[INFO] -----< org.my-project.authentication:authentication-parent >------
[INFO] -------< org.my-project.authentication:authentication-service >-------
[INFO] -----< org.my-project.authentication:ucs-authentication-service >-----
[INFO] -------------------< org.my-project.base:base >-------------------
[INFO] -------------------< org.my-project:common-parent >-------------------
[INFO] ------------------< org.my-project:java-extension >-------------------
[INFO] ----------------< org.my-project:common-junit-parent >----------------
[INFO] --------------< org.my-project:common-dynamic-logging >---------------
[INFO] ------------------< org.my-project:pooled-rest-client >-------------------
[INFO] ------------------< org.my-project:api-client-library >-------------------
[INFO] ------< org.my-project.configuration:configuration-parent >-------
[INFO] -------< org.my-project.configuration:configuration-framework >-------
[INFO] ---< org.my-project.configuration:configuration-provider-mongodb >----
[INFO] --------< org.my-project.configuration:configuration-editor >---------
[INFO] --< org.my-project.format.support:content-profile-format-support-parent >--
[INFO] --< org.my-project.format.support:content-profile-format-support >--
[INFO] -----< org.my-project.format.support:format-support-parent >------
[INFO] ---------< org.my-project.format.support:format-support >---------
[INFO] ------< org.my-project.http.stepdefs:http-stepdefs-parent >-------
[INFO] ----------< org.my-project.http.stepdefs:http-stepdefs >----------
[INFO] -------< org.my-project.issuetracing:issuetracing-parent >--------
[INFO] -----------< org.my-project.issuetracing:issuetracing >-----------
[INFO] ---------< org.my-project.jmxsupport:jmxsupport-parent >----------
[INFO] -------------< org.my-project.jmxsupport:jmxnaming >--------------
[INFO] ---< org.my-project.format.support:json-format-support-parent >---
[INFO] ------< org.my-project.format.support:json-format-support >-------
[INFO] --< org.my-project.format.support:jsonld-format-support-parent >--
[INFO] -----< org.my-project.format.support:jsonld-format-support >------
[INFO] ---------------< org.my-project:local-dynamodb-parent >---------------
[INFO] ------------------< org.my-project:local-dynamodb >-------------------
[INFO] ------------< org.my-project:local-elasticsearch-parent >-------------
[INFO] ----------------< org.my-project:local-elasticsearch >----------------
[INFO] ---------------< org.my-project:local-mongodb-parent >----------------
[INFO] -------------------< org.my-project:local-mongodb >-------------------
[INFO] -----------------< org.my-project:local-sftp-parent >-----------------
[INFO] --------------------< org.my-project:local-sftp >---------------------
[INFO] -----------------< org.my-project:local-sqs-parent >------------------
[INFO] ---------------------< org.my-project:local-sqs >---------------------
[INFO] ----------------< org.my-project:localservers-parent >----------------
[INFO] --< org.my-project.microservicesexceptions:microservicesexceptions-parent >--
[INFO] --< org.my-project.microservicesexceptions:microservicesexceptions-interface >--
[INFO] --< org.my-project.microservicesexceptions:microservicesexceptions-exception-builder >--
[INFO] ----------< org.my-project.migration:migration-parent >-----------
[INFO] ---------< org.my-project.migration:migration-framework >---------
[INFO] ---------< org.my-project.migration:migration-dynamodb >----------
[INFO] ----------< org.my-project.migration:migration-mongodb >----------
[INFO] ------------< org.my-project.migration:migration-csv >------------
[INFO] -------< org.my-project.mongodb-client:mongodb-client-parent >--------
[INFO] -----------< org.my-project.mongodb-client:mongodb-client >-----------
[INFO] --< org.my-project.placeholder.stepdef.support:placeholder-stepdef-support-parent >--
[INFO] --< org.my-project.placeholder.stepdef.support:placeholder-stepdef-support >--
[INFO] --------< org.my-project.smd:queuelistener-container-parent >---------
[INFO] ------------< org.my-project.smd:queuelistener-container >------------
[INFO] |  +- org.my-project:java-extension:jar:5.0.3:test (version managed from 5.0.3)
[INFO] --< org.my-project.format.support:separated-value-format-support-parent >--
[INFO] --< org.my-project.format.support:separated-value-format-support >--
[INFO] --------< org.my-project.timemachine:timemachine-parent >---------
[INFO] ------------< org.my-project.timemachine:timemachine >------------
[INFO] ---------< org.my-project.timemachine:timemachine-steps >---------
[INFO] -------------------< org.my-project.apps:apps >-------------------
[INFO] -------------< org.my-project.apps:workflow-common >--------------
[INFO] -----------------< org.my-project.apps:workflow >-----------------
[INFO] ---------------< org.my-project.apps:workflow-web >---------------
[INFO] -------< org.my-project.apps:events-router-state-checker >--------
[INFO] |  +- (org.my-project.core:framework-util:jar:16.1.19:compile - omitted for conflict with 17.0.3)
[INFO] -----------< org.my-project.apps:events-router-rules >------------
[INFO] |  +- (org.my-project.events.common:events-router-common:jar:11.12.21:compile - omitted for conflict with 11.12.20)
[INFO] |  +- (org.my-project.core:framework-util:jar:16.1.19:compile - omitted for conflict with 17.0.3)
[INFO] -------------< org.my-project.apps:router-processor >-------------
[INFO] |  +- (org.my-project.core:framework-util:jar:16.1.19:compile - omitted for conflict with 17.0.3)
[INFO] --------------< org.my-project.apps:service-queue >---------------
[INFO] ----------< org.my-project.assertion:assertion-parent >-----------
[INFO] ----------< org.my-project.assertion:assertion-service >----------
[INFO] ------------< org.my-project.assertion:assertion-api >------------
[INFO] -------------< org.my-project.audit:audit-logger-parent >-------------
[INFO] ---------< org.my-project.audit:audit-logger-servlet-filter >---------
[INFO] -------< org.my-project.backfill:backfill-scripts-parent >--------
[INFO] -------< org.my-project.backfill:patents-backfill-scripts >-------
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  |  +- (org.my-project.core:properties:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:aws-common:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:properties:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:dataset-service:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.ucs:ucs-client:jar:1.2.1:compile - omitted for conflict with 1.1.6)
[INFO] |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project:api-client-library:jar:3.1.0:compile - omitted for conflict with 5.0.3)
[INFO] ---< org.my-project.backfill:patents-backfill-scripts-mongodb >---
[INFO]    |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO]    |  |  +- (org.my-project.core:properties:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO]    |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO]    |  +- (org.my-project.core:aws-common:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO]    |  +- (org.my-project.core:properties:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO]    |  +- (org.my-project.core:dataset-service:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO]    |  +- (org.my-project.ucs:ucs-client:jar:1.2.1:compile - omitted for conflict with 1.1.6)
[INFO]    |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO]    |  +- (org.my-project:api-client-library:jar:3.1.0:compile - omitted for conflict with 5.0.3)
[INFO] -----------------< org.my-project:backup-parent >-----------------
[INFO] ----------------< org.my-project:backup-service >-----------------
[INFO] |  |  |  +- (org.my-project.jmxsupport:jmxnaming:jar:2.0.10:compile - omitted for conflict with 2.0.8)
[INFO] ----------< org.my-project.cel.service:cel-service-parent >-----------
[INFO] --------------< org.my-project.cel.service:cel-service >--------------
[INFO] -------------< org.my-project.copyright:cgs-parent >--------------
[INFO] -------------< org.my-project.copyright:cgs-service >-------------
[INFO] ---------------< org.my-project.copyright:cgs-api >---------------
[INFO] ---------------< org.my-project.copyright:cgs-web >---------------
[INFO] |  +- (org.my-project.core:framework-util:jar:17.0.2:test - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:framework-util:jar:17.0.1:test - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:rest-client:jar:17.0.1:test - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:aws-common:jar:17.0.1:test - omitted for conflict with 17.0.3)
[INFO] ----------------< org.my-project.core:core-root >-----------------
[INFO] -------------< org.my-project.core.check:core-check >-------------
[INFO] --------------< org.my-project.core:framework-util >--------------
[INFO] -------------< org.my-project.core.check:core-base >--------------
[INFO] ----------------< org.my-project.core:web-parent >----------------
[INFO] ------------< org.my-project.core:common-web-parent >-------------
[INFO] ----------------< org.my-project.core:common-web >----------------
[INFO] ------< org.my-project.configuration:configuration-parent >-------
[INFO] ---------< org.my-project.configuration:property-config >---------
[INFO] -----------< org.my-project.configuration:data-utils >------------
[INFO] --------------< org.my-project.core:service-parent >--------------
[INFO] -------------< org.my-project.core:rest-api-parent >--------------
[INFO] -------------< org.my-project.core:workflow-parent >--------------
[INFO] ----------------< org.my-project.core:properties >----------------
[INFO] ----------------< org.my-project.core:aws-common >----------------
[INFO] -----------------< org.my-project.core:auth-dao >-----------------
[INFO] ----------------< org.my-project.core:ldap-auth >-----------------
[INFO] ---------------< org.my-project.core:rest-client >----------------
[INFO] ---------------< org.my-project.core:digest-auth >----------------
[INFO] -----------< org.my-project.core:elasticsearch-common >-----------
[INFO] -------------< org.my-project.core:common-rest-api >--------------
[INFO] -------------< org.my-project.core:dataset-service >--------------
[INFO] --------------< org.my-project.core:identity-model >--------------
[INFO] --------< org.my-project.core:duplicate-identity-checker >--------
[INFO] -------------< org.my-project.core:events-generator >-------------
[INFO] -----------------< org.my-project.core:ftp-util >-----------------
[INFO] ------------< org.my-project.core:elastic-search-client >-------------
[INFO] ---------< org.my-project.core.import:core-import-time >----------
[INFO] --------< org.my-project.core.import:core-import-spring >---------
[INFO] --------< org.my-project.core.import:core-import-jersey >---------
[INFO] --------< org.my-project.core.import:core-import-apache >---------
[INFO] --------< org.my-project.core.import:core-import-google >---------
[INFO] --------< org.my-project.core.import:core-import-jackson >--------
[INFO] --------< org.my-project.core.import:core-import-logging >--------
[INFO] ----------< org.my-project.core.import:core-import-aws >----------
[INFO] --------< org.my-project.core.import:core-import-opencsv >--------
[INFO] --------< org.my-project.core.import:core-import-ehcache >--------
[INFO] ---------< org.my-project.core.import:core-import-cglib >---------
[INFO] -----< org.my-project.core.import:core-import-elasticsearch >-----
[INFO] -------< org.my-project.core.import:core-import-hibernate >-------
[INFO] ---------< org.my-project.core.import:core-import-jaxen >---------
[INFO] ---------< org.my-project.core.import:core-import-jsch >----------
[INFO] ---------< org.my-project.core.import:core-import-jstl >----------
[INFO] --------< org.my-project.core.import:core-import-aspectj >--------
[INFO] ------< org.my-project.core.import:core-import-freemarker >-------
[INFO] ----------< org.my-project.core.import:core-import-xml >----------
[INFO] --------< org.my-project.core.import:core-import-servlet >--------
[INFO] ---------< org.my-project.core.import:core-import-javax >---------
[INFO] --------< org.my-project.core.import:core-import-jayway >---------
[INFO] --------< org.my-project.core.import:core-import-github >---------
[INFO] --------< org.my-project.core.import:core-import-jolbox >---------
[INFO] ---------< org.my-project.core.import:core-import-cukes >---------
[INFO] -------< org.my-project.core.import:core-import-logstash >--------
[INFO] --------< org.my-project.core.import:core-import-minidev >--------
[INFO] --------< org.my-project.core.import:core-import-oracle >---------
[INFO] -------< org.my-project.core.import:core-import-codehaus >--------
[INFO] --------< org.my-project.core.import:core-import-eclipse >--------
[INFO] ---------< org.my-project.core.import:core-import-jboss >---------
[INFO] ---------< org.my-project.core.import:core-import-json >----------
[INFO] ---------< org.my-project.core.import:core-import-yaml >----------
[INFO] ------< org.my-project.core.import:core-import-skyscreamer >------
[INFO] ------< org.my-project.core.import:core-import-flapdoodle >-------
[INFO] -------< org.my-project.core.import:core-import-searchbox >-------
[INFO] --------< org.my-project.core.import:core-import-mongodb >--------
[INFO] ----------< org.my-project.core.import:core-import-smd >----------
[INFO] ------------< org.my-project.core.import:core-import >------------
[INFO] -------< org.my-project.core.check:core-check-deduplicate >-------
[INFO] ---------------< org.my-project.core:core-parent >----------------
[INFO] -------------< org.my-project.delivery:delivery-app >-------------
[INFO] -----------< org.my-project.delivery:delivery-service >-----------
[INFO] |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] -------------< org.my-project.delivery:delivery-api >-------------
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:framework-util:jar:17.0.2:compile - omitted for conflict with 17.0.3)
[INFO] -----------< org.my-project.delivery:delivery-client >------------
[INFO] ----< org.my-project.event.framework:event-framework-parent >-----
[INFO] ----< org.my-project.event.framework:event-framework-common >-----
[INFO] ------< org.my-project.event.framework:event-framework-lib >------
[INFO] ---------< org.my-project.event.framework:event-service >---------
[INFO] ------< org.my-project.event.framework:event-framework-api >------
[INFO] |  |  +- (org.my-project.microservicesexceptions:microservicesexceptions-exception-builder:jar:3.0.1:compile - omitted for conflict with 3.0.0)
[INFO] |  +- (org.my-project.placeholder.stepdef.support:placeholder-stepdef-support:jar:2.0.0:test - omitted for conflict with 2.0.2)
[INFO] |  |  +- (org.my-project.placeholder.stepdef.support:placeholder-stepdef-support:jar:2.0.0:test - omitted for conflict with 2.0.2)
[INFO] |  |  +- (org.my-project.placeholder.stepdef.support:placeholder-stepdef-support:jar:2.0.0:test - omitted for conflict with 2.0.2)
[INFO]    +- (org.my-project:java-extension:jar:5.0.2:compile - omitted for conflict with 5.0.0)
[INFO] --< org.my-project.event.framework.rules:event-framework-rules-parent >--
[INFO] -------< org.my-project.event.framework.rules:ev-patent-rule >--------
[INFO] -------< org.my-project.event.framework.rules:cef-reaxys-rule >-------
[INFO] -----< org.my-project.event.framework.rules:scopus-patent-rule >------
[INFO] ----------< org.my-project.events.common:events-common-parent >-----------
[INFO] ----------< org.my-project.events.common:events-router-common >-----------
[INFO] -------------< org.my-project.apps:events-processor >-------------
[INFO] |  |  +- (org.my-project.jmxsupport:jmxnaming:jar:1.0.1:compile - omitted for conflict with 2.0.2)
[INFO] ----------< org.my-project.events.router:events-router-parent >-----------
[INFO] --------------< org.my-project.events.router:events-router >--------------
[INFO] |  +- (org.my-project.core:framework-util:jar:16.1.19:compile - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:framework-util:jar:17.0.2:test - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:properties:jar:17.0.2:runtime - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:framework-util:jar:17.0.2:runtime - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:aws-common:jar:17.0.2:test - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:rest-client:jar:16.1.7:runtime - omitted for conflict with 17.0.2)
[INFO] |  |  +- (org.my-project.core:aws-common:jar:16.1.7:runtime - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:framework-util:jar:16.1.7:runtime - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:rest-client:jar:17.0.3:runtime - omitted for conflict with 17.0.2)
[INFO] -----------------< org.my-project.events.router:cel-api >-----------------
[INFO] |  +- (org.my-project.core:framework-util:jar:16.1.19:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- org.my-project.api-registry:api-registry:jar:3.0.2:compile (version managed from 3.0.3)
[INFO] |  |  +- (org.my-project.core:properties:jar:17.0.2:test - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:aws-common:jar:17.0.2:test - omitted for conflict with 17.0.3)
[INFO] ------------------< org.my-project.gis:gis-parent >-------------------
[INFO] ------------------< org.my-project.gis:gis-service >------------------
[INFO] |  |  +- (org.my-project.jmxsupport:jmxnaming:jar:2.0.10:compile - omitted for conflict with 2.0.8)
[INFO] --------------------< org.my-project.gis:gis-api >--------------------
[INFO] |  |  +- (org.my-project.jmxsupport:jmxnaming:jar:2.0.10:compile - omitted for conflict with 2.0.8)
[INFO] |  |  |  +- (org.my-project.microservicesexceptions:microservicesexceptions-exception-builder:jar:3.0.1:compile - omitted for conflict with 3.0.0)
[INFO] |  |  |  +- (org.my-project.timemachine:timemachine:jar:4.0.6:compile - omitted for conflict with 4.0.5)
[INFO] |  |  +- (org.my-project.timemachine:timemachine:jar:4.0.6:compile - omitted for conflict with 4.0.5)
[INFO] --------------< org.my-project.inner:inner-parent >---------------
[INFO] --------------< org.my-project.inner:inner-common >---------------
[INFO] ---------------< org.my-project.inner:inner-model >---------------
[INFO] ---------------< org.my-project.inner:inner-ewii >----------------
[INFO] -------------< org.my-project.inner:ucs-common-service >--------------
[INFO] |  +- org.my-project:api-client-library:jar:5.0.3:compile (version managed from 3.1.0)
[INFO] --------------< org.my-project.inner:inner-adaptor >--------------
[INFO] |  |  +- org.my-project:api-client-library:jar:5.0.3:compile (version managed from 3.1.0)
[INFO] ------------< org.my-project.inner:inner-etl-service >------------
[INFO] |  |  +- org.my-project:api-client-library:jar:5.0.3:compile (version managed from 3.1.0)
[INFO] --------------< org.my-project.inner:inner-etl-api >--------------
[INFO] |  |  |  +- org.my-project:api-client-library:jar:5.0.3:compile (version managed from 3.1.0)
[INFO] --------< org.my-project.inner:inner-integration-service >--------
[INFO] ----------< org.my-project.inner:inner-integration-api >----------
[INFO] ---------------------< org.my-project.kews:kews >---------------------
[INFO] ----------------< org.my-project.lov:lov-parent >-----------------
[INFO] ----------------< org.my-project.lov:lov-service >----------------
[INFO] ------------------< org.my-project.lov:lov-api >------------------
[INFO] -----------< org.my-project.metadata:metadata-parent >------------
[INFO] -----------< org.my-project.metadata:metadata-common >------------
[INFO] -------------< org.my-project.metadata:metadata-dao >-------------
[INFO] ----------< org.my-project.metadata:validation-service >----------
[INFO] ---------< org.my-project.metadata:addonsearch-service >----------
[INFO] -----------< org.my-project.metadata:contributor-dao >------------
[INFO] -----------< org.my-project.metadata:metadata-service >-----------
[INFO] ---------< org.my-project.metadata:metadata-api-adapter >---------
[INFO] -------------< org.my-project.metadata:metadata-api >-------------
[INFO] -----------< org.my-project.metadata:contributor-api >------------
[INFO] ---------< org.my-project.event.framework.rules:sample-rule >---------
[INFO] -------------< org.my-project.oa:open-access-parent >-------------
[INFO] ------------< org.my-project.oa:open-access-service >-------------
[INFO] |  +- (org.my-project.core:framework-util:jar:17.0.1:compile - omitted for conflict with 17.0.3)
[INFO] ------------< org.my-project.oa:open-access-workflow >------------
[INFO] |  |  +- (org.my-project.core:framework-util:jar:17.0.1:compile - omitted for conflict with 17.0.3)
[INFO] --------------< org.my-project.oa:open-access-api >---------------
[INFO] |  |  +- (org.my-project.core:framework-util:jar:17.0.1:compile - omitted for conflict with 17.0.3)
[INFO] --------------< org.my-project.oa:open-access-web >---------------
[INFO] |  |  +- (org.my-project.core:framework-util:jar:17.0.1:compile - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:framework-util:jar:17.0.1:compile - omitted for conflict with 17.0.3)
[INFO] ----------< org.my-project.oa:open-access-workflow-web >----------
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:17.0.1:compile - omitted for conflict with 17.0.3)
[INFO] -----------< org.my-project.property:property-parent >------------
[INFO] -------------< org.my-project.property:property-dao >-------------
[INFO] -----------< org.my-project.property:property-service >-----------
[INFO] -------------< org.my-project.property:property-api >-------------
[INFO] -------------------< org.my-project.rp:rp-app >-------------------
[INFO] ----------------< org.my-project.apps:rp-service >----------------
[INFO] |  |  +- (org.my-project.core:framework-util:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:framework-util:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:properties:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:framework-util:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] ------------------< org.my-project.apps:rp-api >------------------
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:properties:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:framework-util:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:properties:jar:17.0.1:compile - omitted for conflict with 17.0.3)
[INFO] |  |  |  +- (org.my-project.mongodb-client:mongodb-client:jar:2.0.1:compile - omitted for conflict with 1.1.12)
[INFO] ------------------< org.my-project.apps:rp-web >------------------
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  |  |  +- (org.my-project.core:framework-util:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:properties:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  |  +- (org.my-project.core:framework-util:jar:16.1.23:compile - omitted for conflict with 17.0.3)
[INFO] |  +- (org.my-project.core:properties:jar:17.0.1:compile - omitted for conflict with 17.0.3)
[INFO] |  |  |  +- (org.my-project.mongodb-client:mongodb-client:jar:2.0.1:compile - omitted for conflict with 1.1.12)
[INFO] --------------------< org.my-project.sdk:sdk >--------------------
[INFO] -----------------< org.my-project.sdk:java-sdk >------------------
[INFO] --------------< org.my-project.sms:shared-messaging-parent >--------------
[INFO] --------------< org.my-project.sms:shared-messaging-common >--------------
[INFO] ---------------< org.my-project.sms:shared-messaging-api >----------------
[INFO] |  |  +- (org.my-project.microservicesexceptions:microservicesexceptions-exception-builder:jar:3.0.1:compile - omitted for conflict with 3.0.0)
[INFO] |  +- (org.my-project.placeholder.stepdef.support:placeholder-stepdef-support:jar:2.0.0:test - omitted for conflict with 2.0.2)
[INFO] |  +- (org.my-project:java-extension:jar:5.0.0:test - omitted for conflict with 5.0.2)
[INFO] |  |  +- (org.my-project.placeholder.stepdef.support:placeholder-stepdef-support:jar:2.0.0:test - omitted for conflict with 2.0.2)
[INFO] |  |  +- (org.my-project.placeholder.stepdef.support:placeholder-stepdef-support:jar:2.0.0:test - omitted for conflict with 2.0.2)
[INFO] -------------< org.my-project.sms:shared-messaging-service >--------------
[INFO] ----------< org.my-project.skip.mgmt:skip-mgmt-parent >-----------
[INFO] ----------< org.my-project.skip.mgmt:skip-mgmt-service >----------
[INFO] ------------< org.my-project.skip.mgmt:skip-mgmt-web >------------
[INFO] -------< org.my-project.reporting:system-reporting-parent >-------
[INFO] ----------< org.my-project.reporting:system-reporting >-----------
[INFO] --------------< org.my-project.web:web-aggregation >--------------
[INFO] -----------------< org.my-project.web:inner-web >-----------------
[INFO] ---------------< org.my-project.web:metadata-web >----------------
[INFO] ---------------< org.my-project.web:property-web >----------------
[INFO] ---------------< org.my-project.web:dashboard-web >---------------

Tuesday 12 September 2023

Load bearing structures

Fast, light lily trotter.
Dall-e Hippo.
Hippo trotter.
For me this is connected with writing software.

Saturday 5 August 2023

Variety build up in a long lived project

My current project has 319 Maven POMs. These have been worked upon by at least five 'generations' of coders, that is under five different software engineering leads. Maven enables different actions depending upon different profiles. This seems like a good idea: you want a new behavior, the work you are doing is new to you and maybe to the project. Name your profile and get the thing you want done. Lets see how that work out over ten years.
find * -name pom.xml | grep -v git |grep -v target \
    |xargs grep -h -a1 "<profile>" \
    |grep "<id>" |tr -d '[ \t]'| \
    sed  's/<[\/]*id>//g'| \
    sort |uniq -c

      2 code-quality
      5 component-test
     64 full
      1 Full
      5 im-elasticsearch
      1 it-test
      1 it-tests
      1 local-it-tests
     28 nightly
      1 owasp
      8 unit

2 code-quality
seems like you want to separate out your Sonar run, maybe combine it with another profile, say security checking, and want the luxury to only run one at a time.
Use full
5 component-test
wat? Is this an integration test or a unit test?
Use full
64 full
short and to the point
Winner!
1 Full
Typo
Use full
5 im-elasticsearch
Might have been a candidate, if inheritted and there was a need for mix-ins, but neither holds. Requires component-test
Use full
1 it-test
Should be plural.
Use full
1 it-tests
IT stands for Integration Test so it-tests is Integration Test Tests.
Use full
1 local-it-tests
Integration tests which are not run on the Continuous Integration server? Wrong.
Use full
28 nightly
These presumably still need to be run on any change. Control periodicity in the Continuous Integration server configuration.
Use full
1 owasp
What are these? Why are they special snowflakes? OWASP tests need to be run on any change.
Use full
8 unit
Unit tests are the default profile.
Delete

Using multiple profiles is a bad idea.

Unit tests should be run in any scenario, and should be run as part of development.

A reasonable distinction between tests is those which require external fixtures off the developers machine and those which do not, or those that take a long time to run, disabling the developer's machine. Some tests attempt to drain resources from the machine they are running on, not great on the developer's machine. There is no inherent advantage to calling out these motivations in the profile name.

What we want is to be able to reason about the whole class, to be able to say true things about all our builds. This is a much greater advantage than an attempt to communiate with other developers through naming.

Use full

Now we know that we have three types of build:

no tests
-Dmaven.test.skip=true
(preferred over -skipTests)
unit tests
Default profile, nothing for you to do.
all tests
-Pfull

We can happily say that the only thing to do when writing code to build our code is to use the profile full.

Variety is sand in the gears of our development machine, however much it may mean to you is wears away at your precious attention.

Monday 17 July 2023

Press Release Driven Development

I realised, as I stumbled forwards with a presentation on my current project, that the presentation can be seen as a Press Release and remembered that I have explicitly followed the Amazon Working Backwards practice twice before.

Write the PR at the start of the project

This informs the team what the point of the activity is, which is not always self-evident.

Internal Press Release: Important System Upgrade Completed

Anticipated release date see: Upgrade Epic Ticket

Today we can announce that we have upgraded the whole of Important System.

In the process we have achieved the following:

  • Extended and upgraded our ability to act upon and monitor all repositories
  • Archived XX repositories
  • Reduced the number of ignored tests from XX to XX
  • Removed all Sonar Bugs and Vulnerabilities
  • Updated all libraries
  • Installed Dependabot to ensure we do not fall behind again

This enables us to take on the strategic work of removing Bad Thing from our estate.

The numeric gaps in the Press Release make it explicit what needs to be measured before and after the project.

Trying to use AI to create an image for work

The idea I want to convey, in a slide deck at work, is that the codebase we are working on has not had enough attention for five years. It has been run by a small number of people who managed to keep the system working, 'kept the lights on' but the team were not the original authors, and was not big enough, and not empowered enough, to make updates, let alone the structural changes required.

I hit upon the idea of The Marie Celeste and her skeleton crew. The original authors had left, the system was still running, but those running it were too few.

HotPot.ai

HotPot.ai is really good. It has a nice freemium on-ramp, generates images fast, and nearly had me spend money with them. I certainly used up my free goes.

I had to choose styles and engines.


Where is the crew?


Where is the crew?


Where is the crew?


Where is the crew? Appears to be a futuristic gun on prow.


Maybe some crew, the one I went with.


Where is the crew?


Too marvel


Too stylised


Too sexualised


I changed the prompt to Crew the ghost ship Marie Celeste or similar, Dalle encorporates the prompt into the filename.


I changed the prompt back to The ghost ship Marie Celeste or similar, I have lost track of the prompts by this stage.


Back to the crew.


Too much emphasis on the word skeleton


I guess it is picking up on 'girls name' + skeleton


'girls name' + skeleton in a different style


Too much skeleton


Back to just the ship


Just the ship in a different style


Skeleton and ship, but I have run out of free goes and the will to live.

creator.nightcafe.studio


creator.nightcafe.studio is a similar, polished service. I liked this creation, though it is more Flying Dutchman than Marie Celest and has no crew, enough to tweet it.


I added the word 'Indian'


Lost the ship


Added 'on the sea'


Too much sea.


Lost the plot now


Back to ghost ship


'Indian skeleton crew on a ghost ship' in a different style


'Indian ghost ship' in a different style


'Ghost ship on the sea'


'Indian ghost ship on the sea' in a different style


'Indian ghost ship on the sea'

Out of credits

Dall-e


The big user benefit of Dall-e is that the prompt is in the file name, so not relying upon my poor memory. Dall-e does not seem to offer a download all option, which the others do.


The Marie Celeste and her Skeleton crew in a fantasy style


The Marie Celeste and her Skeleton crew in a fantasy style


The Marie Celeste and her Skeleton crew in a fantasy style


The Marie Celeste and her Skeleton crew in a fantasy style

Conclusion

I never got the image I wanted, maybe the two ideas are too explicit, but I have an image which I can use. This process was not quick, but it is better than choosing a stock image from the Powerpoint selection.