Blogging cooking

Now blogging my cooking at http://weekend-chef.blogspot.com/ as I could not remember how I last cooked duck.

I blog as I cook, which brings the stress of cooking to a head, but should bring a little more planning and structure to future meals.

A Trigger Servlet for Hudson

Have written a little Trigger Servlet for Hudson, to enable post commit builds on Google Code projects.

By relying upon a shared secret, it enables pre-authentication of the trigger url; which should only be visible to someone with administrator access to the Hudson job and
administrator access to the triggering site, for example the Google Code Administer Source page.

Managed to hit 100% line and branch coverage Yay!.

CSV unification

Unifier on GitHub

A simple problem

Given an excel spreadsheet with three sheets within it, all with approximately the same rows but with differing columns, produce a unified CSV file. If a sheet does not contain a row then insert blank columns. The unique key for each sheet is column 2, called ID, move this column to column 1. No doubt you can see all the issues, as I can in retrospect, but after a week and a bit the job is done. High test coverage enabled me to refactor and add features through to a completely different beast.

Lessons learned

hashcode() for enum constants

The members of an enum inherit their hashCode() method from Object so it varies between JVM invocations. Hence I used

result = prime * result + unificationOption.ordinal();

in CsvTable.hashCode().

Bridging methods

During the compilation of generic code Java quietly generates what are called bridging methods these are visible to Cobertura but not to you, the coder, so Cobertura tries to tell you that you have not exercised these methods by marking the class definition line as not covered. Using the following code to print out all methods

for (Method m : CsvTable.class.getMethods()) {
  System.out.println(m.toGenericString());
}

I discovered the generated bridging methods

public net.pizey.csv.CsvRecord get(java.lang.Object)
public java.lang.Object get(java.lang.Object)

public net.pizey.csv.CsvRecord put(java.lang.String,net.pizey.csv.CsvRecord)
public java.lang.Object put(java.lang.Object,java.lang.Object)

public net.pizey.csv.CsvRecord remove(java.lang.Object)
public java.lang.Object remove(java.lang.Object)

The put(Object key, Object value) cannot be accessed normally, as it is masked by the generic version. So we have to introduce a reflective mechanism of invoking it.

public void testBridgingPut() {
  CsvTable t = new CsvTable("src/test/resources/sheet2.csv", UnificationOptions.LOG);
  Object o = t.get((Object) "1");
  Method method = t.getClass().getMethod("put", new Class[] { Object.class, Object.class });
  method.invoke(t, "jj", o);
}

Finally I had to deal with the generated bridging methods for clone. This is done by not supplying a generic clone method but using the pre-generics signature.

I have come to enjoy Java6 however Generics do feel like an expensive compromise.

Tools and sources

The CSV parser was written by WilliamC and incorporated in a CSV importer in 2000 by MylesC. I started from that framework, but as it was written before Java Collections, let alone Java 6, not a lot remains of the original.
The tools used: GitHub, Hudson CI, Maven, Eclipse and Cobertura.

Sharing and Deployment

The code is at https://github.com/timp21337/unifier.

To include the jar in Maven:

<dependency>
 <groupId>net.pizey.csv>/groupId>
  <artifactId>unifier</artifactId>
  <version>1.0</version>
</dependency>

Deployed to a Maven repository at http://pizey.net/maven2/

Adding a framebuffer to a headless unix server

Some java software, including GWT, expects a framebuffer to be present.
This post shows how to install Xvfb (X virtual Frame Buffer) on a Ubuntu/Debian system. This assumes that you do not currently have an X server running (it is a headless server).

apt-get install xvfb xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic

(the fonts are not needed but their absense is warned about)
Place the following in /etc/init.d/xvfb

case "$1" in
start)
echo "Starting virtual X frame buffer: Xvfb"

/usr/bin/Xvfb :0 \
-auth /etc/X99.cfg -screen 0 1024x768x24 \
-fbdir /var/lib/xvfb \
-extension RANDR &

export  DISPLAY=:0

;;
stop)
echo "Stopping virtual X frame buffer: Xvfb"
echo "noop"
;;
*)
echo "Usage: /etc/init.d/xvfb {start|stop}"
exit 1
;;
esac

exit 0

then

chmod u+x /etc/init.d/xvfb 
echo localhost > /etc/X99.cfg
mkdir /var/lib/xvfb
/etc/init.d/xvfb start

should be good to go.

Using both Http-basic and session based form-login authorisation in Spring Security with wget (--auth-no-challenge)

We recently changed our Spring Security configuration from http-basic to form-login, well actually we used

<http 
auto-config="true" >

which is shorthand for

<http>
<intercept-url pattern="/**" access="ROLE_USER" />
<form-login />
<anonymous />
<http-basic />
<logout />
<remember-me />
</http>

The above, and the rest of the documentation suggests that both http-basic and form-login should work simultaneously. However my wget script broke on the change over. The wget command was

wget  --user=foo@example.org --password=bar --post-data="" \
-d http://localhost/protectedUrl 

This got redirected to the forms login page, as though the authorisation header was being ignored. However alimanfoo was able to use authenticated requests from java to access the protected urls.

Further research revealed that wget only issues authorisation headers in response to a challenge, unless the --auth-no-challenge qualifier is added.

wget  --user=foo@example.org --password=bar --post-data="" \
--auth-no-challenge -d http://localhost/protectedUrl 

You might have thought that supplying username and password meant that you wanted to set them, and this was the behaviour of wget 1.10.2 and prior but no longer.

In fluent manpage we get:

Use of this option is not recommended, and is intended only to
support some few obscure servers, which never send HTTP
authentication challenges, but accept unsolicited auth info, say,
in addition to form-based authentication.

So I am off to explain the use-case to the maintainer list.

Android fonts for Debian/Ubuntu

I first came across the mention of the free, Open Source, Android fonts a while ago; here, I think.
They are clean and easy on the eye.

You can install them with

sudo apt-get install ttf-droid 

which will place fonts in /usr/share/fonts/truetype/ttf-droid

However if all you want is to use a droid font in a web page just use

<link href='http://fonts.googleapis.com/css?family=Droid+Sans' 
rel='stylesheet' type='text/css'>

Bringing WebMacro up for air

WebMacro was the first OpenSource community that I followed, so I love it and still feel a sense of hurt that licensing issues meant that Velocity had to be born.

On top of that I have quite a few WebMacro templates around, and anyway the world needs more than one templating engine.

However, WebMacro went the way of all flesh: The original author moved on, the push to version 1.0 was slow, and I for one stopped following every thread. Versioning was a real mess; (which was more recent 1.0b or 1.0?) Then came the painful, acrid push to 2.0.

I have revisited a few times, cleaning the code of warnings, ensuring tests continue to pass, even regenerating with new versions of javacc.

However what was really needed was a restructuring of the source tree to align with Maven conventions and a thorough cleanout.

The only way to do that used to be to backup the CVS tree, restructure and then ask a SF admin to recreate the repository, which was too big a hurdle.

Recently (well some time in the last ten years) SF have introduced adminrepo, which enables you to lock a repository, restructure it and then replace:

ssh -t timp,webmacro@shell.sourceforge.net create
adminrepo --checkout cvs
cd /cvsroot/webmacro/webmacro
mkdir src/main
mkdir src/main/java
mv src/org src/main/java
cd ~
adminrepo --save cvs
exit


By this means I was able to move current code into the places Maven expects them and move non-core code out to ../contrib.

Then I upgraded from Doug Lea's venerable Concurrent to his java.util.concurrent.

Thanks to a message from Brian Goetz himself I was able to surmount the only real hickup: EDU.oswego.cs.dl.util.concurrent.ClockDaemon becomes java.util.concurrent.ScheduledExecutorService.

Now WebMacro had only one compile-time dependency: the logging system. On inspection it seems that WebMacro's own logging (yes its has a complete logging system built in) should be ripped out in favour of slf4j.

WebMacro 2.2 should be back in the swim.