Using both Http-basic and session based form-login authorisation in Spring Security with wget (--auth-no-challenge)

We recently changed our Spring Security configuration from http-basic to form-login, well actually we used

<http 
auto-config="true" >

which is shorthand for

<http>
<intercept-url pattern="/**" access="ROLE_USER" />
<form-login />
<anonymous />
<http-basic />
<logout />
<remember-me />
</http>

The above, and the rest of the documentation suggests that both http-basic and form-login should work simultaneously. However my wget script broke on the change over. The wget command was

wget  --user=foo@example.org --password=bar --post-data="" \
-d http://localhost/protectedUrl 

This got redirected to the forms login page, as though the authorisation header was being ignored. However alimanfoo was able to use authenticated requests from java to access the protected urls.

Further research revealed that wget only issues authorisation headers in response to a challenge, unless the --auth-no-challenge qualifier is added.

wget  --user=foo@example.org --password=bar --post-data="" \
--auth-no-challenge -d http://localhost/protectedUrl 

You might have thought that supplying username and password meant that you wanted to set them, and this was the behaviour of wget 1.10.2 and prior but no longer.

In fluent manpage we get:

Use of this option is not recommended, and is intended only to
support some few obscure servers, which never send HTTP
authentication challenges, but accept unsolicited auth info, say,
in addition to form-based authentication.

So I am off to explain the use-case to the maintainer list.

Android fonts for Debian/Ubuntu

I first came across the mention of the free, Open Source, Android fonts a while ago; here, I think.
They are clean and easy on the eye.

You can install them with

sudo apt-get install ttf-droid 

which will place fonts in /usr/share/fonts/truetype/ttf-droid

However if all you want is to use a droid font in a web page just use

<link href='http://fonts.googleapis.com/css?family=Droid+Sans' 
rel='stylesheet' type='text/css'>

Bringing WebMacro up for air

WebMacro was the first OpenSource community that I followed, so I love it and still feel a sense of hurt that licensing issues meant that Velocity had to be born.

On top of that I have quite a few WebMacro templates around, and anyway the world needs more than one templating engine.

However, WebMacro went the way of all flesh: The original author moved on, the push to version 1.0 was slow, and I for one stopped following every thread. Versioning was a real mess; (which was more recent 1.0b or 1.0?) Then came the painful, acrid push to 2.0.

I have revisited a few times, cleaning the code of warnings, ensuring tests continue to pass, even regenerating with new versions of javacc.

However what was really needed was a restructuring of the source tree to align with Maven conventions and a thorough cleanout.

The only way to do that used to be to backup the CVS tree, restructure and then ask a SF admin to recreate the repository, which was too big a hurdle.

Recently (well some time in the last ten years) SF have introduced adminrepo, which enables you to lock a repository, restructure it and then replace:

ssh -t timp,webmacro@shell.sourceforge.net create
adminrepo --checkout cvs
cd /cvsroot/webmacro/webmacro
mkdir src/main
mkdir src/main/java
mv src/org src/main/java
cd ~
adminrepo --save cvs
exit


By this means I was able to move current code into the places Maven expects them and move non-core code out to ../contrib.

Then I upgraded from Doug Lea's venerable Concurrent to his java.util.concurrent.

Thanks to a message from Brian Goetz himself I was able to surmount the only real hickup: EDU.oswego.cs.dl.util.concurrent.ClockDaemon becomes java.util.concurrent.ScheduledExecutorService.

Now WebMacro had only one compile-time dependency: the logging system. On inspection it seems that WebMacro's own logging (yes its has a complete logging system built in) should be ripped out in favour of slf4j.

WebMacro 2.2 should be back in the swim.

Deploying large war files to Tomcat under Hudson: use Cargo

Whilst

mvn tomcat:deploy

works for the majority of war files under Hudson, for a large war (34m) the deploy failed with Out of memory error, though not at the shell prompt. Failure to fix by tweeking memory arguments led me to use Cargo instead. Note that the local tomcat is treated as remote to reduce rework when deploying to a remote server.

<build>
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<configuration>
<wait>true</wait>
<container>
<containerId>tomcat6x</containerId>
<type>remote</type>
</container>
<configuration>
<type>runtime</type>
<properties>
<cargo.tomcat.manager.url>http://localhost:8080/manager</cargo.tomcat.manager.url>
<cargo.remote.username>admin</cargo.remote.username>
<cargo.remote.password></cargo.remote.password>
</properties>
</configuration>
<deployer>
<type>remote</type>
<deployables>
<deployable>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<properties>
<context>${project.artifactId}</context>
</properties>
<type>war</type>
</deployable>
</deployables>
</deployer>
</configuration>
<executions>
<execution>
<id>do</id>
<phase>pre-integration-test</phase>
<goals>
<goal>deployer-redeploy</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

Setting up Hudson CI for GWT Development

GWT development requires a 32 bit Sun JDK.

We decided to do this by using a 32 bit VM, which could be moved to another server at a later date.

1. Install VMWare Player (Ensure networking is enabled during installation)
2. Download a 32 bit ubuntu ISO (or 32 bit Debian)
3. Create a new VM from the ISO.
4. Enter VM and establish your ipaddress.
5. Add hudson repo to /etc/apt/sources

   deb http://hudson.gotdns.com/debian binary/
   

6. Install Sun JDK

   apt-get install sun-java6-jdk 
   update-alternatives --config java
   update-alternatives --config javac

7. Install Tomcat

   apt-get install tomcat6
   apt-get install tomcat6-admin
   

   Modify /etc/tomcat/tomcat-users.xml and add
   <user username="admin" password="" roles="manager,admin"/>
   (or add a password and add password to ~/.m2/settings.xml)
   Set Tomcat security off
   

   sed -i "s/TOMCAT6_SECURITY=.\+/TOMCAT6_SECURITY=no/" \
   /etc/default/tomcat6
   

   Increase Tomcat default memory allocation in /etc/defaults/tomcat6
   JAVA_OPTS="-Djava.awt.headless=true -Xmx1024M -Xms1024m"
   
8. Install Hudson

   apt-get install hudson
   

9. Change Hudson port and ajpport by editting /etc/init.d/hudson to add

   HUDSON_ARGS="--httpPort=8081  --ajp13Port=8102" 

10. Install Apache

    apt-get install apache2
    a2enmod proxy
    a2enmod proxy_http

11. Setup /etc/apache2/httpd.conf (or similar)

servername hudson
ProxyPass   /  http://localhost:8081/
ProxyPassReverse  /  http://localhost:8081/
ProxyRequests   Off
<Proxy http://localhost:8081/*>
Order deny,allow
Allow from all
</Proxy>

12. Install Maven

    apt-get install maven2

13. Configure Hudson
    • setup smtp
    • set MAVEN_HOME to /usr/share/maven2
    • set email from address
    • set url to ip address
14. Download GWT toolkit 1.7.1 and extract then install gwt-dev-linux.jar to your local repo

    mvn install:install-file \
    -DgroupId=com.google.gwt -DartifactId=gwt-dev \
    -Dversion=1.7.1 -Dclassifier=linux \
    -Dpackaging=jar -Dfile=gwt-dev-linux.jar

15. Add individual projects to Hudson

Enabling private cross site scripting with a XMLHttpRequest proxy servlet

The problem

You have a list of URLs and want to interrogate them for their Content-Type using javascript in the browser.

You cannot do this because the javascript security model forbids this activity, called cross site scripting, as it could be used to invoke malign code.

The solution

Install a proxy on your server which can make the request on your behalf and relay it back to you. You have not violated the javascript security model and you are free to invoke any url on the net by a call to your own, originating, server.

However the proxy does need to be protected otherwise it could be abused by other sites. To protect the proxy we ensure that it can only be accessed from a page generated by our server, indeed one which has set a named session variable.

The code As implemented within the Melati framework, which wraps the HttpRequest and HttpResponse objects in a Melati object:

private String proxy(Melati melati, ServletTemplateContext context) {
  if (melati.getSession().getAttribute("generatedByMelatiClass") == null)
    throw new AnticipatedException("Only available from within an Admin generated page");
  String method = melati.getRequest().getMethod();
  String url =  melati.getRequest().getQueryString();
  HttpServletResponse response = melati.getResponse();
  HttpMethod httpMethod = null;
  try {

    HttpClient client = new HttpClient();
    if (method.equals("GET"))
      httpMethod = new GetMethod(url);
    else if (method.equals("POST"))
      httpMethod = new PostMethod(url);
    else if (method.equals("PUT"))
      httpMethod = new PutMethod(url);
    else if (method.equals("HEAD"))
      httpMethod = new HeadMethod(url);
    else
      throw new RuntimeException("Unexpected method '" + method + "'");
    try {
      httpMethod.setFollowRedirects(true);
      client.executeMethod(httpMethod);
      for (Header h : httpMethod.getResponseHeaders()) {
        response.setHeader(h.getName(), h.getValue());
      }
      response.setStatus(httpMethod.getStatusCode());
      response.setHeader("Cache-Control", "no-cache");
      byte[] outputBytes = httpMethod.getResponseBody();
      if (outputBytes != null) {
        response.setBufferSize(outputBytes.length);
        response.getWriter().write(new String(outputBytes));
        response.getWriter().flush();
      }
    } catch (Exception e) {
      throw new MelatiIOException(e);
    }
  } finally {
    httpMethod.releaseConnection();
  }
  return null;
}

Software Development with Certainty

My most recent project was to turn an impressive prototype developed by one person into a library, and applications based upon it, that can be simultaneously developed and used by a team.

The development process gains a new dimension once you have published your first version or have your first user. The task is made more tricky due to the absence of the original author.
In other words: situation normal, don't start from here.

The Erewhon applications, Gaboto library and its main dependencies ng4j and Jena are all changing rapidly. New functionality is being added and code and dependencies are being refactored and changed. The challenge is to enable this change without breaking installed systems or at least not breaking them unknowingly. This is ensured by establishing a contract between the code and the design by the use of tests. The tests guarantee that the system actually does do what it claims. Or, more properly, the tests are exactly what the system claims to do.