Installation of OpenClinica on Ubuntu 12.10

Installing a new version of Open Clinica on an Ubuntu 12.10 desktop installation.

The hope is to then install this on a SuSE machine which already has a number of Open Clinica instances installed.

This involves creating a .war file with an instance specific name, we will use pets.war.

Register at openclinica.com. Download .zip

Looks like the installation notes are based on something other than the one true distribution; so taking some notes.

You cannot install the default Postgresql. It won't work.

Ensure you have completely removed all traces of Postgresql 9! The following should show no results:

dpkg -l  |grep postgres

Follow the installation of postgresql instructions at OpenClinica.

sudo chown -R postgres:postgres  /opt/PostgreSQL/8.4/

Unpack and copy to expected location:

sudo mv OpenClinica-3.1.3 /usr/local/
sudo ln -s OpenClinica-3.1.3 oc
cd /usr/local/oc

Create database user and database in Postgres:

sudo su postgres
/opt/PostgreSQL/8.4/bin/psql -U postgres -c "CREATE ROLE clinica LOGIN ENCRYPTED PASSWORD 'clinica' SUPERUSER NOINHERIT NOCREATEDB NOCREATEROLE"
/opt/PostgreSQL/8.4/bin/psql -U postgres -c "CREATE DATABASE pets WITH ENCODING='UTF8' OWNER=clinica"

Generate password with Password gorilla, but do not include punctuation.

Change password:

sudo su postgres
/opt/PostgreSQL/8.4/bin/psql -U postgres 
  ALTER USER clinica WITH PASSWORD 'XXXX';

Undocumented
Create a data directory with the webapp name.

sudo mkdir  /usr/share/tomcat7/pets.data
sudo chown tomcat7:tomcat7  /usr/share/tomcat7/pets.data

The way it is recommended you deploy is pretty awful, to paraphrase:

Unzip the .war copy to tomcat webapps directory, edit in place.

This may work on a Wednesday, but is not the right way®.

So what we will do is unpack the .war, stash in git. then build a new .war.

Ideally this would be by using the Maven War Overlay Plugin, but we might get it to work by creating the war manually.

cd /usr/local/oc
git init
emacs distribution/OpenClinica/WEB-INF/classes/datainfo.properties 

You must modify the following:

dbPass=XXXX
adminEmail=admin@example.com

but you probably also need to setup mail sending.

When you have edited these properties, create a .war file, which we will call pets.war

cd /usr/local/oc/distribution/OpenClinica/
jar -cvf ../pets.war *

Deploy:

sudo /etc/init.d/tomcat7 stop
sudo cp pets.war /var/lib/tomcat7/webapps/
sudo /etc/init.d/tomcat7 start

At this point you will need to allow yourself to manage Tomcat7, as it comes secured. Add the following to /etc/tomcat7/tomcat-users.xml.

<role rolename="manager-gui" />
<role rolename="manager-status" />
<role rolename="manager-script" />
<role rolename="manager-jmx" />

<role rolename="admin-gui" />
<role rolename="admin-script" />

<user username="admin" password="admin" roles="manager-gui, manager-status, admin-gui"/>

Unfortunately there is a problem (from localhost.date.log):

Feb 01, 2013 4:16:04 PM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class org.akaza.openclinica.dao.core.OCContextLoaderListener
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/akaza/openclinica/applicationContext-core-db.xml]: Invocation of init method failed; nested exception is liquibase.exception.MigrationFailedException: Migration failed for change set migration/2.5/changeLogCreateTables.xml::1235684743487-0::pgawade (generated):
     Reason: liquibase.exception.JDBCException: Error executing SQL DROP LANGUAGE IF EXISTS plpgsql;
   CREATE PROCEDURAL LANGUAGE plpgsql;
   SET check_function_bodies = false;
   SET client_min_messages = error;
   SET default_tablespace = '';
   SET default_with_oids = true;
   SET search_path = public, pg_catalog;
   COMMENT ON SCHEMA public IS 'Standard public schema';:
          Caused By: Error executing SQL DROP LANGUAGE IF EXISTS plpgsql;
   CREATE PROCEDURAL LANGUAGE plpgsql;
   SET check_function_bodies = false;
   SET client_min_messages = error;
   SET default_tablespace = '';
   SET default_with_oids = true;
   SET search_path = public, pg_catalog;
   COMMENT ON SCHEMA public IS 'Standard public schema';:
          Caused By: ERROR: cannot drop language plpgsql because extension plpgsql requires it
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1420)
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
 at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:291)
 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:288)
 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:190)
 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:281)
 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:190)
 at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:563)
 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
 at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:276)
 at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:197)
 at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:47)
 at org.akaza.openclinica.dao.core.OCContextLoaderListener.contextInitialized(OCContextLoaderListener.java:38)
 at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4791)
 at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5285)
 at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
 at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
 at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
 at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:618)
 at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:963)
 at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1600)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
 at java.util.concurrent.FutureTask.run(FutureTask.java:166)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
 at java.lang.Thread.run(Thread.java:722)

Apparently this can be fixed:

sudo /etc/init.d/tomcat7 stop
sudo su postgres
psql -U postgres
  DROP EXTENSION plpgsq;
  \q
exit
sudo /etc/init.d/tomcat7 start

At this point I discover that OpenClinica does not work with Postgresql 9.1!! Blogs are difficult to represent a process. I have updated the above so you won't encounter this error.

You may well see the following, which is a success message !

SEVERE: Servlet /pets threw load() exception
javax.servlet.ServletException: missing jspFile
 at org.apache.jasper.servlet.JspServlet.init(JspServlet.java:123)
 at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)
 at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)
 at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)
 at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5027)
 at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5314)
 at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
 at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
 at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
 at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:618)
 at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:963)
 at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1600)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
 at java.util.concurrent.FutureTask.run(FutureTask.java:166)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
 at java.lang.Thread.run(Thread.java:722)

http://localhost:8080/pets/ takes me to a login screen.

You can now login with a username of root and a one time password of 12345678.

Ubuntu setup

After a short toy with the idea that you can develop web service software for Ubunbtu on Windows7, using cygwin, VirtualBox, PuTTY etc I stopped being silly and installed Ubuntu. I had intended to dual boot, but it looks as though Ubuntu was not quite as gentle as it might have been, and Windows got in a tizzy, trying to recover disks, so I blew it away. The joy. It is worth revisiting windows occasionally to re-motivate yourself and renew your passion for opensource.

Setup general packages:

sudo apt-get remove unity-lens-shopping
# Also switch off amazon results on settings >> Privacy


sudo apt-get install openssh-server

#actually I copied these from somewhere else
ssh-keygen

sudo apt-get install git
git config --global user.name "Tim Pizey" 
git config --global user.email Tim.Pizey@psy.ox.ac.uk


sudo apt-get install openjdk-7-jdk

sudo apt-get install eclipse

sudo apt-get install emacs23-nox
sudo apt-get install cvs

sudo apt-get install jedit

sudo apt-get install curl

sudo apt-get install texlive-xetex
sudo apt-get install texlive-latex-extra

#scrollbars in eclipse
sudo su -c 'echo export LIBOVERLAY_SCROLLBAR=0 > /etc/X11/Xsession.d/80overlayscrollbars'

Setup Chrome

wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
sudo sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
sudo apt-get update

sudo apt-get install google-chrome-stable

Setup Haskell:

sudo apt-get install ghc
sudo apt-get install libghc6-zlib-dev
sudo apt-get install cabal-install
cabal update
cabal install cabal-install
sudo apt-get install texlive
cabal install lhs2tex

Jenkins Setup

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

Setup BestMin packages:

sudo apt-get install pgadmin3


sudo su postgres

createuser --superuser --pwprompt dev_user

createdb dolab

exit

sudo apt-get install python

# superstition, maybe
sudo apt-get install python-dev

sudo apt-get install python-pip

sudo apt-get install python-postgresl


sudo apt-get install python-django


sudo apt-get install python-psycopg2


sudo apt-get install gnupg
sudo pip install python-gnupg
sudo apt-get install python-crypto

sudo apt-get install python-yaml


mkdir git
cd git
git clone gitosis@truecolours.nhs.uk:dolab.git

cd dolab/

sudo ln -s /home/timp/git/dolab /usr/local/dolab

./manage.py syncdb

./manage.py runserver 0:8000

Virtualbox networking

Set up an Ubuntu server. call it psy. You should be able to ssh out from the box to the outside world. The problem is getting in.

VBoxManage modifyvm "psy" --natpf1 "guestssh,tcp,,22,,2222"
VBoxManage modifyvm "psy" --natpf1 "guesthttp,tcp,,8000,,8000"
VBoxManage modifyvm "psy" --natpf1 "guestjenkins,tcp,,80,,8080"

[],tcp|udp,[<hostip>],<hostport>,[<guestip>], <guestport>

See also http://www.virtualbox.org/manual/ch06.html. It may be that this is more effective than http://tim-pizey.blogspot.co.uk/2011/08/alfresco-setup.html

A Security Policy Model for Clinical Information Systems

A Security Policy Model for Clinical Information Systems appeared at the 1996 IEEE Symposium on Security and Privacy. It presents the BMA policy model to the computer security community in a format comparable to policies such as Bell-LaPadula and Clark-Wilson. It had some influence on later US health privacy legislation (the Kennedy-Kassebaum Bill, now HIPAA).

A classic paper by Ross Anderson found here, more properly from Ross' own site.

These are just notes and pickings. Though all nine Principles are given.

The paper sets out a model that consciously parallels the Bell LaPadula model for military systems and the Clark - Wilson Model for the world of banking.

Extract from the Hippocratic oath

Whatsoever I shall see or hear in the course of my dealings with men, if it be what should not be published abroad, I will never divulge, holding such things to be holy secrets.

Though this is a different translation to that on Wikipedia.

Consent must be informed and voluntary.

The main new threat comes from abuse by insiders.

Unnoticed failures from data corruption are also new, non-digital information stores tend to fail completely.

The ill-considered aggregation of systems has grown apace, culminating in cloud service provision.

BMA recommends a callback authentication protocol to defend against private investigators impersonating other health professionals.

Principle 1

Each identifiable clinical record shall be marked with an acccess control list naming the people or groups of people who may read it and append data to it. The system shall prevent anyone not on the access control list from accessing the record in any way.

Immediate bad smell: The system shall prevent. This suggests a failure mode where access is allowed. The requirement needs to be framed in the opposite sense: The system shall enable anyone on the access control list from accessing the record. There will be no other mechanism of access.

Patients must be informed of a care team's access control policy when they first enrol, and have the opportunity to restrict access further if they wish.

Principle 2

A clinician may open a record with herself and the patient on the access control list. Where a patient has been referred, she may open a record with herself, the patient and the referring clinician(s) on the access control list.

Apart from the patient himself, only clinicians may have access to his records.

This seems incredibly strong and calls for encrypted data storage.

Principle 3

One of the clinicians on the access control list must be marked as being responsible. Only she may alter the access control list, and she may only add other health care professionals to it.

This may be one principle but these are three separate requirements:

1. Write access to the ACL is controlled by an entry in the ACL.
2. Only one member of an ACL may have write access
3. Only health care professionals may be added.

I can't see one of these standing up. One of the default roles will have to be super user, of course it will.

Where access has been granted to administrators, as in the USA, the result has been abuse.

Confused reference to UK safe havens and their role in difusing dispute.

German and Canadian examples of hospital billing data: aggregation is done prior to billing government.

All legal outputs, required by police, social workers etc, should be produced on paper.

Principle 4

The responsible clinician must notify the patient of the names on his record's access control list when it is opened, of all subsequent additions, and whenever responsibility is transferred. His consent must also be obtained, except in emergency or in the case of statutory exemptions.

Principle 5

No one shall have the ability to delete the clinical information until the appropriate time period has expired.

There is always a need to correct data. Current version, like Wikipedia, would need links to previous versions.

Patient consent is not immutable, but rather a continuing dialogue between the patient and the clinician.

Principle 6

All accesses to clinical records shall be marked on the record with the subject's name, as well as the data and time. An audit trail must also be kept of all deletions.

Remember here subject means data accessor, not patient.

Principle 7

Information derived from record A may be appended to record B if and only if B's access control list is contained in A's.

If there exist two records referring to a patient, with different ACLs, where one record is more restricted than the other, should the existence of the restricted record be shown to users of the less restricted record?

We expect that clinicians will decide in favour of discrete flags that indicate only the presence of hidden information.

Principle 8

There shall be effective measures to prevent the aggregation of personal health information. In particular, patients must receive special notification if any person whom it is proposed to add to their access control list already has access to personal health information on a large number of people.

In this policy model, the primary control is notification, and the secondary control is to keep a list somewhere of who has accessed what record outside their own team.

Principle 9

Computer systems that handle personal health information shall have a subsystem that enforces the above principles in an effective way. Its effectiveness shall be subject to evaluation by independent experts.

White hat job creation scheme?

Ideally a non-NHS accreditation and monitoring organisation. A Trusted Computer Base (TCB) must not itself leak personal data.

X.509 is probably not enough.

Information access audit must be effective, that is it must be quite likely that an intruder will be caught.

Our approach has been to provide two auditors, both of whom have an interest in detecting abuse and acting upon it.

Firstly the patient and secondly an audit office eg the GMC.

'show me the records of all females aged 35 with 2 daughters aged 13 and 15 both of whom suffer from eczema'

Suppose that I walk into a hospital and claim that my demons are bothering me. When asked my name I reply 'John Major'. May the psychiatrist get the prime ministers record and append a diagnosis of schizophrenia?

Can we trust the patient to self identify? How do we treat patients anonymously? What about people who claim identity change?

Re the unified patient record:

The onus is on proposers of such systems to provide a clear statement of the expected health benefits, and to analyse the threats, the cost of added countermeasures and the likely effects of the residual risk.

A Binary Chop Java method to discover memory limits

If you want to discover exactly how many Objects you can store in your JVM with its current memory configuration this little functional style method might help.

  public void testChop() { 
    // -Xmx=256m
    //assertEquals(33554433, between(1, Integer.MAX_VALUE));
    // -Xmx=1966m
      assertEquals(343535293, between(1, Integer.MAX_VALUE));
  }
 
  public int between(int from, int to) {
    if ((to - from) <= 1) 
      return (to);
    int i = from + ((to - from) /2);
    try { 
      Object[] a = new Object[i];
      a[1]="1";
      a=null;
      return (between(i, to));
    } catch (OutOfMemoryError e) {
      return (between(from, i));
    }      
  }

Another thing to note is the speed that this runs at: memory allocation can be very slow.

Obviously, in hindsight, this routine will always take 30 steps.

Implementing Java interfaces: when is it OK to add extra methods?

I have just found myself offended by a usage and I want to write out why.

public interface Stack {
  public int size();
  public boolean isEmpty();
  public Object top()  throws StackEmptyException;    
  public void push (Object element);
  public Object pop()  throws StackEmptyException;   
}

public class ArrayStack implements Stack {

  ......

  public Object atPosition(int i) throws StackOutOfScopeException {
    if (i > tos)
      throw new StackOutOfScopeException("Attempt to pass top of stack");
    else
      return S[i - 1];
  }
}

My claim is that the atPosition should either be in the interface or not in the class, or ArrayStack should be called AugmentedArrayStack or that an intermediate interface called AugmentedStack which extends Stack is needed.

My expectation is that if a class implements only one interface and that interface is named for a Noun and does not have a pertinent name (eg ...Mixin) then the class will only implement methods declared in that interface (and possibly override those inherited from Object).

Preparation for Software Testing Course

• The Impossibility of Complete Testing by Cem Kaner

  Bad software © 1997

  Article 2B October 30, 1998 of the UCC became UCITA which has only been adopted by two states (passed in 2000).

• Exhausting Your Test Options by Douglas Hoffman

  By exhaustive testing a bug was caught, which would not have been spotted by inspection of edge cases.

• The Patriot Missile Failure by Douglas N. Arnold

  Ironically, the fact that the bad time calculation had been improved in some parts of the code, but not all, contributed to the problem, since it meant that the inaccuracies did not cancel.

• A bug and a Crash by James Gleick

  Fortunately, he points out, really important software has a reliability of 99.9999999 percent. At least, until it doesn't.

• Other Papers from Testing Education

Case Study

Based upon The art of the elevator pitch describe your testing project.

Unifier

• Open Source utility aimed at developers and office users.
• A command line utility for the common problem of unifying spreadsheet sheets into one sheet.
• A pure java command line tool.
• A small utility with only 16 classes.
• Code coverage testing has already been completed through test driven development and retrofitting to 100&percent;, however there are is a need to extend testing for unexpected and illegal input.

Melati

• An Open Source ORM framework aimed at developers and office users.
• A DSL for writing database backed systems, with especial focus on web pages.
• A pure java stack on top of any JDBC capable database, typically used with a template engine.
• 4792 files, 180k lines of code, 8 developers over 12 years.
• Code coverage testing has already been completed through retrofitting. Still some benefits to be gained: caching, speed tests, technology changes.