Legally Scientific? (Errors at ESR)

Even New Zealand was soon caught up in the enthusiasm for forensic DNA testing, although it was initially ill equipped to carry it out. Nonetheless the Criminal Investigations (Blood Samples) Act was quickly passed in 1995 "in order that a DNA databank can be created for the investigation of criminal offences", according to Environmental Science & Research Ltd (ESR), the main proponent and beneficiary of the legislation.

As has been the case in many other jurisdictions, enthusiasm for the new technology exceeded expertise and ESR's Mt Albert laboratories soon found themselves at the centre of two high profile cases which serve as textbook examples of what can go wrong in DNA testing laboratories, even those which are fully accredited and meet internationally accepted quality control guidelines.

In 1996, Peter Robert Howse was arrested for the rape of a teenager on the basis of strong evidence which included identification by a witness. Samples taken from Howse and the victim were sent to ESR labs for SLP (RFLP) testing. ESR reported no match, apparently eliminating Howse as a suspect, and he was released.

In 1999 Howse was arrested and tested again in connection with at least three further rapes. In the meantime ESR had switched to STR (PCR) testing. The results strongly implicated Howse in the 1996 rape as well as those being investigated.

The inquiry ordered by the NZ Department of Justice found that ESR had failed to gain a result with the 1996 tests, but laboratory policy had led to 'no result' being reported as 'no match'.

ESR conceded that 'with hindsight' it might have been better if failure to get a result was reported as such, but explained that they adopted a 'conservative' approach to reporting in order to give the accused benefit of any doubt.

One caught up in the testing was a mild mannered Christchurch man who had been assaulted outside his local, the Hagley Arms Hotel, on 23 April 1998. Police had taken blood in order to eliminate his DNA from samples taken from the scene of the assault and sent it to ESR for testing. The result of this test came to be dubbed 'Profile N' by the inquiries which were to follow.

Operation 'Pad' had collected evidence from the February 21 murder of Bill Fleet in his home. There were witnesses to the event and a man had been arrested and charged. Blood stains on a roller door at the murder scene were sampled and sent to ESR for analysis.

Operation 'Rex' concerned an 18 January murder on gang premises at Porirua. Some weeks later Mr Kuka Tiai was arrested and charged. He immediately confessed, saying he had acted alone. Samples were taken from the crime scene and the deceased and sent to ESR.

Samples from under the fingernails of the deceased and from spots on a wardrobe at the scene produced a profile which failed to match either Mr Tiai or his victim. Suspecting an accomplice, police requested that ESR conduct a search of its database for matching profiles.

ESR found two. One was from a rollerdoor sample taken in 'Pad' and the other was Profile N, which ESR claimed was likely to occur in only one New Zealander in 930 million.

The Christchurch assault victim was subjected to 'extensive police inquiries' for more than three months and his financial records were seized. This established what he had claimed all along, that he had not left Christchurch around the time of the murders. In fact he hardly left Christchurch at all.

In the Sharman inquiry which followed, ESR suggested that the assault victim may have a brother who had been at the crime scenes and that this would have greatly increased the chances of gaining a matching profile. When it was pointed out that he had no brothers ESR countered with the possibility of an unknown half-brother. After all none of us can really know what our parents have been up to. Sharman seemed to accept that this was a possibility but the later Eichelbaum-Scott inquiry was less charitable.

Sir Thomas Eichelbaum and Professor John Scott, assisted by an impressive array of legal, forensic and scientific experts, conducted a thorough investigation into the collection, transport, storage, testing and analysis of the samples in question, reporting in November 1999.

Although they never discovered exactly how the mistakes had happened they did determine that the 'Rex' and 'Pad' samples had been accidentally contaminated with DNA from Profile N at an early stage of processing at ESR's Mt Albert laboratories. Extracts from them sent to other laboratories for testing also returned Profile N. Numerous recommendations for improving oversight, record keeping and even laboratory ventilation were made.

During the inquiry ESR was asked to check the results of all testing done in July 1998 for any further instances of Profile N. Although they initially reported that none were found, later retests showed a partial profile consistent with N in another sample from 'Pad'. They also revealed 25 other 'unexplained' profiles in 14 of the 36 samples tested that month.

Justice Action asked Wayne Chisnall, ESR's 'General Manager Forensic', who assured us

In your question #8 you infer that ESR has had failures with DNA. This is not correct:

-The Howse case was all about the success of DNA. Old SLP technology did not conclusively pick up DNA from the accused but the new STR multilocus system did. New technological advances are inevitably going to be more sensitive than their predecessors.

-The Wellington murders you refer to were not DNA failures. The problem was picked up and investigated. Five separate investigations failed to find scientific proof for the cause of the anomalies. No direct evidence of either accidental or deliberate contamination has come to light. Existing facilities and protocols were shown to meet independent quality audit requirements. The Ministerial Inquiry recognising the points above concluded that on the balance of probabilities the apparently anomalous results could be ascribed to accidental contamination of DNA extracts within the ESR laboratory. They made a number of recommendations which ESR has either implemented or is planning to implement. This sits comfortably with our continuous improvement management focus. [Personal communication, 28 April 2000]

ESR senior scientist John Buckleton was more candid. In response to questions about statistical models used by ESR he replied in part

It is intrinsically impossible to estimate error in the model. We use the subpopulation model which is a summary of a very complex human evolutionary process. However we cannot empirically test our estimates with any real belief in the testing process. The closest we can get is our Tippett testing. This however, per se, is a very big question and cannot be answered simply.

ESR reports the probability of a match by chance (in fact we report the LR).

The "match by error" is a seperate source of error and is not part of the population genetic argument. The argument that it may be the larger of the two gains credibility as the "match by chance" gets smaller.

ESR accepts that both avenues are legitimate sources of cross examination.

We are at this time unable to quote an estimate of the potential for "match by error" for any given case. [Personal communication, 27 April 2000]

Perhaps, but given that the inquiry exposed four matches by error in less than two months of ESR testing, it will take several centuries of 'continuous improvement management focus' to reduce the overall error rate of the Mt Albert labs to anything like 1 in 930 million.