Expert Witnesses – Who Needs Them?

 

During the week the News at Six on BBC starts at 6pm and finishes at 6.30pm.  Half an hour every night regardless of what stories are to be told, there will be news to report and it will last 30 minutes.   In much the same way the format of trials such as the present one is laid out in advance.  There will be one expert witness allowed from each side who will investigate the matter put before them, write a report and be cross examined on the contents of that report in the witness stand.  In the current trial each expert has spent the best part of a week each in court defending their assumptions.

Personally speaking from reading the transcripts and witness statements I think we have learned next to nothing from their appearances.  I do however think that Dr Worden’s statistical approach needed to be delivered in order for that approach to be ruled out in its entirety as totally inappropriate for the purposes of determining what transpired to cause the claimants in this case to have such discrepancies in their branch accounts.   Statistics are of no use when examining the cause and effect of what must be abnormal occurrences in a system that everybody agrees works as it should most of the time.

I am sure most will be aware that a normal distribution curve of events is bell shaped and that mean, median and standard deviation can be calculated from that graph.  However the norm will always be skewed when unlikely events take place towards the ends of the bell shaped graph.   Probability plays no part in determining whether or not a particular branch will suffer from communication problems or not and how frequently these will occur.  Nor will it play a part in determining whether or not an intermittent hardware failure will occur.  The reason being is that while you may use historical analysis to determine the number of events that are likely to occur it would be impossible to use the same analysis to quantify the extremes of the amounts involved.  You may arrive at an average but, as we have already seen, the financial amounts involved differ hugely from pennies to millions.

Dr Worden has produced a computer based model of his assertions and challenged the court to use this and adapt the input to what the court considers reasonable in order to provide an outcome based on those factors.  Perhaps they might, perhaps not but there is another way that can be considered.

Monte Carlo simulation allows a computer to generate thousands if not millions of different results by re-iterating the formula using a randomly selected set of inputs limited by their range.   The results are then analysed using standard statistical techniques and again a bell graph of results can be charted.  In financial risk this, or a similar method, is used to determine the likelihood of an event happening.  However risk analysts are not overly concerned about the most likely event they are more interested in any incidents outside the norm that have skewed the shape of the results graph one way or another.

If for instance there is a 10,000 to one chance of an extreme event occurring and you use 10,000 iterations in your Monte Carlo simulation then the likelihood is that that one extreme event will be used in the risk calculation.  Risk analysts will then look at the graph of results, and from my personal experience of this very example, discount the 10,000 to one chance from their interpretation.   Use a million iterations and then that same event will likely occur 100 times and the analysts will leave the effect of that event occurring in their risk estimate.   There is one huge problem though with all of this risk analysis using statistics and that is the determination of the probability factor if you have not enough historical information to determine it.   The 10,000 to one chance in my example was from my experience working in credit risk and it was the ‘probability’ that a AAA rated corporation would fail.  At that time in the 90s from my recollection there were only 7 AAA rated corporations in the world and none of them had ever failed.  So 10,000 was just a guess.  Move forward a few years and hundreds if not thousands of AAA rated entities has been created by the banks to hold debt and as we know the collapse of many of these AAA debt bins led to the financial crisis.  I don’t know what the current probability is now of these failing but it will certainly be less than 10,000 to one.

Going back to Horizon, there is no probability table that can be used.  Up until now there has never been an investigation in to how often the system fails the user and there is one absolutely certain statistic that has been ignored and unless things change within the Post Office will never be found out and that is the number of times that a subpostmaster has had to make good losses in their branches over the years and the amounts that they have lost.   Dr Worden attempted a probability table on this very statistic and it was laughable I am afraid because he has no historical data to support it – no one has.

I am afraid Dr Worden’s efforts were in vain, yet as I said it is good that now exposed to the court they can be discounted as it leaves the uncertainty factor, the balance of probability that what the claimants alleged happen did happen, intact.

The Lowest Common Denominator?

Having now read the ‘expert’s’ reports into matters relating to Horizon errors as well as the preceding testimony and evidence in this and the previous trials, it appears to me that there is a key ingredient missing from the mix of information required to assess not just the outcome of this litigation but also the future of the network and how to fix it as it is clearly broken.

All subpostmasters are not born equal.  They differ in all the ways nature intended but the most important difference between them that has not yet been incorporated into any of the arguments put forward in this trial is the great disparity in levels of intelligence.   Subpostmasters are, like politicians, not required to have any educational qualifications nor do they sit any form of aptitude test in order to be approved to run their branch.

That said there are some mightily impressive figures out there who are clearly highly intelligent and who may have chosen to become a subpostmaster purely for lifestyle reasons rather than a career in high finance or academia.   The very clear and outstanding example of this is Alan Bates, the leader of the claimants, who perhaps could have followed a different career path into litigation and one in which I am sure he would have been most successful – not to say he is not about to be so in any event.

The expert witnesses have agreed upon a list of countermeasures that would and should play a part in preventing Horizon errors creating permanent discrepancies in a branch’s accounts.  One of them is referred to as UEC User Error Correction where the reliance is on the subpostmaster to detect the error whether human or computer and correct it by themselves.  For instance in the Dalmellington error it is accepted that 80 or so branches detected the error and entered a separate transaction to correct the discrepancy that occurred as a result of a computer error (the fact that the transaction that they entered was illegal and a criminal act that other subpostmasters have been imprisoned for is another story).

Seema Misra was imprisoned partly on the assessment of the judge in her trial based on the evidence put before the court that she would have spotted any computer error that may have caused a discrepancy in her accounts.   That is simply not the case and whether a subpostmaster can detect an error in such a sophisticated computer system must certainly be a factor of the intelligence level and capability of the individual concerned.

So that leads me to this true story in which I played a part and which certainly helps form my opinion on the stories I read and hear about subpostmasters unable to run their branch properly and who as a result suffer huge losses.  It is not their fault – it is the fault of Post Office Ltd for allowing them anywhere near a Post Office branch.

Five years or so ago I assisted a potential applicant with his business plan submission to become a subpostmaster in a nearby town.  This gentleman seemed reasonably astute and ran a small convenience store close to my post office.  He was successful in his application and he and his brother expanded his empire by buying and running the new convenience store and PO in a nearby town.

Some three months after he had opened and had been a subpostmaster for that time, I was behind the counter in my post office when one of his assistants came in a with a letter to post, wanting it signed for.   I took the envelope and read the address on the front, it stated (not exact address of course):

ABC Company Ltd

82 High St

That was it.  No town and no postcode.

I explained to the assistant that I could not process the letter as it had not sufficient information in the address.  The assistant took the letter and went back to his boss who appeared quickly hilding the letter and asking me what on earth was wrong with it.  I am usually quite cool and collected in explaining matters like these to customers but the conversation became somewhat heated as he insisted that this was all that was on the letter as a reply address.  I believe that the recipient was a solicitor and my erstwhile friend was insistent that a solicitor would know how to address a letter correctly!

I asked to see the letter which he went away to get and sure enough the letter looked something like this:

Dear Sir

kajshkad k;jsdfk kkjsdf kjh sdkjjhsdfk kkjsdhfkjkkksdf kkkksdfjh

kajshkad k;jsdfk kkjsdf kjh sdkjjhsdfk kkjsdhfkjkkksdf kkkksdfjh

kajshkad k;jsdfk kkjsdf kjh sdkjjhsdfk kkjsdhfkjkkksdf kkkksdfjh

kajshkad k;jsdfk kkjsdf kjh sdkjjhsdfk kkjsdhfkjkkksdf kkkksdfjh

kajshkad k;jsdfk kkjsdf kjh sdkjjhsdfk kkjsdhfkjkkksdf kkkksdfjh

kajshkad k;jsdfk kkjsdf kjh sdkjjhsdfk kkjsdhfkjkkksdf kkkksdfjh

kajshkad k;jsdfk kkjsdf kjh sdkjjhsdfk kkjsdhfkjkkksdf kkkksdfjh

Please reply to :

ABC Company Ltd

82 High St

 

 

He pointed to the last lines and said “There you are.  See what I mean?”

At that point I turned the page of the letter over and the text of it continued

 

Eyemouth

Berwickshire

TD14 3GH

 

 

This is NOT a simple mistake.  This is .. well I don’t quite know how to describe it .. certainly NOT the actions of a subpostmaster capable of detecting any sort of error in his branch accounts whether computer or otherwise.   This person should not even be allowed to accept mail from a customer.  Perhaps I wouldn’t even blame the trainer for allowing this person to continue in his role because there are no tests before or after to determine what an individual has taken on board and therefore their level of competence.   I do blame POL though – entirely – for letting this happen.

It is vitally important that this factor – the level of competence of individual subpostmasters – be taken into account when attempting to assess the issues put before the court.   Dr Worden’s attempt at categorising actions subpostmasters would take on detection of a loss in their branch by defining the probability of each action on the amount of the loss is totally absurd and extremely annoying particularly when he confesses in yesterday’s transcript that he had made no contact (perhaps he was not permitted by POL to do so) with any subpostmaster.

My example above is just one of many.  Perhaps the worst, perhaps not.  Any subpostmaster who is or has been a member of various social media forums will know and understand that this rather low level of ability is common within the network although I have no idea of how prevalent it is.

It is of POL’s own doing that this has been allowed to happen – I would suggest the situation has developed more so since the inception of NT and the rush to get offices converted which is now being replaced by the necessity to find enough idiots to take on a post office and maintain the network size at its legislated level for the convenience of the Goovernment.

To fix it POL will have to start with the Lowest Common Denominator and work upwards although they might want to examine the ability and intelligence levels of their own staff first who got them in this mess.