Expert Witnesses – Who Needs Them?

 

During the week the News at Six on BBC starts at 6pm and finishes at 6.30pm.  Half an hour every night regardless of what stories are to be told, there will be news to report and it will last 30 minutes.   In much the same way the format of trials such as the present one is laid out in advance.  There will be one expert witness allowed from each side who will investigate the matter put before them, write a report and be cross examined on the contents of that report in the witness stand.  In the current trial each expert has spent the best part of a week each in court defending their assumptions.

Personally speaking from reading the transcripts and witness statements I think we have learned next to nothing from their appearances.  I do however think that Dr Worden’s statistical approach needed to be delivered in order for that approach to be ruled out in its entirety as totally inappropriate for the purposes of determining what transpired to cause the claimants in this case to have such discrepancies in their branch accounts.   Statistics are of no use when examining the cause and effect of what must be abnormal occurrences in a system that everybody agrees works as it should most of the time.

I am sure most will be aware that a normal distribution curve of events is bell shaped and that mean, median and standard deviation can be calculated from that graph.  However the norm will always be skewed when unlikely events take place towards the ends of the bell shaped graph.   Probability plays no part in determining whether or not a particular branch will suffer from communication problems or not and how frequently these will occur.  Nor will it play a part in determining whether or not an intermittent hardware failure will occur.  The reason being is that while you may use historical analysis to determine the number of events that are likely to occur it would be impossible to use the same analysis to quantify the extremes of the amounts involved.  You may arrive at an average but, as we have already seen, the financial amounts involved differ hugely from pennies to millions.

Dr Worden has produced a computer based model of his assertions and challenged the court to use this and adapt the input to what the court considers reasonable in order to provide an outcome based on those factors.  Perhaps they might, perhaps not but there is another way that can be considered.

Monte Carlo simulation allows a computer to generate thousands if not millions of different results by re-iterating the formula using a randomly selected set of inputs limited by their range.   The results are then analysed using standard statistical techniques and again a bell graph of results can be charted.  In financial risk this, or a similar method, is used to determine the likelihood of an event happening.  However risk analysts are not overly concerned about the most likely event they are more interested in any incidents outside the norm that have skewed the shape of the results graph one way or another.

If for instance there is a 10,000 to one chance of an extreme event occurring and you use 10,000 iterations in your Monte Carlo simulation then the likelihood is that that one extreme event will be used in the risk calculation.  Risk analysts will then look at the graph of results, and from my personal experience of this very example, discount the 10,000 to one chance from their interpretation.   Use a million iterations and then that same event will likely occur 100 times and the analysts will leave the effect of that event occurring in their risk estimate.   There is one huge problem though with all of this risk analysis using statistics and that is the determination of the probability factor if you have not enough historical information to determine it.   The 10,000 to one chance in my example was from my experience working in credit risk and it was the ‘probability’ that a AAA rated corporation would fail.  At that time in the 90s from my recollection there were only 7 AAA rated corporations in the world and none of them had ever failed.  So 10,000 was just a guess.  Move forward a few years and hundreds if not thousands of AAA rated entities has been created by the banks to hold debt and as we know the collapse of many of these AAA debt bins led to the financial crisis.  I don’t know what the current probability is now of these failing but it will certainly be less than 10,000 to one.

Going back to Horizon, there is no probability table that can be used.  Up until now there has never been an investigation in to how often the system fails the user and there is one absolutely certain statistic that has been ignored and unless things change within the Post Office will never be found out and that is the number of times that a subpostmaster has had to make good losses in their branches over the years and the amounts that they have lost.   Dr Worden attempted a probability table on this very statistic and it was laughable I am afraid because he has no historical data to support it – no one has.

I am afraid Dr Worden’s efforts were in vain, yet as I said it is good that now exposed to the court they can be discounted as it leaves the uncertainty factor, the balance of probability that what the claimants alleged happen did happen, intact.