Bob Breeze | HKA
It is not uncommon in large construction projects for disputes to arise which involve several thousand alleged variations (or change orders) and/or alleged defects which need to be considered by the arbitral Tribunal or Court in the formal dispute resolution process chosen by the parties.
In addition to the often-extensive factual evidence, this usually gives rise to the need for expert evidence to be adduced dealing with the cause-and-effect liability issues as well as possible delay and quantum claims which arise as a result.
Parties and their appointed experts in many jurisdictions are now required to adopt a proportionate approach whereby they do not “use a sledgehammer to crack a nut” but this can mean different things in different situations. The concept of proportionality is not formulaic in nature.
Being proportionate must often be set against allegations of abuse of process in that not every claim may be considered on its own merits if shortcuts are taken. Despite this, the English Court at least has recently upheld[1] the established position that in principle, a claimant can pursue a claim which relies on sampling for establishing liability or causation of damage, but great care needs to be taken as to how this is executed. Also, whether the sampled answers can be extrapolated to the unsampled population needs careful attention too. This is in line with the Civil Procedural Rules rule 32.1 which confers powers on the Court to control evidence.
The Technology and Construction Court (TCC) Guide[2] in England requires , before the first case management conference, the parties to give careful thought to expert issues including any “…appropriate or necessary …sampling”. This will often require a discussion between the experts as to an appropriate sampling protocol to be adopted in the proceedings. In these joint expert discussions it is worth bearing in mind what are known as the “Whitford Guidelines”[3]. For a survey to have evidential value it must be shown that (in that case) the interviewees were selected from a cross section of the population, the survey was of a sufficient size to produce relevant results on a statistical basis, and it was conducted fairly.
Statistical sampling is a method whereby it involves the selection of a subset of items from a larger group (or population) of data claimed and uses the results of this sample to estimate the characteristics of the remainder of the population (i.e., the unsampled data claimed).
The use of sampling and extrapolation may make the claim more difficult to establish at trial but this must be balanced against the cost savings on offer by not exhaustively examining every individual claim on its own merits. However, the claimant must still give the defending party an opportunity to properly understand the claim made against it and allow it to defend itself against both the sampled and extrapolated claim.
So, the obvious question which arises is how does one approach a large and technically complex dispute involving thousands of issues on a proportionate basis by using sampling and extrapolation as a preferred method? I have recently been involved with two such cases and make the following comments on the basis that I am not a statistician, but I have used statistics as a tool in the same way that I use Microsoft Excel as a tool, but I do not know how the software works.
The first challenge any expert may face when employed to try to demonstrate cause and effect, is to their credibility, expertise and whether they are suitably qualified to offer an opinion on the matters under consideration. As well as the relevant rules of evidence to be applied, this is often referred to as a “Daubert” challenge following a case in the USA[4] which provided that an expert must be qualified based on “knowledge, skill, experience, training and education” and questioned whether such evidence was admissible in that case. While this case related to scientific expert evidence, it now seems it can equally be applied to other expert evidence (at least in the USA and Canada) such as technical or quantum experts using for instance, sampling and extrapolation as an approach which may be described as a scientific approach.
A Daubert challenge consists of two elements. The first is a question of reliability and the second is one of relevance. In the first such instance, reliability means: did the expert follow a sound method or any acceptable protocols? There are usually at least five tests which are applied to answer this question. They include:
- Whether the theory or technique presented as expert testimony and evidence can be tested;
- Whether the theory or technique has been subjected to peer review and publication;
- The known or potential rate of error;
- The existence and maintenance of standards controlling the technique’s operation; and
- Is there any general acceptance of the approach by industry peers?
The second question is relevance: do the opinions fit the facts of the case? This is usually a matter for the Court or Tribunal to determine.
As for the five tests for reliability, the first one as to whether the theory or technique can be tested is perhaps less applicable to many construction disputes and is more applicable to scientific cases such as medical malpractice or pharmaceutical litigation. In construction cases it is often a question of whether there are other indicators that the expert’s analysis and evidence are reliable.
Likewise, in construction cases the lack of a peer review is not often a successful Daubert challenge due to the less widespread publications for quantum and delay experts than for, say, medical experts. So, for example a delay expert using a “Windows” method which is widely accepted in the industry nowadays would not need to be subject to a peer review.
So the remaining three challenges listed above should be considered by an expert when selecting a methodology. The expert’s methodology should also consider a reasonable amount of other potential causes of loss or damage etc. if they are likely to exist. These other potential causes should be identified in an expert report and an explanation provided as to why they were ruled out.
Assuming these challenges can be satisfied, the next step is to decide how best to go about a sampling and extrapolation exercise. These two do not automatically go together. There are two predominant types of sampling, they are random sampling and judgmental sampling. Random sampling is a widely used statistical method with various techniques of how it should be done, and the results can be extrapolated if done properly.
Judgmental sampling on the other hand, is a non-statistical method used to obtain a broad coverage of the population in a sample. For the results to be extrapolated, it is essential to show the sample was representative and was free of any bias.
As referred to above, the cases where this has been used successfully are often where both parties and/or their experts working together, are involved in the sampling process. By agreeing a sampling protocol, the parties can avoid the viability of the sampling process itself being an issue at trial.
When large complex disputes are sought to be resolved by sampling, a useful guide as to how best to approach the sampling exercise can be found in a document published by the U.S. Department of Health and Human Services Office of Inspector General titled “Statistical Sampling: A Toolkit for MFCU’s”[5] but this is by no means legal guidance. It does, however, provide a useful response to some of the five tests listed above.
The Toolkit provides a step-by-step guide of how to select a statistical sample and calculate a valid statistical estimate. It sets out thirteen steps which can be followed and would be capable of being replicated by an opposing party. It is an important aspect, and a potential challenge to sampling, if it is not possible to replicate what was done in the sampling exercise.
Importantly, the Toolkit provides a means of generating a random sample which is essential if it is to be extrapolated to the remainder of the population (claim data). A useful suggestion which is made is that if a party is planning to seek assistance from a statistician, this should be done at an early stage in the sampling process.
However, while it may well be advisable to employ an expert statistician at an early stage in this process, it is by no means essential provided great care is taken along the way, the right steps are taken, with the aim to achieve up to a 95% confidence level in the results generated.
Done properly, sampling and extrapolation can offer a proportionate approach to case management and will have a realistic chance of success in proving liability or causation in large complex cases. It must not, however, be seen as a shortcut to success and may well involve significant cost and time to achieve the desired results. The claimant still shoulders the burden of proof, and this must never be overlooked.
When one of your cases is in need of a construction expert, estimates, insurance appraisal or umpire services in defect or insurance disputes – please call Advise & Consult, Inc. at 888.684.8305, or email experts@adviseandconsult.net.