|[Home] [Databases] [World Law] [Multidatabase Search] [Help] [Feedback]
England and Wales High Court (Chancery Division) Decisions
You are here: BAILII >> Databases >> England and Wales High Court (Chancery Division) Decisions >> Pyrrho Investments Ltd v MWB Property Ltd & Ors  EWHC 256 (Ch) (16 February 2016)
Cite as:  EWHC 256 (Ch)
[New search] [View without highlighting] [Printable PDF version] [Help]
Strand, London, WC2A 2LL
B e f o r e :
| (1) Pyrrho Investments Limited
(2) MWB Business Exchange Limited
|- and -
|(1) MWB Property Limited
(2) Rick Aspland-Robinson
(3) Keval Pankhania
(4) Richard Balfour-Lynn
(5) Jagtar Singh
Clive Freedman QC (instructed by Simons Muirhead & Burton) for the Second Defendant
Andrew George QC (instructed by Taylor Wessing LLP) for the Fourth Defendant
The other parties were represented by their solicitors, but did not address the court
Hearing date: 2 February 2016
Crown Copyright ©
Master Matthews :
- The parties shall by 4.30pm on Friday, 26 February 2016 give standard disclosure by simultaneous exchange of documents by lists and categories.
Any requests for inspection or copies of disclosed documents shall be made within 14 days of service of the lists and shall be responded to within 7 days of receipt of the request.
- The parties shall seek to agree, if possible, the scope of the reasonable search to be carried out by each party for electronic documents. Further:
i. the Defendants shall respond to RPC in relation to electronic disclosure (in particular RPC's letters dated 25 February 2015 and 6 March 2015) by no later than 4.30pm on Friday 13 November 2015;
ii. the Claimants shall by no later than 4.30pm on Friday 20 November 2015 provide the Defendants with a reply to the responses received pursuant to paragraph ll.i above.
- In the event that the parties have not reached agreement in relation to the scope of the reasonable search for electronic documents to be carried out by any party by 4.30pm on Friday 27 November 2015 any outstanding issues shall be considered and determined at the Disclosure CMC referred to below.
- The Parties shall within 7 days of the date of this order seek to fix a further CMC (the "Disclosure CMC") for the first mutually convenient date after 27 November 2015 with a time estimate of 2.5 hours.
- The purpose of the Disclosure CMC shall be to determine:
i. any issues remaining in relation to electronic disclosure, in accordance with paragraphs 11 and 12 above; and
ii. (as between the Claimants and the Third Defendant) the appropriate mechanism to be adopted in relation to documents currently held in quarantine by Stroz Friedberg which have been the subject of correspondence between the Third Defendant and the Second Claimant.
- Any issues referred to in paragraph 14.i shall be dealt with first in time at the Disclosure CMC and the First, Second, Fourth and Fifth Defendants need not attend the remaining part of the Disclosure CMC (which shall be given over to dealing with the issue referred to in paragraph 14.ii). In the event that there are no issues arising under paragraph 14.i for determination at the Disclosure
CMC the First, Second, Fourth and Fifth Defendants need not attend the Disclosure CMC."
(The Disclosure CMC contemplated by those directions was in part held on 2 February 2016 and the order referred to in paragraph 1 above was made. The hearing contemplated by paragraph 14.ii has yet to be held.)
"(1) When giving standard disclosure, a party is required to make a reasonable search for documents falling within rule 31.6(b) or (c).
(2) The factors relevant in deciding the reasonableness of a search include the following -(a) the number of documents involved;
(b) the nature and complexity of the proceedings;
(c) the ease and expense of retrieval of any particular document; and
(d) the significance of any document which is likely to be located during the search.
(3) Where a party has not searched for a category or class of document on the grounds that to do so would be unreasonable, he must state this in his disclosure statement and identify the category or class of document."
"20 The extent of the reasonable search required by rale 31.7 for the purposes of standard disclosure is affected by the existence of Electronic Documents. The extent of the search which must be made will depend on the circumstances of the case including, in particular, the factors referred to in rule 31.7(2). The parties should bear in mind that the overriding objective includes dealing with the case in ways which are proportionate.
21 The factors that may be relevant in deciding the reasonableness of a search for Electronic Documents include (but are not limited to) the following -(1) the number of documents involved;
(2) the nature and complexity of the proceedings;
(3) the ease and expense of retrieval of any particular document. This includes:(a) the accessibility of Electronic Documents including e-mail communications on computer systems, servers, back-up systems and other electronic devices or media that may contain such documents taking into account alterations or developments in hardware or software systems used by the disclosing party and/or available to enable access to such documents;
(b) the location of relevant Electronic Documents, data, computer systems, servers, back-up systems and other electronic devices or media that may contain such documents;
(c) the likelihood of locating relevant data;
(d) the cost of recovering any Electronic Documents;
(e) the cost of disclosing and providing inspection of any relevant Electronic Documents; and
(f) the likelihood that Electronic Documents will be materially altered in the course of recovery, disclosure or inspection;
(4) the availability of documents or contents of documents from other sources; and
(5) the significance of any document which is likely to be located during the search."
"25 It may be reasonable to search for Electronic Documents by means of Keyword Searches or other automated methods of searching if a full review of each and every document would be unreasonable.
26. However, it will often be insufficient to use simple Keyword Searches or other automated methods of searching alone. The injudicious use of Keyword Searches and other automated search techniques -(1) may result in failure to find important documents which ought to be disclosed, and/or
(2) may find excessive quantities of irrelevant documents, which if disclosed would place an excessive burden in time and cost on the party to whom disclosure is given.
27. The parties should consider supplementing Keyword Searches and other automated searches with additional techniques such as individually reviewing certain documents or categories of documents (for example important documents generated by key personnel) and taking such other steps as may be required in order to justify the selection to the court."
"The court must seek to give effect to the overriding objective when it-(a) exercises any power given to it by the Rules; or
(b) interprets any rule,
subject to rules 76.2, 79.2, 80.2 and 82.2."
None of the last-mentioned rules is relevant to the present case. The overriding objective is set out in rule 1.1(1), and is well-known. It is "enabling the court to deal with cases justly and at proportionate cost". This is amplified in rale 1.1(2), and includes, so far as practicable, saving expense and dealing proportionately with the case.
"1. This judgment concerns a serious practical problem for the case management of disclosure which is now occurring on a regular basis. The reason is that, since certainly the beginning of this decade, increasing numbers of public bodies and private businesses, not to mention individuals, have gone over to creating, exchanging and storing their documentation and communicating with each other entirely by electronic means. The end result is that an enormous volume of information is now created, exchanged and stored only electronically. Email communication, word processed documents, spreadsheets and ever increasing numbers of other forms of electronically stored information ("ESI") now often form the entire corpus of the documentation held by companies and individuals who become involved in litigation. So the incidence of paper disclosure is becoming less and less prevalent though in some cases it may still be critical, and the incidence of the disclosure of electronically stored information, or ESI as it is known, is becoming more and more so.
2.. What is more, the volume of the ESI, even in small organisations is immense, often, as in the case of email, because of the huge quantities of documents created (including wide-scale duplication) and the fact that the documents can exist in many different forms and locations so that they are not readily accessible except at significant cost. It is also commonplace for many individuals to have more than one email account - business, personal, web- mail (for example, Yahoo, Gmail, Hotmail etc.) When ESI is available, metadata (literally data about data) associated with it can easily be unintentionally altered by the very act of collection, which in some circumstances can have a detrimental effect on the document's evidential integrity. What is more, ESI can be moved about nationally and internationally, indiscriminately and at lightening speed.
3.. What is the problem with this in litigation? Disclosure is a tripartite exercise of search, disclosure, and inspection, and the problem, when it comes to ESI is often for a party to gauge the scope of a reasonable search for ESI under CPR Rule 31.7 and PD31(2). The problem is how the parties and (if disputed) the court determines what the scope of that search of ESI should be, how it is going to be made proportionate and how it is going to be carried out correctly first time, without the court having to order it to be done again, as has occurred, for example, in Digicel (St Lucia) Ltd and others v. Cable and Wireless Plc and Others 2008 EWHC 2522 (Ch) in which case Morgan J ordered the defendants re-do their ESI search exercise at an additional cost of something like £2 million.
4.. By contrast, except in unusual cases, in the case of paper disclosure, parties usually know what paper they have. Often the problem is merely locating it physically and going through it to produce the documents required by the standard disclosure test. The problem with ESI is that, because of the matters mentioned above in paragraph 1, parties often do not know how much ESI they have, or where it is. They might have a idea as to which servers it is on or which personal computers it is on, or which back-up tapes it is on, but without a great deal more information, it is very difficult for them to know how much documentation will be revealed by searches of the media on which their ESI is stored and how much it is going to cost to search it and what the end result is going to be. A further issue might be that not all forms of ESI are searchable. Therefore, it has to be accepted that any search is not necessarily conclusive as to whether a particular document exists. Equally often the parties do not know where to begin their searches. In the case, for example, of email, the relevant servers are often not in their possession and sometimes not even in the jurisdiction. An ill considered search for ESI may produce far too few documents for review but more likely will produce such volumes that human review of every document is neither proportionate nor practical. Because of this a substantial industry has developed to handle the identification, collection, reduction and organisation for review of ESI. Often, this is carried out electronically, with technology aiding and supplementing human review."
"26. That searching, because it is going to be done in a comparatively simple way, without using specialist software at this stage, is just going to give us the potential numbers of documents. Similarly, doing the same type of search in respect of the MEDS system for the 31 terms but only in respect of each of the key witnesses, will give us the potential number of documents in respect of that as well. It is at that stage, when that crude way of finding out what documents might be in existence is completed, that a service provider will have to be agreed between the parties, and will have to be instructed to look at what the next stage of the exercise should involve and how much it is going to cost, in order to produce a corpus of documents which is reviewable by both parties.
27.. At the moment we are just staring into open space as to what the volume of the documents produced by a search is going to be. I suspect that in the long ran this crude search will not throw up more than a few hundred thousand documents. If that is the case, then this is a prime candidate for the application of software that providers now have, which can de-duplicate that material and render it down to a more sensible size and search it by computer to produce a manageable corpus for human review - which is of course the most expensive part of the exercise. Indeed, when it comes to review, I am aware of software that will effectively score each document as to its likely relevance and which will enable a prioritisation of categories within the entire document set."
So the Senior Master certainly contemplated that specialist software might be brought into play to score the hundreds of thousands of anticipated e-documents for relevance and therefore possible disclosure in the proceedings. But so far as I am aware he said no more than that.
"At present, the population of documents identified is normally searched for relevance by lawyers or paralegals. In the United States electronic searching is beginning to be introduced. Tests have shown that it is more reliable than review by humans. No doubt this will be with us soon."
A footnote to this statement reads:
"There is a judgment of the US District Court for the Southern District of New York which discusses predictive coding in detail: Moore v Publicis Groupe, unreported, February 24, 2012. The judge was Judge Andrew Peck, who has written widely on predictive coding..."
"The decision to allow computer-assisted review in this case was relatively easy - the parties agreed to its use (although disagreed about how best to implement such review). The Court recognises that computer-assisted review is not a magic, Staples-easy-Button, solution appropriate for all cases. The technology exists and should be used where appropriate, but it is not a case of machine replacing humans: it is the process used and the interaction of man and machine that the court needs to examine.
The objective of review in ediscovery is to identify as many relevant documents as possible, while reviewing as few non-relevant documents as possible. Recall is the fraction of relevant documents identified during a review; precision is the fraction of identified documents that are relevant.
Thus, recall is a measure of completeness, while precision is a measure of accuracy or correctness. The goal is for the review method to result in higher recall and higher precision than another review method, at cost proportionate to the 'value' of the case.
The slightly more difficult case would be where the producing party wants to use computer-assisted review and the requesting party objects. The question to ask in that situation is what methodology would the requesting party suggest instead? Linear manual review is simply too expensive where, as here, there are over three million emails to review. Moreover, while some lawyers still consider manual review to be the 'gold standard', that is a myth, as statistics clearly show that computerised searches are at least as accurate, if not more so, than manual review. [ ... ]
Because of the volume of ESI, lawyers frequently have turned to keyword searches to cull email (or other ESI) down to a more manageable volume for further manual review. Keywords have a place in production of ESI - indeed, the parties here used keyword searches (with Boolean connectors) to find documents for the expanded seed set to train the predictive coding software, hr too many cases, however, the way lawyers choose keywords is the equivalent of the child's game of 'Go Fish'. The requesting party guesses which keywords might produce evidence to support its case without having much, if any, knowledge of the responding party's 'cards' [ ... ]
Another problem with keywords is that they often are over-inclusive, that is, they find responsive documents but also large numbers of irrelevant documents. [ ...]
Computer-assisted review appears to be better than the available alternatives, and thus should be used in appropriate cases. "
"Mindful of this highly deferential standard of review, the Court adopts Judge Peck's rulings because they are well reasoned and they consider the potential advantages and pitfalls of the predictive coding software."
"There is simply no review tool that guarantees perfection. The parties and Judge Peck have acknowledged that there are risks inherent in any method of reviewing electronic documents. Manual review with keyword searches is costly, though appropriate in certain situations. However, even if all parties here were willing to entertain the notion of manually reviewing the documents, such review is prone to human error and marred with inconsistencies from the various attorneys' determination of whether a document is responsive. Judge Peck concluded that under the circumstances of this particular case, the use of the predictive coding software as specified in the ESI protocol is more appropriate than keyword searching. The Court does not find a basis to hold that his conclusion is clearly erroneous or contrary to law. Thus Judge Peck's orders are adopted and Plaintiffs' objections are denied."
"66. The evidence establishes, that in discovery of large data sets, technology assisted review using predictive coding is at least as accurate as, and, probably more accurate than, the manual or linear method in identifying relevant documents. Furthermore, the plaintiff's expert, Mr. Crowley exhibits a number of studies which have examined the effectiveness of a purely manual review of documents compared to using TAR and predictive coding. One such study, by Grossman and Cormack, highlighted that manual review results in less relevant documents being identified. The level of recall in this study was found to range between 20% and 83%. A further study, as part of the 2009 Text Retrieval Conference, found the average recall and precision to be 59.3% and 31.7% respectively using manual review, compared to 76.7% and 84.7% when using TAR. What is clear, and accepted by Mr. Crowley, is that no method of identification is guaranteed to return all relevant documents.
67.. If one were to assume that TAR will only be equally as effective, but no more effective, than a manual review, the fact remains that using TAR will still allow for a more expeditious and economical discovery process.
68.. As technology assisted review combines man and machine, the process must contain appropriate checks and balances which render each stage capable of independent verification. A balance must be struck between the right of the party making discovery to determine the manner in which discovery is provided and participation by the requesting party in ensuring that the methodology chosen is transparent and reliable. Ordinarily, as the rules in other jurisdictions provide, this is a matter of agreement between the parties at the outset. Agreement, as Clarke J. said in Thema, gives the parties "an added degree of comfort that a failure of the system to throw up a relevant document will be more likely to be viewed as unfortunate but unavoidable rather than a deliberate act".
69.. Pursuant to the legal authorities which I have cited supra, and with particular reference to the albeit limited Irish jurisprudence on the topic, I am satisfied that, provided the process has sufficient transparency, Technology Assisted Review using predictive coding discharges a party's discovery obligations under Order 31, rule 12."
(1) Experience in other jurisdictions, whilst so far limited, has been that predictive coding software can be useful in appropriate cases.
(2) There is no evidence to show that the use of predictive coding software leads to less accurate disclosure being given than, say, manual review alone or keyword searches and manual review combined, and indeed there is some evidence (referred to in the US and Irish cases to which I referred above) to the contrary,
(3) Moreover, there will be greater consistency in using the computer to apply the approach of a senior lawyer towards the initial sample (as refined) to the whole document set, than in using dozens, perhaps hundreds, of lower-grade fee-earners, each seeking independently to apply the relevant criteria in relation to individual documents.
(4) There is nothing in the CPR or Practice Directions to prohibit the use of such software.
(5) The number of electronic documents which must be considered for relevance and possible disclosure in the present case is huge, over 3 million.
(6) The cost of manually searching these documents would be enormous, amounting to several million pounds at least, hr my judgment, therefore, a full manual review of each document would be "unreasonable" within paragraph 25 of Practice Direction B to Part 31, at least where a suitable automated alternative exists at lower cost.
(7) The costs of using predictive coding software would depend on various factors, including importantly whether the number of documents is reduced by keyword searches, but the estimates given in this case vary between £181,988 plus monthly hosting costs of £15,717, to £469,049 plus monthly hosting costs of £20,820. This is obviously far less expensive than the full manual alternative, though of course there may be additional costs if manual reviews still need to be carried out when the software has done its best.
(8) The 'value' of the claims made in this litigation is in the tens of millions of pounds. In my judgment the estimated costs of using the software are proportionate.
(9) The trial in the present case is not until June 2017, so there would be plenty of time to consider other disclosure methods if for any reason the predictive software route turned out to be unsatisfactory.
(10) The parties have agreed on the use of the software, and also how to use it, subject only to the approval of the Court.
There were no factors of any weight pointing in the opposite direction.