By Pradeep Victor, Assistant Vice President, Litigation Solutions, Pangea3
There are generally three ways to conduct a document review:
• Traditional, linear “eyeballs on every document” review,
• Production without reviewing any documents,or
• Use of technology to help facilitate the review.
The traditional, and most common, method involves human reviewers manually coding documents for relevancy. Because the reviewers typically code large sets of documents in serial fashion, linear review can be costly, often up to 70% of the total cost of litigation, and time consuming.
Producing without reviewing any documents is rare - typically appropriate when there are equal sized parties on both sides and when the document population is unlikely to contain much, if any, privileged or confidential documents. Make sure you have an ironclad clawback agreement in place!
Technology-assisted review (“TAR”) generally refers to a set of tools and techniques that assists in either speeding up a more traditional linear review, or through the use of specialized software, assists in the auto-categorization of un-reviewed documents in lieu of human review of those documents.
While the benefits of TAR have been around for several years, the risk-averse legal marketplace has been slow to adopt the technology mainly because of confusion over how to implement it and the lack of judicial acceptance (see, e.g.,Monique Da Silva Moore v. Publicis Groupe & MSL Group (No. 11-cv-01279, S.D.N.Y. Feb. 24, 2012)).
To help navigate this area that some continue to see as confusing, this Top Ten provides the key considerations for in-house counsel when evaluating TAR technology, particularly in light of the recent Da Silva opinion.
1. The Volume of Data Potentially Available for Future Litigation Is Growing at Exponential Levels
Market intelligence firm IDC says that the amount of digital data will grow by 48% in 2012 alone and projects that most companies will be dealing with 50 times the amount of data in 2020 than they had in 2011. It is soon likely the amount of ESI in cases will either take too long or be too expensive for a purely linear review. Indeed, the proportionality requirements of FRCP 26(b)(2)(C) may at some point be found to require parties to use some form of TAR to ensure a just, speedy and inexpensive-discovery process.
2. You May Already Be Using TAR
Buzzwords such as “predictive coding” have been getting a lot of press lately and many essentially equate predictive coding with TAR. TAR however, is broader than that and also includes tools and techniques already used in some document review processes including: document clustering, email threading, the use of keywords to cull and organize the data in order to speed up the review, etc. TAR tools are not mutually exclusive and one should always consider which tools can and should be used in conjunction with the others to produce the most cost-effective results.
3. TAR Does Not Replace Human Review
As the name implies, Technology-Assisted Review requires human input (e.g., attorneys must train the software in order to accelerate and accurately categorize each un-reviewed document). Despite its speed and cost efficiencies, Judge Peck noted in Da Silva Moore that TAR is not a case of "machine replacing humans". Further, despite studies showing the effectiveness of TAR over linear review, some courts may be hesitant to endorse the technology unless they are “quality control verified” by a human. For example, because of the reliance on computer algorithms when using TAR technology, it is possible that a very important document (depending on how the document is written) may be auto-categorized as “non-responsive” and never produced to the opposing party. Minimizing these types of errors is another reason why in-house counsel should have a process that has a robust quality control procedure.
4. Defensibility Depends on the Process Surrounding the Use of TAR, not on the Technology Itself
In Da Silva Moore, Judge Peck noted, “it is the process used and the interaction of man and machine that the courts need to examine….[I]t is unlikely that courts will be able to determine or approve a party’s proposal as to when review and production can stop until computer-assisted review software has been trained and the results are quality control verified.” It comes down to people, process and technology. As Judge Peck further notes, “[a]s with keywords or any other technological solution to eDiscovery, counsel must design an appropriate process, including use of available technology, with appropriate quality control testing, to review and produce ESI.”
5. TAR May Not Be Appropriate for All Reviews
A key driver in the use of TAR is the promise of fairly significant time and cost savings over linear document review. However, one cannot look at these savings in a vacuum as there may also be significant up-front costs to implement TAR. Indeed, Judge Peck opined that size of the discovery and the need for cost effectiveness are just a few of the factors that should be considered when determining whether or not to use TAR. As with every review, critical thought needs to be given as to whether the time and cost savings of TAR, in addition to the other costs of review, will produce an ROI that makes sense and is the most effective solution for your particular litigation.
6. Cost Predictability Could be As Important As Cost Savings
For many clients, cost predictability is often as important, if not more so, than cost savings. Using TAR clients may be able to determine and identify an exact number of documents needed for further review (depending on the accuracy percentage needed) and to therefore allow them to more accurately estimate the cost of the engagement.
7. TAR May Allow For Greater Efficiency in Document Coding
Absent a documented and tested review process, individual reviewers in a linear review often see their documents in isolation to what their fellow reviewers are seeing and as a result, underlying connections may not be seen. Because TAR utilizes fewer reviewers, document patterns may be more readily identifiable. TAR allows reviewers to analyze the most relevant documents first, leading to earlier and faster identification of key documents.
8. Transparency of Process and Cooperation with Opposing Counsel
In general, transparency and cooperation between counsel at the outset of a review may often avoid many potential issues. Judge Peck said e-discovery “requires cooperation between opposing counsel and transparency in all aspects of preservation and production of ESI. Transparency allows the opposing side to be more comfortable with computer-assisted review and reduces fears about the so-called black box of the technology. This court highly recommends that counsel in future cases be willing to at least discuss, if not agree to, such transparency in the computer-assisted review process.” However, it’s important to note that Judge Peck did not rule that such transparency and disclosure was required, just recommended. For now, it is up to the parties to determine whether the level of transparency/cooperation apparent in Da Silva Moore would be appropriate for another matter utilizing TAR.
9. Focus Should Be on the Accuracy of the Training Set
Because TAR ensures the old maxim “garbage in, garbage out” remains true, in Da Silva Moore, Judge Peck approved the use of senior attorneys to review randomly created seed sets which would be used to "train" the TAR technology (via multiple iterations) to identify similar documents. Ensuring seed set accuracy is absolutely critical, therefore, it is more important to develop a process of review that is not dependent on any one reviewer (senior or otherwise) and has the goal of accuracy built in. This process should still allow senior counsel to adequately supervise the review and quality control process.
10. Case Law is Still in the Nascent Stages
The case law surrounding this issue is changing daily. After Judge Peck’s February ruling, plaintiffs attempted to get his recusal however, U.S. District Judge Andrew L. Carter upheld the ruling on April 26 indicating the protocol adopted by Judge Peck “contains standards for measuring the reliability of the process and … [includes] participation by Plaintiffs.” Prior to that decision, on April 23 Virginia state Judge James H. Chamblin issued a final order authorizing the use of TAR in Global Aerospace Inc. v. Landow Aviation Limited Partnership, et al., No. 61040. While a Virginia State Court opinion has no national force of law, it is still precedent setting. In another potentially important e-discovery case, the ongoing Kleen Products LLC v. Packaging Corporation of America, et al., plaintiffs have asked Magistrate Judge Nan Nolan to require the producing parties use TAR in their e-discovery production stating “[t]he large disparity between the effectiveness of [TAR] methodology and Boolean keyword search methodology demonstrates that Defendants cannot establish that their proposed [keyword] search methodology is reasonable and adequate as they are required.” Defendants, however, plan to use their own experts to argue their use of Boolean search techniques is reasonable, precise and cost-effective. Defendants also argue it would be difficult to implement TAR which they refer to as a “relatively new, largely untested methodology,” since they have already started ESI collection. It is important to note that both sides have indicated significant investment in their respective e-discovery tools. This matter is ongoing and the resulting opinion is expected to be as important as Judge Peck’s Da Silva Moore opinion.
Whatever the outcome of these cases, TAR makes most sense where both sides possess and will be required to review large amounts of documents. In Da Silva Moore, Judge Peck considered five factors when he approved the use of TAR: "(1) the parties' agreement, (2) the vast amount of ESI to be reviewed (over three million documents), (3) the superiority of computer-assisted review to the available alternatives (i.e., linear manual review or keyword searches), (4) the need for cost effectiveness and proportionality under Rule 26(b)(2)(C), and (5) the transparent process proposed by [defendant].” Although far from set rules, these factors, along with the other considerations in this Top Ten, are helpful if you consider the use of TAR in your future reviews.