Instead of Fighting About Discovery, Set up First and Second Pass Review to Produce Responsive ESI

Reviewing electronically stored information (“ESI”) for productions should be a streamlined process to identify responsive information. Parties often find new and exciting ways to derail document review. Consider the employment discrimination case of Bird v. Wells Fargo Bank, where the Defendant took the following positions:

It would take 6-8 weeks for it to collect the requested ESI, which did not include reviewing the data for privilege, privacy, or confidentiality;

Defendants reserved the right to ‘shift all fees and costs incurred in the collection, review, and production of ESI to Plaintiff and Plaintiff’s counsel.’ 1

There were many problems with the Defendants’ view of their discovery obligations, especially on cost-shifting. Magistrate Judge Erica P. Grosjean took the Defendants to task over “the legally unsupportable position that it is not under any obligation to provide electronic discovery unless and until there is full agreement on search terms.”2  There were more unsupportable arguments by the Defendants, such as they would seek sanctions if the Plaintiffs did not provide search terms by a date set by the Defendants. The Court referred to this conduct as being neither professional nor a good faith meet and confer.

The Court issued the following order for producing discovery:

The Defendants were to ‘immediately’ design and implement a discovery plan to diligently and in good faith produce documents consistent with this order and schedule its production on a rolling basis;

Produce responsive documents consistent with the discovery rules;

Disclose the scope of their search, including ‘search terms, custodians or other limitations’;

The parties were not to engage in further meet and confers;

Plaintiffs could file a motion for sanctions if the Defendant’s searches did not meet their discovery obligations; and

Electronically stored information shall be produced in a format that extracts the text and preserves the metadata from native files.

This is an extreme case, but instead of turning discovery into trench warfare, it is usually cheaper to focus on setting up document review to produce responsive data.

Document review should further the goal of Federal Rule of Civil Procedure Rule 1 to “secure the just, speedy, and inexpensive determination of every action and proceeding.” One way to meet that goal is to leverage predictive coding with first and second pass review in responding to discovery requests.

Preparing for Document Review

There are multiple steps in order to prepare for document review to be as efficient as possible. An admin should create issue coding for responsive discovery requests, either by request number (RFP001), or by categories based on similar requests (RFP – Breach of Contract). This is to make identifying responsive ESI easier and for objecting with specificity if necessary.  Known privileges should also be added as coding options before beginning review.

The other key step is having an experienced attorney begin general document review to rate ESI hot, warm, or cold for examples of ESI that are relevant, marginally relevant or technically responsive, or irrelevant.

Screenshot of Everlaw ESI document relevance rating

A prediction model can also be created for ESI that is rated “hot” or “warm” and issue coded as “responsive” to any of the discovery requests.

First Pass Document Review

Attorneys conducting “first pass document review” are doing the hard work of identifying ESI that could be responsive from data that is irrelevant or non-responsive. The goal is for the attorneys who are trying the case, preparing for depositions, or engaged in motion practice, to not have to wade through extensive documents to find what is relevant to the claims or defense in the case.

One strategy for responding to discovery requests is for an admin or knowledgeable attorney to create saved searches for each discovery request. These searches should be given names that are easily identifiable, such as RFP001 – Contract Claims.

The search terms for each request are taken directly from the request for production. If the requesting party has done their job right, they have crafted a narrowly tailored request for production that asks for ESI between specific individuals, time periods, with concepts that can be searched for in the database.

Screenshot of RFP search terms for document production in Everlaw

We don’t live in a perfect world, and sometimes a request for production is a strange house of mirrors where a requesting party wants all emails containing highly common phrases that are horrible search terms. In such situations, letting the other side know their requests will likely contain a substantial number of false positive hits is a wise course of action.

If the other side is not cooperative, that is not a free pass to skip responding to a discovery request. The producing party still has an obligation to object and make a good faith effort to produce responsive documents.

After a search has been created for each request for production, these searches can be used to create assignments for the review team. Here are the advantages for creating an assignment for each RFP saved search:

  1. A dynamic assignment will update with any new data that is added to the case that is a “hit” to the search terms. Reviewers can then self-assign additional ESI to review for responsiveness;
  2. The producing party is able to identify with specificity any data that is responsive that needs to be withheld for privilege;
  3. Proportionality arguments can be easier to make, as the producing party can state the exact number of files or emails that are “hits” to the search terms;
  4. If there is a motion to compel, the producing party can address each specific request for production on the size of the dataset, or other relevant information to the motion.

The reviewing attorneys should rate highly relevant documents as hot, responsive documents as warm, and irrelevant documents as cold. There are other ways to code, but whatever model a team follows, be consistent. If a document is rated as hot or warm, it should also be coded as responsive to specific requests for production. Anything privileged should be coded properly.

There is a high probability that ESI responsive to one request for production, could be responsive to another. Reviewing attorneys should keep the big picture in mind and issue code whether the ESI is responsive to any other requests. In order to avoid reviewers being assigned documents that have already been reviewed by another attorney, the review criteria for subsequent assignments can contain an exclusion for ESI that has already been coded responsive to any discovery request.

Screenshot showing exclusions in review criteria for assignments

The reviewing attorneys should provide feedback on the search terms used to find responsive ESI. Additional inclusion or exclusion criteria can be used to help focus on the relevant data. Moreover, completely new searches could be warranted, if terms of art are discovered in review that could impact the scope of the search. Moreover, if reviewing attorneys report they have found newsletters, marketing emails, eBay alerts, or similar irrelevant information, those communications can be searched for and bulk tagged as both cold and irrelevant.

Applying Predictive Coding

Reviewing attorneys have multiple options for leveraging predictive coding to focus on relevant or responsive information.

If search terms yield a large number of hits, applying a predictive coding model to the hits can help narrow the dataset in an assignment. This first requires training the predictive coding model and testing it against the data. This can be done by focusing on data that is not well reviewed within the database or data that is well reviewed. If the predictive coding has been well trained, this can help narrow the searches to data with a high precision of being responsive.

Screenshot of Everlaw predictive coding of documents in first pass review

Another option is as attorneys near finishing their first pass review, is to crosscheck the database with the prediction model for ESI rated hot or warm and coded as responsive to the Requests for Production. The predictive coding can identify responsive data that was not identified by the search terms, but is nevertheless responsive.

Screenshot showing Everlaw predictive model for ESI

Second Pass Document Review

Quality assurance is the name of the game with second pass review. There are multiple approaches for second pass review. One tactic is to make sure what has been coded as responsive actually is responsive. Another method is to focus on ESI from the first pass review that has been rated hot, warm, and responsive to specific discovery requests before being produced to the other side. More experience attorneys could also apply confidentiality or attorney’s eyes designations to responsive ESI.   

Another option is to focus on identifying any privileged communications that could be within the responsive dataset. This requires conducting searches over the dataset for attorney email addresses or other key phrases that would contain privileged information.  

It is a good quality assurance practice to sample the ESI from the first pass review that was rated cold and irrelevant. While the reviewing attorneys most likely made the correct judgment call, sampling the data, or running a prediction model against the cold ESI, is a good way to “trust, but verify” the first pass review.

Final Review Before Production

Responsive ESI identified from the first and second pass reviews can have one validation before production. This can be done with searches within the assignment for specific issue coding, such as proper designations, or known privileged information. After the ESI has been checked, it can be added to a “pre-production binder” for production. A binder is an arbitrary collection of documents in Everlaw that can be shared with your colleagues on Everlaw.

Document Review Conclusion

Discovery can be complicated, but it is more complicated without a plan. Leveraging search terms, assignments, and predictive coding can help attorneys have a dynamic workflow to find responsive ESI to discovery requests.

1Bird v. Wells Fargo Bank, 2017 U.S. Dist. LEXIS 49675, at *7-8 (E.D. Cal. March 31, 2017)
2Bird, at *16