Partner Michael H. Rogers and Associates Stephen C. Boscolo and Grace Harmon are the authors of the article “TAR and Feather Them: How Principles of TAR Can Influence Use of GenAI” published by New York Law Journal.   As Generative AI (GenAI) becomes increasingly integrated into document collection and courts have yet to weigh in on its use, this timely article discusses how courts and litigants should look to the established validation and transparency requirements used for Technology Assisted Review (TAR).

Highlighting that widespread adoption of GenAI for document review “is coming, if it has not already arrived,” the authors identify two key TAR guardrails “can and should” be applied to GenAI review: results validation, where a party must disclose metrics demonstrating that TAR has successfully collected some critical mass of relevant and responsive documents, and seed-set transparency, where a party must disclose how a TAR model was trained.

Noting that courts are increasingly willing to require producing parties to validate TAR processes and disclose the result to receiving parties, the authors contend that established TAR validation processes “can be readily applied to GenAI.” In In re Uber Techs., Inc., Passenger Sexual Assault Litigation, for example, a California federal judge ordered the producing party to validate that its TAR process was producing relevant and responsive information and to disclose the results, including procedures designed to ensure a certain recall rate was achieved. The authors argue that, once GenAI has reviewed a set of documents and identified a subset as responsive, human reviewers should review a sample of the remaining documents to “determine whether GenAI has collected an acceptable amount of the responsive documents in the set.”  As with TAR, Michael, Stephen, and Grace contend that requiring validation procedures for GenAI “is not only useful, but necessary, in complex cases involving novel technology.”

The authors also address seed sets, which typically contain documents coded as responsive or non-responsive that TAR models use to classify documents.  Importantly, the effectiveness of TAR “is largely dependent on the accuracy of its seed set,” since “if seed documents are not coded correctly, TAR’s ability to identify responsive documents is vastly curtailed.”  While courts are split on whether requiring a party to disclose a seed set is appropriate, cases such as Progressive Cas. Ins. Co. v. Delaney highlight that courts are sympathetic to requests for increased transparency.  Similarly, Michael, Stephen, and Grace underscore that transparency is especially critical in assessing the use of GenAI.  They contend that, because GenAI models typically require prompts, the prompt used “faces a similar vulnerability as a TAR seed set—if it is deficient, any coding performed by a GenAI model will also likely be deficient.” Given the novelty of GenAI and the risks still associated with its use, including hallucinations, they argue that the need for transparency is even greater than it is for TAR.

Anticipating a future where GenAI “is likely to become a staple of document review,” the authors conclude that the “goal of an opposing party should be to ensure that such a review captures all responsive and relevant documents.”