Confirming Recall Adequacy With Unbiased Multi-Stage Acceptance Testing - ComplexDiscovery Archived Content

ARCHIVED CONTENT

A post shared by Rob Robinson (@complexdiscovery) on Jun 6, 2017 at 4:56am PDT

Editor’s Note: Published by Bill Dimm of Hot Neuron, the paper Confirming Recall Adequacy With Unbiased Multi-Stage Acceptance Testing and a link to a brief overview of the paper describe a new method for requiring much less document review to demonstrate adequate recall achievement in eDiscovery. The paper will be presented by Mr. Dimm as part of the DESI VII Workshop in London, UK.

Confirming Recall Adequacy: A New Approach

Abstract and Complete Paper by Bill Dimm

The adequacy of an e-discovery production has traditionally been established by using random sampling to estimate recall, but that requires review of approximately 400/ρ documents, where ρ is the prevalence, which can be burdensome when prevalence is low. Accept-on-zero testing is sometimes suggested as an option requiring less review at only about 12/ρ, but in practice, it is biased and is likely to fail when recall actually is adequate. This paper proposes a multi-stage acceptance testing procedure that avoids bias and actually works in practice. The amount of document review required with the new method depends on the level of recall actually achieved. It is typically around 200/ρ or 100/ρ, but can be as low as 25/ρ if the actual recall is substantially higher than the minimum required. This dependence on the recall achieved may motivate producing parties to aim for higher recall since the additional document review put into pushing recall higher will be at least partially offset set by a reduction in review effort needed to confirm the adequacy of the result.

The Complete Paper (Bill Dimm/Hot Neuron LLC)

Read the complete explanation of the article at Substantial Reduction in Review Effort Required to Demonstrate Adequate Recall

Source: Dimm, William. Confirming Recall Adequacy With Unbiased Multi-Stage Acceptance Testing. London, UK: Bill Dimm, Hot Neuron, LLC, 2017. Web. 6 June 2017. ICAIL 2017 DESI VII Workshop.

Additional Reading

• ICAIL DESI VII Workshop (June 12, 2017, King’s College, London, UK)
• Technology-Assisted Review on ComplexDiscovery