ARCHIVED CONTENT
You are viewing ARCHIVED CONTENT released online between 1 April 2010 and 24 August 2018 or content that has been selectively archived and is no longer active. Content in this archive is NOT UPDATED, and links may not function.Extract from commentary by Bill Dimm
NOTE: The following letter is from Dr. Bill Dimm, the founder and CEO of Hot Neuron LLC. He has developed the algorithms for conceptual clustering, near-dupe detection, and predictive coding used in the company’s Clustify software. He is responding to commentary from Information Retrieval expert Gordon Cormack, published here following the ACEDS’ webinar How Automation is Revolutionizing E-Discovery.
We talked about the history of the industry, but also about judicial decisions and methodologies that are less than a year old. We talked a lot about the state of judicial acceptance, which is of clear importance to practitioners and is not well understood — the results from our poll at the beginning of the webinar illustrated this point. We talked about the problem of vendors putting overly optimistic spin on reporting about judicial decisions. We talked about the Cormack and Grossman JOLT study because it is the study that judicial opinions rely upon, and we’re not aware of any subsequent study comparing the quality (not merely the cost) of TAR results to those of human review. In spite of its importance in judicial decisions, the JOLT study does not seem to be well understood. We pointed out that it studied only the systems that had the best performance, not systems that were in any sense average or typical. Should courts be relying on a study of the best systems to make rulings on the use of systems bearing little resemblance to the systems studied? This is not an issue that is resolved by a Bonferroni correction (which merely ensures that statistical significance claims about the systems studied are valid for those systems). It is important for litigants to know how strong the evidence and judicial support for TAR really is. We pointed out that a mere month ago Judge Hedges proclaimed that TAR has not really been challenged yet.
Read the complete article at ACEDS Commentary: Bill Dimm
