Thu. Dec 7th, 2023

Content Assessment: Double eDiscovery Vision? New Specification Helps Solve Longstanding Email Deduplication Problem



A short assessment of the qualitative benefit of the recent announcement by the EDRM of its new Email Duplicate Identification Specification, EDRM MIH.

Editor’s Note: The Electronic Discovery Reference Model (EDRM) is a global organization that has been at the forefront of establishing standards and best practices for eDiscovery and legal technology since 2005. With members in over 145 countries, EDRM has created numerous practical resources that are widely used by corporations, law firms, government agencies, and educational institutions to improve eDiscovery processes and workflows.

The just-announced EDRM Cross Platform Email Duplicate Identification Specification demonstrates EDRM’s role as a leader in developing consensus-based standards that help address key challenges in eDiscovery and legal technology. By providing a common framework that can be adopted across different platforms and tools, the specification enables a more efficient and cost-effective way to identify duplicate emails in cross-platform contexts.

Media Announcement Summary

Double eDiscovery Vision? New Specification Helps Solve Longstanding Email Deduplication Problem

ComplexDiscovery Staff

For years, a major pain point has plagued eDiscovery professionals – the inability to efficiently identify duplicate emails across different platforms. While deduplicating data within a single system was achievable, performing cross-platform deduplication remained an unsolved challenge without a clear solution.

That frustrating status quo may now change thanks to a new innovative specification developed by a standards body devoted to advancing the industry.

The Electronic Discovery Reference Model (EDRM) is an organization that has established itself as a leader in eDiscovery best practices since 2005. With members around the world, EDRM serves as a neutral body where competitors come together to collaborate on projects that tackle common problems holding the industry back.

One such nagging issue was the lack of options for cross-platform email deduplication. When Beth Patterson, an EDRM member and Director at consultancy ESPconnect, flagged this persisting problem, she knew EDRM’s unique position could help drive an answer.

EDRM pulled together a project team from its extensive membership network. It included eDiscovery experts like Craig Ball, product developers like Stephen Stewart of Nuix, technologists like Murali Baddula of Law In Order, and lawyers determined to find an efficient and workable approach.

The team conceived a simple but effective solution – using the hash value of an email’s Message ID metadata field, called the EDRM MIH. This new approach would not replace but rather complement existing vendor email deduplication methods by enabling cross-platform duplicate identification.

After intense work over 18 months, the project team finalized the specification. It provided a common framework for implementing the EDRM MIH approach that could integrate with existing vendor tools and workflows.

With the specification complete, EDRM encouraged platforms to adopt it. Leading the charge were top vendors Reveal, Relativity, EDT, and Nuix, competitors who had collaborated within EDRM on developing the spec.

George Socha, VP at Reveal, noted they were proud to be the first to implement the specification, helping improve eDiscovery. Cristin Traylor, a director at Relativity, touted the project as an excellent example of cooperation driving positive change.

The ability to solve a longstanding struggle through collaboration earned praise across the sector. Jo Sherman, CEO of EDT, highlighted that it was rewarding for competitors to jointly achieve a tangible outcome benefiting everyone.

For EDRM’s CEO and Chief Legal Technologist, Mary Mack, the project showcased EDRM’s power to drive real progress through neutral standards development. She said EDRM is encouraging widespread adoption of the specification.

With the new specification now a reality thanks to EDRM’s determined working group, eDiscovery professionals finally have an efficient way to tackle the cross-platform duplicate identification challenge. It represents a triumph of cooperation and community to solve an entrenched issue holding the industry back.

Read More on the New Specification

Assisted by GAI and LLM Technologies

Additional Reading

Source: ComplexDiscovery


Generative Artificial Intelligence and Large Language Model Use

ComplexDiscovery OÜ recognizes the value of GAI and LLM tools in streamlining content creation processes and enhancing the overall quality of its research, writing, and editing efforts. To this end, ComplexDiscovery OÜ regularly employs GAI tools, including ChatGPT, Claude 2, Midjourney, and DALL-E3, to assist, augment, and accelerate the development and publication of both new and revised content in posts and pages published (initiated in late 2022).

ComplexDiscovery also provides a ChatGPT-powered AI article assistant for its users. This feature leverages LLM capabilities to generate relevant and valuable insights related to specific page and post content published on By offering this AI-driven service, ComplexDiscovery OÜ aims to create a more interactive and engaging experience for its users, while highlighting the importance of responsible and ethical use of GAI and LLM technologies.


Have a Request?

If you have information or offering requests that you would like to ask us about, please let us know, and we will make our response to you a priority.

ComplexDiscovery is a distinguished digital publication that delivers journalistic insights into cybersecurity, information governance, and eDiscovery developments and technologies. It adeptly navigates the intersection of these sectors with international business and current affairs, transforming relevant developments into informational news stories. This unique editorial approach enables professionals to gain a broader perspective on the intricacies of the digital landscape for informed strategic decision-making.

Incorporated in Estonia, a nation celebrated for its digital innovation, ComplexDiscovery OÜ adheres to the most rigorous standards of journalistic integrity. The publication diligently analyzes global trends, assesses technological breakthroughs, and offers in-depth appraisals of services involving electronically stored information. By contextualizing complex legal technology issues within the broader narrative of worldwide commerce and current events, ComplexDiscovery provides its readership with indispensable insights and a nuanced understanding of the eDiscovery industry.