From the trigger point for audits, investigations, and litigation to the conclusion of cases and matters with the defensible disposition of data, there are countless ways data discovery and legal discovery professionals approach and administer the discipline of eDiscovery. Based on an aggregation of research from leading eDiscovery educators, developers, and providers, the following eDiscovery Processes and Tasks listing may be helpful as a planning tool for guiding business and technology discussions and decisions related to the conduct of cybersecurity, information governance, and legal discovery-related eDiscovery projects. The processes and tasks highlighted in this listing are not all-inclusive and represent only one of the myriads of approaches to eDiscovery.
The workflow of the electronic discovery process includes initiation, legal hold, collection, ingestion, processing, analytics, predictive coding (technology-assisted review), review, production/export, and data disposition.
Initiation #
The initiation stage involves preliminary planning, project organization, project scoping, and the creation of a statement of work (SOW) and project estimates. The client reviews the SOW and estimates, and upon acceptance, the SOW is executed.
Legal Hold #
The legal hold stage involves scoping the legal hold, identifying and preserving electronic data in accordance with the legal hold plan, issuing legal hold notifications, and documenting the legal hold process.
Collection #
The collection stage involves scoping the collection, identifying the collection assets, developing a collection plan, conducting onsite collection, documenting and certifying the collection, and shipping/transferring the collection.
Ingestion #
The ingestion stage involves processing the received electronic data, conducting an ESI reception audit, creating a chain of custody, reviewing the collection log and notes, and preparing the electronic data for processing.
Processing #
The processing stage involves processing the electronic data according to specification, reporting the processing status to the client, identifying any exception or hold files (EHF), and completing the processing of the electronic data.
Analytics #
The analytics stage involves analyzing the electronic data, preparing an analytics repository, conducting an exploratory analysis, conducting a preliminary filtering hit report, and filtering the electronic data according to specification.
Predictive Coding #
The predictive coding (technology-assisted review) stage involves planning the predictive coding specification, moving the electronic data into a predictive coding application, conducting the predictive coding, and reporting the results of the predictive coding.
Review #
The review stage involves planning and preparing the review technology and staffing, conducting the review, and reporting the results of the review.
Production/Export #
The production/export stage involves planning and executing the production/export of the electronic data, conducting quality control (QC), and delivering the electronic data according to specification.
Data Disposition #
The data disposition stage involves planning the data disposition, evaluating the active discovery, making a decision to conclude the project, and disposing of the data in accordance with the client’s guidance.
The electronic discovery process involves a structured and systematic approach to managing the vast amounts of electronic data involved in modern litigation, ensuring that all relevant information is identified and reviewed in an efficient and effective manner.
The Workstream of eDiscovery: Process and Task Listing #
Complete Checklist: eDiscovery Processes and Tasks
The Workstream of eDiscovery | |
Process | Task |
Initiation
|
Preliminary Planning |
Project Organization | |
Project Scoping | |
Statement of Work (SOW) | |
Project Estimates | |
Client Assessment of SOW and Estimates | |
Assessment Acceptance | |
Execute SOW | |
Legal Hold
|
Legal Hold Scoping |
Legal Hold Asset Requirements | |
Legal Hold Plan Development | |
Client Assessment of Legal Hold Plan | |
Assessment Acceptance | |
Issue Legal Hold Notifications | |
Identify and Preserve ESI Per Legal Hold Plan | |
Track Legal Hold Notifications | |
Communicate and Execute Legal Hold Release | |
Release Preserved ESI To Data Retention Policy | |
Document Legal Hold Process | |
Collection
|
Collection Scoping |
Collection Asset Requirements | |
Collection Plan Development | |
Client Assessment of Collection Plan | |
Assessment Acceptance | |
Onsite Collection | |
Collection Documented and Certified | |
Collection Shipment/Transfer | |
Ingestion
|
Processing Specification Planning |
Processing Specifications Defined | |
Analytics Specification Planning | |
Analytics Specification Defined | |
Client Review of Project Plans | |
Project Plan Acceptance | |
ESI Received | |
ESI Reception Audit | |
Chain of Custody Check | |
Reception Reporting | |
Collection Log and Notes Reviewed | |
Transmittal Letter Reviewed | |
Client Notification of ESI Reception | |
Actual and Estimated Volumes Aligned | |
Modification of Estimates and SOW | |
Client Upload Approval | |
ESI Upload into eDiscovery Platform | |
Processing
|
ESI Processed According to Specification |
Processing Status Reported to Client | |
Exception and Hold File (EHF) Identification | |
Client Guidance of EHF Handling Protocol | |
Further Processing of EHF Per Protocol | |
ESI Processing Completed | |
Cyber Discovery (Artificial Intelligence)
|
Preparation – Initiation of Cyber Discovery Process |
Planning – Model and Protocol Planning | |
Training – Selection, Building Testing, and Training | |
Tuning – Validation and Evaluation | |
Discovery – Adaptation, Deployment, and Maintenance | |
Response – Cyber Discovery Understanding | |
Analytics
|
ESI Analyzed According to Specification |
Analytics Repository Preparation | |
Client Exploratory Analysis of ESI | |
Client Early Case Assessment of ESI | |
Filtering Planning | |
Filtering Specification Defined | |
Client Confirmation of Filtering Specification | |
Preliminary Filtering Hit Report Preparation | |
Acceptance of Initial Filtering Results | |
Complete Filtering Per Specification | |
Analytics Reduction Completed | |
Predictive Coding (Technology-Assisted Review)
|
Predictive Coding Specification Planning |
Predictive Coding Specification Defined | |
Client Review of Predictive Coding Plans | |
Predictive Coding Plan Acceptance | |
ESI Moved into Predictive Coding Application | |
Predictive Coding Accomplished Per Specification | |
Review
|
Review Technology Specification Planning |
Review Technology Specification Defined | |
Review Staffing Specification Planning | |
Review Staffing Specification Defined | |
Review Technology and Staffing Plan Developed | |
Client Evaluation of Review Plan | |
Client Acceptance of Review Plan | |
Review Technology Prepared According to Specification | |
Review Staffing Executed According to Specification | |
ESI Prepared for Review | |
ESI Volume and Expectation Alignment Verification | |
ESI Promoted to Review Application | |
Review Conducted Per Plan | |
Review QC and Validation | |
Review Results Reported | |
Review Results Acceptance | |
ESI Prepared for Production/Export | |
Production/Export
|
Production/Export Specification Planning |
Timeframe and Budget Planning/Update | |
Timeframe and Budget Plan Acceptance | |
Production/Export Execution | |
Production/Export QC | |
Production/Export Delivery Per Specification | |
Data Disposition
|
Hosting Requirement Planning |
Active Discovery Evaluation | |
Data Disposition Option Planning | |
Client Decision To Conclude Project | |
Data Disposition Per Client Guidance | |
Project Concluded | |
AI – Preparation
|
Cyber Discovery Goals |
Data Collection and Ingestion | |
Data Exploration | |
Data Processing | |
AI – Planning | Model and Protocol Planning (AI+Experts) |
AI – Training
|
Model and Protocol Selection and Building |
Model and Protocol Testing and Training | |
AI – Tuning
|
Model and Protocol Validation |
Model and Protocol Evaluation | |
AI – Discovery
|
Model and Protocol Adaptation (Adjustment) |
Model and Protocol Deployment (Execution) | |
Model and Protocol Monitoring (Monitoring) | |
AI – Response | Cyber Discovery Action |
TAR 1.0 – Simple Active Learning
|
ESI Moved Into Technology-Assisted Review (Predictive Coding) Application |
Establish a Random Control Set of ESI | |
Review and Code Control Set for Relevance | |
Continue Training (Establish, Review, and Code) Until Sufficient Number of Relevant Documents in Control Set | |
Establish a Seed Set of ESI (Random and/Judgmental Sampling) | |
Review and Code Seed Set for Relevance | |
Apply Machine Learning Algorithm to Suggest Best Learning Documents | |
Review and Code Suggested Best Learning Documents and Add to Seed Set | |
Repeat Application of Machine Learning Algorithm With Seed Set Until Stabilization Occurs (Based on Accuracy of Relevance Prediction for Documents in Control Set) | |
Apply Learning Algorithm to Categorize or Rank All Documents | |
Prepare for Review All Documents Categorized as Relevant or Ranked Above Cut-off Score | |
Validate the TAR (Predictive Coding) Process | |
TAR 1.0 – Simple Passive Learning
|
ESI Moved Into Technology-Assisted Review (Predictive Coding) Application |
Establish a Seed Set of ESI (Random and/Judgmental Sampling) | |
Review and Code Seed Set for Relevance | |
Apply Machine Learning Algorithm to Evaluate Whether Documents Are Relevant | |
Evaluate Effectiveness of Training (Number of Overturns and Machine Learning Unclassifiable Documents) | |
Continue Training (Establish, Review, and Code) with Larger Seed Set Until Training Effectiveness Deemed Sufficient) | |
Apply Learning Algorithm to Categorize or Rank All Documents | |
Prepare for Review All Documents Categorized as Relevant or Ranked Above Cut-off Score | |
Validate the TAR (Predictive Coding) Process | |
TAR 2.0 – Continuous Active Learning®
|
ESI Moved Into Technology-Assisted Review (Predictive Coding) Application |
Establish a Seed Set of Relevant Documents (Judgemental Sampling) | |
Apply Machine Learning Algorithm to Collection to Suggest Most Likely Responsive Documents | |
Review Suggested Documents and Provide Feedback for Machine Learning Algorithm | |
Repeat Application of Machine Learning Algorithm Against Collection Until Few, If Any, Suggested Documents Are Relevant | |
Prepare for Review All Documents Categorized as Relevant | |
Validate the TAR (Predictive Coding) Process | |
TAR 3.0 – Cluster-Centric CAL
|
ESI Moved Into Technology-Assisted Review (Predictive Coding) Application |
Form Conceptual Clusters of Collection | |
Establish a Seed Set of Relevant Documents (Judgemental Sampling) | |
Apply Machine Learning Algorithm to Cluster Centers and Sort By Relevance Score | |
Review Small Number of Cluster Centers with Highest Relevance Score and Repeat Application of Machine Learning Algorithm Until Few Relevant Clusters Remain | |
Apply Machine Learning Algorithm Against Collection | |
Determine Whether Produce Documents Without Review, Produce Documents with High Relevance Scores Without Review and Perform Standard CAL on Remainder of Documents, or Review All Documents for Production Using Standard CAL | |
Validate the TAR (Predictive Coding) Process | |
TAR 4.0 – Hybrid Multimodal IST* Predictive Coding (*Intelligently Spaced Training)
|
ESI Communications for the Scope Definition of Discovery, Relevance, and Related Review Procedures |
Conduct Multimodal Early Case Assessment (ECA) | |
Random Sample to Determine Prevalence | |
Training Select (Iterative) to Determine What Documents to Use to Train the Machine (Human Function) | |
AI Document Ranking (Iterative) to Appropriately Rank Documents (Machine Function) | |
Multimodal Review (Iterative) to Find New or Irrelevant Document for the Next Round of Training with Multiple Techniques That May Include Search by High Ranked Documents, Mid-Ranked Uncertain Documents, Random and Judgemental Sampling, and Ad Hoc Searches Not Based on Document Ranking | |
Zero Error Numerics (ZEN) Quality Assurance Tests to Validate the TAR (Predictive Coding) Process | |
Phased Production, Where Relevant Documents are Reviewed Again and Produced | |
References:
- Haslam, A. (2022). eDisclosure Systems – Buyers Guide. 2022 ed. [ebook] Allvision Computing. Available at: https://complexdiscovery.com/the-edisclosure-systems-buyers-guide-2022-edition-andrew-haslam// [Accessed 17 June 2022].
- European Union Agency for Cybersecurity, 2020. Artificial Intelligence Cybersecurity Challenges. [online] European Union Agency for Cybersecurity. Available at: https://digital-strategy.ec.europa.eu/en/library/report-artificial-intelligence-cybersecurity-challenges [Accessed 17 June 2022].
- EDRM | Empowering the Global Leaders of eDiscovery. 2022. EDRM. [online] Available at: https://edrm.net/ [Accessed 17 June 2022].
- Robinson, R., 2022. Considering Cyber Discovery? A Strategic Framework. [online] ComplexDiscovery. Available at: https://complexdiscovery.com/ [Accessed 17 June 2022].
- All, A., 2014. Data Discovery Is Changing Business Intelligence. [online] Enterprise Apps Today. Available at: http://www.enterpriseappstoday.com/business-intelligence/data-discovery-is-changing-business-intelligence.html [Accessed 17 June 2022].
- Grossman, M. and Cormack, G., 2013. The Grossman-Cormack Glossary of Technology-Assisted Review. Federal Courts Law Review, [online] 7(1). Available at: https://www.fclr.org/fclr/articles/html/2010/grossman.pdf [Accessed 17 June 2021].
- Wikipedia. 20221. Insight. [online] Available at: https://en.wikipedia.org/wiki/Insight [Accessed 17 June 2022].
- In: Lexico (Oxford). 2022. Intelligence. [online] Available at: https://www.lexico.com/definition/intelligence [Accessed 17 June 2022].
- Dimm, B. (2016). TAR 3.0 and Training of Predictive Coding Systems. [online] Cluster-text.com. Available at: http://www.cluster-text.com/papers/TAR_3_and_training_predictive_coding.pdf [Accessed 17 June 2022].
- Grossman, M. and Cormack, G. (2016). Continuous Active Learning for TAR. Practical Law, [online] (April/May). Available at: https://plg.uwaterloo.ca/~gvcormac/caldemo/AprMay16_EdiscoveryBulletin.pdf [Accessed 17 June 2022].
- Losey, R. (2018). e-Discovery Team ®. [online] e-Discovery Team ®. Available at: https://e-discoveryteam.com/ [Accessed 17 June 2022].
- Macauley, C. (2016). Forensic Matters: Further advances in TAR. [online] KordaMentha. Available at: https://www.kordamentha.com/News-and-Insights/Further-advances-in-TAR [Accessed 17 June 2022].
- Socha, G. and Gelbmann, T. (2005). Electronic Discovery Reference Model. [online] Edrm.net. Available at: https://www.edrm.net/frameworks-and-standards/edrm-model/ [Accessed 17 June 2022].