How It Works
You’ve identified ESI for a number of custodians over a date range and now have hundreds of thousands, if not millions of documents to review. Reviewing every document is far too expensive and time consuming, so it is time to leverage technology assisted review and more specifically, predictive coding.
With BlackStone’s Technology Assisted Review (TAR), a review set of 1,000,000 documents can quickly be culled to 100,000 – 200,000 documents and reviewed at a small fraction of the normal review cost. This is mainly done through machine learning with Brainspace, which identifies responsive and privileged documents from massive sets of data after being programmed using a small seed set.
BlackStone’s TAR experts manage the entire process. First, they cull the ESI based on data types and search terms. They ensure that the seed set is representative of all the data and is coded for responsiveness and privilege. Brainspace then produces an algorithm to identify responsive documents based off this set which BlackStone’s experts sample and refine to achieve desired results. Once this is done, the complete review is authorized and Brainspace’s Artificial Intelligence takes over.
The final result is a much smaller number of documents to be produced based on an agreed upon level of certainty and percentage of relevant documents.
The practice of predictive coding greatly cuts cost and time for our clients. Recently, BlackStone used predictive coding to review 3 million documents for under $200,000 within a month’s time for a patent litigation. In 2011, before predictive coding was a common practice, BlackStone was featured in the New York Times for reviewing 1.5 million documents for less than $100,000.
Price will vary depending on a few factors:
- The extent to which the full data set can be culled before reviewing
- Whether you choose to have one of our experts code the seed set or do it yourself
- The area of knowledge required of an expert coding the seed set
Using Brainspace, the information space of the collection is spanned by the training examples more rapidly, leading to faster learning – and better understanding by the trainer and review manager of what the collection holds. Brainspace uses logistical regression to power predictions, which has a number of advantages over other predictive coding approaches.
- Utilizes a range of published and internal experiment results for predictive accuracy
- Quick to build model and predict relevance of documents
- Utilizes prioritized ranking over the target documents instead of binary partitioning
- Provides reasonable predictions of the probability that a document is relevant
- Produces a model that allows human interpretability
Brainspace also helps to measure the user’s cost-benefit tradeoff, guiding the review manager on acceptable levels of precision and completeness. This helps the review manager determine when to pass the culled data set on to attorney reviewers for review in Relativity.
Diversified Active Learning Allows Brainspace to determine whether documents are similar to the training batch. Furthermore, it determines whether documents are similar or dissimilar to other unlabeled documents in order to actively learn from them.
Other Tools and Techniques
BlackStone offers a large number of analytics tools, some common and some more sophisticated, which will assist with the culling, prioritization, and tagging of documents to avoid unnecessary linear review.
- Search Term and Privilege Filter Consultation: BlackStone’s eDiscovery consultants have a breadth of knowledge and experience using platform search tools to generate and apply search terms. We will review search term lists and provide useful feedback and suggestions to maximize the effect of the terms. Our team also possesses a great deal of privilege knowledge and will apply that expertise to preparing and using privilege filters built in conjunction with client and counsel.
- Email Threading and Grouping: Conversation threads are identified to eliminate the need of re-reading the same original emails multiple times. Email review can be limited to the most inclusive conversations with the knowledge that all members of that thread that are removed from review and subsequent production would be duplicative of the reviewed content.
- De-Dupe & De-NIST: The most current methods are used to identify duplicate documents as well as documents that cannot be reviewed, such as system files.
- Clustering: BlackStone can assist the client with identifying highly relevant documents with greater accuracy than simple search terms. Documents that are generally considered to be potentially relevant will receive a higher level of scrutiny to find a smaller review set with a greater likelihood of relevance. Conversely, our clustering tool can take docs which counsel has labeled as highly relevant and locate additional documents that are very similar to those.
- Machine Translation: In the event that there are non-English documents which require review, BlackStone has the ability to mass translate those documents in over 100+ languages within the review platform environment, thus eliminating the need to hire expensive, specialized reviewers for a small subset of documents.
- Productivity Review: BlackStone applies scripts and QC logic searches during review to analyze and manage the productivity of its reviewers. Review rates, work hours, accuracy, and trends are pulled and analyzed daily. Any deviation from the workflow is immediately adjusted for in real time, and not at the end of the review.
“On top of the timely and courteous responses from BlackStone to our numerous (often last-minute) requests, there wasn’t a single mistake in the many jobs they returned to us. It was a huge stress relief.”