Extract from an article by Bill Dimm
TAR won across the board, as in previous iterations of the challenge. Only one person submitted queries for the medical industry topic. His/her revised query did a better job of finding relevant documents, but still returned fewer than 3,000 documents and fared far worse than TAR — the query was just not broad enough to achieve high recall. Three people submitted queries on the law topic. One of those people revised the query a few times and got decent results, but still fell far short of the TAR result, with review of 6,000 documents from the best query finding fewer relevant documents than review of half as many documents with TAR 3.0 SAL (TAR 3.0 CAL did even better). It is unfortunate that the audience was so small, since a larger audience might have done better by learning from each other’s submissions. Hopefully I’ll be able to do this with a bigger audience in the future.
- Predictive Coding Technologies and Protocols: Overview and Survey
- TAR, Proportionality, and Bad Algorithms (1-NN)