News Announcement from the National Institute of Standards and Technology (NIST)
To Measure Bias in Data, NIST Initiates ‘Fair Ranking’ Research Effort
New initiative has long-term goal of making search technology more evenhanded.
A new research effort at the National Institute of Standards and Technology (NIST) aims to address a pervasive issue in our data-driven society: a lack of fairness that sometimes turns up in the answers we get from information retrieval software.
Software of this type is everywhere, from popular search engines to less-known algorithms that help specialists comb through databases. This software usually incorporates forms of artificial intelligence that help it learn to make better decisions over time. But it bases these decisions on the data it receives, and if that data is biased in some way, the software will learn to make decisions that reflect that bias too. These decisions can have real-world consequences, influencing what music artists a streaming service suggests and whether you get recommended for a job interview.
“It’s now recognized that systems aren’t unbiased. They can actually amplify existing bias because of the historical data the systems train on,” said Ellen Voorhees, a NIST computer scientist. “The systems are going to learn that bias and recommend you take an action that reflects it.”
As a step toward confronting this problem, NIST has launched the Fair Ranking track this year as part of its long-running Text Retrieval Conference (TREC), which is taking place this week at NIST’s Gaithersburg, Maryland, campus. Proposed and organized by researchers from Microsoft, Boise State University and NIST, the track — essentially an incubator for a new area of study — aims to coordinate research around the idea of fairness. By finding appropriate ways to measure the amount of bias in data and search techniques, the organizers hope to identify strategies for eliminating it.
“We would like to develop systems that serve all of their users, as opposed to benefiting a certain group of people,” said Asia Biega, a postdoctoral researcher at Microsoft Research Montreal and one of the track’s co-organizers. “We are trying to avoid developing systems that amplify existing inequality.”
While awareness of the trouble that biased data creates is growing, there are also many different ways of defining and evaluating fairness in datasets and search tools. In order to keep the research effort focused, the organizers have chosen a particular set of data used by a specific search tool: the Semantic Scholar search engine, developed by the nonprofit Allen Institute for Artificial Intelligence to help academics search for papers relevant to their field. The Allen Institute provided TREC with a 400-gigabyte database of queries made to Semantic Scholar, the resulting lists of answers it returned, and — as a measure of each answer’s relevance to the searcher — the number of clicks each answer received.
The organizers also have elected to concentrate on a problem that often crops up both in commercial search engine results and in scholarly searches for research papers: the appearance of the same answers at the top of a list every time after running a particular search term.
One problem with academic searches is that they often return a list of papers by the best-known researchers from high-ranked institutions. In one way, this approach to ranking makes sense, as conscientious scientists want to show they have reviewed the most relevant past research before claiming to have discovered something new. Similarly, when we use an internet search tool, we might not mind if we see a well-respected company at the top of our search for a product we know little about.
While this result is fine if we are looking for the most popular answer, it is problematic if there are great numbers of worthwhile answers. To be sure, some of us do scroll through pages of results, but by and large, most people who use search tools never look beyond the first page.
“The results on that first-page influence people’s economic livelihood in the real world,” Biega said. “Search engines have the power to amplify exposure. Whoever is on the first page gets more.”
A fair algorithm, by the research track’s measure, would not always return the exact same list of articles in the same order in response to a query, but instead would give other articles their fair share of exposure. In practice, this would mean more prominent articles might still show up more frequently, and the less prominent ones less so — but the returned list would not always be the same. It would contain answers relevant to the searcher’s needs, but it would vary in ways that would be quantifiable.
This is of course not the only way to define fairness, and Voorhees said that she does not expect a single research project to solve such a broad societal problem as this one. She does say that quantifying the problem is an appropriate first step, however.
“It’s important for us to be able to measure the amount of bias in a system effectively enough that we can do research on it,” she said. “We need to measure it if we want to try.”
The Fair Ranking track is open to all interested research teams. NIST will make the official call this December for participation in 2020 TREC, which will take place from November 18-20, 2020, in Gaithersburg, Maryland.
Read the original news announcement at To Measure Bias in Data, NIST Initiates ‘Fair Ranking’ Research Effort
- U.S. Leadership in AI: A Plan from NIST
- Focusing on Artificial Intelligence? Seven Questions For Vendors With AI Products
Generative Artificial Intelligence and Large Language Model Use
ComplexDiscovery OÜ recognizes the value of GAI and LLM tools in streamlining content creation processes and enhancing the overall quality of its research, writing, and editing efforts. To this end, ComplexDiscovery OÜ regularly employs GAI tools, including ChatGPT, Claude, Midjourney, and DALL-E, to assist, augment, and accelerate the development and publication of both new and revised content in posts and pages published (initiated in late 2022).
ComplexDiscovery also provides a ChatGPT-powered AI article assistant for its users. This feature leverages LLM capabilities to generate relevant and valuable insights related to specific page and post content published on ComplexDiscovery.com. By offering this AI-driven service, ComplexDiscovery OÜ aims to create a more interactive and engaging experience for its users, while highlighting the importance of responsible and ethical use of GAI and LLM technologies.
Have a Request?
If you have information or offering requests that you would like to ask us about, please let us know, and we will make our response to you a priority.
ComplexDiscovery OÜ is a highly recognized digital publication focused on providing detailed insights into the fields of cybersecurity, information governance, and eDiscovery. Based in Estonia, a hub for digital innovation, ComplexDiscovery OÜ upholds rigorous standards in journalistic integrity, delivering nuanced analyses of global trends, technology advancements, and the eDiscovery sector. The publication expertly connects intricate legal technology issues with the broader narrative of international business and current events, offering its readership invaluable insights for informed decision-making.
For the latest in law, technology, and business, visit ComplexDiscovery.com.