New Scientist coverage of our [Weber and Castillo 2010] SIGIR paper

The New Scientist magazine featured our upcoming SIGIR paper; it will be presented in Geneva next week:

Demographic data can help, say Ingmar Weber and Carlos Castillo at Yahoo Research Barcelona, Spain. For example, they say that when US women type in the search term "wagner", they are most likely to be thinking of the 19th-century German composer. US men, on the other hand, may well be thinking about the makers of spray painters.

By giving a search engine some basic demographic information, such as age, gender and educational background, it is possible to boost the engine's chances of identifying user intent correctly, say Weber and Castillo. That personal information can be gleaned when people sign up to the other services, such as email, that search engines provide.

To check their theory, the researchers analysed data collected from Yahoo account holders through its search engine over a 12-month period. They then identified ambiguous search terms by looking at searches for which the top few results concerned wildly divergent concepts, and recorded which result the user chose to click on. By running those ambiguous search terms through their demographically modified search engine, they managed to get the chosen link to appear as the top-ranked result 7 per cent more often than in the standard Yahoo search.

Ingmar Weber and Carlos Castillo: The Demographics of Web Search. SIGIR 2010.

Update 2010-07-13: the paper was also posted in Slashdot.

Update 2010-07-14: the paper was also recommended by The Economist (free exchange blog).