On Friday Google admitted that it uses whitelists to manually override its search algorithms, more than a year after its European corporate counsel denied the existence of whitelists when defending the company against antitrust complaints in the EU.
According to Search Engine Land, Matt Cutts – the head of Google's webspam team – told a Silicon Valley conference that Google uses "exception lists" that prevent certain algorithms from affecting certain sites. There is no global whitelist, Cutts said, but for some algorithms, Google will make an exception for sites it believes have been wrongly demoted on its search pages.
Whitelists play a big role in the Google antitrust complaint brought by Foundem, a UK-based vertical search engine. Foundem is one of three companies whose complaints are being formally investigated by the European Commission. Foundem essentially vanished from Google in 2006, after Mountain View introduced a new algorithm that demoted vertical search engines. But some vertical search engines were not affected by the new algorithms. They were apparently whitelisted –even though the existence of whitelists was denied. Foundem produced emails from Google's AdWords team indicating that whitelists are used on search-ad algorithms. The company was in discussions with the AdWords team because it had also been demoted in Google's search ad system. Foundem fought its removal for more than three years, but it didn't return to Google's "organic" search until it went public with its story at the end of 2009. Foundem was granted an audience with Google's Search Quality Team, and the UK outfit has always said that it was then manually whitelisted.
Though the existence of blacklists is denied, Google clearly has blacklists in place as well – or at least something very similar to blacklists. Last month, Matt Cutts publically admitted that the company will manually demote sites."We have confirmed that Google's webspam team is willing to take action manually – for example, if we get a spam report, off-topic porn, things like that," he said.
Google spent years refusing to acknowledge that these manual interventions exist – and, in some cases, totally denying them. Google spent nearly a decade telling the world that its search engine was completely objective, and it has only recently begun to admit that this is not the case, presumably as a result of the EU investigation.
According to Search Engine Land, Cutts did not use the term whitelist. He called them "exception lists." But the two terms mean the same thing. "It is clear to us that Google no longer likes to use the term 'whitelisting'," Foundem has told us in the past. "But this was their word, not ours. In the context of lifting our AdWord penalty, Google themselves called it “whitelisting', and in leaked quality reviewer guidelines from 2003, Google used the same word 'whitelisting” when talking about immunity from search penalties.