The dark world of search engine manipulation

When it comes to directing people’s attention, there has never been anything as powerful as today’s vast online search engines, and when it comes to search engines, nobody can touch Google, whose sites handle an estimated 88 billion queries a month, roughly two-thirds the world total.

Users don’t have any idea how Google decides the order in which it presents search results, which is too bad, since that ranking is the most consequential thing Google does.

That’s because search engines may look far and wide, but their users do not. If your company doesn’t show up at or near the top of Google’s results, it’s invisible.

A survey last May by the online advertising network Chitika found that the No. 1 search result drew over one-third of all traffic the results generated—twice as much as No. 2, three times the traffic of No. 3. Being on the first page of rankings was critical. Even No. 10, at the bottom of page 1, drew nearly two and a half times the traffic of No. 11, at the top of the second page.

So any retailer that wants to reach customers online cares intensely about its rankings, and is eager for ploys to ensure prominence. Hence the business of SEO—search engine optimization.

SEO is focused on figuring out Google’s rankings and giving Google what it’s looking for. Which is what? Only Google knows, and its search methodology is about as widely shared as the formula for Coke.

What is known is that Google puts great weight not just on traffic flows, but on how well regarded a particular site is, and tries to measure that regard by calculating the number, and to some extent, the quality of other sites that link to it. Google likes to think it is reflecting some prevailing judgment of a site’s value.

But that judgment can be counterfeited. One extraordinary instance, uncovered recently by The New York Times, involved JC Penney, the venerable Main Street retailer. Apparently, for months Penney was the top-ranked site if you searched for terms as disparate as “skinny jeans,” “home décor,” “area rugs,” “dresses,” and “table cloths.” Penney even pulled more traffic for “Samsonite carry on luggage” than Samsonite’s own site.

How come? Penney’s success was traced to an SEO consultant who had, essentially, contracted with more than 2,000 web pages that had no discernible purpose apart from linking to sites like Penney’s—for pay. Penney’s fortunes rose thanks to this virtual ballot-stuffing.

In a second case, The Wall Street Journal reported that retailer Overstock.com had been caught offering discounts to college students and faculty for linking to Overstock from various search terms, among them “gift baskets” and “bunk beds.”

Overstock’s rankings soared because the links came from sites with the “.edu” suffix reserved for schools. Google apparently assigns great weight to .edu sites, since they rarely link to commercial entities and their endorsements are thought to be especially credible.

Google’s response in both instances was terrible and swift. It took undisclosed measures that, in Penney’s case, led to its average position for 59 search terms plummeting from 1.3 to 52 within two weeks. Overstock had been at or near the top for dozens of keywords, but within days had plummeted to the fifth or six page of results, the functional equivalent of vanishing.

The tales are disturbing on several counts. First, the vulnerability of the rankings to manipulation. With 300 million domain names to police, Google is preposterously outgunned.

Second, the quiet, unchallengeable ferocity of the response. You don’t have to support fraud to agree with the Times reader who posted: “Was anyone else spooked by Google virtually eliminating a company from existence by removing it entirely from search results?”

Third, the non-transparency of the whole search business. What could be more opaque? The retailers’ actions certainly seem wrong, but says who? If the principles guiding search shape public awareness in sweeping ways, shouldn’t we know what they are?

Besides, Google itself routinely features its own spinoffs—Google Product Search, Google News and YouTube—high up on its results. Is that OK? Why do I get three “Google Maps” links on page 1 when I type in “driving directions?”

Now Google is incorporating recommendations from your social media “friends” to personalize the search results you get. Who authorized Google to help itself to that information? And precisely how will your so-called friends’ opinions alter the rankings you see?

Google is an extraordinary company, and its credo of “don’t be evil” is impressive. But it’s difficult to think of another private, profit-seeking entity that has ever exercised such vast power over what the world thinks about and pays attention to.

That’s a profoundly public function, and with it comes an obligation of accountability that Google has yet to shoulder.

-30-

About these ads

6 responses to “The dark world of search engine manipulation

  1. Excellent column Ed. Surveys very nicely the nexus of things and players we see evolving before our screen-fixed eyes– of technology, governments and the private players: corporations and consumers.

    Your article got me sucked into trying to make a diagram/list of it:

    TECHNOLOGY–WWW
    -Web technology (current constraints & options)

    COMPANIES
    -Quasi-monopoly power of dominant search engines (Google)
    —their search algorithms and susceptibility to ‘gaming’ by others
    —their “reputation” and effects on consumer use habits on ad-accessible IT-Web
    -Other companies’ ability in ‘gaming’ the system and need to use the quasi-monopoly arrangement to succeed in business
    -Media companies who envy the current quasi-monopolist (Google) and their potential routes to challenge the today’s dominance by Google.

    CONSUMERS
    -for the advertising purposes of business
    —consumers’ eyeballs & other attention-directing organs
    -what is their behavior and who has all the data (and which part of available information on usage is privately controlled by corporations)
    -what consumers (citizens) really might seek: want?/need?/dream of/be willing to pay for? etc. (people as citizens–as well as ‘consumers’–remains the ultimate focal point for some of us)

    GOVERNMENTS
    -existing rules of the game(s) about ownership/control & business practices allowed or prohibited/monitored in USA and other countries
    -competition/conflicts among the above players over additional legislation in big economies (too big to ignore for biz & pattern-setting for other countries)

  2. Mr Wasserman,
    A small correction to your excellent article, Google doesn’t assign any additional weight to .edu websites. I realize the source of your information was the Wall Street Journal, and I have contacted their writer as well to correct this. I am an SEO specialist from Minnesota, and our agency has does SEO for some of the largest companies in the world.

    I believe your source was this line in the WSJ article:

    “There is big money in .edu links because they are ‘trusted sites’ in Google’s eyes,” said David Harry, president of Reliable SEO, a SEO specialist based in Canada.

    This is not true.

    Take a look at this video by Matt Cutts, probably the most famous of Google employees aside from Larry Page and Sergey Brin. He is head of the Google Webspam team, and is the best source for information about SEO and Google’s policies.

    At exactly one minute, he mentions that the value of an .edu website (education site) is the same as a .com (commercial) website.

    While this was supposedly the case in the past, this is not true today, and in fact that YouTube video was posted on Jan 14, 2010.
    SEO is a changing tactic, anything printed is ancient history, anything 6 months or older on blogs is modern history and more current is present time. I realize this makes it very difficult to report on, however Google is the best source for updated information. Also if you have an upcoming article on SEO, please send it to many SEOs for opinions. This is similar to reporting on the economy, a well recognized name in economics can be a great source, however everything that even the most famous of economists mentions will be debated by other economists. That is just the nature of the topic.

    If you require a list of SEO experts, please let me know. There are not that many and they are well recognized in the field. Mr. David Harry is not on that list as a well recognized professional at this time.

    Thank you,
    Alexander Pokorny
    Three Deep Marketing

  3. edwardwasserman

    Thanks for this. As Spock would say, “Fascinating.” I don’t pretend to understand even a fraction of the logic that guides search engines, but I do see a basic fallacy in the underlying premise they operate from: That they detect and ratify a judgment about the value and popularity of a site that arose spontaneously from the Internet, independently of their own influence. The fallacy is that the single most powerful determinant of a site’s popularity is its search engine ranking. Seems to me there’s an inescapable circularity to the logic that the search engine designers no doubt wrestle with but can’t fundamentally change.

  4. Pingback: What to do When Your Online Reputation is Maliciously Attacked | Just Another Wordpress Blog

  5. Nice post. I was checking continuously this blog and I am impressed!
    Extremely useful info specially the final part :) I handle such info a lot.
    I was looking for this certain information for a long time.
    Thanks and good luck.

  6. excellent points altogether, you just gained a new reader.
    What may you suggest in regards to your put up that
    you just made some days ago? Any positive?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s