Google explains why all sites may harm your computer

This is Google’s response for informing all their users that every site on this world wide web is harmful to their computer. “Very simply, human error,” they confessed. It’s a huge mistake and definitely shaken people’s confidence a little. But by being truthful about the whole incident without using the word “whoops” (like Dreamhost) is good PR still.

“This site may harm your computer” on every search result?!?!

What happened? Very simply, human error. Google flags search results with the message “This site may harm your computer” if the site is known to install malicious software in the background or otherwise surreptitiously. We do this to protect our users against visiting sites that could harm their computers. We maintain a list of such sites through both manual and automated methods. We work with a non-profit called StopBadware.org to come up with criteria for maintaining this list, and to provide simple processes for webmasters to remove their site from the list.

We periodically update that list and released one such update to the site this morning. Unfortunately (and here’s the human error), the URL of ‘/’ was mistakenly checked in as a value to the file and ‘/’ expands to all URLs. Fortunately, our on-call site reliability team found the problem quickly and reverted the file. Since we push these updates in a staggered and rolling fashion, the errors began appearing between 6:27 a.m. and 6:40 a.m. and began disappearing between 7:10 and 7:25 a.m., so the duration of the problem for any particular user was approximately 40 minutes. (Source: Google Blog)

I was a little troubled yesterday and had to use Yahoo for searches but I wasn’t too concern. Here’s StopBadware’s side of the story.

Why Ubuntu is better than Windows XP

Here’s a screenshot of a website for Windows XP:

Trojan in Windows XP

(Trojan in Windows XP.)

There is an animation of a Windows scanning utility discovering lots of trojan, an anti-spyware tool would be offered at the end. It tricks users into install something that they claim secure.

Here’s the same website in Ubuntu with the same animation:

Trojan in Ubuntu

(Trojan in Ubuntu.)

The screenshot would probably look realistic to a naive Windows XP user. And when you install the additional checking tool, who knows, you may end up with a virus.

I tried installing it in Ubuntu and couldn’t. Me sad.

[Thanks Irene for showing me the site.]

AVG LinkScanner pretends to be IE6, screws up analytics

Speaking of Internet Explorer, AVG has been disguising as Internet Explorer to visit websites. Web developers and webmasters aren’t too pleased. AVG’s LinkScanner is estimated having to be downloaded by more than 20 million people. The LinkScanner attempts to disguise itself as a real live human click claiming itself to be Internet Explorer 6. It just screws up web analytics.

AVG disguises fake traffic as IE6

…webmasters who rely on log files for their traffic numbers may be unaware their stats are skewed. And others complain that LinkScanner has added extra dollars to their bandwidth bill.

…(Paid AVG) appears that scans now use these agents as well:

  • Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
  • User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
  • User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)

…the first agent is by far the most common. Which is bad news for webmasters. That’s also the Internet Explorer 6 user agent. Unlike the other two – and the original “1813” agent – it’s a perfectly valid agent that may turn up with real clicks.

AVG’s chief of research Roger Thompson says the for-pay LinkScanner is only using the IE6 user agent

In an effort to fix this problem, one web master advocates redirecting AVG scans back to AVG’s site. “Many webmasters simply tell LinkScanner to scan AVG’s site instead, so their site gets marked as malware free every time – while AVG gets handed the extra bandwidth cost,” says the webmaster of TheSilhouettes.org.

But this assumes that AVG is using a unique agent. And at the moment, it’s not. The send-it-back-to-AVG method may redirect legitimate clicks as well.

Which gets to the heart of the matter: AVG’s security philosophy is fundamentally at odds with webmaster peace of mind. The company wants to scan search results, and it wants to scan them in a way that’s difficult to distinguish from real traffic. “In order to detect the really tricky – and by association, the most important – malicious content, we need to look just like a browser driven by a human being,” AVG chief of research Roger Thompson has told us.

And if that causes problems for webmasters, Thompson says, so be it. “I don’t want to sound flip about this, but if you want to make omelets, you have to break some eggs.”

Clearly, the company doesn’t fully realize the importance of web analytics.

“In order to make an omelet you have to crack some eggs. But a good omelet has cheese, ham, peppers, mushrooms and all sorts of other ingredients which AVG seem to have forgotten about.”

But AVG continues to say it’s working to solve the problem – including the bandwidth issue. Referring to LinkScanner’s new IE6-like user agent, Thompson told us, “We intend to leave those in place until we can find the right balance point which will allow us to continue to provide the best possible protection for our customers, without imposing too much extra bandwidth on websites.” (Source: The Register)

[ad#highlight]

I was reading what the chief of research had to say – “If you want to make omelets, you have to break some eggs.” Sir, that’s just a nonsensical comparison, when you break eggs, you don’t get your neighbors to pay for them.

Bandwidth is a clear issue. It is not free. This solution is another example of how innocent parties are penalizes just because of a tiny number of visited sites infected with malware.

(By the way, I go around disguised at GoogleBot as my user agent. But that won’t hurt your bandwidth, I have reasons for doing so.)