Ok - I really couldn't make much sense of that article - but the general
premise seems to be that MSN Bot is crashing open source sites by spidering
them..or something like that. Err..I thought it was a spider and that was it
was supposed to do. A little bit of googling on MSN Bot would tell you that
- Its in beta. You cannot view the results yet(techpreview.search.msn.com
comes up now and then when they have a new release)
- A bit of googling on webmaster forums would also show that webmasters
grudgingly accept that the MSN Bot is pretty polite and follows robots.txt
religiously..and also supports extra directives where you (the webmaster)
can specify how frequently you should be crawled
But the article does deserve some reading - I've been reading it for 15 mins
and still can't make sense of it. A good example is the following para
<quote>
..One of the things I have discovered about Microsoft is how innovative they
can be. In my research and activities, I have uncovered some remarkable
things about a lot of companies and people. Would you find it surprising if
some open source advocates were on the payrolls of commercial companies?
What about discovering that all those articles about MSN search technology
competing with Google simply gave MSN a search bot for manipulating the
knowledge base of the Internet? The question is strictly one of conjecture;
just a question for the sake of argument and to stimulate discussion about
things people can do with technology..
</quote>
Huh?
Sriram
---------------------------------------------------
I blog at http://www.dotnetjunkies.com/weblog/sriram
----------------------------------------------------
-----Original Message-----
From: Bharathi S [mailto:sbharathi@xxxxxxxxxxxxx] ;
Sent: 30 September 2004 10:23
To: Indian Linux Users Group - Chennai
Subject: [Ilugc] Fwd: MSN Search engine in the battle to kill Linux?
On Thu, 30 Sep 2004, Frederick Noronha (FN) wrote:
---------- Forwarded message ----------
http://lxer.com/module/newswire/view/23328/index.html
I could not track the source of old content showing up atthe top of
google searches on my name. According to the number of hits Ito find out
generate, I have a significant mind share on Google. Finally, we
tracked down the problem and found that MSNbot was hammering and
multiplying links old and irrelevant content, pushing that above
recent articles about Linux in Government, case studies, etc.
I also found other open source advocates on the web trying
why they couldn't stop MSNbot using a standard robots.txtdirective in
the root directory of their web sites. Some of thecomplaints related
to increased fees for bandwidth.the content
In this article, I discuss the issue. I will also post the fix we
discovered by accident. BTW, a standard redirect will not work.
I'm not very pleased to discover Microsoft is manipulating
on the Internet.
Tom Adelsten
Bye :)
--
Bharathi S