[AG-TECH] Mailing List Search Engine

[Hiroyuki Komatsu]

Thank you!

Then I'm getting emails every one second (i.e. wget -w1).
After updating the search engine, I'll annouce to this ML.

--
Hiroyuki Komatsu

At Wed, 2 Oct 2002 12:10:53 +0100,
Richard Lawley wrote:
> You can make wget ignore the robots.txt file by placing robots=off in
> your .wgetrc file (from the wget manual:
> http://qcd.nersc.gov/utilities/wget.html#SEC29)
> 
> Richard
> 
> > And I have been failed to get new emails from an ANL web site
> >
> (http://www-unix.mcs.anl.gov/web-mail-archive/lists/ag-tech/maillist.htm
> l)
> > using by 'wget' command, because of restriction by 'robot.txt'
> (http://www-unix.mcs.anl.gov/robot.txt).
> > 
> > I'd like ANL admins to permit me getting the emails by 'wget' or tell
> me another site which does not restrict 'wget'.
> > 
> > -- 
> > Hiroyuki Komatsu
>