~ Bots lore ~
         Petit image    Bots
lore
Version January 2005
  
BOT WRITING, BOT TRAPPING & BOT WARS

This is a 'living' workshop on bots trapping and reversing, you will find elsewhere on my site other "broad" site protection (and de-protection :-) lore. As ~S~ deep writes in his bot-essay: "There are many Perl bots available on the net, but I'm fairly certain that you will not find one that does exactly what you want. There's also a "convention" amoung bot writers not to give bots source code to people who do not understand them - it's considered irresponsible. Of course, once you've learned how to build bots, you can be as irresponsible as you like". This is exact. Anyway we will find more knowledge only if we create it ourself at the same time. Your own contributes and work are necessary. The material presented here should be more than enough to "get you started". Write your own bots, publish the code so that others may improve them. Reverse the code of the existing bots. Awaiting your own contributions
Skip introduction and go directly to
[Iliad fetchbot]   [Juno autoresponder]    ["our" essays!]   [our friends' essays!]

An introduction - explanation for those that do not know...

The term "bot" is, according to DeadelviS, a short for "robot", which sounds much cooler than "program"

As Andrew Leonard explains, like mechanical robots, bots are guided by algorithmic rules of behavior - if this happens, do that; if that happens, do this. But instead of clanking around a laboratory bumping into walls, software robots are executable programs that maneuver through cyberspace bouncing off communications protocols. Strings of code written by everyone from teenage chat-room lurkers to top-flight computer scientists, bots are variously designed to carry on conversations, act as human surrogates, or achieve specific tasks - such as seeking out and retrieving information. And... bots can also be used as weapons.

These pages of mine may regards all sort of web robots: spiders, wanderers, and worms. Cancelbots, Lazarus, Automoose. Chatterbots, softbots, userbots, taskbots, knowbots, mailbots. MrBot and MrsBot. Warbots, clonebots, floodbots, annoybots, hackbots, and Vladbots. Gaybots, gossipbots, gamebots. Skeleton bots, spybots, and sloth bots. Xbots, meta-bots. Eggdrop bots... as you can see the terminology is far from being simple... basically, though, the idea is to allow you to learn enough to WRITE your own search bots. Information searching through commercial engines is less and less efficient, as you'll soon realize it is of paramount importance, for a ~S~, to learn how to bulid his OWN 'home made and tailored" specific bots. You'll be amazed at the whealth of really strong signals you'll be able to find among the noise as soon as you use your own bots, even if they will be imperfect and surely wont be 100% state of the art...

It's up to you to help us with your own work or to keep what you'll dvelop for yourself: it is my intention to offer you enough material on this page to allow you to start. Choose the konwledge path or choose the dark path, it's up to you.
Since I'll NEVER charge money for accessing my site I would like to ask you the only "money" that's worth something on this web of ours: knowledge for all!

I want you to contribute with YOUR knowledge!
Remember, if you build on other people's shoulders, you must offer your own shoulders for others to build upon!

The powerful Iliad fetchbot


Hey! I wanna see a real bot in action before joining this section!

Yessir! Here is a good and powerful 'fetchbot', very useful for seekers and searchers alike. And if you knew nothing of this stuff you'll be fascinated (and even if you already knew... :-)
You'll now (at once if I were you) approach the "iliad" Searchbot (a very useful one, btw, was at iliad@algol.jsc.nasa.gov, is now at iliad@prime.jsc.nasa.gov):
Send an email to:
iliad@prime.jsc.nasa.gov
write into the SUBJECT part of your email (into the subject field, duh!):
iliad query
write into the TEXT part of your email (that's your letter, duh!):
?Q: internet bots automated retrieval
(for instance... and you'll -most probably- get quite a lot of interesting material about bots from this mighty useful Searchbot... whatt'd'ya say?
If you're stuck email the same address with the word help both in Subject and in text (a pretty poor help will you get :-(



The powerful Juno autoresponder


Hey, this is great! I wanna taste another email-bot, just for fun!

Yessir! Please go ahead: have a look at friend autobot: Send an email to:
autobot@junoaccmail.org
write into the SUBJECT part of your email (into the subject field, duh!):
send index
or if you want to have a laugh at some 'scarecrow' copyright propaganda, write - always in the SUBJECT field - the following:
send Copyrights

Hey, this is gorgeous! Now, before I start working on my own, let me please see and touch the code of a "real" bot!
Yessir! Please go ahead: enjoy the following essays!
All the code you may need to start working on your own!

Essays: how to build your own bots
PHASE ONE (16 July 1999)

this essay (perl_es1.htm): Perl@usa.net ~ How to reverse a "free" service has been written by [blue] in July 1999 for the removing banners section, read and enjoy, let's hope you'll write afterwards your own perl-bots and send them here so that others can ameliorate and give feedback...
PHASE TWO (22 July 1999)

this essay (rt_bot1.htm):The HCUbot: a simple Web Retrieval Bot in Perl has been written by deep in July 1999, read and enjoy! Let's hope you'll write afterwards your own perl-bots and send them here so that others can ameliorate and give feedback...
PHASE THREE (14 September 1999)

this essay (botcgi.htm):Mirbot 1.0: a very special kind of a Robot has been written by The Mystical Friend in September 1999, read and enjoy! Let's hope you'll write afterwards your own perl-bots and send them here so that others can ameliorate and give feedback...
PHASE FOUR (14 September 1999)

this essay (rt_bot2.htm):The HCUbot (Version 2.0): a simple Web Retrieval Bot in Perl has been written by deep in July 1999 and updated and ameliorated in September 1999, read and enjoy! Let's hope you'll write afterwards your own perl-bots and send them here so that others can ameliorate and give feedback...
PHASE FIVE (21 September 1999)

this essay (sono_bot.htm):spider.r: a handy search tool and intro to REBOL has been written by sonofsamiam in September 1999, read and enjoy! Let's hope you'll write afterwards your own rebol-bots and send them over here so that others can ameliorate and give feedback...
PHASE SIX (March 2000)

[ftpbot1.htm]: A small ftp fetcher bot
by DarkWyrm
This bot searches a FTP site for a particular file (in Perl)
PHASE SEVEN (May 2000)

[plbtgrab.htm]: Source code for a spam bot (Kevin's spider) (in Perl)
by Kevin Jobson
Automatical link searching
PHASE EIGHT (September 2000)

[scan_reb.htm]: A simple REBOL scanner ways to retrieve hidden files, pages, zips, images
by -Sp!ke
Automatical link sniffing
PHASE NINE (October 2000)

Check [mysearch.zip]: ~ 20233 bytes A search bot in Visual Basic by Shoki (see [shokiwcd.htm])
PHASE TEN (February 2001)

Check [wf_add.htm]: Adding engines to WebFerret by Laurent (The guts of a search engines parser) Advanced
PHASE ELEVEN (April 2001)
[perlbot.htm]: HOW TO FOOL SSL DOWNLOAD OBSTACLES (spelunking into https "secure" servers)
by DigJim, Very Advanced essay
PHASE TWELVE (April 2001)