Rockwell, 1922
~ Combing lore ~

searchlores.org
         Petit image    Combing for
information
Version March 2002
[Recent essays]
[What is combing?]
[The "double combing" approach]
[An old case study]
[Combing resources]
See also the ad hoc section [Combing resources]


to basic
Back to basic
  
No_commercial!
Searchers against Smut
  
Petit image
Back to advanced

Some (more or less recent) essays about combing

  1. [whitemea.htm]: Proxy Logs - The Other White Meat, by Finn61 part of the combing [section]
    "So now you should have some large lists of URL's you can scan for that hard-to-find document or program"
    March 2002
  2. [Web wizard searching techniques, anti-advertisement galore and software reversing tips], a draft of fravia+'s session at HAL2001 in August 2001 (see the 'how to search' part).
  3. [Combing: The art of sailing in pure water] by Loki, October 2000 (A little of "methodology" about the information)
  4. [Simple combing techniques] by Fravia+, October 2000 (part of a conference held in Milan for the Linux day)
  5. [The importance of Webrings for combing purposes], by Lorenzo Gatti

Finally, you may find useful to peruse my obsolete 'lesson' [combing and klebing techniques] (November 1997!)
Visit (and use) the [Combing resources] page as well!
What is combing?

Combing is a very effective search strategy: basically, instead of simply searching, you search those that have already searched. This will give you a quick 'jumpstart' possibility.
Let's begin with the beginning: usually a good seeker does not search directly a specific target: you search people that have already searched the web for years for that target. The web is so big and deep that you'll always have some weirdo that has spent three years of his life cataloguing all possible variants of the Yak2 russian fighter plane, if you see what I mean.

Note that if the target has enough signal-power, you may even search among the noise for people that have searched people that have in turn searched for that specific target... :-)

That's combing, in a nutshell.
You may usefully comb on usenet or on the thousand many private messageboards dealing with your target stuff or on private homepages, or on ad hoc webrings, or some useful referrals lists, or applying klebing (i.e. referral based), or luring techniques. You may have to recur to social engineering as well. Stalking maybe an important option too, and you may have to put on the web some clever "honeypots" to stalk your targets through a klebing approach.
You may comb directly or you may use combing bots or [scrolls]
You may also use various older net resources like the continuously updated "Top 100" or "Top 1000" URL-locations; all kind of ftp searches and the various "vigilant filters" and automated server loggings.

Obviously combing is an important technique for whatever interest you may have, quite useful in order to spare an incredible lot of Internet searching hours.

Combing is an ART! Read the essays
The "double combing" approach

As Jeff realized and pointed out, simple combing techniques can give incredibly accurate results.
I think if a person really thinks about this and puts together some good keywords you can really find some terrific links to info thru BOOKMARKS ...and ALL THE WORK has already been done for you!......all with headings and sometimes alphabetized...:)
Just look at the following example: google... search... bookmarks fravia...
http://www.cs.umass.edu/~lmccarth/bookmarks.html ....mostly lots of info on crypto...
Indeed this kind of very simple combing approach (a combing querystring on a local search engine) can give impressive results. Try it out (here for instance: bookmarks proxies on crosswinds' homepages) and enjoy this kind of fishing right now! Yet you'll discover more advanced combing techniques reading the following...

[Combing resources], originally compilated by Rumsteack in February 2000

An old "case study": combing commercial smut depots
death to the pornodealers

Warning: you better set the option "autoload images" OFF inside your Netscape settings, else you'll pretty soon regret having accessed this kind of sites... you will not loose anything... NONE of the images they carry is worth loading... should you really want nice "sexually explicit" images (for free of course, and please excuse the pathetic euphemism), then visit the many artists that expose their own work on the net... on the sites we are going to destroy (see how in the CGI-reverse engineering page) you'll not even find any "pornography" whatsoever, only fetid smut.

Let's start with a typical "combing" approach, I will not hyperlink because I do not want this site spidered along these links, but you may cut and paste the following URLs: 
Top1000 counter
http://www.hitbox.com/wc/world2.html          ;TOP1000 "normal", example for useful combing
http://www.hitbox.com/wc/adult.html           ;TOP1000 "adult", main entrance
http://www.hitbox.com/wc/top10.adult.html     ;top 10 smut commercial
http://www.hitbox.com/wc/top2100.adult.html   ;top > 2100... understand the "name" 
                                              ;approach




Webcounter
http://www.digits.com/top/both_adult_100.html ;top site has here "only" 540000 a day
http://www.digits.com/top/comm_adult_100.html ;top site has here "only" 124000 a day




Etcetera... you understand the trick now... here are some other ones
various smut counters:

http://www.xxxcounter.com/home/ 
http://www.web21.com/
http://www.sexhound.com/index.cgi?from=16818			this one uses CGI! :-)
I do not want this page catalogued inside the smut information retrievers, therefore the above links are not hyperlinked... cut and paste them in order to use them.

For combing purposes you may also use:
1) the usual search engines (which give incredible results at time!)
2) ftp search, looking for "hidden" subdirectories with relevant names
3) the "big page provider" search engines (like the ones on geocities or mygale)
As you can see from the above short information,
1.1) many "counters' statistics" betray quite a lot of useful information... if, for instance, you are interested in jellyfishes (it's an example!) you would be well advised, instead of searching the web for ages, to have a quick look at all the pages that inside the counters' statistics, fall under the counter's main categories "biology" or "science"... pretty soon you would find the "golden link" you were looking for...
1.2) We need MANY addresses of SMUT dealers in order to find the many that utilise a CGI-script (or other attackpoints) in order to know "from which site" they got the query... as you'll see on the redcgi reverse engineering page of this section, this opens the way to their doom!
2) As anybody that uses redftp search already knows, the ftp search approach (that fishes hidden directories) can fish incredible (if tricky to interpret) results.
3) For other combing purposes (not for smut dealers, of course) you may use also the search systems specific to the big free pages providers...have a search at redhttp://www.geocities.com/search/ and you'll understand what I mean
Combing on the Usenet
(See the ad hoc usenet search page)
Usenet combing can work "on the fly" or "regularly" through the "Vigilant" filter at
filter@vigilant.bc.ca
I'll show you for instance one of my queries:
FIND how-to-search tutorial manual
		NOT spam
		NOT top position
		NOT advertising
		MAX 8
Such a query would give you useful information about "searching techniques" on the Web, you may of course construct how many queries you like and *register* (for free) by the vigilant filter, in order to get the results of your usenet queries emailed to you every day or week or month.

Usenet query can also be done through the two big Usenet "depots": Dejanews and email query, that are explained elsewhere on my site. Many of the main red search engines allow such querying too, using the services of either Dejanews or emailquery.

Good luck, good hunt!
to basic
Back to basic
  
No_commercial!
Searchers against Smut
  
Petit image
Back to advanced

(c) 2000: [fravia+], all rights reserved