The web is huge, of course, but that's not a real problem. The problem is that time
and space have
a different meaning on the web. Add to this the truth that everything that 'happens'
is carved forever: try to pull something 'off the web' and you will soon realize that
you wont be able
to do it. Everything you write and publish will defy eternity, carved in electrons:
the very moment
you put something on the web, someone, somewhere, will make a copy out of it. It is
bound to reappear, somewhere sometime: indestructible and redoutable powers of the
void.
Time is different too. As anyone that has real web-experience knows, it happens that
something you wrote, or published, remains unanswered - and apparently uncared of -
for months, or years... and then,
all of a sudden, when you almost forgot it yourself, a dozen persons begin
contacting you
out of the void, with an enormous and for you inexplicable
interest in what you wrote so long ago.
There are in fact micro-communities, working along similar paths,
that ignore reciprocally their existence for an inexplicable long time. This has
of course to do with both the vastness of the web and the fact that people do not
know how to search.
Furthermore, the web is truly international, in
a depth that even those that did travel a lot (mostly) underestimate. Some of the
readers interacting with you may have problems, ideals and aims so 'alien' from your point
of view that you cannot even hope to understand them. On the other hand this
multicultural and
truly international collaboration may bring some sparkles of fresh breath
in a world of cloned
Euro-American zombies who drink the same coke with the same bottles, wear the same shirts,
the same shoes (and often enough the same pants),
and sit ritually in the same McDonald's in order
to perform their compulsory collective and quick "reverse shitting".
The web is truly huge: at the moment there should be around 32 milliard/billions (indexable)
pages on the web, according to
most (self-proclaimed) experts. Less than one half of thel are indexed by the main search engines How many there really
are is anybody's guess.
Moreover there's an "invisible" (or "deep") "hidden databases"
web (made out of
dynamic, not persistent, pages,
the content of which can only be found by a direct query) which could be up to 500 times bigger than the "visible" (or "surface") web.
Fact is that the main search engines (in this very moment, on
a continuously moving landscape, Google, Yahoo, MSN and Gigablast) cover at
best one third
of the indexable web, and probably far less. The pace of growth being amazing as well,
there is no hope that
the main search engines will ever be able to cope with the www.
Hence search engines are not enough (by a long shot not enough)
in order to search this huge bulk
of scattered information,
therefore different methods MUST be used to search the web.
But for searchers the huge number of sites and the incredible pace of growth are
not motives of despair. In fact
even if the web will continue to increase with the same incredible pace it has registered in the last
years, its DIAMETER will remain low. Thus searchers will always be able to find what they
are looking for, provided they know how to search and how to program their own
[bots.htm] and
[scrolls].
At the moment the diameter of the web is still around 19 'clicks'.
Since the average number of links per page
is seven (on some sites you'll find hundreds of links, on some others: none), given the
presumed dimensions of the web you should be able to hop from any Internet
site to any other one using - on average - just 19 clicks.
NOTA BENE: This limit will not increase
much with the growth of the web (there's a logarithmical correlation): it may
increase to a maximum of 21 or 22 links.
Therefore (this is the important corollary):
a seeker that knows his skills will always be able to find what he's looking for in
a relatively short time, no matter how big the web is or will be, this being
the "first optimistical law of seeking" :-)
This said, the real chances that you can reach at all (I mean, not "on average")
- from any random site -
any random site clicking only forward are just around 25%.
The structure of the web, from a searcher's point of view, is - to say
the least- pretty weird, as you'll see in the picture below.
Yet the structure of the web is of paramount
importance (from a searcher's point of view, eh) in order to decide which techniques
you should apply when searching and where i.e. in which areas.
Imagine a huge bulk of around 7/8 milliard (billions) pages,
all mutually interconnected. This is the 'Nucleus', the very CORE of the web,
a bulk of strongly interconnetted pages, sites,
usenet newsgroups, messageboards, you name it.
This is the 'web' everyone knows, where you
happily browse from link to link smearing all your personal data along,
as we'll see later (and counter) in the '[Anonymity] ' labs. It is not easy
to represent the web threedimensionally. The Nucleus is far from being a compact and uniform 'ball' of mutually interconnected sites, you should think at a
fractal like entity, with almost 'organic' features, with spaghetti-similar
'tubes' that quickly
connect some areas while leaving 'link-holes' in many places, it would probably
look like a chump of Gruyere (the commercial advertisement banners being the cause of its
bad smell :-)
This image I have made could be of some help...
As you can see there are four main different areas: 'bulk', 'hidden databases', 'outside linkers'
and 'outside linked'.
Different techniques are used to access these different areas.
An important area of the web is made of
hidden databases. These are pages that the Nucleus points to, and that may
(or may not) point back to the Nucleus. Yet
for commercial (or other access-restrictive) reasons visitors of sites located here
are supposed to
"pay" (or adhere to some "clan") in order to access them. As you may imagine, these pages
are NOT mutually linked. Fortunately (for us, unfortunately for the commercial bastards)
the web was originally built
in order to share (and neither to hoard nor to sell) knowledge.
And thus the building blocks, the
"basic frames" behind the
structure of the web are still the same. If I may dare a comparison: exactly as it is
pretty easy
to break any software protection written in a higher language if you know
(and use) assembly,
so it is easy to break any server-user delivered barrier to a given
database if you know (and can outflank) the protocols used by
browsers and servers.
As a result let's simply say that it is
relatively easy to access all pages in this area reversing the (simple) perl
or javascript tricks used to keep them "off limit" for zombies and lusers. (You wont
even have to recurr to common exploits à la
"politically correct" :-)
The Nucleus "points" to another area of the web, the "outside linked". The sites in this area
are linked from the Nucleus but do not point back to it. A simple example are the elements of
a
database of images, linked from the Nucleus but not necessarily pointing back to it.
This part of the web, made mostly out of 'storage clumps' and
"non hidden" databases (but not only),
can be searched
and combed with the same searching techniques that we usually
apply when searching the Nucleus. These pages are "outside" the nucleus, yet not
particularly difficult to find.
Like matter and anti-matter, to the "outside linked" pages we spoke of above, correspond
an inversed related part of the web: the "outside linkers" pages. Indeed
all the pages located in this specific area
of the web do "point" to the Nucleus but are not pointed back from it. Imagine as
an example the personal links page of a scientist: lotta interesting links to
the Nucleus yet no need to publicize its existence. A page with information you
may need is there, somewhere, without any
link whatsoever that could bring you to it. Indeed there are no links back from
the Nucleus to these pages.
The "outside linkers" are a part of the web you cannot reach using
"normal" search techniques, since no link whatsoever points to them. Yet they may hoard
knowledge you need. There are, fortunately, some techniques that you can apply in order
to find them, the most simple and common one being 'klebing' (using referrers and luring
techniques to find them).
Anyway, first of all, read on and learn how to seek!
You'll always find what you want
And even when all your searching techniques have failed,
when all your cunning approaches did not
catch anything, your lonely searches seems endless, when all your tricks have brought
you no reward...
even when a rude database dares deny you access,
even when your target has been pulled off the web, jailed, destroyed, censored, annichilated
by the powers that be... even in those
dire moments you will always know that you can find what you are looking for even if it is no more there!
Yes, as strange as it may seem to you, this is the
"second optimistical law of seeking" :-)
How do you find your disappeared target? You travel trough time.
How do you travel trough time? You take advantage ofthe fact that EVERYTHING THAT HAS
BEEN PUT ON THE WEB ONCE WILL LIVE ON COPYCATTED ELECTRONS FOR THE ETERNITY, either trough
the usual & well-known public '[time machines]' (basically huge
caches or 'photosnaps' of the web at a given time),
or through more 'gray' (and difficult) channels and alleys
that you'll discover and take advantage of in due time.
Anyway be assured: you are going to be a seeker, you will be able to
find everything, even targets that do not exist any more.
Given that everything you put on the web will live forever on copycatted electrons,
we can derivate a third law...
Plagiarism and Anti-plagiarism
A
Plagiarist
is someone that 'appropriates'
informations or thoughts or data he finds on the web without giving
due reference to the original Author of the data / thoughts / information he is using, trying to
'give the impression' he is the real Authors of such data / thoughts / information.
Unfortunately (for the Plagiarist) on the web it is as easy to plagiate as it is to find out
the plagiarism, using ad hoc searching techniques.
In fact the web is incredibly 'stiky': try searching for the first line of this very page: "Knowing how to find anything you want on the web, once you will have learned it, will give you power."
You will most probably (on the slippery web you may always 'slip' :-)
landhereagain.
This is the reason those that create content should never fear to give it out for
free! A correct 'restitution' of the paternity of any given snippet is possible using
ad hoc searching techniques (like the ones regarding the 'disappeared web'
we have pointed to in the previous chapter).
This is the reason we don't need to fear being plagiarized by
self described 'Super searcher experts', 'internationally respected Internet
trainers' and/or 'Revered Authorities
on Web search engines' nor any
other of the commercial hot baloons that abound on the web (And that you should
learn to evaluate (or rather 'devaluate' :-)
on your own: they will NEVER be
as able as we are in providing NEW content. It would take them a whole life of (heavy)
studies just to reach those levels we have already abandoned. Besides I firmly believe that
the very moment you cease to do something for fun, joy or interest and you begin doing it
for 'money', your value decreases... :-)
The above explains also - incidentally - the reason you may (and should) want to
contribute to THIS site:
your work will never be plagiarized without being discovered (and eventually
triggering a well-deserved punishment :-)
This is still the realm of the web of old, and no trespassers are allowed from
the commercial sargassos seas.
Your contributions may be used by others, they may be teached, learned, shared,
developed or built upon, but they will
always remain YOURS, we'll together take care of that :-)
Thus the
"third optimistical law of seeking" reads: 'On the web
EVERYTHING - and at the same time
NOTHING - can be plagiated' :-)
Ethical problems
Knowing how to find anything you want on the web, once you will have learned it,
will give you power. As always, that power can be used for 'evil' deeds and/or for
'good' deeds (let's leave aside, for now, the rather complex question of what would be 'evil' and
what 'good', just use your own parameters).
A similar problem arised in my previous 'page of reverse engineering' (1995-2000),
a site that dealt specifically
with software reverse engineering techniques and tools. The idea was to convert young
crackers (i.e. people
interested almost only in breaking software protections) into software reverse
engineers, something
that the [world needs
badly], especially given the many [malwares] practices around.
The experiment worked only in part, hence the decision a couple of years ago to freeze that site
Spreading knowledge on the web is indeed a difficult act of balance.
There is for instance a special section, ported from the previous site,
called 'ideale.htm', that was indeed intended as a sort of small
'introduction' to various "lores of destruction" (yet
purposedly not server-attack oriented).
There are indeed enough informations (and 'angles') there to allow anyone
with average searching skills to find out, on other parts of the web,
everything he may need to wreak havoc on a server and then some.
Not that anyone would really need help for that: a simple search like [+directory +indexing +bugtraq] will "turn some mighty big stones", as
a contributor pointed out some time ago.
This raises an interesting paradox.
Basically it is now clear that those that do learn how to search, which is what this
site is intended to teach, will be able to find on the web -if needs be- ANY KNOWLEDGE,
and thus anyone of
the million possible exploits, some of them 'so recent that
your server administrator wont even understand what's going on'.
Thus, once learned how to search they COULD turn out to be
potential dangerous little script kiddies.
Yet -at the same time- there are reasons to believe (and hope)
they won't be 'simple' script kiddies anymore once they realize what incredible
power the sheer fact of being capable to search gives them.
No kidding: we came to the conclusion -for me quite unexpected
until a couple of years ago- that a good seeker can be MUCH MORE
DANGEROUS than a good cracker or a good hacker or a good viri writer
or a good reverser, because HE CAN BE ALL OF THEM AT ONCE,
lacking their in-depth understanding of their relative specific field,
but easily compensating this with his evaluation and searching skills.
That's one of the reasons of our insisting (through the [reality
cracking] section and various ramblings)
on the ETHICAL side of the coin: may the "dangerous" seekers we contribute to
create endanger the enemies of poetry, knowledge,
diversity and tolerance and annoy instead the apostles of compelled behaviour,
consumistic slavery,
monolithical thinking and religious fanaticism (be it under the forms of catholic, jewish or islamic
intolerance).
This said, I do believe also that seekers are and will be
open-minded per definition, and thus pretty difficult to force
into any pre-definite, pre-digested or 'obligatorial' way of thought.
Therefore ethic should kiss their shoulders 'semi-authomatically'
when they seek. Seekers will grow ethically for the simple fact they seek,
this evolution into an
ethical seeker being
the "fourth optimistical law of seeking" :-)
In fact, judging from the firewalls' loggings, most attacks seem to come
from idiots that DO NOT KNOW how to search... hence they are using common
exploits and can easily be stopped (and if necessary directly and immediately punished :-)
Anyway, first of all, read on and learn how to seek!
Final advices
But this introduction is getting far too far... go back to the
[entrance] and click on the
main logo... your long trip into the lore of searching is about to begin.
As you'll discover perusing my site, there
are many more sections. For security reasons
some of them like the [PHP Lab] are located
on a different
server while some other will become accessible only
once you yourself will
have gathered enough searching - and hopefully [ethical] skills.
Yes, there are indeed many pages on this site... you are not
compelled to follow any logical path. You may peruse everything at
will, you are welcome. Do not be scared, nor paralysed if you don't
understand everything immediately, knowledge is like one of the chill
white wines
bottled in the ancient lagoons I come from: you should sip it slowly
and knowingly, else it won't do you no good.
Back to the entrance
(c) III Millennium: [fravia+], all rights
reserved
As you can see there are four main different areas: 'bulk', 'hidden databases', 'outside linkers'
and 'outside linked'.