The dark internet is basically the part of the internet or web that search engines cannot reach. In plain terms, if Google can’t find something it doesn’t exist.

Imagine looking for information on autoresponder memberships. You might search and search and search, but never find this site:

The reason you might not find that site is very simple. Google might hot have this site indexed. That means that Google hasn’t visited it yet. (Aside: How Google indexes your site.) Or, it might mean that Google cannot reach the site. Maybe you have password protection in place.

Back in October 2007, some people over at Webmaster World started talking about how Google was indexing sites that were part of the dark internet. Speculation about this showed up in other places. Here’s what a few people said:

1. Maybe Google is using the Google Toolbar to dig into sites

2. Maybe Google is using Google Analytics to find a backdoor

3. Maybe someone is linking directly to sites and Google’s following

No matter how you look at this, it sounds like Google’s trying to add some light to the dark internet. Maybe they are trying to use a mix of form variables and GET to drill into pages. Maybe they are using new, sophisticated robots to drill for oil.

It’s also possible that Google is finding some “secret” ways into site through RSS feeds or perhaps Gmail. There are many such potential holes.

I don’t really need to discuss exactly how Google might shine light on the dark internet. There are plenty of smart people looking at how it might be done. Remember, Google’s mission is to organize the world’s information.

My real point is very simple and extremely targeted. Look, the dark internet will continue to be under attack. Google, and many other companies, want to expose and index and exploit your content. Dark is bad. Light is good. You’re fighting with a multi-billion dollar company.

Yes, I want people to learn about autoresponder memberships but I don’t know how Google’s going to help me or hurt me along the way. I have to stay on my toes about their dark internet plans, and so do you.

My final caution is this: Anything on your web server might be open and exposed to the world. Even with security — lot’s of it — you might be at risk. If it’s on a server, you might find it on Google tomorrow. Understand this. Prepare for it. Google will index first, ask questions later, and mea culpa along the way.

Do no evil? You tell me.

Be Sociable, Share!