In the beginning, there was the Netscape home page and it was good. Okay, so much for my imitating the bible. But seriously, when the World Wide Web was first created in the early 1990s, the only way to find new web pages was to check on the Netscape home page, which featured new web pages and a directory of existing ones. Back then, there were just a few hundred web pages and the concept of SEO hadn’t even been born yet.
Actually, just for the purists out there, technically, there was a way to find web pages before the Netscape home page existed. It was called Archie and it is credited as the world’s first search engine.
The thing is, Archie wasn’t even created as a search engine. It was more like the modern Way Back Machine that we use today to find archived versions of websites. In fact, that’s where the name originated – it was supposed to be something about archives, but it got shortened to Archie. Archie also wasn’t much used by the general public. Instead, it was mostly used by academics (though to be fair, it was mostly academics on the Web when it first started).
The Days of Alta Vista and Yahoo!
The history of SEO really begins when websites like Alta Vista and Yahoo! first began to appear. This really represented the first radical shift in SEO since there was real competition to get yourself noticed in the two big directories online. Until now, people pretty much put up a web page and it was noticed by the rather tiny community that existed on the web at the time.
Yahoo! and Alta Vista worked in two very different ways and those two ways gave rise to two common ways of doing SEO that we know today.
Alta Vista was a true crawl based search engine of the kind we know and love today. It used a bot to search for new sites to add to its database and was considered to be the most comprehensive search engine on the web for a number of years.
Yahoo! by comparison was designed as a web directory. It was intended to be human controlled and in order to get listed, you had to request a listing. Most sites, at least in the early days, were listed whenever a request was made, but the fact that you had to make a request and that Yahoo! could decline your listing meant that getting into Yahoo! was a big deal. It’s kind of like getting into DMOZ today.
By the way, these days Yahoo! uses a similar automated system to that used by everyone else, though the Yahoo! Directory is still human controlled.
The Porn Industry Makes a Mess of SEO
By this time, the concept of SEO had started to become entrenched – with hundreds of thousands of web pages already available, it had gotten to the point where you had to work to get noticed. People started experimenting with keywords and the Meta tag was considered the single most important part of SEO work.
The porn industry, which has always been on the cutting edge of technology (let’s face it – if you’re trying to sell something rather expensive which can be had for free in many places, you need to innovate to stay competitive) noticed this thing about meta tags and took advantage of them. They started stuffing their Meta tags with unrelated keywords in an effort to rank higher in all search results. The idea was to pull in curious random surfer rather than just those looking for the stuff.
This effort by the porn industry led to the second major revolution in SEO, the dropping of Meta tags as an important way to index web sites.
Google Comes On the Scene
Early in the 21st century, Google was created and seemed to Yahoo! to be a minor annoyance. However, the engineers behind Google were able to build it into the world’s most popular search engine through a number of different methods.
First, they dropped the Meta tag thing which was destroying other search engines and began to crawl entire web pages instead. Second, they created Adwords and Adsense. More than anything else, those two systems changed SEO forever. Until now, people paid a flat fee for advertising or they paid a per view price.
Adwords introduced the concept of pay per click. Adsense introduced the concept of advertising on thousands of sites all at once. Combined, these two services created an entire industry which we know today as being the pay per click industry. No longer were you just trying to rank for the free search engine results, but you also tried to rank for paid results.
Black Hat SEO Takes Off
Black hat SEO has always been around, for as long as there have been search engines. However, the whole cat and mouse game that exists today where black hat SEO experts find some kind of a loophole and Google figures it out and closes it really started to take off at this time.
Every time that Google tried to change their algorithm to fight the black hat SEO people, they’d come up with a new idea. A whole new terminology was created at this time. Terms like Google Bowling and Keyword Stuffing were invented as a result of Google’s efforts to create a better search engine.
Social Media Takes Off
Finally, the most recent change has been the advent of social media. It has really taken the concept of SEO and stood it on its ear because until now, you were trying to get Google to notice you. Now, for the first time, because of social media, you need to try to get human beings to notice you too and recommend you to their friends.
Where Is It Going in the Future?
The Panda update could be called another way SEO has changed, but I see that as more of an incremental thing where Google has simply been fighting the battle against black hat SEO. I see SEO moving increasingly in the direction of content focused more on being human friendly and more social media oriented, at least until the next big thing comes out, whatever that is.