How Do Search Engines Work - Web Crawlers

It is the search engines that finally bring your website to the notice of the prospective customers. Hence it is better to know how these search engines actually work and how they present information to the customer initiating a search.     

There are basically two types of search engines. The first is by robots called crawlers or spiders.  

Search Engines use spiders to index websites. When you submit your website pages to a search engine by completing their required submission page, the search engine spider will index your entire site. A ‘spider’ is an automated program that is run by the search engine system. Spider visits a web site, read the content on the actual site, the site’s

Meta tags and also follow the links that the site connects. The spider then returns all that information back to a central depository, where the data is indexed. It will visit each link you have on your website and index those sites as well. Some spiders will only index a certain number of pages on your site, so don’t create a site with 500 pages! 

The spider will periodically return to the sites to check for any information that has changed. The frequency with which this happens is determined by the moderators of the search engine. 

A spider is almost like a book where it contains the table of contents, the actual content and the links and references for all the websites it finds during its search, and it may index up to a million pages a day. 

Example:  Excite, Lycos, AltaVista and Google. 

When you ask a search engine to locate information, it is actually searching through the index which it has created and not actually searching the Web. Different search engines produce different rankings because not every search engine uses the same algorithm to search through the indices.  

One of the things that a search engine algorithm scans for is the frequency and location of keywords on a web page, but it can also detect artificial keyword stuffing or spamdexing. Then the algorithms analyze the way that pages link to other pages in the Web. By checking how pages link to each other, an engine can both determine what a page is about, if the keywords of the linked pages are similar to the keywords on the original page.

Sitewide link selling question.

There was a thread started on a popular Forum asking the following Question.

(I Have Removed All Names)

Hi folks, Is it acceptable to increase the price of a sitewide when PR increases, or should the agreed price be set for the duration of the contract, even if the directory has no PR?

 

1st Reply:

 

You don’t cancel a subscription on people who are already existing subscribers just because you gained a little PR, just to try and extort more money from them or raise your prices. A subscriber should stay subscribed until they themselves cancel out.

That is at least my view on it and how I see it. Opinions will vary. (I guess)

 

2nd Reply:

 

During the contract period that should stay the same.

When the link is up for renewal you can discuss the price again.

Thanks

 

3rd Reply:

 

Yes, I don’t think its good idea to cancel any subscribed customer without letting them know BEFORE they actually buy a link.

I have changed the payment options several times. It was initially $10 one time fee. Then $15 one time fee.

But those prices were when my directory pages were PR3. Now I charge monthly for the same pages, except they are all PR4.

Those who have bought their links on my directory never get to pay more just because the PR got high. It wouldn’t be fair for them unless you mention and let them know beforehand.

 

4th Reply:

 

I also agree that a subscriber should ALWAYS stay subscribed until they themselves cancel out, even if your site gains a lot of pagerank. Long term subscribers/customers are too valuable. If my site went from pr3 to pr6 I would not try to increase the money unless they cancelled and then tried to subscribe again. That is just the way I prefer to do business. Also a generous reputation will bring you more happy customers in the future.

 

5th Reply:

 

If you have integrity, then you’ll keep your subscribers onboard at the price they paid until their subscription period runs out. With anyone NEW that joins up, you can charge more.

Remember, PR can go up and PR can go down…will you be refunding people when your PR drops midstream?

 

6th Reply:

 

Contracts must be honoured till expiry–while PR is temporary, reputation and honesty are permanent.

 

7th Reply:

 

Hello….

Don’t fool around like that….

You are given the option at paypal for the subscription length of term…

Make it 3 month intervals then when PR updates

If they want to go at it again then it’s all good and your new price is set.

 

My Conclusion:

I think the best response had to be post 7.

If I sign a Subscription I expect it to last for 1 year unless I’m told otherwise.

Whenever you sell anything you should make it clear what the terms are before a transaction takes place.

Some would say that the buyer should make sure of the terms before buying and after all there do say buyer beware.

To avoid confusion both parties should ask questions and keep a copy of the details as proof if needed for the future.

Never use “click here” as link text.

You should never use CLICK HERE for a link to your website, always try and use a keyword as a navigation link, Look at the Example below.

To View my directory Click Here the search engines will not give you any benefit for the keyword “Click Here”.

You should use: To visit my Directory notice the only keyword used is “Directory”

If you want to get the users Attention try GET Backlinks!

Note that “GET” is left out of the hypertext; it’s not recommended putting verb phrases in link text.

Remember to try and link your main keywords, say your website is a So Related Blog and your main keyword is Seo Blog your text link should look like this.

Come and look at our Seo Blog where you can find some useful Seo related advice as well as Deeplinks Directories list, free Phpld Templates and much more.

Submit 1 and get extra links - 7 featured and 5 regular

Web directories are one of the best ways of looking at quality collection of websites with great content. These web directories can be a good recourse locator for those visitors who are searching for specific information on the internet. SEO or search engine optimization is one of the most wanted features every website would like to run their content with. Using several SEO techniques, websites increase their daily page views which helps in achieving higher page ranks. Even the web directories are built using SEO techniques. So, if you would like to gain more popularity for your website then you must try submitting your website URL with description to this SEO Web Directory.

Quality web directories would review your website content and only after complete satisfaction, the webmasters of these web directories would manually approve your website link to be listed amongst other (related) existing links. Manual approval is much required for quality control as not every site is professionally built with good amount of relevant content. Mainsearches Directory is one of the best web directories available on the internet today. It has structured categories and sub categories with fantastic collection of good website links.

There are plenty of other web directories available on the internet as well which offer web link submission. Before you even submit your link to any web directory do look for the existing links. Opening one or two would be sufficient to give an idea of a website content standard. If you are not satisfied with the content quality you can always move on to next friendly Web Directory. To get the maximum benefit of a web directory submission, you should try for either featured links or regular links which would cost either an annual or monthly fees.

With a new concept in which website owner gains more with single featured link, more and more opportunities are now available for people to add up to 7 features links in price of one. Similarly, you will get 5 additional regular links free in price of one. This is a great way to submit more than one URL for your websites into such good quality web directories. You may not get such a lucrative opportunity again so take full advantage of the existing offer and make the most of it. This will not only help the page ranking of your website but will also fetch you good amount of visitors from the search engines.

How To Test For A Pagerank Update.

Ok, due to the fact that very few people will actually read all of the posts in the current (and the past, and most likely the future) “PR Update” threads, I thought I’d post a nice little example of how to make sure something is actually happening.

When PageRank is updated (talking visual update here, anyone who thinks they’re being intelligent by pointing out that behind the scenes PR is updated constantly should be slapped), Google bases it on a single snapshot from a specific point in time (same goes for BL updates, btw). That point in time is usually a few weeks prior to the actual export. When you are looking at PR, it is calculated by all using all of the links that Google knows about at that time. Any links you get to your site after than snapshot won’t be counted, even if you get them prior to the actual PR itself being exported.

One of the spin-offs of this is that any WebPages that didn’t exist before the update will not have PR until the next update. This is important, especially with the current out-of-sync datacenter situation that Google has at this time. Currently there are certain datacenters that are still showing pre-August PageRank values for sites. What this means is that if someone looks at a webpage at different times, if the pre-August PR is different from the latest PR, sometimes they will see different results. Same thing when using a multi-dc PR checking tool… some DC’s will have one value, while others will have the older, last year values.

People who do not realize this, and check their PR say, every couple of months using a multi-dc tool, might see what they think is a PR shift, and assume that the difference in values means a recent change. Hence, a new flurry of “PR Update” threads get spawned.

However, while this makes it slightly more difficult nowadays to check for PR updates, it does not make it impossible. You cannot reliably use pages that had existed going back 2 updates, but you can always use newly created pages, ones that did not exist prior to the last snapshot date. A good source of tracking down pages that we can pinpoint a date on and assume Google knew about the page back then, is popular blogs.

Therefore, before starting a PR Update thread, it is probably a good idea to base it around observations where you can actually show before and after data, rather than just “I see changes” types of responses.

By Webmaster Tools