Difference between revisions of "Learn/Get-Your-Website-Indexed"

(→‎Is your website and its pages indexed?)
 

(91 intermediate revisions by 9 users not shown)

Line 1: Line 1:
−
{{ListKeeprBox
+
{{DISPLAYTITLE:Get Your Website Indexed}}
−
| ListKeepr = KristinaWeis
+
{{ArticleTemplate2
 +
| Writer = KristinaWeis
 
| Name = Kristina Weis
 
| Name = Kristina Weis
−
| Image = Image:KristinaMongoose.png
+
| Header = Get%20Your%20Website%20Indexed
−
| Header = How%20to%20Help%20Your%20Website%20Get%20Indexed
+
| Subhead = Being indexed is the first step to good SEO
−
| Subhead = Being indexed is the key to showing up in search engines
+
| Bitly = http://bit.ly/IndexYourSite
−
| Bitly = http://bit.ly/NotSure
+
| Date = August 17, 2010
 
}}
 
}}
 +
==Being indexed is the first step to good SEO==
 +
----
 +
Before a website can be found in search engine results, the search engines must first discover and index that website.
 +
<br />
 +
The sooner a website gets indexed, the sooner the webmaster can begin applying SEO strategies (see [[SEO: The Basics|Learn/SEO:-The-Basics]]) to boost the site higher in search engine results. Appearing in the top 10 to 20 search results can, in turn, attract more visitors to the site.
  
−
It's something that every website must go through. Going from a new, obscure website that can only be found by someone who knows its URL to one whose pages are indexed and can be found by people searching for relevant keywords in search engines. After creating a solid website, being found by search engines is the first step...
+
If a site owner doesn't do anything to help search engines find a new website, it can take a long time for it to be indexed and show up in search results, let alone rank high for relevant search terms.
  
−
The sooner a website can get indexed, the sooner it can begin ranking well in search engines ([[SEO: The Basics|SEO]]) and receiving more traffic and business.
+
You can speed up getting more of your pages indexed. But first, let's see where your site stands now.
  
−
==Is your website and its pages indexed?==
+
==Are Your Web Pages in the Google Index?==
 +
----
 +
* '''You can use the information on AboutUs pages''' to learn how many pages of a website have been indexed by the three major search engines, and when the search engines last crawled and indexed your website's home page. You can find it by clicking "Web Presence" in the left-hand navigation on a website's page here on AboutUs.
 +
: A zero beside one of the search engine names indicates that the engine has not yet indexed any of the pages in this website. If you see question marks instead of numbers, or if you want to make sure the numbers are up-to-date, click "Refresh these numbers".
 +
: Reminder: To find a website's page on AboutUs.org, enter www.aboutus.org/MyWebsite.com in your web browser's navigation bar. You can also find the page by entering the domain name - MyWebsite.com or MyWebsite.org - in the search bar at the top right of any AboutUs.org page.
  
−
*Try searching in Google for "site:yourwebsite.com" and see what turns up. This isn't 100% accurate, but it can give you a rough idea of what you're working with. This will tell you which pages are in the index, and usually if it's a little off it will show less pages than are actually in the index.
+
* '''Search in Google for "site:YourWebsite.com" and see what turns up.'''
−
** Note that with this command, there's no space between the colon and the URL. A search for www.example.com returns URLs that begin with www and a search for example.com returns URLs for all subdomains. (So, site:MySite.com returns URLs such as www.MySite.com, puppies.MySite.com, and cats.MySite.com).
+
: This isn't 100 percent accurate, but the search will tell you how many pages are certainly indexed. Normally if the number is wrong, Google is showing fewer pages of your site than are actually in its index.
−
*If you've submitted an [[Creating an XML Sitemap|XML sitemap]] via Google Webmaster Tools you can find the most accurate number of pages in your sitemap that Google has indexed.  This will tell you the number of pages in Google's index out of the total number of pages in your sitemap, but it won't tell you ''which'' pages are indexed.
+
: Note that you don't insert a space between the colon and the site's domain name when you search.
−
*[[Online_Visibility_Audit]]
 
  
−
==Ways to Help Your Website and its Pages Get Indexed Sooner==
+
* '''If you've submitted an [[Creating an XML Sitemap|Learn/XML sitemap]] via [http://www.google.com/webmasters Google Webmaster Tools],''' you can use these tools to find the most accurate count of pages that Google has indexed.
−
* Have a website with unique, '''valuable content''' that you would find valuable enough to serve in search results if you were a search engine. I hope this is obvious. A website without quality content is unlikely to get indexed no matter how hard someone tries the other tactics below.
+
: This will tell you how many pages in your sitemap are in Google's index, but it won't tell you ''which'' pages are indexed.
−
* Get '''inbound links''' to your site. The primary way that search engines find new sites and web pages to add to its index and search results is by following links from other websites. Links without a NoFollow attribute (often called DoFollow links) are best, but NoFollow links may still be helpful (particularly ones from popular places like [[Twitter]]).
 
−
** Ask a friend or related website to link to your site from theirs or in a blog post
 
−
** Get a [[NoFollow|DoFollow link]] from AboutUs.org
 
−
** Create additional web presences for your website. There's a list of directories and similar sites in my [[Reputation Management: Neutralize the Bad with the Good|Reputation Management]] article.
 
−
* If your site is brand new and doesn't have any pages in search engines yet, submitting it to search engines can help. Here is [http://www.google.com/addurl/?continue=/addurl Google's submission form]. Don't, however, waste your time and/or money with so-called search engine submission services.
 
−
* Creating an [[Creating a Simple Sitemap|'''HTML sitemap''']] can help search engines crawlers find and index more pages on your site, once they've found your website and "sitemap" link.
 
−
* An [[Complex Sitemap|'''XML sitemap''']] can also help search engines index more of your web pages, and in addition it allows you to tell search engines which pages are most important to you.
 
−
* Setting a '''custom crawl rate''' in Google Webmaster Tools asks Google to crawl pages more quickly, which can be helpful for very large websites. If you do this, make sure that it doesn't hinder the performance, speed or usability of your site.
 
  
−
==Double check:==
+
==Get Your Website Indexed Faster==
 +
----
 +
* Your website should have '''unique''' and '''valuable''' content.
 +
: I hope this is obvious. Look at your website from the perspective of someone managing a search engine. Is the content so valuable that a search engine professional would want to send people to your site? Keep in mind that a website without quality content, or with low-quality content that's available elsewhere on the web, is unlikely ever to be indexed, no matter how hard you push indexing tactics.
 +
<div style="float:right; margin-left:1em;"><YouTube>Zcu3fp5G5EI</YouTube></div>
 +
* Get people to create '''inbound links''' to your site.
 +
: The primary way that search engines find new sites and web pages is by following links from other websites. To the right is a video of Matt Cutts (of [[Google]]) explaining how search engines use links to evaluate a web page's usefulness.
 +
: It's best to get links that don't have the [[Glossary/NoFollow|NoFollow]] attribute -- often called "DoFollow" links. But even links with the NoFollow tag may be helpful.
 +
: Ideas for getting more links to your website:
 +
::* Ask a friend or someone at a related website to link to your site from their site, or in a blog post.
 +
::* Create additional web presences for your site. You can find suggestions for where to do this in  my article about [[Reputation Management: Neutralize the Bad with the Good|Reputation Management]].
 +
::* Submit a press release with a link to your website. [[Christian Bullock]] suggests using at least one paid wire service (he likes [[PRLeap.com]]) in addition to free services.
 +
* If your site is brand new and hasn't yet been indexed by the search engines, '''submitting it to search engines''' can help.
 +
: Use the website submission forms for [http://www.google.com/addurl/?continue=/addurl Google], [http://siteexplorer.search.yahoo.com/submit Yahoo] and [http://www.bing.com/webmaster/SubmitSitePage.aspx Bing], which are the largest search engines. '''Don't waste your time and/or money on so-called search engine submission services.'''
 +
* Creating an [[Creating a Simple Sitemap|'''HTML sitemap''']] can help search engines crawlers find and index more pages on your site, once they've found your site and its "sitemap" link.
 +
* An [[Creating an XML Sitemap|'''XML sitemap''']] can also help search engines find and index more of your web pages. This kind of sitemap has the advantage of allowing you to tell the search engines which pages on your site are most important, so Google will crawl them more often.
 +
* Setting a '''custom crawl rate''' in Google Webmaster Tools allows you to ask Google to crawl pages more quickly. This is especially helpful for very large websites. If you do this, watch your site's performance carefully. If your site slows down or becomes difficult to use, dial down the crawl speed.
  
−
If none of your website's pages are showing up in search engines, make sure you aren't accidentally telling them not to.
+
==Make Sure You're Not Telling Search Engines to Ignore Your Site==
 +
----
 +
If none of your website's pages are showing up in search engines, make sure you aren't [[Learn/Don't-Block-Search-Engine-Crawlers|accidentally telling the search engines ''not'' to index them]].
  
−
*Look at the page source (in most web browsers, clicking ctrl and the letter U will do this) for one of your web pages and make sure that there isn't something like <meta name="ROBOTS" content="'''NOINDEX'''" /> in there. If a "NOINDEX" tag is there, you will need to remove it because that tells search engines to go away and not put that page in its index.
+
* Look at the page source for one of your web pages and make sure that something like <meta name="ROBOTS" content="'''NOINDEX'''" /> doesn't show up. If there's a "NOINDEX" tag, you will need to remove it. That tag tells a search engine not to place that page in its index.
−
*Look at yourwebsite.com/'''robots.txt''' and make sure you aren't telling search engine spiders to stay out and not index a certain type of page with something like this: User-Agent: * Disallow: /Special/ (This as an example, tells all search engines not to index pages on AboutUs.org that have a URL like AboutUs.org/Special/something.)
+
: You can check the page source by pressing the "ctrl" key and the "U" key simultaneously if you're using a Windows computer. If you're using an Apple, press "command" and "U."
−
* [http://www.google.com/support/webmasters/bin/answer.py?answer=35120 '''Crawl errors'''] can slow down the rate at which search engines are able to spider and index a website. You can find these in Google Webmaster Tools, along with more details about the specific errors they are experiencing. For very large websites with lots of crawl errors, search engines may decide to stop and not index the entire site.
 
−
* There's no good way to know for sure if Google or another search engine has '''penalized''' or banned your site, but... If an established website -- one that's been around for a while and has some decent inbound links -- has noticed a drop in traffic from that search engine and/or doesn't have pages indexed in search engines (like a Google search for "site:example.com" turns up nothing), a search engine penalty may be possible. Check [http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=35769#3 Google's quality guidelines] for a list of things to avoid and that they may penalize sites for.
 
  
 +
* Enter YourWebsite.com/robots.txt in the navigation bar of your browser, and check for text in your [[Learn/How-To-Use-Robots.txt|robots.txt file]] that includes the word "Disallow." For example, text reading "User-Agent: * Disallow: /" would tell all search engines to refrain from indexing any pages on that website.
  
−
<big>'''See [[Learn|more articles]] like this one.'''</big>
+
* [http://www.google.com/support/webmasters/bin/answer.py?answer=35120 '''Crawl errors'''] can slow down the rate at which search engines are able to crawl and index a website. You can find information about crawl errors in Google Webmaster Tools. If a search engine gets lots of crawl errors while it's trying to index a very large website, the crawler may simply stop indexing partway through the site.
 +
 
 +
* There's no good way to know for sure if Google or another search engine has '''penalized''' or banned your site. But if you have an established website -- one that's been around for a while and has some decent inbound links -- and you notice a drop in traffic from a search engine, or a site:example.com search shows you no results, it's possible that your site has been penalized for some reason. Check [http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=35769#3 Google's quality guidelines] to learn what actions can cause the company to penalize a website.
 +
{{LearnBottomBio
 +
| Writer        = KristinaWeis
 +
| Name          = Kristina Weis
 +
| Image        = Image:KristinaBluesFest.png
 +
| AuthorWebsite = AboutUs.org
 +
| ShortBio      = Kristina is a community manager for [[AboutUs.org]] who talks with a lot of website owners who are trying to promote their business online. <small>Have a question? [[Kristina Weis#Contact_Kristina|Contact me]].</small>
 +
}}

Latest revision as of 06:25, 4 December 2013

By [[User:|]] on

Being indexed is the first step to good SEO

Before a website can be found in search engine results, the search engines must first discover and index that website.
The sooner a website gets indexed, the sooner the webmaster can begin applying SEO strategies (see Learn/SEO:-The-Basics) to boost the site higher in search engine results. Appearing in the top 10 to 20 search results can, in turn, attract more visitors to the site.

If a site owner doesn't do anything to help search engines find a new website, it can take a long time for it to be indexed and show up in search results, let alone rank high for relevant search terms.

You can speed up getting more of your pages indexed. But first, let's see where your site stands now.

Are Your Web Pages in the Google Index?

  • You can use the information on AboutUs pages to learn how many pages of a website have been indexed by the three major search engines, and when the search engines last crawled and indexed your website's home page. You can find it by clicking "Web Presence" in the left-hand navigation on a website's page here on AboutUs.
A zero beside one of the search engine names indicates that the engine has not yet indexed any of the pages in this website. If you see question marks instead of numbers, or if you want to make sure the numbers are up-to-date, click "Refresh these numbers".
Reminder: To find a website's page on AboutUs.org, enter www.aboutus.org/MyWebsite.com in your web browser's navigation bar. You can also find the page by entering the domain name - MyWebsite.com or MyWebsite.org - in the search bar at the top right of any AboutUs.org page.
  • Search in Google for "site:YourWebsite.com" and see what turns up.
This isn't 100 percent accurate, but the search will tell you how many pages are certainly indexed. Normally if the number is wrong, Google is showing fewer pages of your site than are actually in its index.
Note that you don't insert a space between the colon and the site's domain name when you search.
This will tell you how many pages in your sitemap are in Google's index, but it won't tell you which pages are indexed.

Get Your Website Indexed Faster

  • Your website should have unique and valuable content.
I hope this is obvious. Look at your website from the perspective of someone managing a search engine. Is the content so valuable that a search engine professional would want to send people to your site? Keep in mind that a website without quality content, or with low-quality content that's available elsewhere on the web, is unlikely ever to be indexed, no matter how hard you push indexing tactics.
  • Get people to create inbound links to your site.
The primary way that search engines find new sites and web pages is by following links from other websites. To the right is a video of Matt Cutts (of Google) explaining how search engines use links to evaluate a web page's usefulness.
It's best to get links that don't have the NoFollow attribute -- often called "DoFollow" links. But even links with the NoFollow tag may be helpful.
Ideas for getting more links to your website:
  • Ask a friend or someone at a related website to link to your site from their site, or in a blog post.
  • Create additional web presences for your site. You can find suggestions for where to do this in my article about Reputation Management.
  • Submit a press release with a link to your website. Christian Bullock suggests using at least one paid wire service (he likes PRLeap.com) in addition to free services.
  • If your site is brand new and hasn't yet been indexed by the search engines, submitting it to search engines can help.
Use the website submission forms for Google, Yahoo and Bing, which are the largest search engines. Don't waste your time and/or money on so-called search engine submission services.
  • Creating an HTML sitemap can help search engines crawlers find and index more pages on your site, once they've found your site and its "sitemap" link.
  • An XML sitemap can also help search engines find and index more of your web pages. This kind of sitemap has the advantage of allowing you to tell the search engines which pages on your site are most important, so Google will crawl them more often.
  • Setting a custom crawl rate in Google Webmaster Tools allows you to ask Google to crawl pages more quickly. This is especially helpful for very large websites. If you do this, watch your site's performance carefully. If your site slows down or becomes difficult to use, dial down the crawl speed.

Make Sure You're Not Telling Search Engines to Ignore Your Site

If none of your website's pages are showing up in search engines, make sure you aren't accidentally telling the search engines not to index them.

  • Look at the page source for one of your web pages and make sure that something like doesn't show up. If there's a "NOINDEX" tag, you will need to remove it. That tag tells a search engine not to place that page in its index.
You can check the page source by pressing the "ctrl" key and the "U" key simultaneously if you're using a Windows computer. If you're using an Apple, press "command" and "U."
  • Enter YourWebsite.com/robots.txt in the navigation bar of your browser, and check for text in your robots.txt file that includes the word "Disallow." For example, text reading "User-Agent: * Disallow: /" would tell all search engines to refrain from indexing any pages on that website.
  • Crawl errors can slow down the rate at which search engines are able to crawl and index a website. You can find information about crawl errors in Google Webmaster Tools. If a search engine gets lots of crawl errors while it's trying to index a very large website, the crawler may simply stop indexing partway through the site.
  • There's no good way to know for sure if Google or another search engine has penalized or banned your site. But if you have an established website -- one that's been around for a while and has some decent inbound links -- and you notice a drop in traffic from a search engine, or a site:example.com search shows you no results, it's possible that your site has been penalized for some reason. Check Google's quality guidelines to learn what actions can cause the company to penalize a website.


Retrieved from "http://aboutus.com/index.php?title=Learn/Get-Your-Website-Indexed&oldid=38951147"