Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

search engine web site indexing?

Posted on 2011-03-09
9
Medium Priority
?
317 Views
Last Modified: 2012-05-11
Hello
I have put my website live yesterday, but neither google nor yahoo is showing it when I searched today. Is there any more steps to do for the website to be added to the search engines, or does it take time to add the website to the db of these search engines?
0
Comment
Question by:lordrt
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
9 Comments
 
LVL 10

Expert Comment

by:gavsmith
ID: 35081241
It takes time for their bots to crawl your site. Although you may be able to speed things up a bit by logging into google and looking through the tools... they have a tool which allows you to track your site, they give you a txt file to put in your webspace to prove it's yours. I think it even lets you know if there are broken links.
0
 
LVL 10

Accepted Solution

by:
gavsmith earned 1000 total points
ID: 35081264
Found it www.google.com/webmasters/tools/

Regards
Gav
0
 
LVL 29

Assisted Solution

by:fibo
fibo earned 1000 total points
ID: 35081434
Well, being indexed overnight is not the rule.
You need to make a distinction between Google and others (Yahoo, Bing etc).

1. Bing & Yahoo

For these engines, the simplest action is to submit the url of your site to their engine.
Bing: http://www.bing.com/webmaster/SubmitSitePage.aspx
Yahoo: http://siteexplorer.search.yahoo.com/submit (will require that you register - anyway you would do to access to their webmasters tools)
With these sites you can also register to get access webmasters tools and submit the urls or sitemaps.

2. Google

For Google: you can register to get access to webmasters tools, and even submit sites or sitemaps, and even get analytics.
But that is not enough to get your site indexed by Google!
Google will not accept to index a site or a url if it is not part of the "reachable web universe". More precisely Goggle will not index a page/url if none of the pages that Google knows points to this url.
So you must take care that there is a link to your site from a page (or a site) that Google knows. And once this is done Google spiders will explore the rest of your site.

One easy step if you have some corporate or personal page that Google knows: place there a link to the new site, then resubmit the url of this page to Google... GG will explore it, take note of the new link, and spider the new link (initially just the linked-to page, later the links on that page).
You can reach a similar result by placing a link to the new site as a signature to posts you make on forums known by Google, then resubmit the url of one or 2 pages where this signature is visible.

3. Do not overflood!

Do not oversubmit!
If you make too many submits to any of the engines (ie: submitting the same page repeatedly, submitting lots of pages from the same site), chances are high that this engine will consider your submissions as spam and place them low in its priority list.
Just find the right balance between informing and spamming!
Personally, for a new site I usually create a sitemap (GSiteCrawler is the tool I use), then upload it to GYB (Google Yahoo Bing), then update my own page with the new link, and resubmit it to GYB, then the next day submit the site url to GYB. That is enough, all spiders will start indexing pages within days, and will complete the spîdering of the sites within a few weeks.
B-) except that in some occasions Bing seems to refuse to index the site!
0
Plesk WordPress Toolkit

Plesk's WordPress Toolkit allows server administrators, resellers and customers to manage their WordPress instances, enabling a variety of development workflows for WordPress admins of all skill levels, from beginners to pros.

See why 2/3 of Plesk servers use it.

 
LVL 4

Author Closing Comment

by:lordrt
ID: 35081537
thanks for the suggestions, hopefully will get my website to show on these search engines quickly enough
0
 
LVL 29

Expert Comment

by:fibo
ID: 35085486
B-) Glad we could help. Thx for the grade and points.

While spiders start exploring your site, help them to find faster the content of your site by
1 - Generating a sitemap, and uploading it to your site
2 - Using webmasters tools of each of GYB, give them the link to the sitemap file on your site
3 - Edit your robots.txt file on site by adding the line
Sitemap: http://www.example.com/sitemap.xml
Note that the sitemap file must be a complete url with http
In addition to GYB, most other spiders will use this information to speed up theit spidering of your site, see
http://www.sitemaps.org/protocol.php#informing

As I said, I love GSiteCrawler to generate sitemaps http://gsitecrawler.com/ and somehow automate the process of updtaing them on the site. (This program is free, although donations are possible and welcome - I have no relation with them)
0
 
LVL 4

Author Comment

by:lordrt
ID: 35091612
thanks again fibo, my site is dev in drupal, and it does provide a module which automatically creates a sitemap for the website when it is activated in it
0
 
LVL 4

Author Comment

by:lordrt
ID: 35091843
Just a little problem thou, the www part is missing when I lookup my site in the search engines, it only gives me, e.g., example.com, while my site has the www.example.com format... anything I might have missed?
0
 
LVL 29

Expert Comment

by:fibo
ID: 35092271
yes.
seems your site answers to both www and non www

now is the time to decide which of the 2 you want, then
1 - check that the link to your home page is to the selected option (you might need to make one change in your site parameters and one chage in the html code of your template

2 - put a rewrite 301 in your htaccess: this will tell all search engines to use only the right address
RewriteEngine On

RewriteCond %{HTTP_HOST} ^mydomain.com [NC] 
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [L,R=301]

Open in new window

within weeks, your non-www indexed pages will disappear and appear only as www

3 - If in some engine there still remains some non-www after some weeks... I would ignore them, because the rewrite will do the job.
You might however check in Google what is still there after 8-12 weeks. If some page remains, just ask Google to remove it

4 - Additional actions
They are not really needed nor useful for your problem, but they may speed up things and/or give you the warm feeling you are acting:
- register with Google webmasters tools, and submit both www and non www URLs of the site, then tell webmasters tools wich one is the preferred (this would not work with others spiders, while the redirect 301 will)
- generate a sitemap of the www site as defined earlier (then upload it, change your robots.txt etc)
0
 
LVL 4

Author Comment

by:lordrt
ID: 35093259
thanks again fido
0

Featured Post

Understanding Web Applications

Without even knowing it, most of us are using web applications on a daily basis. Gmail and Yahoo email, Twitter, Facebook, and eBay are used by most of us daily—and they are web applications. We often confuse these web applications tools for websites.  So, what is the difference?

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article was originally published on Monitis Blog, you can check it here . Today it’s fairly well known that high-performing websites and applications bring in more visitors, higher SEO, and ultimately more sales. By the same token, downtime…
CTAs encourage people to do something specific to show interest in your company, product or service. Keep reading to learn why CTAs should always be thought of as extremely important, albeit small, sections of websites.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

610 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question