Avatar of lordrt
lordrt
 asked on

search engine web site indexing?

Hello
I have put my website live yesterday, but neither google nor yahoo is showing it when I searched today. Is there any more steps to do for the website to be added to the search engines, or does it take time to add the website to the db of these search engines?
Web DevelopmentContent Management

Avatar of undefined
Last Comment
lordrt

8/22/2022 - Mon
gavsmith

It takes time for their bots to crawl your site. Although you may be able to speed things up a bit by logging into google and looking through the tools... they have a tool which allows you to track your site, they give you a txt file to put in your webspace to prove it's yours. I think it even lets you know if there are broken links.
ASKER CERTIFIED SOLUTION
gavsmith

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
SOLUTION
Bernard Savonet

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
lordrt

ASKER
thanks for the suggestions, hopefully will get my website to show on these search engines quickly enough
Bernard Savonet

B-) Glad we could help. Thx for the grade and points.

While spiders start exploring your site, help them to find faster the content of your site by
1 - Generating a sitemap, and uploading it to your site
2 - Using webmasters tools of each of GYB, give them the link to the sitemap file on your site
3 - Edit your robots.txt file on site by adding the line
Sitemap: http://www.example.com/sitemap.xml
Note that the sitemap file must be a complete url with http
In addition to GYB, most other spiders will use this information to speed up theit spidering of your site, see
http://www.sitemaps.org/protocol.php#informing

As I said, I love GSiteCrawler to generate sitemaps http://gsitecrawler.com/ and somehow automate the process of updtaing them on the site. (This program is free, although donations are possible and welcome - I have no relation with them)
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
lordrt

ASKER
thanks again fibo, my site is dev in drupal, and it does provide a module which automatically creates a sitemap for the website when it is activated in it
lordrt

ASKER
Just a little problem thou, the www part is missing when I lookup my site in the search engines, it only gives me, e.g., example.com, while my site has the www.example.com format... anything I might have missed?
Bernard Savonet

yes.
seems your site answers to both www and non www

now is the time to decide which of the 2 you want, then
1 - check that the link to your home page is to the selected option (you might need to make one change in your site parameters and one chage in the html code of your template

2 - put a rewrite 301 in your htaccess: this will tell all search engines to use only the right address
RewriteEngine On

RewriteCond %{HTTP_HOST} ^mydomain.com [NC] 
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [L,R=301]

Open in new window

within weeks, your non-www indexed pages will disappear and appear only as www

3 - If in some engine there still remains some non-www after some weeks... I would ignore them, because the rewrite will do the job.
You might however check in Google what is still there after 8-12 weeks. If some page remains, just ask Google to remove it

4 - Additional actions
They are not really needed nor useful for your problem, but they may speed up things and/or give you the warm feeling you are acting:
- register with Google webmasters tools, and submit both www and non www URLs of the site, then tell webmasters tools wich one is the preferred (this would not work with others spiders, while the redirect 301 will)
- generate a sitemap of the www site as defined earlier (then upload it, change your robots.txt etc)
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
lordrt

ASKER
thanks again fido