Solved

A description for this result is not available because of this site's robots.txt

Posted on 2016-09-26
13
103 Views
Last Modified: 2016-09-29
Hi, When I search Google with a new site I put up, rockypointfishingservices.com, I get this message: A description for this result is not available because of this site's robots.txt

But on Friday I added a robots.txt file to the root. http://rockypointfishingservices.com/robots.txt

What can I do? Is it just that Google hasn't re-browsed me? Thanks for any advice.
0
Comment
Question by:mel200
  • 8
  • 3
13 Comments
 
LVL 35

Assisted Solution

by:Terry Woods
Terry Woods earned 500 total points
ID: 41817028
Your robots.txt file currently contains:
User-Agent: *
Disallow: /wp-content/plugins/
Disallow: /readme.html

Open in new window


You don't want to block your plugins folder from robots. Details explained here:
https://yoast.com/wordpress-robots-txt-example/

It might be easiest just to install the Yoast plugin; the author has clearly put some thought into it, and it's free.
1
 

Author Comment

by:mel200
ID: 41817111
Thanks!
0
 

Author Comment

by:mel200
ID: 41817114
I've already got an SEO plugin, I really don't want to add plugins if I can avoid it. I've made the robots.txt file simple:

User-agent: *
Disallow:

If anyone else can chime in, I'd appreciate another point of view as well. Thanks!
0
 
LVL 35

Expert Comment

by:Terry Woods
ID: 41822414
In the Google Webmasters tools, there's a "robots.txt Tester" tool and also a tool for asking Google to load the site as a robot "Fetch as Google". Both these are under the Crawl section of the tools.

Have a play with those, in case Google needs to be told that the robots.txt has changed, or in case it gives any other clues as to what's happening.
1
 

Author Comment

by:mel200
ID: 41822445
Thanks, will try that!!
0
3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

 

Author Comment

by:mel200
ID: 41822458
OK, that says this, does it mean that message will go away soon?

capture
0
 
LVL 35

Accepted Solution

by:
Terry Woods earned 500 total points
ID: 41822491
Maybe. You might try using "Fetch as Google" to fetch the site too, as once fetched I think you get an option Submit to Index, which may help.
1
 

Author Comment

by:mel200
ID: 41822568
Interesting. At first I got this "redirected" announcement, then I submitted and got this. The domain is registered and hosted at Bluehost.  It's a wordpress installation. What might I have done wrong, any idea?

screenshot v2
0
 

Author Comment

by:mel200
ID: 41822574
Um... never mind that last. It's because I selected http://www.... as my preferred domain.
0
 

Author Comment

by:mel200
ID: 41822592
I did it again on the www. version. It said Status: partial.

I read this, which seems to indicate again a problem with the robots.txt file.
https://productforums.google.com/forum/#!topic/webmasters/3X0X8c3c39U
0
 

Author Closing Comment

by:mel200
ID: 41822852
Finally got it, thanks for your help!
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Although it can be difficult to imagine, someday your child will have a career of his or her own. He or she will likely start a family, buy a home and start having their own children. So, while being a kid is still extremely important, it’s also …
This code takes an Excel list of URL’s and adds a header titled “URL List”. It then searches through all URL’s in column “A”, looking for duplicates. When a duplicate is found, it is moved to the top of the list. The duplicate URL’s are then highlig…
The purpose of this video is to demonstrate how to add AdSense Ads to a WordPress Website, and how to set up WordPress to automatically place Ads in Sidebars. This will be demonstrated using a Windows 8 PC. Log into your AdSense account. : Cli…
This Micro Tutorial will demonstrate how to add subdomains to your content reports. This can be very importing in having a site with multiple subdomains.

914 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now