Robots.txt Exclude URL's

Posted on 2011-03-05
Medium Priority
Last Modified: 2012-05-11
I have links like this:


I want to keep these links from being included in anything that creates a sitemap and I want to keep Google from following them.  I tried to add the add_cart.php to the robots.txt but I'm still seeing them when I generate a site map. Would a no follow rule on the link work?
Question by:cstormer
1 Comment

Accepted Solution

AlanNMMX earned 2000 total points
ID: 35043959
Hi, you could add the following meta tag to the head section of the add_cart.php file. This would stop bots from indexing this page.
<meta name="robots" content="noindex" />

Open in new window


Featured Post

Easily Design & Build Your Next Website

Squarespace’s all-in-one platform gives you everything you need to express yourself creatively online, whether it is with a domain, website, or online store. Get started with your free trial today, and when ready, take 10% off your first purchase with offer code 'EXPERTS'.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Over time, the online landscape has altered considerably, but that’s nothing compared to the up-and-coming trends that will shape the web design industry in the coming year. Keep reading to find out which trends will shape B2B web design in 2018.
I recently worked on a Wordpress site that utilized the popular ContactForm7 (https://contactform7.com/) plug-in that only sends an email and does not save data. The client wanted the data saved to a custom CRM database. This is my solution.
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.
Suggested Courses

607 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question