Avatar of hankknight
hankknightFlag for Canada asked on

robots.txt: exclude all pages that have ? in url

How can I prevent robots from following dynamic pages liks this?

http://www.example.com/page?item=value
Web DevelopmentWeb Languages and Standards

Avatar of undefined
Last Comment
dangereux

8/22/2022 - Mon
dangereux

User-agent: *
Disallow: /page

or to disallow all pages (not recommended)
User-agent: *
Disallow: /
ASKER
hankknight

ASKER CERTIFIED SOLUTION
dangereux

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
See how we're fighting big data
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question
Experts Exchange has (a) saved my job multiple times, (b) saved me hours, days, and even weeks of work, and often (c) makes me look like a superhero! This place is MAGIC!
Walt Forbes