<div>
User-agent: *<br />
Disallow: /page<br />
<br />
or to disallow all pages (not recommended)<br />
User-agent: *<br />
Disallow: /
</div>


User-agent: *
Disallow: /page

or to disallow all pages (not recommended)
User-agent: *
Disallow: /

<div>
No, my question is how to limit crawling of ALL pages that have ? in it.<br />
<br />
<a href="http://www.example.com/page?item=value" rel="ugc">http://www.example.com/page?item=value</a><br />
<a href="http://www.example.com/?foo=bar" rel="ugc">http://www.example.com/?foo=bar</a><br />
<a href="http://www.example.com/cgi-bin/run.pl?item=value" rel="ugc">http://www.example.com/cgi-bin/run.pl?item=value</a><br />
<a href="http://www.example.com/script.php?sid=456345635654656" rel="ugc">http://www.example.com/script.php?sid=456345635654656</a><br />
<br />

</div>


No, my question is how to limit crawling of ALL pages that have ? in it.

http://www.example.com/page?item=value [http://www.example.com/page?item=value]
http://www.example.com/?foo=bar [http://www.example.com/?foo=bar]
http://www.example.com/cgi-bin/run.pl?item=value [http://www.example.com/cgi-bin/run.pl?item=value]
http://www.example.com/script.php?sid=456345635654656 [http://www.example.com/script.php?sid=456345635654656]



<div>
<div class="content wysiwyg-content">
How can I prevent robots from following dynamic pages liks this?<br />
<br />
<a href="http://www.example.com/page?item=value" rel="ugc">http://www.example.com/page?item=value</a>
</div>
</div>


How can I prevent robots from following dynamic pages liks this?

http://www.example.com/page?item=value

robots.txt: exclude all pages that have ? in url

Web development includes all aspects of presenting content on intranets and the Internet, including delivery development, protocols, languages and standards, server software, browser clients, databases and multimedia generation.

Web Development

Web development can range from developing the simplest static single page of plain text to the most complex web-based internet applications, electronic businesses, and social network services using a wide variety of languages and standards, including the familiar HTML, JavaScript and jQuery, ASP and ASP.NET, PHP, ColdFusion, CSS, PHP, Flex and Flash, but also the implementation of a broad list of standards including XML, WSDL, SSDL, VoiceXML and many more.