<div>
<div class="content wysiwyg-content">
<blockquote>
<br />
1 - (optional and often overlooked) one pointer to your sitemap (or to a single sitemap index if you want to indicate several sitemaps files).<br />
The sitemap is just suggestions to help the spiders to spider your site and to speed their discovery of it. They bring NO guarantee that the pages you indicate will be eventually indexed, not even that they will be spidered in the next hours. And of course the sitemap is not a restricted list of what should indexed,nor that nothing else should.<br />
So, what's wrong with this pointer in robots.txt not being used in the next 24h?<br />

</blockquote>
<br />
Maybe my robots.txt can just say look at sitemap.xml and nothing else<br />
<br />
I only want 8 pages in google and the rest of the pages on the website, I would like hidden<br />
<br />
robots.txt<br />
allow: page1<br />
allow: page2<br />
allow: page3<br />
<br />
deny the rest
</div>
</div>



1 - (optional and often overlooked) one pointer to your sitemap (or to a single sitemap index if you want to indicate several sitemaps files).
The sitemap is just suggestions to help the spiders to spider your site and to speed their discovery of it. They bring NO guarantee that the pages you indicate will be eventually indexed, not even that they will be spidered in the next hours. And of course the sitemap is not a restricted list of what should indexed,nor that nothing else should.
So, what's wrong with this pointer in robots.txt not being used in the next 24h?


Maybe my robots.txt can just say look at sitemap.xml and nothing else

I only want 8 pages in google and the rest of the pages on the website, I would like hidden

robots.txt
allow: page1
allow: page2
allow: page3

deny the rest

allow 8 pages and deny the rest

Search Engine Optimization (SEO) is the process of improving the visibility of web content in search engines using an understanding of search engines' processes and algorithms. Also known as "natural" or "organic" search, it is distinct from paid web advertising. Performance is typically measured by testing the appearance of a website for targeted search terms, and more specifically, ranked positions in search for important keywords. Techniques focus on content optimization, technical optimization and link building.

Search Engine Optimization (SEO)

PHP is a widely-used server-side scripting language especially suited for web development, powering tens of millions of sites from Facebook to personal WordPress blogs. PHP is often paired with the MySQL relational database, but includes support for most other mainstream databases. By utilizing different Server APIs, PHP can work on many different web servers as a server-side scripting language.

The successor to Active Server Pages, ASP.NET websites utilize the .NET framework to produce dynamic, data and content-driven web applications and services. ASP.NET code can be written using any .NET supported language. As of 2009, ASP.NET can also apply the Model-View-Controller (MVC) pattern to web applications