If your webhost denies you from uploading the robots.txt file to the server,And you wish to block the crawlers,from indexing few of your webpages,then ROBOTS META TAGS are the best alternative for you.
Structure:
The "ROBOTS META TAGS" looks as similar as any other meta tag,and is also included within the <head> section of your page.
<meta name="robots" content=""index,follow">
The above code means,the Crawlers will index all of your pages,and will also let them to crawl links contained within it.
<meta name="robots" content=""index,nofollow">
The above code means,the Crawlers will index all of your pages,and will not let them to crawl links contained within it.
<meta name="robots" content=""noindex,nofollow">
The crawlers will neither index the page,nor will crawl links contained within it.
<meta name="robots" content=""noindex,follow">
The crawlers will not index the page,but will crawl links contained within it.
0 comments:
Post a Comment