ITWissen.info - Tech know how online

robot exclusion standard (RES)

Many websites do not want to appear in the search engine results pages( SERP).

To prevent this, you can mark websites in such a way that the robots accessing them recognize that the marked website must not be captured. This is done with RES marking, which stands for Robot Exclusion Standard (RES).

The behavior of the robots can be influenced by header entries and the ASCII file robots.txt in the root directory of the web server. If the robots.txt file is stored in the web server, it prohibits the robots from accessing the web page.

Informations:
Englisch: robot exclusion standard - RES
Updated at: 15.11.2009
#Words: 96
Links: search engine (SE), search engine result page (SERP), resume message (GSM) (RES), header (HDR), file
Translations: DE
Sharing:    

All rights reserved DATACOM Buchverlag GmbH © 2022