A rule must always be preceded by the User-agent directive. It is possible that an empty line break follows the User-agent directive.

A rule is too long.

Yes

The rule exceeds acceptable length (1024 characters).

Incorrect name of the main mirror of the site.

Yes

The name of the main mirror of the site in the Host directive contains a syntax error.

Badly formed URL of the Sitemap file.

Yes

URL of the Sitemap file must be specified as a complete URL, including http://. E.g., http://www.mysite.ru/sitemap.xml

Wrong format of Crawl-delay directive.

Yes

Wrong time specified in the Crawl-delay directive.

Several Crawl-delay directives have been discovered.

Yes

Only one Crawl-delay directive is allowed.

Wrong format of Clean-param directive.

Yes

The Clean-param directive specifies one or more parameters that the robot will disregard, as well as the path prefix. The parameters are
delimited with '&' and separated from the path prefix by a space.

A rule must always be preceded by the User-agent directive. It is possible that an empty line break follows the User-agent directive.

A rule is too long.

Yes

The rule exceeds acceptable length (1024 characters).

Incorrect name of the main mirror of the site.

Yes

The name of the main mirror of the site in the Host directive contains a syntax error.

Badly formed URL of the Sitemap file.

Yes

URL of the Sitemap file must be specified as a complete URL, including http://. E.g., http://www.mysite.ru/sitemap.xml

Wrong format of Crawl-delay directive.

Yes

Wrong time specified in the Crawl-delay directive.

Several Crawl-delay directives have been discovered.

Yes

Only one Crawl-delay directive is allowed.

Wrong format of Clean-param directive.

Yes

The Clean-param directive specifies one or more parameters that the robot will disregard, as well as the path prefix. The parameters are
delimited with '&' and separated from the path prefix by a space.

Warnings

The list of warnings that may be generated while parsing robots.txt.

Warning

Yandex extension?

Description

An invalid character may have been used.

Yes

A special character different from "*" and "$" was discovered.

Unknown directive was discovered.

Yes

A directive not described in the rules for using robots.txt was discovered. It is possible that this directive is used by robots that belong to other search engines.