# Robots.txt: what it is and how it works · Zorraquino
> Robots.txt is a file that tells compatible crawlers which paths they may or may not crawl.
URL: https://www.zorraquino.com/en/dictionary/internet/robots-txt.html

# Robots.txt

Robots.txt is a file that tells compatible crawlers which paths they may or may not crawl

What it means, how it is used and where it comes up, with its related concepts.

[The whole glossary](https://www.zorraquino.com/en/dictionary)

## What it is and how it works

It is published at the root of the site and does not work as a mechanism for protecting information. Blocking a URL in this file does not guarantee it will be removed from the index. To apply a noindex directive, the crawler normally needs to be able to access the content that contains it.

Topics

[SEO, content and GEO](https://www.zorraquino.com/en/dictionary/category/seo-contenidos-y-geo)

## Related concepts

- [Noindex](https://www.zorraquino.com/en/dictionary/digital-marketing/noindex.html)
- [URL](https://www.zorraquino.com/en/dictionary/internet/url.html)
- [Duplicate content](https://www.zorraquino.com/en/dictionary/internet/duplicated-content.html)