# Web crawler: what it is and how it works · Zorraquino
> A web crawler is a program that moves through internet resources to discover and process information.
URL: https://www.zorraquino.com/en/dictionary/internet/web-crawler.html

# Web crawler

A web crawler is a program that moves through internet resources to discover and process information

What it means, how it is used and where it comes up, with its related concepts.

[The whole glossary](https://www.zorraquino.com/en/dictionary)

## What it is and how it works

It can follow links and use other discovery sources, such as sitemaps. Each service has its own rules for access and processing. To make a site accessible to authorised crawlers, it is worth reviewing responses, links, directives and content availability.

Topics

[SEO, content and GEO](https://www.zorraquino.com/en/dictionary/category/seo-contenidos-y-geo)

## Related concepts

- [Duplicate content](https://www.zorraquino.com/en/dictionary/internet/duplicated-content.html)
- [Evergreen content](https://www.zorraquino.com/en/dictionary/digital-marketing/evergreen.html)
- [Core Web Vitals](https://www.zorraquino.com/en/dictionary/core-web-vitals)