NEW: AI VISIBILITYSee whether ChatGPT, Gemini, Perplexity and Claude recommend your business

SEO glossary

What is crawling and indexing?

Crawling is how search engines discover your pages; indexing is how they store them so they can appear in results.

Crawling is when search engine bots follow links and sitemaps to find pages. Indexing is when the search engine processes a page and adds it to its database. Only indexed pages can rank.

Why it matters

A page that can't be crawled or isn't indexed won't show up, however good it is. Blocked pages, broken links, missing sitemaps and duplicate content can all get in the way.

Example

A new article that nothing links to and that isn't in your sitemap may take a long time to be discovered, or may not be indexed at all.

What to do

Keep an up-to-date sitemap, link new pages from existing ones, don't block important pages in robots.txt, and use Search Console to see which pages are indexed and why others aren't.