When managing a website composed of several dozen pages, the first challenge is not creating content, but knowing precisely what already exists. A page published two years ago, an outdated contact form, a forgotten landing page that still loads a tracking pixel without a consent banner: these blind spots pose concrete problems, from SEO to CNIL compliance.
Creating a complete list of the pages available on a website is an operational prerequisite, not a theoretical exercise.
Trackers and CNIL compliance: the real reason to inventory each page
It is often thought that a page inventory serves only for SEO. In practice, the most pressing reason lies elsewhere. The CNIL reminds us that the obligation of consent applies not only to cookies but also to pixels, local storage, fingerprinting, and advertising identifiers. Each active URL on a site can load different scripts.
A product page created in 2022 may incorporate an audience tool configured before your CMP (consent management platform) was updated. If no one has checked it since, this page may drop trackers without valid consent. The compliant banner displayed on the homepage does not automatically cover all URLs in the domain.
In practical terms, a page-by-page audit must verify the technologies loaded before and after consent, and not just list the discovered URLs. The CNIL specifically recommends timestamped captures of each version of the banner and a configuration history maintained by the CMP. In other words, a useful page list includes an exploration date and a compliance status.
By browsing the list of pages of Bla Bla Bla, it is clear how a structured sitemap facilitates this type of verification: each URL is identifiable, and specific sections that require targeted checks can be quickly identified.
XML Sitemap and internal crawl: two complementary approaches to list a site’s pages
The sitemap.xml file is the obvious starting point. It can be found by adding /sitemap.xml to the root of the domain. It lists the URLs that the site administrator wants to make visible to search engines. The problem is that it only contains what has been willingly included.
Orphan pages (those that receive no internal links) do not appear in the sitemap or in navigation. An internal crawl with a dedicated tool scans the site as a search engine robot would, following each link. This is the only method to discover forgotten URLs.
What the sitemap does not show
- Noindex pages that remain accessible via their direct URL but are excluded from the sitemap by convention
- Old redirected URLs (301/302) whose destination has changed or that point to a deleted page
- Pages dynamically generated by filters or URL parameters, often absent from the sitemap but indexed by Google
Combining sitemap and crawl covers nearly all active URLs. It is recommended to conduct a complete crawl at least once a quarter on a medium-sized site and to compare the results with the declared sitemap.
Site: operator on Google and Search Console to check indexing
Once the internal list is established, it is compared to what Google actually knows. The command site:mondomaine.fr in the search bar displays the indexed pages. This method has its limitations: Google does not always return the totality of indexed URLs, especially on large sites.
The Search Console provides a more reliable indexing report. It distinguishes between indexed pages, those intentionally excluded (noindex, canonical pointing elsewhere), and those rejected by Google for technical reasons. The gap between submitted URLs and indexed URLs reveals issues: duplicate content, server errors, excessive loading times.
URL parameters and ghost pages
URL parameters (sorting, filters, session identifiers) sometimes generate hundreds of variants of the same page. Google can index them separately, which dilutes the authority of the domain. In Search Console, the “URL Parameters” section allows you to indicate which ones Google should ignore.
Feedback varies on this point: some sites see a rapid improvement in crawl budget after cleaning up parameters, while others notice no significant change. The impact depends on the volume of affected pages and the site’s crawl frequency.
Classifying pages by level of dependence on consent-subjected tools
Not all pages on a site present the same risk in terms of compliance. A static institutional page without analytical tools poses no problem. A conversion page that loads a remarketing pixel, a heatmap tool, and a tag manager is at the opposite end.
An operational classification involves sorting URLs into three categories:
- Pages without trackers subject to consent (legal notices, static pages, certain blog pages)
- Pages with audience tools exempted under conditions (strictly anonymized audience measurement, according to CNIL guidelines)
- Pages loading advertising trackers or third-party tools requiring explicit consent before deployment
This sorting immediately identifies priority pages for a compliance audit. We start with the third category, where the risk of sanction is highest.
Proof of compliance is not limited to a visible banner. It implies that refusal is as simple as acceptance, and that withdrawal of consent remains possible at any time. Each page on the list must be verifiable against these three criteria.
Creating a complete inventory of a site is not a one-time task. Pages evolve, scripts change, CMPs get updated. An up-to-date sitemap, regular crawls, and compliance tracking by URL form the foundation of a well-managed site, both for SEO and for meeting legal obligations.
