METHOD 1.0.0 · CHECK SET 2026-08-30

Site Architecture Auditor methodology

A bounded, same-origin model of how visitors and crawlers can move through a website—and where its internal links fail to support important pages.

What this tool measures

The auditor measures site structure: discoverable internal links, shortest crawl depth, sitemap-only pages, relative link authority, hub behaviour, close topic overlap and plausible missing links. It does not replace the Page Checker’s page-level SEO inspection or the Agent Readiness Checker’s bot-access and protocol audit.

Every report records its URL budget. “No issue detected” means no issue was observed inside that bounded sample, not that the complete site is faultless.

Crawl process

  1. 01

    Establish the public origin

    The starting URL must be public HTTP or HTTPS. Private, loopback, local and metadata hosts are rejected. Redirects are followed with a strict hop limit.

  2. 02

    Read crawl instructions

    The site’s robots.txt is read for the auditor user agent and its declared sitemaps. The parser applies the most specific available matching user-agent group and records blocked paths.

  3. 03

    Crawl breadth first

    HTML pages are discovered through same-origin links in breadth-first order. The shortest observed click path becomes crawl depth. URL fragments and query strings are removed for graph identity so tracking parameters do not create false pages.

  4. 04

    Use the sitemap as a second view

    Unlinked sitemap URLs can use remaining crawl budget. Pages found this way with no crawled inlinks are marked as sitemap orphans.

  5. 05

    Build and score the graph

    Links between crawled pages form a directed graph. Relative authority and hub scores are calculated, then page findings, topic overlap and link opportunities are added.

Authority and hub metrics

Relative authority (PageRank)

Graph measurement

An iterative PageRank calculation estimates how internal link equity is distributed within this crawl. Values are normalised from 0–100 for the report. They are not Google PageRank and cannot be compared across different crawls.

Hub score (reverse authority)

Graph measurement

The link graph is reversed and scored again. High values identify pages that connect prominently to other structurally important pages. This is a useful hub proxy, not a judgement of editorial quality.

Crawl depth

Deterministic measurement

Depth is the shortest observed same-origin link path from the starting page: start = 0, directly linked = 1. A sitemap-only page has no observed link-path depth.

Scoring

The score begins at 100 and deducts bounded penalties for observed structural risks. Critical findings have the largest impact, warnings a moderate impact, and opportunities a small impact. Repeated findings are capped so one large template-level issue cannot reduce the score without limit.

SignalTypical impact
Broken or unreachable pageCritical
Sitemap orphan or priority-page riskHigh
Depth over three, linked noindex, redirect, dead endModerate
Close topic competitionModerate group penalty

Labels are descriptive rather than industry benchmarks. The crawl budget, start URL and site type materially affect the result.

Check definitions

Broken or unreachable page

Deterministic measurement

A crawled URL returned HTTP 4xx/5xx or could not provide usable HTML. A temporary network failure can look identical, so confirm before removing links.

Sitemap orphan

Deterministic within sample

A page was discovered in XML sitemap data but received no link from another crawled page. Links outside the crawl budget or JavaScript-only links may change this classification.

Deep page

Deterministic within sample

An indexable page required more than three observed clicks from the starting page. The threshold is a prioritisation heuristic; not every deep archive or utility page needs promotion.

Linked non-indexable page

Deterministic measurement

A page with internal inlinks declared a noindex directive. This can be intentional for workflows and utility pages; the report asks for confirmation rather than assuming an error.

Dead end

Graph measurement

An indexable HTML page exposed no followed same-origin destination in the crawled graph. Global navigation removed by scripts or consent states may produce false positives.

Priority-page risk

User-defined overlay

A URL explicitly nominated before crawling is missing from the observed graph, deeper than two clicks, or has low relative authority. This overlay prevents high-value commercial pages disappearing inside a site average.

Close topic competition

Experimental heuristic

Pairs indexable pages when title and H1 terms overlap strongly. It identifies candidates for editorial review; it does not prove search cannibalisation or recommend consolidation automatically.

Limits and safe interpretation

  • The crawl is capped at 10, 25 or 40 HTML pages and stays on the final URL’s exact origin.
  • Query strings and fragments are removed. Sites where query parameters define essential pages need specialist crawling.
  • The crawler reads response HTML; links and content created only after browser interaction may be absent.
  • robots.txt handling covers practical allow/disallow patterns but is not a full search-engine implementation.
  • Sitemap indexes and URL sets are sampled within safety and crawl budgets.
  • PageRank and hub values are relative to one report and should not be compared across sites or crawl sizes.
  • Topic competition and link suggestions are disclosed heuristics, not ranking predictions.
  • Authentication, consent walls, rate limiting and intermittent hosting errors can limit results.

Use the map to choose a change, verify that change in context, then rescan with the same start URL and budget.