An orphan candidate is a public URL found in one inventory but not reached through the site's rendered internal links. It may still be discoverable through a sitemap, an external link, a historical analytics export, or a direct visit. The useful question is not whether a crawler label says "orphan"; it is whether the URL belongs in the current information architecture.
The problem compounds at scale. A Laravel application with auto-generated landing pages, a Next.js site with dynamic routes created by user submissions, or a WordPress site with hundreds of draft-to-published posts that were never cross-linked—each creates orphan pages silently.
Compare inventories, not one crawl
Build two sets:
- Known public URLs: published CMS records, application routes, sitemap URLs, landing-page reports, Search Console exports, and migration maps.
- Linked URLs: canonical internal destinations extracted from rendered pages, menus, breadcrumbs, pagination, and body content.
Normalise scheme, host, path, trailing slash, percent encoding, and irrelevant tracking parameters before comparison. Redirecting sources, canonical variants, non-HTML assets, and intentionally private URLs should not be counted as orphan content.
The first pass is set subtraction:
func orphanCandidates(known, linked map[string]struct{}) []string {
candidates := make([]string, 0)
for url := range known {
if _, found := linked[url]; !found {
candidates = append(candidates, url)
}
}
sort.Strings(candidates)
return candidates
}
The hard work happens before this function: crawl rendered HTML, resolve relative href values, stay on the intended hosts, ignore fragments, and retain the source page, anchor, placement, status, and canonical for each edge. A raw string search through HTML will miss relative links and can count URLs that only appear in scripts or data.
Prioritising which orphans to fix
Do not invent a score from word count. Review candidates first by current business purpose, index status, impressions or visits, known external links from a tool that actually supplies them, conversion history, freshness, and whether a better replacement exists. The Search Console API does not provide a per-URL external-link count, so that signal needs another documented source.
Keep platform reports read-only
Laravel: Report published models missing from the crawl graph, but leave remediation to the content or product owner. Do not append random links directly into stored body HTML.
Next.js: Compare the route or CMS inventory with links extracted from built and rendered pages. A regular expression over MDX source alone will miss links emitted by layouts and components and may count content that never publishes.
WordPress: Export published permalinks, then compare them with destinations found by a crawl of rendered posts, menus, and archive templates. Core WordPress does not maintain a reliable reverse-link index in postmeta, so a generic SQL query against an assumed metadata key will miss ordinary links in post content and templates.
Choose one remediation
| Decision | Use when |
|---|---|
| Integrate | The page is current, useful, and belongs in a hub, category, related-content, or navigation path |
| Redirect | One genuinely equivalent successor exists |
noindex | The page still serves users or a campaign but should not be a search landing page |
404 or 410 | The resource is gone and has no suitable replacement |
| Leave as-is | The isolated access pattern is intentional, documented, and still accurate |
Do not redirect unrelated pages to a parent merely to retain an assumed signal. Do not add every candidate to a global footer. A valid remediation gives the page a truthful place in the site or removes it cleanly.
Re-run both inventories after each migration, bulk publish, routing change, or taxonomy cleanup. Unknown orphan candidates deserve review; intentional isolated pages deserve documentation.