Crawler / audit evidence

ILCrawler - Technical SEO crawler and audit workbench

Mixed-language crawler/audit platform for turning raw crawl, rendered, Lighthouse, resource, issue, and export evidence into reviewable SEO handoff data.

GoRustPythonNode.jsNext.jsPostgreSQLS3

What this proves

System behavior and guardrails

  • Go raw crawling with bounded concurrency and batch spooling
  • Rust idempotent importer for large raw evidence sets
  • Browser rendering and local Lighthouse workers
  • Operator UI for projects, runs, issues, exports, and health

Project scope

ILCrawler

Mixed-language crawler/audit platform for turning raw crawl, rendered, Lighthouse, resource, issue, and export evidence into reviewable SEO handoff data.

Private data and credentials are intentionally omitted. The architecture and operator surface are shown only where they support the implementation claim.

Operator surface

Selected interface evidence

Redacted ILCrawler interface evidenceRedacted ILCrawler interface evidenceRedacted ILCrawler interface evidence

Need a related technical system or diagnostic path?