Soft 404s: fix missing pages without breaking empty categories

A soft 404 is a URL that Google treats as an error even though the server returned a success response. Start by opening the URL in Search Console and checking the rendered page. It may show an error message, missing main content, or an empty result caused by a failed request. Google's soft 404 guidance covers these checks.

An empty-looking page is not always missing. An existing category with temporarily unavailable products can still help visitors if it explains the situation and offers accurate alternatives. Decide what the URL represents before changing its status.

Compare the affected page with a working one

Check the exact URL in URL Inspection and compare its rendered main content with a healthy page from the same template. A word count or phrase match can help triage a large export, but Google does not publish a phrase list or percentage formula. A short page can be legitimate; a long custom error page is still an error.

Find candidates in a URL list

A conservative script can label review candidates without pretending to reproduce Google's classifier:

import requests
from bs4 import BeautifulSoup

ERROR_PHRASES = ('not found', 'no results', 'nothing here', 'no matches')

def triage_empty_response(url):
    try:
        response = requests.get(url, timeout=10)
        text = BeautifulSoup(response.text, 'html.parser').get_text(' ', strip=True)
        return {
            'url': url,
            'status': response.status_code,
            'words': len(text.split()),
            'error_language': [p for p in ERROR_PHRASES if p in text.lower()],
        }
    except requests.RequestException as error:
        return {'url': url, 'fetch_error': str(error)}

Use the output alongside the route type and rendered screenshot. The script checks the initial response only, so render pages that depend on JavaScript before deciding whether their content is missing.

Match the response to the URL state

In Laravel, keep a no-results search page distinct from a missing route:

public function search(Request $request)
{
    $query = $request->get('q');
    $results = Product::search($query)->paginate(20);

    if ($results->isEmpty()) {
        return response()->view('search.empty', [
            'query' => $query,
            'suggestions' => $this->fallbackSuggestions($query),
        ], 200);
    }

    return view('search.results', compact('results', 'query'));
}

Apply noindex to internal-search results in the template or response headers if they should not appear in Search. Do not block those URLs in robots.txt before Google can read the directive.

For categories, check the intended state as well as the product count. Zero products does not establish that a category is permanently gone, and out-of-stock products do not guarantee that the empty page remains useful. Keep a 200 response only when the category still exists and the page gives visitors accurate information. Return 404 or 410 when the category has been removed without a replacement.

Reject invalid and out-of-range page numbers

Check zero, negative, malformed, and over-limit page numbers. A valid framework-level not-found response must produce a real HTTP 404, not a 200 app shell that changes into an error after JavaScript runs:

export async function getStaticProps({ params }: { params: { page: string } }) {
  if (!/^[1-9]\d*$/.test(params.page)) {
    return { notFound: true };
  }

  const pageNum = Number(params.page);
  if (!Number.isSafeInteger(pageNum)) {
    return { notFound: true };
  }

  const { posts, totalPages } = await getPostsPage(pageNum);

  if (pageNum > totalPages) {
    return {
      notFound: true, // Returns 404
    };
  }

  return {
    props: { posts, page: pageNum, totalPages },
  };
}

After the fix, check the live status and rendered content for the affected template. Include the first page, the last valid page, and a URL beyond the end so the same empty response does not recur under another page number.