Technical SEO · Quick reference

Indexing signals reference: robots, noindex, canonicals and sitemaps

Reachability, crawling, indexing and ranking are separate questions. SiteAudit inspects one page and bounded checks for its host. Use this reference to interpret its findings, then use Search Console to investigate Google’s actual crawl and indexing decisions.

Access and indexing controls

Choose the intended audience first. A public guide and an authenticated account page need different access and indexing policies.

Access and indexing controls
Signal	Role	Review point
HTTP status	Describes the response served to the request.	Confirm the final URL serves the expected HTML, not an error or challenge page.
robots.txt	Controls crawling for matching user agents.	Reachable robots.txt is not proof that the scanned page is allowed by every rule.
Meta robots: noindex	Asks supporting engines to exclude a crawlable HTML page.	Allow the crawler to read it; remove it from a page intended for search.
X-Robots-Tag	Can deliver indexing instructions in response headers.	Inspect headers as well as HTML; it can also apply to non-HTML resources.
Authentication	Restricts access to content.	Use for private content; robots directives are not access control.

Reference: Google: Block indexing with noindex

Page identity and metadata

The scanner reads the first matching HTML fields without rendering JavaScript. Check the actual page purpose and look for duplicate declarations when the output is unexpected.

Page identity and metadata
Field	Purpose	Next check
Title	Names the page for people and search systems.	Make it specific to the page; presence does not assess its quality.
Description	Provides a possible search snippet summary.	Describe the content accurately; Google may use another snippet.
Canonical	Signals a preferred equivalent URL.	Use a reachable production URL matching the content; it is a signal, not a guaranteed selection.
Robots	Provides page-level indexing instructions.	Review noindex and any response-header instructions together.
Language / viewport	Describes document language and mobile viewport.	Check rendered content and mobile usability separately.
Social metadata	Supports link previews.	Optional; presence is not a ranking measurement.

Reference: Google: Canonical best practices

Discovery and host evidence

Use stable public URLs in a sitemap. SiteAudit checks the conventional /sitemap.xml location and reports references from robots.txt without following those references.

Discovery and host evidence
Evidence	What it establishes	What remains to check
Sitemap XML	The fetched XML and sitemap root can be validated.	Included URLs, completeness and Google’s fetch result.
Internal links	Provide paths for people and crawlers to discover content.	Check that important pages have ordinary links from relevant public pages.
DNS A / AAAA	Address records for the exact scanned host.	Endpoint reachability and any differing responses across addresses.
DNS CNAME	An alias for the exact scanned hostname, when published.	Verify the intended hosting destination; direct address records can be valid without an alias.
Mail DNS records	Presence on the exact host only.	A missing record on www may be intentional; delivery and parent-domain policy are not established.
DNS CAA	Presence of certificate issuance policy records at the scanned name.	Applicable alias and parent-name rules, issuer authorization and DNSSEC are not evaluated.

Reference: Google: SEO starter guide

Compare a clean URL and a tracking variant

When two URLs serve equivalent content, a canonical can identify the clean page. Keep your internal links and sitemap consistent with it. Do not remove parameters that change the resource without checking equivalence.

<link rel="canonical" href="https://example.com/guides/caching">

Inspect the final response headers and the HTML head on both variants.
Open the canonical destination and confirm it is equivalent and intended for indexing.
Use a live URL inspection, then compare your declared canonical with Google’s selected canonical.

From scan to solution

Turn a website audit into a clear next step

SiteAudit brings six checks together for one page and its host. Use the findings to investigate access, metadata, indexing directives and technical configuration. These guides explain how to prioritize the results and where a single-page check needs to be followed by a wider review of your website.

Technical SEO