Dynamic Serving Tests

Test 1

Aim, determine if there are crawling differences with/without the HTTP Vary response header.

1 & 2 serve the same content to both Googlebot/Googlebot (smartphone), adding a Vary header in the latter. 3 & 4 serve different content based on the user agent, the latter also includes a Vary header.

• 1, No dynamic serving, no Vary header
• 2, No dynamic serving, Vary header
• 3, Dynamic serving, no Vary header
• 4, Dynamic serving, Vary header

Test 2

Aim, determine if URLs served only to Googlebot (smartphone) are used for discovery.

Both tests use dynamic serving, the latter of the two also uses the HTTP Vary header. The mobile version of both tests link to unique URLs not seen by Googlebot, the unique URLs are available for both Googlebot/Googlebot (smartphone) to crawl.

• 1, Dynamic serving, no Vary header
• 2, Dynamic serving, Vary header

Test 3

Aim, determine if URLs crawled only by Googlebot (smartphone) are indexed.

Both tests use dynamic serving, the latter of the two also uses the HTTP Vary header. The mobile versions of both tests link to unique URLs not seen by Googlebot. The unique URLs are crawlable by Googlebot (smartphone), however Googlebot will receive a HTTP 403 (Forbidden) response code.

• 1, Dynamic serving, no Vary header
• 2, Dynamic serving, Vary header

Test 4

Aim, determine if meta robots noindex tags served to Googlebot (smartphone) are actioned.

Both tests use dynamic serving, the latter of the two also uses the HTTP Vary header. The mobile versions of both tests include a <meta name="robots" content="noindex" /> tag that isn't in the desktop HTML.

• 1, Dynamic serving, no Vary header
• 2, Dynamic serving, Vary header

Test 5

Aim, determine if rel="canonical" tags served to Googlebot (smartphone) are actioned.

Tests 1 & 2 have no rel="canonical" tag on the desktop content, but do on the mobile version and the latter also has a Vary header. Tests 3 & 4 have a different rel="canonical" tags on desktop and mobile versions, the latter also includes a Vary header. Tests 5 & 6 return HTTP 403 (Forbidden) to Googlebot, Googlebot (smartphone) can crawl the URLs which both include a rel="canonical" tag, the latter also has a Vary header.

• 1, Dynamic serving, no Vary header
• 2, Dynamic serving, Vary header
• 3, Dynamic serving, Vary header
• 4, Dynamic serving, Vary header
• 5, Dynamic serving, Vary header
• 6, Dynamic serving, Vary header

Test 6

Aim, determine if HTTP X-Robots-Tag noindex headers served to Googlebot (smartphone) are actioned.

Tests 1 & 2 serve HTTP X-Robots-Tag noindex response headers to Googlebot (smartphone), the latter also includes Vary headers.

• 1, Dynamic serving, no Vary header
• 2, Dynamic serving, Vary header

Test 7

Aim, determine if HTTP Link rel="canonical" response headers served to Googlebot (smartphone) are actioned.

Tests 1 & 2 serve HTTP Link rel="canonical" response headers to Googlebot (smartphone), the latter also includes Vary headers.

• 1, Dynamic serving, no Vary header
• 2, Dynamic serving, Vary header

Test 8

Aim, determine if anchor text seen by only Googlebot (smartphone) has an impact on rankings.

Tests 1 & 2 use dynamic serving, the latter uses Vary headers. The mobile versions of each test link to a URL not seen by Googlebot with anchor text unrelated to the content on the linked URL. The unique URL linked to by the mobile versions of the content is accessible to Googelbot (smartphone), however Googlebot will receive a HTTP 403 (Forbidden) error trying to crawl the unique URLs.

• 1, Dynamic serving, no Vary header
• 2, Dynamic serving, Vary header

Test 9

Aim, determine if any content from any of the aforementioned tests uniquely served to Googlebot/Googlebot (smartphone) appear in the in index via desktop or mobile search.