Quick tipFix itContent HubMarketing Hub

Page shows an error in SEO recommendations

Robots.txt blocks are a common cause.

Subscription
All tiers with pages
Permissions
Pages access
Time
2 min · Intermediate
Last verified

Short answer

HubSpot's KB: if a crawler can't index a page, it returns a crawling error. Rescan from the page's Optimization tab. For Scan blocked by robots.txt on an external page, add the HubSpot Crawler user agent as an exemption.

1. In HubSpot

The relevant screen in my demo portal.

SEO Recommendations tab with Scan new URL and a note about over 40 SEO best practices
Recommendations (1), what the scan checks (2), and Scan new URL (3).

2. Worth knowing

The same errors can come from external tools like Semrush crawling HubSpot pages.

3. Related

SEO recommendations and robots.txt.

Sources

New field notes by email

One email when a new article publishes.

Still stuck?

If this did not fix it, the problem is probably specific to your setup. Book a call and we will look at it together.

Book a call with Sam