Meaning of Ad Crawler Errors/404 and Practical Solutions

You're here because you've got the "You have ad crawler errors, which can result in lost revenue." notice on your AdSense.

Google doesn't only want to serve their advertisers, but also the publishers who are helping them to place ads in their content.

This is the very reason they've sent you this message. Google wanted you to be aware that a few things might be wrong with your blog, website, links, pages, or app - where your AdSense codes are being placed.

The system is aware of some pages, that have the potential to, are not running ads. And this is a result of your own errors on those pages to which you may be losing money.

Hence, you're informed and required to take the appropriate actions.

What are the Ad Crawler Errors?

When you receive the Ad Crawler Error notice, it signifies the situations where Google crawlers/bots find it difficult to or unable to serve your page with the right amount of ads, where needed, or at worst find no keywords to measure the right ads for that crawled pages.

We're both aware that Google carefully chooses where their ads show up within our content, with or without auto mode, even if you've put codes on the very page.  

In other words, it doesn't matter, sometimes, where you wish ads to run on certain pages. If Google doesn't care for those pages, perhaps because people don't visit them or no natural traffic for them, they won't let ads show up.

This was well explained in, "11 Reasons AdSense Ads are Not Showing and Solutions".

But where Google cares for pages that people are visiting but no quality content, enough information or none at all, is present, to measure the ads people should be served with, they realize they're losing 32% of the earning potential of the pages while you're losing 68%.

Consequently. they will call your attention to this mess and expect immediate action. That's why you got this notice.

What Usually Leads to Ads Crawling Errors

One main reason you'll be served with the ads crawler error is if you have error 404 which is blatantly a cruel error not only to you but also to Google and your readers.

Let me explain.

Error 404 directly means your pages can't be found but people are visiting the page using existing links. In place of the potential content, they're greeted with blank pages or pages displaying "errors 404".

Wondering how it's possible for people to be visiting a page that doesn't exist on your blogs or websites?

Originally, the page was once alive. You might have once written a post or created a page but later deleted, hid, unpublished, or reverted it to draft.

Whereas the page/post has been ranked earlier, shared somewhere on the net, or with a few people before being deleted or hidden.

Henceforth, if anybody clicks on its link, he will be greeted with the error 404.

Whereas, Google only displays ads on regularly visited pages. And if the error 404 persists, what you will get in return, on your AdSense page, is the "You have ad crawler errors, which can result in lost revenue." warning.

This error means that someone is using your ad code on a different site without your permission. In this situation, it may that you, by mistake, shared your content, containing your AdSense code, on another platform not approved for AdSense.

Your AdSense code can only show ads on the approved website, blogs, or apps. If for any reason, those codes are input on a different website, blog, or app, other than the originally approved one, Google won't be able to serve the ads there and consequently will send you this error notice.

How to Undo or Correct Ad Crawler Errors

Of course, you can ignore this notice if you're sent one by Google. However, you're not doing yourself any good

This is because Google frowns at error 404 just like violating other Google publisher policies.

It'll be ridiculous, on the part of the search engine, to rank your empty pages for their users. It's not only losing them the advertisers and income but also the readers.

The long-term effect of this is loss of ranking. The next algorithm update may not smile at your blog or website if you turn deaf ears to this warning.

Hence, it's important you handle things before it's too late.

Below are the steps to take if you must stop the ad crawler/4040 errors.

Follow Google Warning Notice to Locate the Affected Links

On your Adsense dashboard, right after the "You have ad crawler errors, which can result in lost revenue", you will see two buttons. One is "Action", and the other is "Dismiss".

You don't want to hit dismiss because you don't want to ignore this warning as earlier noted. Hence, click on the "Action" link.

On the resulting page, you should see the number of pages affected by this error.

In our case, here, only one page was affected.

Below that section, you should see a short explanation of what happened to this page.

In this case, the error type is Crawler: Page not found. 

Of course, there may be other related errors (usually resulting in not being able to crawl and serve suitable ads).

And right below that should be the website's URL affected. Clicking on this will take you to the actual link with the error on the website or blog.

If the affected link is clicked, you will be greeted with, Oops; Sorry, but the page you were trying to view does not exist "404" or a related message stating that the page is no longer or not found on the server.

If you take a moment to check through this page, apart from this error and other on-page details, you won't see any Google ads served on the page or around its corners.

This shows you that Google has discontinued the ads on it because it's not serving any purpose. 

However, it can be restored, and it has the potential to be served with ads.

What to Do After Locating the Affected Link

If the page was hidden or reverted to draft, you can just republish and bring it alive again. All it takes are just the click of a few buttons.

The problem will sooner or later be rectified and you, your reader, and Google can smile again.

However, if the page had been deleted, no other choice than to look for all the sources of the links and get them all zapped (deleted).

To do this, take the steps below:

1. Search for the link around your blog and see where it has been included before among other posts
2. This may be difficult if done page by page. A quick way is to copy the affected link, paste it, and search for it on Google.
3. Google will show you all posts or pages where the affected links are already exist
4. You can now trace each of these posts on your blog. Then, remove the affected links
5. For the links personally shared on social media, delete them from your social pages.
6. For those shared by others, on social media, or elsewhere, you can request a manual deletion from their ends.

You Can Redirect the Pages to Existing Posts

One of the very reasons we delete posts is to avoid duplication of content thereby allowing a better post to rank.

If you have any existing or can write one now, you should redirect that traffic, getting to empty, pages, to them.

Conclusion

Ad Crawler Error might have not been as you might have imagined. You'll be losing earnings as long as you have it. Readers will be disappointed not seeing what they'd come for. All these might not matter if the pages affected have no significant results compared to the overall performance of your blog/website.

However, you don't want to see it as something little because Google will sooner or later respond to this error by dropping your posts/blog on the SEARP.

Do you have this error present on your Google AdSense dashboard right now? Have you ignored such in the past without a drastic effect on your earnings and ranking? Share your experience with us in the comment section below.