We have been studying carefully the forums and results from the various link analysis tools that we use and other information available around the web. Our conclusion is based on this analysis and also informed by discussions with other SEO professionals is as follows:-
Google has a process for crawling, it crawls, caches then analyses web pages, the process follows a regular pattern for established pages, so a directory listing page may be crawled once per month, whilst a newspaper home page several times per hour. When links are discovered they are recorded.
We understand that when you add a disavow link or domain to your webmaster tools, an extra process is added to the above routine, being a check if a link is disavowed, if it is, then it is treated differently, either its ignored or more likely recorded with a different status. Something in the way that a link can have a no-follow attribute disavowed links have a disavowed status.
This means that when you add a disavow tool, it doesn’t instantly get implemented but is used to control how links are recorded as part of Google’s regular crawl. So if your links are from weaker pages they will take longer to get re-crawled, whilst stronger links will be re-crawled quicker. It does explain why pinging the links you disavow can potential lead to a faster results showing and why recoveries are spread over a number of weeks, matching the time to crawl the majority of the links added to the disavow section of Google Webmaster tools.
Tips for best disavow practice
Canonical version – if you have multiple versions of your domain name know as the canonical versions for example http:// http://www. and https:// remember that these are all treated as unique entities by Google and you, therefore, need to upload your disavow to each version, so three times !
In theory, you should just disavow the links that point to each version, but in practice, we use one file and upload it to all versions, links change over time so better to adopt a blanket approach.
The frequency of updates – A very busy site with high traffic and a large number of links, that are growing all the time should update their disavow on a weekly basis, for other sites at least monthly.
Now the Google Penguin is updating real time it is important to be on top of your disavow file at all times and quickly respond if there is an unexpected drop in the rankings of any phrase.
How to create a disavow file – How to work out which links are good, and which ones are toxic? That is the main issue! We use and recommend Linkresearchtools which analyses your links and automatically produces a disavow file for your use.
If you have links from obscure sites with thin pages such as profile pages they may get crawled even less frequently than normal meaning a recovery can take months due to the lag in crawl times.
Even if your site has not been hit by a penalty manual or algorithmic Google recommends that you use the disavow tool to be pro-active, this does seem to make a lot of sense particularly as the quality criteria are for ever rising and the along with it the need to have cleaner and better back link profiles
The impact of disavowing is of course heavily influenced by your overall situation, if your site has manual penalties or algorithmic ones. Sometimes you can have both which makes it hard to see what is going on.
If you are suffering from Penalties talk to us, we may be able to help you recover, read more here about our services for recovery.
References
1.. Our blog post on using Linkresearchtools