In my previous post (How to Tell if You’ve Been Attacked by a Penguin – Diagnosing Traffic Decreases) I discussed how to tell if you’ve been hit by one of Google’s Penguin updates. Once you’re pretty sure you’ve been affected by Penguin, what do you do next? Well, you could consult a professional. There are many great consultants or agencies that can handle Penguin recovery tasks for you but this kind of work can be very labor intensive (read, expensive).
There are also many services that claim to handle much of the work you. I have to admit that I’m a little skeptical of how well they evaluate the quality of each and every link that you have coming to your site.
Some of these services include:
Again, I have not used any of these services so I can’t comment on how effective they are or their success rate. What I have done, with success, is a mostly manual review of backlinks to a site and then sent requests for link removals to webmasters followed by a link disavow request to Google. Now, there are likely lots of other tools you could use for this, but I’m going to show you my favorites. Disclaimer: I have no financial incentive of any kind to recommend any of these services; they are simply tools that I believe in and have used with success.
Step 1: Assemble a List of Every Link to Your Site
I strongly suggest that you not rely on any one source of links. No one source, not even Google will give you all the links to your site so I use the tools below to research backlinks. I’d also recommend dumping any manual lists of links you have into a spreadsheet. If you’ve ever hired a link builder and they gave you a list of links they built for you, include that list. Essentially what you are trying to build is a list of every possible link to your site.
I export lists of every back link from all the sources below into a single spreadsheet. Essentially you want a single column in Excel of every link you have. It might cost a few bucks but I recommend subscribing for a least a month to all of the below.
You also want to export all of your backlinks from Google Webmaster Tools. There’s a bit of a trick to this one as many people export the wrong links. You want to go to Google Webmaster Tools, the Search Traffic > Links To Your Site section and under “Who Links Most to Your Site” section click the more link
After that you don’t want the “Download This Table” button, you want “Download More Sample Links”. This will give you the actual links to your site rather than just a sample of the domains linking it.
OK, after you have assembled your massive list of all the links you have pointing to your site you want to remove any duplicates. I finally found a super easy way to do this in Excel and I can’ believe I did this stuff manually for years (or paid someone to do it).
Put all of your links in one column in Excel and make sure you have a heading in the column. Then it’s a simple as highlighting all of your data including the heading, go to Data > Advanced Filter and “filter in place” making sure to check Unique Records Only and BOOM Excel goes to work removing all duplicates from the list. Note that Excel always keeps the first instance it finds and deletes subsequent ones.
Step 2: Evaluate Your Links
Once you have your list of links this is where I prefer to use one particular tool – LinkResearchTools Link Detox Tool. It’s not perfect but I’ve found it’s about 98% accurate in judging the quality of a link. I’ll have to perhaps go into the intricacies of all the things you can do with this tool in a future post but here’s the basics at least. There may be other tools out there you can use for this but this is the one that I’ve found to be the most effective and efficient.
The first thing you are going to get in your report is an overall ranking of the “riskiness” of your links.
This mostly gives you an overall sense of the task ahead of you and the number of links that you may have to request removal for or disavow.
The next thing you’ll see is the full breakdown of the risk or spamminess ratings for every single link pointing to your site. These ratings range everywhere from “Very Low Risk” to “Deadly Risk” and a variety of shades of riskiness in between. I’d recommend looking at any link with a rating of “Moderate – 200” or higher. Note that you can filter or sort this report a ton of different ways and export it to a CSV or even straight to a Google Disavow Links formatted text file.
I prefer here to actually export the whole list to a .csv and then set some conditional formatting on the “Risk” column to call out the Moderate, High, Very High, and Deadly Risk. I also add fields to the sheet for tracking my evaluation of the accuracy of the risk rating, when I contacted the webmaster for both a first and second time and what action they took. I’ve attached an example of the template I use, complete with formatting HERE.
This is where the leg work begins and the part that really does, in my opinion, require an experienced SEO. You need to go through each and every link, and yes that may mean 1000s of links, to judge for yourself if the risk rating is accurate. Why? You simply can’t trust an algorithm, no matter how good it is, to accurately assess the quality of websites (no, that’s not a thinly veiled shot at the Goog). If you haven’t been around SEO for a long time you may not know a good link from a bad at a glance and it’s going to end up taking you forever to assess the links. At any rate, that’s really your next step. You need to do this! You don’t want to be asking for links to be removed if they are OK but the software has rated them as risky. Good links are hard enough to come by; don’t be flushing that value away unless you are pretty sure.
Step 3: Ask for Your Links to Be Removed
So, you’ve rated all your links. What’s next? Unfortunately Google wants to make link disavowal a painful process. Are they doing this because they are mean and petty? Well, no. I get this question a lot and unfortunately the answer is, spammers. If Google made it easy to build tons of crappy links to see if they work to rank a site, and if not, just disavow them and start fresh, spammers would have a field day testing to find the loopholes in Google’s link quality rating. To help prevent that, they make it slow and painful to remove links and they throw in a few curves just to make sure you actually do the work.
The next step is finding contact information for all of the sites for which you want the links removed. You are going to need to contact every single one you can find contact info for and ask (or beg) for the link to be removed. How do you do this? There are a few options: you can manually go to every site and look around for contact info, check the WHOIS info, etc. to find the info if it’s available, or hire a good contractor on a site like ODesk to do this manual work for you, or you can automate. As much as the software Scrapebox has been used for a lot of spammy purposes it’s actually pretty good (and I think a legitimate use) to use it to harvest email addresses for domains. Usually this is used to spam websites in a particular vertical but you can dump your list of domains in and have Scrapebox scrape all those sites for contact emails.
From there you need to contact each and every site to ask for your link to be removed. There are any number of email solutions you can use to send these emails in a relatively automated fashion. One I have been exploring (and will review shortly) is Pitchbox. I’ve only seen a beta version (live version coming very soon) and it allows you to upload a list of email addresses and craft an email campaign that includes dynamically generated emails, tracking of who has responded, and an automated follow up to those that do not respond on the first flight.
Essentially you want to contact each and every site and either get the link removed, or record the website owners response or lack thereof and the date(s) you contacted the owner. Some sites will ask you to pay for link removals. This one is up to you. Some say don’t pay, just record that the site asked for money as the response, and some say go ahead and pay for the webmaster’s time and get the link removed. Please also note that this has created, in a few rare instances, a market for crappy sites to lnk to you on purpose, let you know about the link through a “neutral” third party, then ask you for money when you want the link removed.
Step 4: Beg Google’s Forgiveness (Disavow Your Spammy Links)
You’ve evaluated all your links. You’ve asked for them to be removed. Now what? Well, this is when you can finally ask Google (for the first time) to stop counting the links you don’t want counted against your site. Why do I say the “first” time? Because Google wants to make link disavowal a hard thing to do, they will frequently reject your first request and make you go through the whole requesting removal step two or three times before they actually let you disavow.
One of the keys to success is clearly documenting what you did to ask to get each link removed. The other is actually getting some links removed. That’s one of the primary factors Google looks at to see if you actually did the work – do you have less links now then you did when the Penguin part of the algo flagged your site for lower rankings?
Once you are ready to ask for link disavowal you can do so with the Google Disavow Links Tool. Note that you need to be logged into Google and be the “owner” of the site that you want to disavow for as far as Google Webmaster Tools is concerned. You can see detail instructions of how to format a disavow request here but in essence you want to:
- Take each link (or domain) that you want to disavow the links for and create a line for it in a .txt file (or an Excel sheet that you will later save as a .txt) and add at least one line of comments preceded by a # character telling Google what you did and when to try to get the links removed
- Add a line to disavow either the particular link or links you want to disavow or ask to disavow all the links from the entire domain. The entire domain is likely going to be your default here but there will be a few cases where you are either sure you only have a single link from a domain, or where you want to disavow one particular link but leave others. In the case of a single link just paste the whole URL including the http:// into the line following the comment. For the domain include the command “domain:shadyseo.com” where of course shadyseo.com is replaced by the domain. You do not need to disavow subdomains separately, the “domain:” command should cover all links from a top-level domain.
Here’s Google’s example of how a disavow should be formatted
One thing they don’t explicitly mention and which I found through trial and error is that when you are using the “domain:” command to disavow all links from a domain you must use the format “shadyseo.com” not “shadyseo.com/” or the disavow tool with reject your request and let you know there are errors in the file.
Continue going through each and every link flagged in your sheet for removal for each link. This is another task that you can possibly farm out to a less expensive worker through a site like ODesk, as long as you trust them and are pretty thorough with doing QA on the work they provide.
After you are sure you have your disavow request correctly formatted (if working in Excel make sure to save it as a .txt file) you can submit the request to the Google Disavow Links Tool.
The next question I usually get is – how do I know if my request has been accepted? Well, in many cases you will never know for sure. If you received a notification in Google Webmaster Tools that a Manual Action was taken against your site, submit a Reconsideration Request at this time. Google will usually, but not always, send you a response when they have reviewed your request and let you know if it was accepted or if you need to go back and do more work removing links. If there was no manual action aimed at your links, they will usually tell you that too. Unfortunately if you had no message in GWT and you were likely affected by an algorithmic adjustment to your rankings from the Penguin part of the algo, you will likely never get a response from Google if your link disavow request is accepted. All you can do at this point is watch your Google organic traffic daily. If you see a sudden positive spike with no other explanation, you can assume they accepted and the part of the algo depressing your rankings no longer affects you.
How long does it take for Google to look at my request?
Again, unfortunately, that depends. After chatting with a bunch of my colleagues and from my own experience, it can take anywhere from about ten days to three months. If you don’t see any movement after that, try try again.
Oh, and by the way, don’t expect to bounce back all the way to your previous rankings and traffic. If before you had 10,000 links and Google now counts 8,000 of those as spammy, removing those links should eliminate the “penalty” affecting your site but if you successfully remove or disavow them, you’re only going to come back to where you would have ranked with the 2000 good ones in the first place. I have seen link profiles where I’d recommend getting rid of or disavowing 99% of the incoming links so that means essentially that site needs to start from scratch, as if it essentially has almost no links and is starting almost as a new site.
See now wasn’t that easy. Nothing to doing link removal and disavows, right? 🙂
Unfortunately for many sites Penguin is here to stay and I suspect it will only continue to get better at detecting the kinds of links Google does not want you to build. What links are those? Really any link built just (or even mostly) for SEO is one Google does not like. If they had their way NONE of them would count, and they are going to keep working to find them and discount or penalize them.
If you have not been hit by Penguin DO NOT BE COMPLACENT! Ask anyone building links for you what they are doing. Get second and third opinions on whether those tactics are sustainable. Just because you haven’t been hit yet does not mean you won’t! At the outset estimates were that the algo would allow up to about 80% “bad” links and penalize those above that. The estimate is now 50%. Ask yourself what happens when that changes to 40% or 20%… How sure are you your links are 80-90% the kind of links Google “likes”?
I know that if your site has been around a while and has used the types of link building that worked in 2003 or 2006 or 2009 it feels like a lot of wasted money and effort to now do a ton of work to get rid of those links, but if you don’t start working on it now, how soon until you get attacked by a Penguin?
Get on it!