2 easy ways to get rid of referral spam in Google Analytics
Nothing sets me off quite like the realization I've done a ton of work for nothing. We spend a considerable amount of time making sure the web products we build achieve our clients' business goals, make them money, keep their customers coming back and all that good stuff.
We also preach the importance of making data-driven decisions when it comes to the web. You can't improve what you don't measure. And you sure as hell can't measure properly with bad data.
So you can imagine the violent thoughts that filled my head the first time I logged into Google Analytics to put together a report for one of our sites, looked at the site's referral traffic sources and saw stuff like best-seo-offer.com, buttons-for-website.com and – wait for it – buttons-for-your-website.com.
This is referral spam. And in terms of high-quality, potentially lead-worthy traffic, it doesn't mean jack.
Referral spam is a collection of faceless bots that crawl the web, collecting information about sites and the people who use them. They can wreak havoc on a site's analytics by inflating traffic numbers, which gives webmasters inaccurate data and a massive headache.
They're a lot like the current lineup of 2016 presidential candidates: a bunch of awkwardly-named robots offering nothing but a highly-distorted view of reality. But I digress.
Some spam, like the infamous Semalt, offers a way to remove your site from the crawl-fest. (I cringe even linking to Semalt. Although the flimsy "why can't we be friends" explanation of their crawler is pretty hilarious.)
But for the most part, it's up to you – the almighty webmaster on the verge of a nervous breakdown and/or alcoholism because the analytics report is due in an hour – to mount an anti-spam offensive.
Luckily, there are a couple of easy ways to fight referral spam from the comfort of your Google Analytics account.
Pros: Instagram filters make your brunch photos look all artsy or whatever. Google Analytics filters are your first line of defense against referral spam because they can stop bad traffic before it gets processed into your analytics data. Think of them as a permanent sentry guarding your account.
Cons: Filters, while powerful, can also be a little risky, because once your analytics data gets processed through the filters you set up, there's no way to undo it. It's permanent. Therefore, before applying any filters to your analytics account, we recommend creating at least one additional reporting view, leaving your original view unfiltered. If you mess up creating a filter and inadvertently turn away good traffic, you'll always have an unfiltered reporting view as a backup. For Integrity's website, we've actually set up three reporting views: unfiltered, a "master" view (for pulling reports) and a test view to experiment with different filters.
To create a new view, go to the Admin tab of your Google Analytics account and select Create New View from the View dropdown menu on the right side of the page. You can create up to 25 views per property, so feel free to go a little nuts.
How to get rid of referral spam URLs with Google Analytics filters:
- In the Admin tab, make sure you're on the reporting view you want to filter, then select Filters from the View column on the right side of the page.
- Click the red + New Filter button at the top of the page.
- Make sure the "Create new Filter" radio button is selected and give your new filter a descriptive name. (This ain't Instagram.)
- Select Custom for Filter Type.
- Select the Exclude radio button and Campaign Source from the Filter Field dropdown menu.
- Enter the URLs of the referral bots in the Filter Pattern field as they appear in your referral report. Separate multiple URLs with a vertical bar (ex: buttons-for-website.com|semalt.com). Note, there is a 255-character limit in this field, so if you still have more spam URLs to enter, you'll need to create a new filter.
- If you'd like, you can select Verify This Filter to get an idea of how your filter will affect your current reporting view's data, based on the past week of traffic.
- Hit the blue Save button, and you're done. From now on, traffic from the crawler URLs you entered should no longer appear in the analytics data of your selected reporting view. You can always go back and edit or delete filters, but remember, once traffic is filtered, it can't be undone. It also doesn't apply to historical data.
How to filter out ghost referrals:
What if I told you some referral spam never actually visits your site? Plot twist. But it's true. Certain bots, using what I'm certain can only be black magic, ping Google's tracking service based on random tracking IDs. If your site's tracking ID comes up, congratulations, you've just been haunted by ghost referral spam that never even made it to your site. The good news is filters work for these bots, too.
- Follow steps 1-3 above.
- Select Predefined for Filter Type, then Include Only from the Select Filter Type dropdown menu.
- In the Select Source or Destination dropdown menu, select Traffic to the Hostname.
- Choose That End With from the Select Expression dropdown menu on the far right.
- In the Hostname field that appears, enter your site address, minus the "www."
- Click Save.
Pros: Google Analytics segments can fill the one big gap left by filters: getting referral spam out of your historical data. Like filters, they're easy to apply (and you only have to set them up once), but unlike filters, they're not permanent. You can enable and disable at will, with no lasting effects on your data. They are also available for any web property in your Google Analytics account, which is especially handy for segmenting out bots across multiple website analytics profiles.
Cons: Since segments are only temporary, you need to be cognizant of whether or not they're enabled when viewing your data, so you don't inadvertently misinterpret the numbers. Also, you do need to apply them each time you open up your analytics; it's not automatic like filters are.
How to get rid of referral spam with Google Analytics segments:
- On any view in your Google Analytics, click Add Segment.
- Click the red + New Segment button.
- Enter a descriptive name for your new segment.
- Head on down to the Advanced section of the segment menu and select Conditions.
- Configure your segment to filter Sessions and to Exclude traffic.
- Select the Source/Medium and Contains parameters.
- Start entering the URL of the referral bot in the text field, and it should auto-populate. To add multiple URLs, click the Or button on the right side of the segment setup window and repeat steps 6 and 7.
- Click the blue Save button when finished. Your segment will be applied immediately. You may need to turn off the default All Sessions segment so just the one you created is applied. If you want to edit your segment, click the caret icon on the far right of the segment's bar and click Edit.
It's important to note that neither filters nor segments can completely outsmart referral spam URLs. There inevitably will be new spam addresses that pop up in your reports in the future; you'll just have to edit your filters or segments to add them to the block list.
If you haven't noticed by now, we're just a tad obsessed with collecting high-quality, actionable data so that we can keep building amazing things on the web. Looking for that attentiveness to detail on your next big project? Let's talk.
The latest from Integrity
Why Your Site Needs a UX Audit
We have all been in the position of visiting a brand’s site and quickly giving up. This may be because you were unable to quickl...
Martech Innovation: Prepare Your Small Business for a Post-Pandemic World
There is no getting around it. Small businesses are the backbone of the U.S. economic engine. At a micro-level, they power local...