So, How Do You Identify Bot Traffic?

January 18, 2021
by
|
9 min
 reading time
Tired of reading? Then listen to this post:

Traffic generation is one of the biggest marketing challenges of all time. The more traffic your website gets, the better off your company is. Of course, businesses use different traffic generation techniques including (but not limited to) paid traffic. The top 5 enterprises spend $77.3 million per month on Google Ads alone. It is a business’s worst nightmare when it realizes that it receives bot traffic.

In reality, more than half of the internet traffic is generated by bots and 15% of all PPC clicks are fraudulent. According to Google, 56.1% of its ads are never seen by a human. Bot traffic is a serious concern and threat to businesses and advertisers all over the world.

Identifying bot traffic and reporting it to your ad network (so you aren’t charged for invalid clicks) is the only (and the best) solution. If you can’t identify and report bot traffic, you'll pay for it. This article shows you how to identify bot traffic for PPC optimization.

But first…

What Are Bots?

A bot, also known as an internet bot, is a software application that performs repetitive tasks with extreme accuracy. Bots are automated and don’t need human intervention. Web crawlers are one of the best examples of bots that crawl the web, analyze web pages, and collect and store information as needed.

Not all bots are bad. There are tons of bots that are useful in several ways. For example, chatbots are used by businesses to automate customer support. On the other hand, there are bots that are used for deception, cheating, and fraud over the internet. Generating invalid traffic via bots to exhaust competitor ad budget is an example of a bad bot.

What is Bot Traffic

A bot can visit your website, it will scroll, click, and spend time on your website just like a real human visit. The traffic resulting from this kind of action is called “bot traffic”. Your analytics tool will record bot traffic and it won’t be able to differentiate between a bot and a human.

Bots that are used for click fraud are usually part of a large bot farm that is a complete system of several bots that generate thousands of clicks and internet traffic.

Methbot is a classic example of a bot farm that had a very sophisticated infrastructure. It had 1900+ dedicated servers and more than 800K IP addresses, all working towards click fraud. What should be a red flag. It was used for click fraud where it generated fake websites with ads. These invalid clicks ended up costing advertisers $3 million per day consistently by generating 14 billion ad views per day.

Bot traffic isn’t just limited to PPC, as they generate traffic of all types. Bots can visit your website, scroll pages, and even click links without you having a hint of it. All this is a loss of time, effort, and money on your end, as your marketing efforts will not result in a proportionate number of leads and conversions.

Why Do You Have to Identify Bot Traffic?

Detecting and identifying bot traffic is essential for several reasons.

Ruins Your Analytics

First, bot traffic ruins your analytics data. The metrics that you use to monitor the health of your website and traffic don’t show the real picture. For example, bot traffic has a high bounce rate and low session duration. This has dire consequences.

If you choose to unpublish a blog post because it has a low session duration and high bounce rate, you might decide your target audience doesn’t like it (and thus, you will not write anything on the same topic ever again). What if your blog post was receiving bot traffic, though?

If that is the case, you might end up making a wrong decision based on bot traffic. You never know, this blog post might turn out to be a goldmine for your business should you have filtered bot traffic.

When your analytics is flawed, it will hurt your business in several aspects as you'll be making the wrong decisions.

Increased Cost of Tools

Second, bot traffic increases the cost of tools and apps that you are using. Most of the marketing, sales, and other tools that businesses charge you on the basis of your traffic. Here is an example from Hotjar pricing:

More traffic means a higher cost. You need to get rid of bot traffic if you want to pay less for several tools and apps in the market.

PPC Fraud

One of the key reasons why you have to identify bot traffic is to avoid PPC fraud. Bots are extensively used in click fraud to sabotage competitors’ ad campaigns and to earn money. Fraudsters create fake websites, ads, and apps to make heaps of money by selling fake clicks and ad impressions.

Identifying and excluding IP addresses that send invalid clicks to your ads will optimize campaigns and ad costs. It will also improve ROAS, conversions, and several other metrics.

How to Identify Bot Traffic

Differentiating between bots and humans is getting increasingly complicated, as bots replicate human behavior. Your analytics tool (e.g. Google Analytics) can’t detect bot traffic but it can help you identify it. Here is how to do it:

1. Look for Traffic Spikes

If your analytics show an instant spike in traffic for any given day without you taking any initiative for a traffic increase, it is bot traffic.

Here is an example:

bot traffic spike

Without bots meddling with your data, your traffic will increase gradually over time. But when you see a sudden spike, you need to further analyze the traffic on that given day to identify bot traffic.

2. Look for the Right Metrics in Your Analytics

When identifying bot traffic, the three most crucial metrics that you must inspect include: bounce rate, session duration, and the number of sessions per user.

Here is how to identify bot traffic using key metrics in your analytics tool:

metrics for bot traffic activity
  1. Bot traffic has a high bounce rate
  2. Traffic from bots isn’t distributed rather all the visits generate simultaneously within a few minutes
  3. Bot traffic has a low average session duration as its purpose isn’t to spend time on your website. It comes and leaves instantly
  4. Bots don’t click on links, especially internal links. Their entry and exit page are the same and thus the number of sessions per user will be 1
  5. Pages per session will be one too as they don’t move to other pages
  6. All the users appear to be new.

Here is an example of how bot traffic looks like in Google Analytics:

bot traffic metrics

Easy to detect, right?

All the bots visited the website between 8 and10 am, they showed a 100% bounce rate and zero average session duration. This is evident that the traffic was generated by bots.

3. Analyze the Traffic Source

Bot traffic is usually direct. This is because bots don’t find your website in search engines and they rarely originate from a referral. However, bots can hit your website via a referral traffic source but that’s rare.

Once you have analyzed the key metrics, head to the traffic source to see where all those new users came from:

bot traffic source

Out of 56, 51 users aren’t human and these 51 users were nothing but bots.

4. Look at Location and Language

If your website receives traffic from a country or region  you aren’t targeting, it could be a sign that bots visited your website. For example, if you offer plumbing services in a small area in Austin Texas, and analytics show that you are receiving traffic from Germany – that’s bot traffic.

Similarly, if you are targeting a specific country and you are using country-code TLD and you start getting traffic from another country that you aren’t targeting means you are getting bot traffic.

The same is the case with language.

If your website is in your local language, the bot traffic can be identified by looking at the language. If the users have any other language than your website’s language, it is a clear sign you are receiving bot traffic that needs to be excluded.

5. Analyze Your Server Performance

A degraded server performance with a spike in traffic during those hours is another way to identify bot traffic. Bots hit your website for a short period of time and this slows your server as it gets hard for it to handle such a high volume of traffic in a short period of time.

Denial of Service (DoS) attack is usually used to exhaust your website’s resources and crash or even shut down your server by sending heaps of bot traffic. If you ever experience such an issue, you should definitely search for bot activity.

6. Search for Suspicious IP Addresses

Tracing IPs that send a lot of traffic in a short period of time to your website is one of the best ways to identify bot traffic. If a single IP address sends a lot of traffic, you need to block it so that visits from the IP aren’t tracked by your analytics tool.

Excluding IP addresses to safeguard your website from bot traffic is the best way as IP address is the basic constituent of bots.

7. Watch Out for Poor Conversions

If all else fails, consider your website’s conversion rate.

If you notice that you are receiving a lot of traffic but the conversion rate is extremely low, you need to dig further as it is an indication that you are getting bot traffic.

Bots don’t convert (that’s the whole idea behind them, right?)

They just visit your website and leave. If your conversions sink instantly coupled with a spike in traffic, this is another indication of bot traffic.

Also, your conversion rate increases or decreases gradually as you incorporate CRO techniques across your website. It doesn’t fluctuate overnight. Monitor conversions and look for a sudden plunge.

So, What Is There to Do?

Before bots ruin your analytics, decision-making, and cost you money, you need to take action.

Identify bot traffic, find their IP addresses, and exclude them from your analytics. You'll then see the real picture and true traffic metrics. Using a tool like ClickGUARD will help you identify and exclude bot traffic and invalid clicks on complete autopilot. You won’t have to spend time and resources on identifying traffic from bots, leave it to ClickGUARD and it will handle everything for you.

Jason is a passionate data-driven specialist with extensive PPC & SEO experience. When not writing about SEM he can be found surfboarding the wildest ocean waves of the Argentinian coast.