The Traffic Bot: The Real Engine of the Internet?

If you’ve ever spent any time in Google Analytics, it should come as no surprise that nearly half of the global internet traffic now is non-human – in 2021, it was believed that more than 42% of all online activity was driven by the collective of traffic bots roaming wild through the cracks of the World Wide Web.

So…why do bots spend so much time online? What exactly are they up to, besides spreading propaganda and watching cat videos?

We’re going to step back today, and take a look at some of the main reasons why there’s so much bot traffic – and why some people even pay good money for bot traffic to their website.

The Good, The Bad, and The (Bot) Ugly

Good Bots

Some bots, we need. The internet as we know it simply wouldn’t work without them, and lots of things we take for granted would be really difficult to achieve without the ‘good’ bots.

Bots are used to complete a massive variety of repetitive tasks, and can process data on a massive scale. Here’s a few examples of the kind of bot traffic that makes 21st century life easier.

Web crawlers

Also known as spider bots, crawlers are one of the commonest (and most useful) bots on the internet. Your modern search engine experience is only possible as a result of web crawler activity. Every time a web page is uploaded to the internet, within a few weeks it’s (hopefully) been visited by a web crawler. 

The crawler records any useful data about the page and adds the information to the existing data that’s held about that website in the search engine index. 

Web crawlers are also used by SEO tools such as SEMRush, SE Ranking and Moz to analyze and explore search engine results, allowing the user to optimize their content to rank higher in the search results.

Site health monitoring bots

Rather than crawling the wider web, these bots are used to focus on specific websites as a ‘health’ assessment tool. For larger sites, these ongoing checks are essential, to make sure your website is performing to a standard that keeps your visitors happy and engaged. Page load time and responsiveness issues, broken links, and unoptimized images can all be quickly identified and solved with the help of site monitoring and maintenance bots.

Copyright bots

These are used to police creator platforms like YouTube and TikTok, to ensure that creators aren’t using copyrighted materials without permission. Using content recognition software, these bots analyze massive swathes of audio and visual content looking for waveforms and patterns that match the protected materials in their database.

While indisputably effective (try uploading a video of yourself singing ‘Hotel California’ to YouTube and see what happens), there’s a growing consensus that these bots are generating unacceptably high levels of false positives, and unfairly penalizing original creators whose work bears a faint resemblance to previously copyrighted material.

Bad Bots

Unfortunately, for every useful bot that’s trying to make the internet a better place, there’s a ‘bad’ bot that’s been designed for something less wholesome.

Let’s take a look at some typical ‘bad’ bot behavior, before we focus on the murky gray area inhabited by traffic bots.

Ad Fraud Click Bots

These bots are a massive problem for digital advertising (one which ClickGUARD solves every day for marketers all over the world). These bots are designed to click on PPC ads, wasting ad budgets and ruining marketing campaigns. Increasingly, these bots are designed to evade detection by mimicking human behavior – scrolling, visiting website pages in random order, spending more than a couple of seconds on a page. The worst kind of bot traffic for your ads.

DDoS Attack Bots

A Denial-of-Service (DoS) attack is an attempt to shut down a website by overloading it with interactions, queries, and other data. The server becomes overloaded by bot traffic, and incapable of responding to genuine requests from human users. 

A Distributed-Denial-of-Service (DDoS) attack takes things a step further, by distributing the source of the attack across multiple devices and networks. This makes the attack much harder to shut down. 

DDoS attack bots will often be distributed via a botnet – malware that’s been maliciously installed on users’ devices without their knowledge or consent after they’ve visited a dodgy website. The botnet master can then harness all the devices on the botnet – mobilizing bot traffic to attack a website of their choosing.

Shopping Bots

These bots used to carry out Denial-of-Inventory attacks against online retailers. The concept is a simple one – shopping bots simply target high demand items, and ‘add to basket’, until the online inventory is exhausted. Genuine shoppers see the item is sold out, and leave the site. 

The inventory is only tied up in the bot’s basket for a short period of time (usually around 10-15 minutes) – but when these items are targeted in a sustained, automated attack, it’s easy for the bots to keep customers from accessing your stock.

These are just a few of the bot attack methods we encounter every day at ClickGUARD, along with other ‘favorites’ like ransomware, scrapers, and phishing bots. You don’t want these bad bots anywhere near your website OR your ads, if you can help it.

But what about traffic bots?

What Is A Traffic Bot?

A traffic bot is a piece of computer code designed to emulate human visitors to a specific page or website. This increase in traffic can be useful for various reasons, which we’ll explore below.

Traffic bots are used to emulate both paid traffic (from PPC ads) and organic traffic (from search engine results pages). It’s easy to find companies offering organic bot traffic as a service – there’s a whole industry dedicated to providing ‘quality traffic’ for your website. There’s even a thriving arbitrage industry, where industrious middlemen and women buy and sell traffic to make a profit.

But why would anyone want to buy traffic in this way?

Who Needs an Organic Traffic Bot?

Site Flippers

Selling websites is big business. Just like IRL, online commercial property comes in all shapes, sizes, and conditions, and there are big bucks waiting for those who are willing to spend the time and money ‘fixing up’ and selling on a website or online business that’s been neglected.

If a website has a lot of traffic, it’s easy to demonstrate to a potential buyer the gains they could make by publishing ad content on the site. So from a seller’s point of view, the temptation to artificially inflate traffic metrics using organic traffic bots is strong.

Black Hat’ SEO peddlers

Despite Google’s categorical assertions that ‘traffic to a website isn’t a ranking factor’, the confusion caused by the correlation between high traffic volume and high search result rankings is enough to convince lots of people that boosting traffic = better SEO. 

Consequently, there are still plenty of SEO ‘experts’ – and customers – willing to spend good money for organic bot traffic, ‘Google bombing’ web pages in the belief that Google’s algorithms will treat these inflated levels of traffic as a signal of the website’s authority, and boost their rankings accordingly.

Unethical marketers and agencies

Advertisers aren’t exactly regarded as paragons of virtue. There’s a general perception of deceitfulness – and in some cases, the mistrust is justified. Sad to say, the dishonest practice of inflating website visitor numbers with traffic bots is still very much alive and well in 2023.

Anyone with a basic understanding of digital marketing knows that traffic is an empty vanity metric. If none of your visitors convert, there’s little business benefit to boosting visitor numbers. However, many business owners don’t have the time to study these basic tenets of marketing. 

And even when the truth of their inflated traffic becomes obvious, the deceitful marketer or agency will seek to blame the low conversion rates on other factors, such as the product or services offered, or a poorly optimized landing page. However they spin it, you can be sure it’ll provide further opportunities for their marketing ‘services’.

As you can see, there’s very little use for an organic traffic bot that doesn’t involve some kind of dishonest business practice. 

Mostly, organic traffic bots are used to present an appearance of healthy, high traffic. Paid traffic bots, however, are only really used for financial gain.

Paid Traffic Bot – A Game Where No One Wins

Using traffic bots to inflate paid traffic via PPC ads is only ever going to benefit two groups of people. The publisher hosting the ads, or the advertiser’s competitor.

It may be tempting for publishers who already monetize their sites via Adsense to consider boosting their profits a little, by using a spot of sneaky bot traffic just to bump the numbers. After all, there’s plenty of online material to be found advising publishers on the best ways to do this, and no shortage of companies advertising their ‘quality traffic’.

If this is a course of action you’re considering, you’d be advised to take a closer look at Adsense’s ever-increasing vigilance and tighter controls around paid traffic bots. Much of the guidance online for wannabe fraudster publishers either predates or ignores these recent developments. Suffice to say, publisher ad fraud is no longer the easy-win, get-rich-quick game that some people like to portray.

While content-driven ad fraud is still unfortunately a lucrative pursuit for those willing to dedicate themselves to building a criminal empire, it’s not advisable as a casual side-hustle for publishers. You’ll just get an Adsense ban.

Bot Stats

Despite the best efforts of Google/Adsense (and others), ‘bad’ bot traffic has been shown to be steadily on the rise since 2018. As of December 2021, ‘bad’ bot traffic is thought to account for 27.7% of all internet traffic, almost double the levels of ‘good’ bot traffic, which were estimated to be 14.6% of the total. 

Much of this increase can be attributed to the growth of eCommerce and other financial transactions which now take place online – a trend which was only accelerated by the Covid-19 pandemic. 

Black Friday – Bargains For Bots

The Thanksgiving holiday period is typically a profitable time of year for online retailers, thanks to Black Friday weekend. 2022 was no exception, with shoppers eager to purchase all over the U.S. 

Slightly less welcome for retailers was the news that approximately 20% of their Black Friday weekend traffic was bot-driven. Shopping bots, DDoS attacks, ad injections and search engine redirects are thought to have cost more than $360 million in wasted ad spend.

For many businesses and industries, however, protection from bot attacks is a concern all year round. Healthcare, finance, and travel are just a few examples of industries that are continually targeted with bot traffic for click fraud and ad fraud purposes.

Beat the Bot Traffic with ClickGUARD

So, what’s the best way to keep traffic bots away from your business?

There’s a range of options, including hosting-provider solutions, robots.txt, and CAPTCHA forms. However, most of them are less than effective. Robots.txt, for example, is basically optional, and offers no real safeguard against malicious bot activity. Other solutions may provide a barrier of sorts, but either;

• Only provide coverage for your website, without protecting your PPC ads
• Aren’t sophisticated enough to stop 4th generation bots that can mimic human behavior

If you and your business are unfortunate enough to be consistently targeted by malicious bot activity, then your only real option is to invest in dedicated fraud protection software, such as ClickGUARD.

Bot Traffic – FAQs