Traffic Bots: A Brief History

January 19, 2021
by
|
9 min
 reading time
Tired of reading? Then listen to this post:

The war against the machines has always been a subject that fascinated screenwriters and movie-goers alike. Sure, they haven’t yet made a movie about traffic bots, but we’re here for it.

The scenario pretty much always follows the same pattern: humans build machines, machines develop consciousness of their own, and then they rebel against their human masters to take over the world. End of story, cue in the credits. 

Though not nearly as captivating as that, the truth is that a war against a criminal faction of Internet bots is currently underway -- and it has been since the ‘90s. Sure, things have changed a lot since then, and the war against malicious machines is more complex (and expensive) today. 

No worries, Arnold Scharzenegger isn’t dropping by to terminate anything in real life any time soon. We’re talking about bots here. The kind that runs cyberattacks and ruin your PPC budget. It’s a real fight, y’all, and it’s a painfully costly one!

But in order to better understand the fight, we must first answer this question : ‘What are traffic bots and where do they come from?’

What Are Traffic Bots, Even?

An Internet bot (also known as a web bot, robot or, simply, bot) is a software program that runs automated tasks over the internet and performs repetitive functions much faster than human users could. With this ability to perform repetitive tasks quickly, bots can be used for the good, and the bad. Just like carbs can, for example. 

These days, Internet bots account for almost half of the activity on the Internet. Simply put, nearly 50% of all internet traffic is essentially non-human.

bot traffic percentage

Traffic Bots Part I: Once Upon a Time, on the Interwebz

The earliest version of internet bots can be traced back to 1988 and the emergence of IRC, or Internet Relay Chat (if you remember that, you’re kinda old, sorry). The early IRC bots developed by Jyrki Alakuijala, Greg Lindahl and Bill Wisner had only one specific task to perform – preventing the server from closing down a channel due to inactivity.

Today’s most famous internet crawler, the Googlebot, was initially developed in 1996 and followed in the footsteps of WebCrawler, a bot used by AOL and Excite, which was created in 1994 by Brian Pinkerton with the purpose of indexing web pages for search engines (and if you have ever used any of the aforementioned names, you’re probably ancient). 

Quite obviously, these are pretty positive examples of what bots can do. Like, can you even imagine life without being able to Google all those dates you supposedly learned in history class? Ew. 

Chatbots are another good use for bots. We might not always like “discussing” with Messenger bots, but they do a terrific job of saving a lot of money and making Customer Care a little easier and more efficient.  

For example, KLM Royal Dutch Airlines uses a chatbot that allows customers to receive boarding passes, check-in reminders and  other information that is needed for a flight and as a direct consequence has reported that customer engagement has grown since its development and implementation. That’s quite awesome, right?

Traffic Bots Part II: The Takeover

Good bots are, well, good. You can quote ClickGUARD on that. 

Malicious bots, on the other hand, are defined as self-propagating malware that infects its host and connects back to a central server. The server then functions as a command center for a network of compromised computers and similar devices, also known as a botnet. Easy-peasy stuff. 

Some of the earliest botnet programs started to pop up around 1999. Sub7 and Pretty Park are the ancestors to the present-day Trojans and Worms and were released into the IRC network to secretly listen to malicious commands. 

As bots evolved and became more skilled, they started to be used for DDoS attacks and eventually transferred from IRC to HTTP and SSL. 

Traffic Bots: The Rise 

A DDoS attack is a malicious attempt to make a server or a network unavailable to its users by generating large volumes of traffic against a target. This temporary over-saturation ultimately results in its suspension or interruption. 

If you think that sounds serious, just wait til you hear the latest findings of a 2021 report by global information services company Neustar: amid worldwide changes driven by COVID-19 pandemic, DDoS attacks increased by 151% in the first half of 2020 compared to the same period in 2019. Moreover, the consequences of a successful attack on your business range from short-term loss of business to severe long-term effects like loss of online brand reputation. 

DDoS statistics

While the effects of successful DDoS attacks are a major concern for online businesses, they’re not the only bot-nightmare one should consider.. 

Bad bots are responsible for a range of malicious activities, includings: 

  • Web Scraping
  • Credential Stuffing
  • Spam Content
  • Password Cracking and Email Address Harvesting 
  • Click Fraud.

While some bot issues are industry-specific, bad bot problems run across all industries. What’s even more worrisome is that the high variety of bad bots out there and their increasingly high levels of sophistication makes it tricky to build defensive systems against this threat. 

Traffic Bots: Bad Bots as a Service

Two things are perfectly clear by now: there are good bots and there are bad bots. 

But last year brought on something new: rebranded bad bots. These bad boys tried to make themselves look good. Or, more likely, the agencies and companies behind them tried to do so. 

Enter the traffic bots, a bot used specifically and exclusively for increasing the amount of traffic on a website. You’d think that’s great, but when your traffic comes from fake sources, your bounce rate and rankings will not like it. 

Of course, there are a lot of SEO agencies out there. Most of them offer a complex spectrum of website traffic improvement tools. SEO services, PPC campaigns, social media marketing, and more. But one should be wary of those promising fast and spectacular results because, well, no agency can guarantee you ranking or traffic, especially not specific numbers in specific time frames. Chances are that whoever can promise you shocking overnight growth is either using traffic bots or won't deliver to their promise.

You might want it all to happen “fast”, but sooner or later, your site will pay the price of hiring shady agencies and freelancers who use suspicious tactics. 

Risks to Your Business 

This new wave of bot operators can cause an array of damage. 

The first (and most obvious) effect of drowning a webpage in bot traffic is inaccurate analytics data. Page views, bounce rate, session duration, geolocation of users and conversion rates are of little use in interpreting the performance of a website if metrics are heavily influenced by bot activity.

The second most painful effect is that bot traffic doesn’t convert. Bots will not be making purchases on your website or start using your software. Not just yet, anyway. 

Instead, they will land on your site, scramble your numbers, and potentially turn your ad spend inside out (when bots are used for fake clicks, for example). 

In the era of digital marketing and PPC ads, click fraud can take many forms. From accidental clicks by genuine customers, to sophisticated scripts designed to click ad campaigns also known as, you guessed it - traffic bots, this non human traffic costs advertisers billions every year. Without good bot detection software, this bot activity can cost advertisers a large proportion of their ad budget.

Breaking (Traffic) Bots Like a Boss

OK, now you know what bots and traffic bots are, you’ll probably ask yourself: : “How can I protect my business from these bad boys?”

The bot problem is real for every website and mobile app.

In order to make informed business choices, the capability to shrewdly distinguish between traffic created by real human users and the big bad bots is essential.

The most effective way to stop bad bot traffic is with a bot management solution or, to protect your ad budget, a click fraud protection tool. 

Another thing you should start doing TODAY is regularly analyzing your traffic for strange patterns and ask yourself the following questions: 

  • Have your bounce rates increased significantly ? The effect of bots targeting a single page can be an unintended lift in the bounce rate and a decrease in the number of pages per session. 
  • Is there an unexpected increase or decrease in session duration ? 
  • Do you get abnormally high page views? Correlated with traffic spikes for an hour or two (that accounts for almost all your organic traffic of the day), these could be a sign that there are bots clicking through your website. 
  • Has there been a sudden surge in traffic from an unexpected location? Particularly from obscure areas or a foreign country where not a large number of people are fluent in the native language of the website ? This spike can be an indication of bot traffic. 

Be that as it may, we must still remember: prevention is the best medicine, and bot traffic makes no exception from this rule. 

There is no doubt that bots have a role to play in shaping our future. And if you think we can escape it, check out Boston Dynamics’ videos on YouTube. 

With enough safeguarding (and ClickGUARDing, he, he), technology can really make our lives better. Traffic bots and click bots are not part of the plan, though. So let’s squash them by being prophylactically (and forensically) smart about it  

Jason is a passionate data-driven specialist with extensive PPC & SEO experience. When not writing about SEM he can be found surfboarding the wildest ocean waves of the Argentinian coast.