How To Detect Click Fraud With Google Adwords To Avoid Invalid Clicks - Detection Monitoring Prevention Software Tools

NOTE: If you want to learn more about the tool below, please visit www.teasoftware.com

This story has two parts, one is about how we traced Google Adwords to be sending up to 80% of garbage traffic to our client websites, and the other is about how we measured visitor engagement from the remaining 20% of the real traffic.

As a founder of Shopping Cart Elite, I hear the following pain points every other day from new, existing and potential customers over and over again. Here, are the top ones:

  • I tried advertising with Google Adwords, and it was a waste of money, I had no sales!

  • I am barely breaking even by advertising with Google Adwords. How do some of my competitors spend $30,000 per month and stay profitable?

  • How do I know which keywords to bid on to ensure the best return on my investment?

After hearing this pain point from practically all of our customers, we decided to address it once and for all. Our goal was to develop software that can measure every visitor engagement on the website, identify and conclude what they did on the website. Then we would generate a detailed report with our findings. The software application that we started developing is called Traffic Quality Analyzer and it is natively built inside Shopping Cart Elite.

WHY GOOGLE ADWORDS CONVERSION TRACKING DOESNT WORK

Even certified Google Adwords Search Engine Marketing agencies will tell you that the first month will not result in many sales as it will be more trial and error to tweak the campaign. Then they will need a few more months to make it profitable for you. But what they dont tell you is how they will measure and tweak your campaign over those few months. What agency will do is monitor your direct sales using Google conversions tool. Using this information they will modify your campaign. But if you get no sales because your visitors are not engaging with your website due to price, poor product selection, poor website design, etc., then you will put all the blame on Google or the marketing agency that managed your campaign, and miss out on the biggest advertising opportunity for your business.

The strategy for optimizing your Google Adwords campaign using Google conversions is used by the majority of the Adwords advertisers. This is a black and white strategy which rarely works out, even for companies who have deep pockets.

Now here is what happened while we were developing the Traffic Quality Analyzer. We stumbled upon some data which SHOCKED US! It had to do with the biggest internet marketing subject that is being IGNORED by the search engine industry.

Seriously, look up the subject about Clickfraud. No one is talking about, and Google shut everyone up by settling the class action lawsuits. Google even openly admits to Clickfraud, you can read many of their settled cases here, and here is a legitimate source supporting that claim.

Google claims they fight Clickfraud, but there is a problem with their claim. You see there is no way to fight Clickfraud without tracking visitor engagement. Theoretically Google can eliminate most of Clickfraud by not charging the advertiser if they have a slight suspicion of Clickfraud, but if they do that it will be on Googles dime. Google makes close to a billion dollars per month from Adwords advertisers, and if they did what I suggested, they would be taking a 20% - 30% loss of revenue.

Google Clickfraud system does work, but since there is no way to audit their findings, it is very biased. Think about it, Google wants to charge you money, and you want them to charge you less. Even if Google gives a refund for what they claim they found as Clickfraud, how do you know that they dont owe you more money? Google Adwords Clickfraud system is flawed. There is a need for an independent Clickfraud detection system to challenge Googles findings. Here, is a metaphorical example, Google is like the Federal Reserve. We all take a leap of faith by trusting that the US Federal Reserve will continue to print an unlimited amount of money at the expense of the tax payer (YOU) and use it wisely without ever being audited.

VISITOR ENGAGEMENT PROBLEM

Why am I talking about Clickfraud so much? Well the thing that we stumbled upon that shocked us was bot traffic that looked 100% human. One of our clients bought traffic that specifically indicated that it will guarantee real visitors to your website for pennies on a dollar. When we started implementing our Traffic Quality Analyzer it passed all the tests to indicate that this is a high quality visitor. The problem was no one was buying anything, so after thoroughly investigating it, we found that it was a highly sophisticated bot that acted just like a real human. Thats right, the bot was programmed to be a human visiting your website and engage with your content. In addition, the bot was infected with millions of computers which make it almost impossible to detect it.

So in conclusion our Traffic Quality Analyzer cant work unless we first identify bad traffic and eliminate it before measuring its engagement.

We tried to find a solution to solve this problem. After an extensive research, I found only two companies (Adometry and AdsafeMedia) that offer software that does half a decent job in identifying click fraud, but they charge thousands of dollars per month and its a pain to implement. The few smaller companies that I found had very basic technology that would only track duplicate IPs and nothing else.

DEVELOPING OUR OWN CLICK FRAUD DETECTION SYSTEM

Since, we had no other options we decided to develop our own Clickfraud and Bot traffic analyzer, which would become a part of the Traffic Quality Analyzer. We came up with a ranking system which would identify several factors such as:

First we would measure individual visitors and to identify a rogue bot or known offender

  • Browser Type (empty or outdated)

  • Known Offender IP

  • IP that is less likely to be a real visitor

  • No mouse movement

  • Random non link clicks

  • Login history

  • Registration history

  • Checkout success

  • Quote success

  • Sent a Contact Us Message

  • Time on site

  • Returned to site

  • Bounce Rate

  • JavaScript Enabled or Disabled

  • Refer is on Whitelist or Blacklist

  • On-site Page views

  • Type relevant search keyword

  • Triggered a relevant search from dropdowns

  • Click through frequency

  • History of Browsing

Second we would analyze the traffic over a period of time to create a big picture per campaign

  • Frequency of the same IP visit from a paid refer

  • Frequency of visits by hour throughout the day

  • Visits from the similar IP sets

  • Visits from a Geographic location

  • Daily Revisits of the same IP and was the behavior similar

SHOPPING CART ELITE CLICKFRAUD DETECTION vs. GOOGLE CLICKFRAUD DETECTION

Here, is why our system works better than Google. We take the side of our client, not Googles. If we have even a bit of suspicious that the traffic is Clickfraud we will red flag it while Google will ignore it. Our Clickfraud threshold is close to 0% as we have zero tolerance to suspicious traffic while Google threshold can range between 5%-20%. Google takes the approach of innocent until proven guilty while we take the approach guilty until proven innocent. Google also has no way to measure engagement so their system can never work to detect 100% of Clickfraud. You can take Googles word that their system detects all Clickfraud, but dont fail to ignore the fact they openly say that Clickfraud can go undetected, and you will be charged for it regardless. Even if they later find out it was Clickfraud you will still lose money. Weve found Clickfraud to be as high as 80% from some publishers on the Google Adwords Content Network.

Now lets imagine that Google will somehow match our Clickfraud threshold and implement a zero tolerance policy. The Clickfraud problem will still remain because Google is unable to measure Key Performance Indicators (KPI) of the website. They cant understand if the visitor behavior is useful for the advertiser or not because every website is different and has different goals.

Remember earlier in the post I said we found bots that acted like humans. Neither our Clickfraud detection nor Googles can detect all bot traffic alone. You must measure visitor engagement on top of the Clickfraud detection in order to detect 100% of Clickfraud traffic.

How do we do it? Well bots are still bots and even though they act like humans, they usually do not make human decisions. Even though, bots might engage with your website content, they do it in a way that makes their behavior very suspicious. For example, a bot would click on contact us (which gives it ranking points), but then it submits a spam message (losing all of those points), or a bot will go to a product page and engage on an area which has no content. Google is unable to see if a spam email was sent to you by that visitor, or which visitors are valuable to your business. Bottom line is Google is in the business to send you traffic while minimizing the damage, they are not in business to do this type of analysis to save you money and improve your campaign. Google position is for you to hire a search engine marketing agency to do this job for you, except they wont.

Basically what we did was put traps all over the website template. We implemented them in many different areas so we can trigger Key Performance Indictors. Then we compare the data between real visitors vs. bot visitors.

We red flag any suspicious visitor behavior and the store owner can actually watch a recorded video of that visitor when he browsed the website. YES, our Traffic Quality Analyzer actually records each visitor website activity and makes a movie out of it.

If the store owner has identified a bot in that video, he can submit that to us, and we will tweak our algorithm to address those bots in the future.

RESULTS FROM GOOGLE ADWORDS CAMPAIGN

Using this system we started tracking different campaigns on a few of our client websites, and here are the results.

  • Google Shopping produced an average of 6% of Clickfraud

  • Google Adwords (Google Search) produced an average of 16% Clickfraud visits

  • Google Adwords Content Network was all over the board. It ranged between 25% - 80% of Clickfraud

We do not know if Google reimbursed our clients for the above Clickfraud as we only analyzed the data. But my guess is we wouldnt know the details anyway because Google doesnt disclose details. Our goal was to identify rogue traffic, generate a report for the client to request a refund for it, and allow our client to modify his own campaign by tweaking keywords or blocking the specific publisher.

Wouldnt you sleep better at night knowing that 100% of the traffic that you paid for is real? I know I would.

Also, note that the results above are only Clickfraud and not engagement. We believe the Clickfraud numbers would be higher if we also measured engagement against the same traffic. Engagement will not only help detect bot traffic, but it will also detect the quality of real visitors traffic. It will tell you if they are engaging with your content, and it will give you specific details about the results. This will also allow you to tweak your campaign accordingly.

CLICKFRAUD PLUS ENGAGEMENT CHECK

We are very excited to roll this feature out to our clients because not all traffic is equal. More broadly, if overall traffic for the site is down, but the number of engaged visitors are up, that still means the site is doing well. By implementing this feature, we will be able to give the exact number of visitors that actually matter to your business. For example, if you have a small website and you get only 200 visitors per day, our algorithm may show you that only 20 visitors are real and engaging with your content. You will then be able to see the quality of their engagement by watching a movie of their activity that we recorded. Then you may decide to advertise with Google Adwords or Facebook, and all of a sudden you see 50% engagement from that campaign, you will be a lot more comfortable spending money on that advertisement than you otherwise would.

Next time you see a 50% bounce rate on your website dont come to conclusions that people are leaving your website. It might very well be bot traffic.

TRY OUR CLICKFRAUD DETECTION SYSTEM

We are still tweaking the Clickfraud detection system and the engagement system of the Traffic quality analyzer, and it is currently in private beta. If you spend more than $1,000 per month on Google Adwords and would like to try out our system, fill out the form here.

P.S. I will make another post soon about the details of the engagement algorithm.
By: Shopping Cart Elite