In marketing, measurement is always a thorny issue. Traditional marketers sometimes shun measurement, in part because it’s downright tricky. For example, how can you accurately measure how many people see a billboard or a magazine article?
Web marketing makes things easier, because along came web analytics and a host of related tools to rely on. However, measuring reach can be difficult.
For example, at Capulet, we recently ran a small blogger outreach campaign for the Clean Air Foundation. We approached a bunch of bloggers to write about their Mow Down Pollution program, and some of them did.
Yet, how many people actually read those dozen or so blog posts? And how many will read them over the next year (or five)? That’s reach. And while there are a few tricks, there’s a lot of guesswork.
Reducing the Guesswork
A company like Alexa has, for years, been trying to reduce that guesswork. We’ve also been trying out Alexa competitor (heh) Compete. Both use communities of users who agree to install browser plugins and contribute their usage data to the company’s database. I’m no statistician, but I’m pretty sure there are significant issues with this method of data collection. But let’s ignore that issue for a minute.
I wanted to check out Compete’s data. I have access to the stats for a varied group of websites, so I thought I’d run some comparisons between the relatively empirical Google Analytics and Compete.com’s estimates.
The following table shows monthly visitor totals for nine websites, and the ratios between Google Analytics and Compete.com’s numbers.
| Type of Site | Analytics | Compete.com | Ratio |
| Blog #1 | 117227 | 32944 | 3.55 |
| Blog #2 | 2030 | 1879 | 1.08 |
| Blog #3 | 15050 | 688 | 22.5 |
| Blog #4 | 51007 | 6159 | 8.28 |
| Community site | 6610 | 619 | 10.6 |
| Services company | 1270 | 569 | 2.23 |
| Software company #1 | 30656 | 5161 | 5.94 |
| Software company #2 | 1143 | 1715 | 0.66 |
| Static site | 28653 | 3528 | 8.12 |
| Total | 253646 | 53262 | 4.76 |
I’d hoped that I might be able to find a consistent ratio between the two, but clearly they’re all over the place.
In the past, I’ve suggested that these services might be useful for comparisons. We talked to Vanessa Fox about this in our ebook. Here’s what she said:
All of the services are fairly notoriously unreliable. They all use different methods for gathering data that make them fairly inaccurate
by their nature. (Alexa, for instance, uses the Alexa toolbar, which is skewed towards a certain demographic of users.) However, a couple of ways any of these tools are useful are for trending over time and comparisons. If you use one tool to gather data on these two things, then while the data will be unreliable, it should be equally unreliable over time or between sites, so the trending should be fairly accurate.
Judging from my admittedly small data set, I’m not going to rely on Compete.com for comparisons anymore. Consider the table above. According to Google Analytics, blog #3 is roughly seven times more popular than blog #2. According to Compete.com, blog #2 is three times more popular.
Compete offers some explanations for this variance, but I can’t imagine why the ratios would be so wildly different.
There’s still probably value in, as Vanessa suggests, tracking trends over time. However, I wouldn’t use Compete.com for much else.
Add To My Data Set
If you’d like to contribute monthly visitors for a website from Google Analytics and Compete.com, please leave them a comment. If you don’t want to, you don’t need to name the site. Just indicate what kind of site it is.
I looked at the last year’s worth of data in Google Analytics, and dividing by twelve to get an average monthly total.
DarrenBarefoot.com 
Remember that apparently Compete focuses entirely on the US market. Their main numbers and “ranking” are entirely based on US visitors. They also show “global visitors” as a separate number but that doesn’t seem to be included in rankings at all. It’s definitely very irritating for sites whose traffic is mostly non-US, like mine.
Alexa isn’t that hot, either. They update every week or so but I find Quantcast to be the most accurate (well, likely because it’s actually tracked by included code and not based on toolbar installs and ISP traffic.)
Nick: Yep, I’m aware of the US issue, but I’m not sure why that would skew these results. None of the sites listed have a particularly local audience.
You are entirely right about this. None of the Web Metrics companies are any good. The only one who has the entire picture is the NSA / AT&T…
Darren, I was just looking at some local sites here in Newfoundland last night. I was using both Compete and Alexa. Like you, I couldn’t seem to find a correlation.
I will say that some of the local sites I was looking at were Newfoundland based but they seemed to do well with Compete.
Frustrating.
On a side note, I blogged about the Clean Air Foundation but it didn’t seem to get picked up by the Google search. I wonder how many others didn’t get picked up.
I didn’t add my website because I didn’t want to look like I was trying to get a plug in for the Mow Down pollution item.
I just wonder why stuff like that happens.
GK
Gary: Hmm…that’s weird. How long has your site been around?
If you like, I can update this post with a link to your site. That might help get the Google spiders (and others) there faster.
I’ve had the site up since Mar 30, 2006. With long periods of blogging wasteland!!!!
Gary: Hmm…weird. I’m not sure what to recommend. Anybody else have an idea?
@Gary: try this query: http://www.google.ca/search?hl=en&rlz=1B3GGGL_enCA176CA231&q=thinking+about+mowing+the+lawn&btnG=Search&meta=
@Darren: I haven’t investigated Alexa for many sites, but I know it’s completely off for my ConsultantJournal.com site. I get way more traffic than they suggest. I don’t know where they get their numbers.
> Both use communities of users who agree to install browser plugins and contribute their usage data to the company’s database.
That’s exactly where both services lose me and become nothing more than the measurement of what a certain set of people visit. There’s nothing reliable to indicate that their viewing preferences are reflective of wider web use, and they end up feeling like snake oil.
CDarren, I’ve been struggling with exactly this for my media relations team this week! I’ve ended up with a combination of Compete and Xinureturns.com.
We’re trying to assess whether a writer from a particular site is worth providing a media visit for, and need info to support or refute their traffic claims.
We’re going to use Compete to compare vs a site of known value, plus add in things like length of time the site’s been up, incoming link count, total pages indexed, etc.
Certainly none of these metrics is reliable in and of itself, but I’m hoping that in combination they’ll give a reasonable assessment of the site. If anybody else has other suggestions, I’d be all ears too. (I’ve found Alexa to be a total joke for the vast majority of my checks – way too specific a user base.)
@Gary: Have you got a Google sitemap on your server? I’ve had good luck with a couple of clients getting more effective (and more frequent) spidering with one. If you set up for Webmaster Tools, you can even request a spider – though I haven’t’ found that process to be all that reliable.
Paul
This is great comparisons. Thanks for sharing it.