Monday, Mar 24, 2008

Unique visitors is a dumb metric

Design Thinking, Validation by Austin Govella

Unique visitors is a dumb metric

Dumb, aggregate metrics like unique visitors and total page visits give no real insight into your site's success. You need behavior-based metrics.

4 Comments

Lots of organizations try to measure success by measuring unique visitors. Typically, these groups have some sort of advertising-based business strategy where they believe the more eyeballs they reach, the better they’re doing.

But that’s not true.

For sites supported by ad revenue, unique visitors tell you how big your check will be that month. If every visitor is worth $1 in ad revenue, and you have 1 million visitors this month, then your check will be $1 million dollars.

What’s missing, what’s not being measured is whether or not your business model is successful. “But I have a million dollars”, you say. But having a million dollars isn’t your business model.

Let’s say this month, you have 1 million visitors, they all earn you $1 in ad revenue, and you get your check for $1 million dollars. And let’s say your site sucks, so none of them came back, and they told their friends to never go. Next month you have no visitors.

Each visitor still earns $1. And you go to the bank to deposit your check for $0.

What’s missing, what’s not being measured is the behavior your organization relies on to make money. There’s something all those unique visitors do, and it’s the doing that’s necessary for the success of your business.

In marketing terms, unique visitors measures how many potential customers you have acquired. However, in order to make money, you have to convert those potential customers into actual customers.

Broad metrics like unique visitors and total page views give you broad aggregate data that’s just about useless. What you really want is a behavior-based metric that measures the behaviors that earn you money.

A lot of sites driven by ad revenue need the same behavior in order to be successful. A user learns about the site, they visit the site, and then they return to the site again and again. The visitor’s return visits are what really drive ad revenue. Instead of measuring unique visitors, a better measure of your site’s success is return visitors.

Performance metrics should always measure the behaviors your organization relies on to survive. Your metrics should always be behavior-based. Aggregate metrics like unique visitors and total page views don’t tell you much of anything.

Talk About "Unique visitors is a dumb metric"

Register or Log In to comment

shawn carey

shawn carey said:

I can see the point you’re making with this but I would argue that behavior based metrics is good to watch in addition to visitor metrics. That creates a ratio which makes attracting advertisers even less work. If you site has a shopping cart, then the ratio could indicate maybe 45% of your visitors bought something. If you’re running a support site, maybe 73% of your visitors asked for help. Then watching that 73% grow or shrink becomes important.

Sat, Apr 05, 2008 at 12:17 PM

Austin Govella

Austin Govella said:

Shawn,

I’m totally with you here. In fact, the next post explains behavior-based metrics a bit more, and I think they have two key qualities:

1. They measure the desired user behavior.
2. It’s comparative, so you have the kind of ratio you’re talking about.

Sat, Apr 05, 2008 at 07:10 PM

Paul Trumble

Paul Trumble said:

Austin,

I would hope this isn’t news to anyone who works with analytics. There is a definition of ‘hits’ which has been attributed to Jim Sterne, How Idiots Track Success. I think he really meant hits, but most would agree it applies to visitors almost as much. Measuring and analyzing web traffic is all about identifying and measuring various conversion points. You want your design to cause some sort of behavior, to achieve some goal.

I have responsibilities for both analytics and user experience, and analysts who are dedicated to each. I’ve always found a lot of synnergy between the two, but it seems like the qualitative and the quantitative haven’t met up very often. That seems to be changing though, last fall at eMetrics in DC, there was a lot of talk about qualitative research methods, usability, and getting at the “why” not just the “what”. They even had a presentation by an anthropologist. I spent this morning taking another look at the IA Summit agenda, and was surprised to see how many programs were dedicated to measuring and tracking. Two domains seem to be trying to come together.

I hope to see you at the Summit.

Mon, Apr 07, 2008 at 03:15 PM

Austin Govella

Austin Govella said:

Paul,

That’s refreshing to hear. And I think you’re right about it being the obvious. I’m still not sure why the metrics people use are so skewed some places.

I think I’ll be doing a side thing at the Summit on the ‘UX health check’. It’s a qualitative way to derive a longitudinal, quantitative measurement of your product’s experience. That’s lots of big words, but it makes total sense when you see it.

Catch me at the Summit, and I’ll fill you in. I’m interested in getting people’s responses.

Tue, Apr 08, 2008 at 01:36 AM

Register or Log In to comment

Hot topic? Subscribe to a feed of this post's comments.