Estimating online audiences: Understanding the limitations of competitive intelligence services
If you're not the site owner, you'll have to use competitive intelligence services to estimate the traffic. These services a wide range of methodological approaches, generally recruiting panel members using an opt-in mechanism, sometimes mixing in anonymized data bought from Internet service providers. Three of the leading free services of this type are Alexa, Compete, and Quantcast.
This study examines the methodology of these four methods of measuring online audiences. It compares the results from all three services using a panel of 18 Web sites that have agreed to share analytics data with the author.
Results show high variability in the quality of the estimated audience data from competitive intelligence services. Using analytics as the reference, Compete underestimated traffic by more than 300 percent. Quantcast underestimated traffic by more than 200 percent. Confounding this pattern were occasional overestimates of traffic by these services. Alexa data, which is presented differently than Compete and Quantcast data, showed one significant correlation (page views) with analytics data, while three other variables showed no relationship to data from analytics.
Overall, these services performed poorly. But are the data worthless? The report concludes with recommendations for understanding the limitations of competitive intelligence services and appropriately reporting audience activity based upon differing methodologies.