Gav ([info]blufive) wrote in [info]stats_weenie,
@ 2005-02-27 11:12:00
Previous Entry  Add to memories!  Tell a Friend!  Next Entry
Entry tags:statsinthenews, websidestory

Snapshot: WebSideStory and "Firefox gains slowing"

[About WebSideStory]

For-pay internet stats outfit WebSideStory have released one of their infrequent "data spotlight" columns, and it's causing some minor waves. This is presumably because Firefox is flavour of the month, and this column is titled "Firefox Gains Slowing". [note: this appears to be a static page with no archive; it seems likely that, at some indeterminate point in the future, the content I am quoting from will be replaced by something completely different]

The growth rate of Firefox’s usage share on the web has slowed in recent months [...] Firefox’s market share grew just 15 percent in the five weeks leading up to Feb. 18, 2005, the latest benchmark available. In the previous six weeks before that, the browser grew at 22 percent clip. These figures compare with a 34 percent increase between Nov. 5 and Dec. 3, which coincides with the release of Firefox Version 1.0 on Nov. 9.

Anyone see the mathematical flaw here? The text above is built on an assumption of steady geometric growth. That assumption is highly suspect - in my experience, [Edit:] no browser's market share has ever grown exponentially no browser with significant usage levels has managed sustained exponential growth in market share [end edit] in recent (Internet time) history. Usage of the internet itself may have done so, in the mid to late 1990s, and that might have dragged the browser du jour up at a similar rate, but after 1998? No chance. I speculated on some possible reasons why back in November. So I did some number crunching. Here are their headline raw numbers:

Browser         18 Feb 05  14 Jan 05  3 Dec 04   5 Nov 04
IE 	 	89.85       90.28      91.80      92.89 
Firefox 	 5.69        4.95       4.06       3.03 
"Netscape"*      2.47        2.64       2.83       2.95

[*They've actually labelled the last row "Non-Firefox Netscape and Mozilla browsers". Personally, I think lumping Netscape 4 and Mozilla 1.7 into the same group is fairly braindead, but...]

Unfortunately, the date intervals are irregular - 28, 42 and 35 days. So directly comparing the growth in each interval is pretty bogus in itself (especially with the galumphing great demographic elephant called "Christmas" slap in the middle). Here's the growth numbers I calculated from their data:

Date        Interval    Usage	growth%	 Growth/day	
05 Nov 04     -	         3.03
03 Dec 04     28	 4.06    34.0      0.037
14 Jan 05     42	 4.95    21.9      0.021
18 Feb 05     35	 5.69    15.0      0.021

The "growth%" column is obviously what they're quoting in their article. I calculated it using the formula 100x((usage2/usage1)-1). For that number to be "constant" requires geometric growth in usage from one datapoint to the next.

The "growth/day" column is the average growth per day over the preceding interval - calculated with the formula (usage2-usage1)/interval. I think it shows a much more realistic picture (based on these numbers). Namely: after an initial surge in the first interval, growth in Firefox usage slowed, but then stabilised to a linear increase at a lower level. Since the first interval included the Firefox 1.0 release and a huge publicity push, this seems reasonable.

[Update: some extrapolation]

Back in December 2004, it seemed Firefox was a lock to reach 10 percent by mid-2005, ahead of the reported year end goal of the Mozilla Foundation. Given the latest growth rates, the year end target still appears attainable, but a mid-year achievement is unlikely

Assuming continued constant linear growth at the rate seen in the third interval, Firefox will hit 10% in September 2005. Of course, that assumption is almost worthless...

[Disclaimer: this is all based on a grand total of four data points, compiled using unknown methodology from an unknown data set. Guesswork-a-go-go, in statistical terms]

[28-Feb-2005 update: following a prod from David Naylor, WebSideStory have amended their report to more accurately reflect the numbers. Looks more like a mistake than anything deliberate...]




(Post a new comment)


[info]dash_aitch
2005-02-27 12:54 pm UTC (link)
I guess the questions would be... Would websidestory have any particular reason to spin the story against FireFox? Is it (unintentionally) questionable statistical analysis; or are they perhaps taking after stocks/currency reporting*?

* In .au, at least, stock prices and currency fluctuations are routinely reported in the media using an irregular interval - one day will be skipped, usually exaggerating rise or fall.

The wording of the report makes it sound as if they're talking about a marked drop in growth over a period of months, rather than a few weeks. The sidebar makes an aside about how this is consistent if you expect an initial burst from early adopters, but doesn't make mention of the concerted FireFox advertising/awareness campaign.

In short... are they lazy, mistaken or malicious? Or do they just have their own take on how to analyse browser trends?

Basically, the data seems less important than the way it's being presented. Perhaps websidestory think the uncertaintly will prompt people into purchasing deeper analysis. Being cynical (which I am notably wont to do :)) one could suggest they know many IT managers will want to justify products which only work in IE. Such people have the money and inclination to purchase offical-sounding reports which excuse them from potentially expensive rebuilds.

Other than that I'm not sure where the money would come from...

(Reply to this) (Thread)


[info]blufive
2005-02-27 02:09 pm UTC (link)

To semi-randomly wander across the subjects you raised:

I suspect that the variable interval is not particularly significant. Note that the intervals are all multiples of one week. [fx: kicks self for not noticing this earlier.] That's a good thing - it means that they are either reporting daily stats for the same day of the week (Friday), or aggregate totals for whole weeks starting or ending on those dates. From a statistical POV, either would be good practice. I suspect that the middle interval is so long because they've deliberately avoided the figures for the christmas/new year period, which is major a demographic effect.

My first instinct is that it's just sloppy writing by someone with minimal training in rigorous statistical mathematics. Lord knows, there are lots of such people out there. On the other hand, if they're savvy enough to report consistently on one day of the week, and avoid christmas, they shouldn't be making this kind of dumb mistake. [Peter Principle sub-theory: competent statisticians, but the manager/marketer who wrote the text is a klutz.]

To enumerate other possibilities that spring to mind:

There are large numbers of fanboys on both sides of the Mozilla/Microsoft divide - it could be that our writer is one of them, looking (conciously or otherwise) to put the opposition down. Happens innumerable times a day in web forums/mailing lists/offices/newspapers/bars all over the world.

<cynicism class="anti-microsoft">Alternatively, Microsoft might have handed them a large cheque</cynicism>. There's currently no evidence whatsoever to support that theory, but I would point out that Microsoft have cited WebSideStory to rebut stats from other sources, for example in this story:

Microsoft has disputed these numbers [showing firefox gaining rapidly], claiming that they do not represent corporate users.

"It doesn't jibe with what WebSideStory shows, and what neither of these count is corporate intranets where users aren't actually hitting the Web," Gary Schare, Microsoft's director of product management for Windows, said of OneStat's statistics.

(Reply to this) (Parent)

Exponential growth
(Anonymous)
2005-02-27 03:28 pm UTC (link)
The Mozilla Suite's usage was growing exponentially for years before Firefox became so popular. The total of all Gecko-based browser usage continues to approximately double each year. It has since the beginning of 2001, soon after Netscape 6.0 debuted. The best set of data to see this trend is TheCounter's global browser usage data.

It may not be fair to assume Firefox usage will continue to grow exponentially, but if it does not continue to do so, the growth rate *is* slowing down and beoming linear.

(Reply to this) (Thread)

Re: Exponential growth
[info]blufive
2005-02-27 06:26 pm UTC (link)
The total of all Gecko-based browser usage continues to approximately double each year. It has since the beginning of 2001, soon after Netscape 6.0 debuted. The best set of data to see this trend is TheCounter's global browser usage data.

What version of TheCounter are you looking at? I certainly don't see any such trend:

            7/2001  1/2002  7/2002  1/2003  7/2003  1/2004  7/2004  1/2005
TheCounter   0.3     0.9     1.3     2.1      -*     2.0**   2.8    6.7***

*July 2003 is trashed, it's in the middle of a huge data anomaly running from May 2003-December 2004, which may also taint the Jan 2004 figure.
**January 2004 is in the middle of one of their regular browser-detection reshuffles (which lasted from January-April 2004) - this figure is highly unreliable.
***January 2005 is affected by another browser-detection reshuffle, though this number is probably better than the January 2004 one.

I should add that I'm not entirely happy with their detection of Gecko browsers for the entire period. Even taking the figures at face value (and ignoring all the browser-detection screwups and missing data) this is hardly a text-book example of exponential growth.

It may not be fair to assume Firefox usage will continue to grow exponentially, but if it does not continue to do so, the growth rate *is* slowing down and becoming linear.

Is exponential growth faster or slower than linear growth? Answer: neither. Like comparing apples and oranges.

Also, are we counting Firefox or Gecko? Gecko does show something which looks like exponential growth in stats from some sources, between about July 2003-Present, though doubling roughly every 6 months, not 12.

            7/2003  1/2004  7/2004  1/2005
w3schools     7.2     9.7    15.2    24.4
ranking.pl    1.8     2.3     3.9     7.9  

So, on that point: mea culpa, I'll add notes to the original post accordingly.

However, WebSideStory are pretty certain they're talking about Firefox and, based on my observations of various sources, Firefox's growth in the last 14-15 months has been remarkably linear, with an increase in growth rate in November 2004, around the 1.0 release, which has since settled back to a slightly lower linear trend, but still higher than the pre-1.0 growth. Usage of the mozilla suite has been pretty much stable over the same period.

I stand by the gist of the original post: saying "firefox growth is slowing down" and then quoting numbers based on an assumption of exponential growth as evidence is (at best) mistaken or (at worst) deliberately misleading. The mathematics are fundamentally flawed.

(Reply to this) (Parent)

Typo (Numbero?)
(Anonymous)
2005-02-27 05:59 pm UTC (link)
You should change 18 Jan 05 in your second set of numbers to 18 Feb 05, perhaps?

(Reply to this) (Thread)

Re: Typo (Numbero?)
[info]blufive
2005-02-27 06:21 pm UTC (link)
Oops. Yes, it would appear that I'm a complete dolt for typos this weekend. Thanks for the catch.

(Reply to this) (Parent)

Browser Recognition
(Anonymous)
2005-03-07 11:56 pm UTC (link)
Hello!

Since you're into statistics I thought maybe it would interest you that I've produced a little table showing how the different statistics services cope with browser recognition:

http://home.student.uu.se/dana3949/test/

David Naylor
(d .dot naylor @tt telia .dot. com)

(Reply to this) (Thread)

Re: Browser Recognition
(Anonymous)
2005-03-08 12:00 am UTC (link)
I just realized one interesting thing: OneStat have previously reported Firefox separately from other Mozilla browsers in their press release, but the free statistics service doesn't separate Firefox from the rest.

(Reply to this) (Parent)(Thread)

Re: Browser Recognition
(Anonymous)
2005-03-08 11:33 am UTC (link)
I have now produced a blog post on how I went about when testing the browser recognition:

http://naylog.blogspot.com/2005/03/browser-recognition-of-statistics.html

/David

(Reply to this) (Parent)(Thread)

Re: Browser Recognition
[info]blufive
2005-03-08 08:47 pm UTC (link)
thanks for all that. I keep meaning to add a few more "useful site" links to this place, and that's a keeper...

(Reply to this) (Parent)


Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…