Kinja Data Team

it's A/B test time, my dudes

About two weeks ago, we changed how we post to the homepage. Max wrote about the new set-up and the reasoning behind it, and asked me to take a look at what the impact was.


Unfortunately, this was a uniquely bad time to do direct 'before & after' comparisons. We only have one week of data, and that week had a holiday weekend. We made bumbling changes to our tracking code and a flipped on a successful new traffic acquisition tool at about the same time. And two weeks before was the biggest vacation week of the year.

Still, we'll try to parse out some of the changes here by looking at our seven other owned & operated sites as a control group. We don't yet have enough data, so please take these with the appropriately-sized grain of salt.

Top Line Numbers


So there's no way around it: the top line numbers were really bad last week, compared to the week before. Unique pageviews were off almost 20%, compared to a 4% decline for our other sites.

However, it's not clear that this is anything other than a slow newsweek. We actually had one of our other sites do even worse last week. While it doesn't look good, we're going to keep on monitoring this before drawing conclusions.


The story on the amount of total time spent on the site is similar: Gawker was down 6%. That was, again, the second-worst among all the sites, and way off our company-wide average of +10%.

Gawker's "time on page" numbers made an improvement that was slightly better than our other sites, but again, not enough to tell us anything meaningful. We are obviously very happy to see this up across the board, as it has been a focus for us in general.


Breakdown by Source

Our primary assumption was that traffic to articles decreased mostly due to 'direct' traffic- users who go directly to are not seeing as many new articles. The data generally backs this up: our drop in direct traffic was 50% worse than traffic through other channels. The changes to our other sites didn't show meaningful differences.


Adoption of Newsfeed

To mitigate the above concern, Gawker kept the old, rapid-fire front page as the Newsfeed for readers who want the full firehose of content.


Adoption so far has been really good: 10 days in, we're already seeing it pull in over 20% as many entrances as Gawker's front page. I'd like to see this eventually reach parity.


We're continuing to do analyses on the effect of this change, including looking at the distribution of pageviews between 'hits' and regular articles, and if there's a long-term change in the amount of direct traffic to Gawker. All of that will require more time to collect data, so check back here in the future!

[photo credit]

Share This Story

Get our newsletter