Kinja Data Team

it's A/B test time, my dudes

Every day, the Kinja team works to iterate on the platform Fusion Media Group uses for its network of media sites. As part of our job on the data team, we provide product managers and developers with information to help them create new features that improve users’ experiences, enhance editorial’s storytelling tools and provide the business with what it needs to grow.

On any given day, we use A/B testing to evaluate the effects specific features have on these areas (and boy, do we love A/B testing). But sometimes, it’s helpful to take a step back, smell the roses and look at the progress made over a longer period of time. For us data people, that means it’s time to crunch numbers.


At the end of 2017, the Kinja team tasked us with evaluating how a handful of the enhancements and changes launched in the last year affected some of the user behavior metrics. Today, we’re sharing some of the interesting findings from that analysis and what they say about Kinja in 2017 and moving forward.

Migrations, migrations, migrations

If you follow FMG closely (👋), you probably know that our network of sites has grown over the last year. We launched Splinter and Earther; migrated The Root, Very Smart Brothas, The Onion and The A.V. Club to Kinja; and spun out The A.V. Club’s food coverage into its own top-level site, The Takeout. As part of welcoming these sites to the fold, the Kinja team faced a handful of feature requirements to ensure the migrations went off without a hitch for authors and users.


The A.V. Club was one of the largest of these migrations. With an active community and large backlog of comments, our migrations team worked to ensure posts, comments, and users were ported over to Kinja. The product team offered commenters the ability to claim their old accounts and bring them over to Kinja to allow for a smooth transition to the new platform. Of those who claimed their account, 74 percent of them went on to comment on The A.V. Club or its verticals.


Slide to the left

While feature requests may originate with a single site, the product team validates that this is a feature that would benefit the entire network before building. We found this when the the product team launched slideshows in early October to support the The Onion editorial team’s content approach. Our analysis showed just how many of the sites across our network embraced the new feature.


Between launch of slideshows and the end of November, Gizmodo, Lifehacker and Kotaku published 75 percent of the slideshows across our network. This is a clear example of the strength of Kinja as a publishing platform, through which a tool can be adapted and embraced across the platform to serve each brand’s voice.


With varying use cases for slideshows comes slideshows of varying lengths. Most of the slideshows in our analysis had four slides, although we had nine slideshows published in October and November with more than 20 slides. We also found that three in four users would complete slideshows with up to 15 slides, indicating that the sweet spot for length and engagement is around there.


Nineteen percent of readers who visited a page with at least one slideshow interacted with it. On The Onion, that number reached 36 percent, likely due to their recurring “The Week in Pictures” posts.

Feature this

A number of enhancements were made throughout 2017 to our featured templates, which house many of our long-form stories on Kinja (like this one). Between February and November last year, one percent of editorial’s stories published on Kinja used the featured template, which is most likely because this style of storytelling is used for special posts beyond the day-to-day articles.


Video continues to be another primary focus for FMG and to support the growing amount of video content, the Kinja team released a video template, similar to the featured post, in mid-October. This post design, shown here for an A.V. Club video, prominently features a video, along with some text providing context, and related videos.

About 10 percent of both The A.V. Club’s and The Onion’s posts published between the launch of the video template and the end of the year used this post type. Below the video player, there’s a module that recommends related videos to watch, which we found performed well. We found that 13.6 percent of users who visited these video article templates on The Onion clicked to watch a related video.


Wait, there’s more!

Of course, these are only a handful of the features launched on Kinja in the last year. For instance, we also released story types to allow for common groups of articles for readers to easy discover more content they enjoy. For some sites, story type pages have been an effective way for readers to navigate the site, with five percent of pageviews on The A.V. Club occurring on story type pages.


We also experimented with new ways of reaching our readers, such as with the Commerce Bot on Facebook, which alerts subscribers of our deals posts based on categories in which they’re interested. Out of all the categories available for users to subscribe to, we found that the tech category had the highest number of active and lifetime subscribers.


What’s next?

As always, the Kinja team is going to continue to launch new features in 2018 to improve its storytelling tools and the overall user experience. It’s not my place to spoil any surprises, but there’s a lot to look forward to in 2018 when it comes to Kinja.


On our end, the data team will continue to support Kinja and all of those making improvements to the CMS day in and day out. For instance, we’ve collaborated with the product team to run 21 A/B tests so far in 2018 (for context, we ran an average of 18.5 tests in Q1 for the last two years). We’re also working on developing an internal tool that will allow stakeholders throughout the company to more easily monitor the 2018 goals related to the performance of the platform as new features are launched throughout the year.

Kinja Data Engineering

Share This Story

Get our newsletter