Kinja Data Team

it's A/B test time, my dudes

Google’s stock price made a crazy jump last week, at least partially predicated on mobile search. The major change is ‘deep linking’- mobile search results that take people directly to an app or to content within an app. Google mentioned that Etsy was seeing a boost in traffic from this feature, leading to Etsy stock increasing in value by about $700 million the same day.


Google has offered a somewhat similar feature for web publishers: the mobile search carousel. If you search for a topic on your phone, you’ll notice that results from a website are now grouped together and are easy to horizontally scroll through.

I really like this feature: it clearly improves the overall experience of searching on my phone. This has been live for several weeks, so I wanted to see if the improved experience led to a change in traffic.

In addition, ‘MOBILEGEDDON’, a well-branded change to the search algorithm on mobile, also happened a few weeks back. It makes mobile-friendly sites rank relatively higher, and other sites rank lower. Fortunately we’ve been blessed with the ‘mobile-friendly’ tag, as you can see in the carousel image.

I was hoping to test changes in traffic, but there are a few challenges. Namely:

- I can’t A/B test Google’s search results.

- Since we sell news, our top landing pages are different every week, so I can’t compare one search or page over a period of time.


- There’s a long-term trend towards more mobile search.

In order to try to isolate the impact on our mobile search, I took a look at how much of our Google search traffic was on mobile, compared to the year before. These are 7-day rolling averages.


We bottomed out at about 46% mobile (so 46/100 search visitors are on phones), and saw a bump starting around May (Mobilegeddon) up to 51%. So it initially looked like there was an impact on traffic.


However, we know that there’s a long term trend towards more mobile usage. So I compared this year’s data to last year’s data, trying to isolate if there has been a meaningful difference.


Here the story doesn’t look so clear: year-over-year, the growth in mobile search traffic has been about 8-10% before and after these events, and that hasn’t meaningfully changed. That spike we saw this year, we saw last year too. So on further consideration, we don’t see that these changes move the needle for us. Nevermind! Maybe there are big changes elsewhere, but we aren’t seeing them.

So the question remains, why do we see this annual spike mobile search in May?

As it turns out, it’s really easy to tell when there are holidays in the US from our traffi: we see big jumps in the % of traffic from mobile. That’s the more likely explanation of the bump we saw in May (and a good warning about the dangers of before-and-after testing!).


The one other bump was due to a big, sad story that people wanted to check right away on their phones.


Is anyone else out there seeing bigger changes to their traffic from the mobile updates? Any app developers that are benefiting from deep linking? Let me know in the comments.

[photo credit]

Share This Story

Get our newsletter