Monday, November 04, 2013

Experiments not Features



One of the more interesting conversations that piqued my interest while watching the live feed from this week’s Web Summit was Jay Bregman, co-founder of Hailo, explaining their evolving methodology of ‘Experiments Not Features’. 

During the summit’s ‘Going Global’ session Bregman explained some aspects of Hailo’s business model. (His comment about Hailo eventually becoming a platform for many more service transitions than taxi rides is also something worth a blog post all of its own some time.)

Hailo works effectively as a network because “it is local and it appears local”. What that means for the business is that each of their markets operates slightly differently. The overall global network learns a little from each different market, which then feeds into refinements to the master model for more efficient and effective roll-out in future cities.

Because Hailo needed to be a global company from the offset, they deliberately planned not to have one centralised headquarters. Rather their model is to have “centres of gravity” in Asia, North America and Europe. This makes them more sensitive to the specific localised needs of the cities they are serving.

There are three primary implications arising from that corporate structure: firstly they need superb integrated processes and secondly they need excellent internal communications. The third implication relates to their underlying technology. As they are a global network, they have to look as coherent as possible to all of their consumers. Therefore it is mission-critical that they can identify which proposed technological deviations/innovations are truly beneficial for any local market, and which are merely personal preferences. This is where their experimental methodology kicks in.

Bregman claims Hailo has “had to build one of the best A/B mobile testing platforms”. This is allowing them to run massive large scale tests – with the vision that ultimately they can test everything. So if the Dublin operation wants to modify some aspect of the technology platform in future, rather than request a new feature, they would simply be able to run a live test. Then, if they can prove that it produces better results in their city, they can implement it.

On one hand reading this just sounds like common sense – A/B testing is a not new methodology. But the fact that today’s technology now facilitates companies runnig large scale live experiments in real-time is becoming profoundly disruptive. Behind-closed-doors development of digital services design is counterproductive, iterating in public is becoming the proven methodology for success.

It is worth thinking through how this approach effects product development roadmaps and workflows, increases the pace of innovation, and ultimately delivers greater responsiveness to customer needs.

Here is the video of the Web Summit ‘Going Global’ session which features Jay Bregman co-founder of Hailo, and Niklas Adalberth co-founder of Klarna, moderated by Niklas Zennstrom co-founder of Skype.



PostScript: for a complementary take on this subject, have a read of Dan Frommer’s post from the same date: ‘The Best Part Of Twitter’s New Design Is That It’s Experimenting In Public’.