Software development

BUILD, MEASURE, LEARN - Feedback and analysis in digital service development projects

Analytics, feedback, agile

How are successful digital services or products built? Is careful specification and documentation the key to success? Do rigid processes help optimise tasks?

The one thing that successful projects have in common is a strong culture of doing things together. The best services are built by multidisciplinary teams that bring together technical expertise, user-centered design and a deep understanding of business. Ideally, the customer and end users are a part of the team, too.

Writing comprehensive specification documents is slow and very expensive and a rapidly changing environment makes managing them a practical impossibility. If you have something complex to communicate, do it in person. The optimal solution is usually found via discussion that takes in a wide variety of viewpoints.

In iterative development, where the service or product is continuously evolving, constant monitoring of progress is of critical importance. To speed up the feedback process, the client and product owner must have access to the latest build at all times. This way users and other stakeholders are able to test ideas and features quickly.

A continuous feedback cycle - i.e. reacting to feedback instead of following plans to the letter - is a vital component of steering a successful digital service development project. Methods for collecting feedback vary, depending on the phase the project is in. It's important to meet real end users early in the design process. It's enough to have a design or prototype to show them, as Aino and Olli explain in their earlier post

When it's time to start actually building the product, it's important to get a minimum viable product (MVP) out as soon as possible. This way you'll really grasp the power of feedback from real users. Using a variety of methods to collect feedback and reacting to it is especially important when the product is being launched.  

A condensed version of the manifesto for agile software development:
    •    Individuals and interaction instead of methods and tools
    •    Functional software instead of comprehensive documentation
    •    Client cooperation instead of contract negotiations
    •    Reacting to changes instead of sticking to plans

But how do these ideas, as well as feedback and analytics work in real projects?


Case: MTV Uutiset mobile application

MTV3 is a major television channel and one of the most important news outlets in Finland. The mobile app for the MTV Uutiset news service is native to three platforms - iOS, Windows Phone and Android - and a great example of a service where user feedback and analytics played an important role in the development project. Before the service was launched, a multidisciplinary team was consulted to determine which aspects of the application needed to be measured and how the data could be best utilized in the decision-making process. Generally a team needs to have at least an analyst, an application developer and an interaction designer to even have this conversation. It doesn't make sense to use analytics to measure the obvious or needs that other feedback channels have indicated. Generally the development team has a fairly clear vision of the service's core features.

We want to measure the amount of users and produce statistics that can be compared to other platforms, such as web. Analytics is used to try to understand the end user's behaviour in the application. This knowledge can then be used to confirm the functionality of the design solutions and steer the user towards content he or she may not have found. Advertising analytics is used to monitor the fulfillment of business aims.

Just analysing available information does not tell us the whole truth, however. User feedback and the applications' built-in error reporting help prioritise the continued development of the app. The review feedback received from application stores is rarely directly useful  

The Uservoice service is used to collect informal user feedback and requests for features that users can vote on. Direct user feedback steers and adds to interpretations based on analytics. It also cuts down on the risk of coming up with erroneous assumptions based solely on usage volumes. Insights produced via analytics may provide conflicting data on the same application, depending on the platform. 

The content creators at MTV3 News are also involved in the app's development. Ensuring efficient penetration of news items is paramount to the channel's journalists. The dialog with the newsroom concerns which features best support this goal on each platform.

In addition to user and use case -based needs, analytics can be used to monitor how the service works technically. These may be response times the developers are going for, general performance levels or sufficiently tight data security. The service's lifecycle has an impact on how these goals are prioritised in relation to adding new features. 

Feedback channels are mined and analysed daily to fulfill a variety of needs. Depending on the platform, as well as publication-related delays in platform stores, new data is fed into the service or application development cycle only once every two to four weeks.

The most common challenges a software project built on Lean Startup principles faces come up during contract negotiation. In this case the contract specified the features the service should have on a fairly general level, without an excess of detail. A better understanding of the content and the service being produced was generated in workshops and discussions with the client, as well as via an early prototype. 

We produced a test version of the app very quickly. We released new features all the time, as a part of a continuously functional app. This way of working strengthened the bond of trust between the involved parties. We defined a set of criteria for approving features as a part of the continuous development and demonstration of the application. At the same we also created a basis for colelcting user feedback and analytics data. 

A relationship bult on mutual trust meant that we didn't have to rely on contract clauses to get things done during the project.

The end result is an awesome news app.

Yrjö Kari-Koskinen
Janne Kääriäinen
Ville-Matti Toivonen

This is the fifth blog in a series written by experts at Futurice for Talouselämä magazine's partner blog. The original Finnish version is here.