What we are learning about measuring impact, and where we want to get

Futurice’s social responsibility program Spice is ambitiously aiming to make Futurice a better company and the world a better place by sponsoring employees’ free time open source contributions, and now also social inclusion and digital open education efforts. The actual meters for proving the program’s impact have been missing though.

To change this, I’m working three months in a project funded by Tekes, developing measures for the Spice Program’s impact on Futurice and its employees. The focus has been on impact in the company context, rather than on wider societal impact and the making the world better place part.

Although doing good is worthy in its own right, and social responsibility activities can be supported by moral arguments like “it’s the right thing to do”, they can also be assessed from more strategic perspectives, as social responsibility initiatives can create positive business effects, too. To assess and improve these, you need to have metrics of course.

In this case, assumptions about the benefits of the Spice Program have been the starting point for meter development. The main assumptions we are testing are:

  • supporting recruitment by attracting talented people,
  • learning new skills that are valuable at work,
  • strengthening employee engagement.

In short, social responsibility activities might be positively related to attracting, motivating and retaining talented people, as well as competence development, and thus very focal HC (human capital) topics. These can also have financial value due to lower costs related to employee turnover or recruitment. In addition, and related to these, social responsibility initiatives may feed innovative company culture and lead to service innovations, not to forget the improvements in the company brand image. But as the schedule is quite tight, focus in creating metrics has been primarily in learning, recruitment and employee engagement.

What have we learned so far?

In January we sent a short survey on learning to Spice program participants to give us a better idea whether or not they have learned professionally relevant skills via Spice activities during the past 12 months, how frequently they have contributed and what kind of things they’ve learned. Because the Spice program contributions are made on participants’ free time, we were also interested if they’ve been able to apply those skills at work.

The survey was sent to 54 Spice Program participants, and 37 of them answered to it. Some graphs about the results were introduced in recent blog post, and according to responses, participating in Spice Program activities had really improved their professional skills. Assessed on a scale of 1 (strongly disagree) to 5 (strongly agree), 22 participants (59 %) agreed strongly and 14 participants (39 %) agreed on this one, and the mean was 4,57. (And yes, calculating mean from ordinal data is actually silly, but it gives you the idea.) Respondents also described their learnings in detail, and the list is quite extensive.

Participants mostly agreed (21 people, 57 %) or strongly agreed (12 people, 32 %) that they had been able to use those learned skills at work (mean 4,14). The frequency of contributions was assessed on weekly, monthly or less than monthly basis. This categorization seems too vague since 24 participants had contributed less than monthly anyway, and frequency didn’t seem to be related to learning outcomes or applying skills at work. One respondent described successfully applying learned skills “I haven't made many contributions - but those I have made I have been able to use directly in my project work at Futurice, as well as gain deeper knowledge of the open source tools we use daily”.

Next time, application of skills needs to be asked more closely to get a more complete picture of how participants are using the acquired skills at work, and if they think they aren’t able to use them at their work, why is that. It’s probably mostly related to the type of the contribution and the kind of work participants have. Professional skills gained via open source contributions might be more directly applicable in many work projects at Futurice than skills learned through code school teaching, for example.

Some people might even be participating in Spice activities because their current work tasks are not challenging and meaningful enough, and perhaps these kind of positions don’t offer as many options to apply new skills, either. And finally, even though company sponsored Spice Program contributions are made on one’s spare time and in projects that interest participants, so being able to use those skills at one’s work is merely a nice bonus from company perspective, not something you should require.

To conclude from the first results: Spice participants are really learning skills valuable at work, but there’s still more potential to apply those increased professional skills in their work projects. The impact Spice Program has on recruitment and employee engagement will be unveiled later on. In the meantime, feel free to share your thoughts on these topics with me.



This blog post is licensed under a Creative Commons Attribution 4.0 International License. Attribution to Futurice Oy (www.futurice.com).