Posted on 3rd December, 2012 by LEO Learning Web Team
The appearance of Towards Maturity’s benchmark report in Autumn is getting to be a regular event in the e-learning calendar, and one that rarely disappoints. This year’s report is no exception.
Towards Maturity is held in high regard by the learning community – above and beyond the benefit participating organisations get from taking part in its benchmarking activity. In an industry that suffers from a distinct lack of analyst focus, this not-for-profit body’s research provides a unique source of valuable information, not only about what organisations are actually doing to apply technology to the business of learning but also, importantly, about the results they are achieving. As the story gets stronger, the reports also seem to improve in quality.
Year on year, we find in these pages a consistent and growing wealth of evidence for the business results that can be realised from investments in learning technologies. The picture that emerges makes a far more compelling argument for the efficacy of technology as an aid to learning than anything marketing people like me can cook up.
The report feels more considered and coherently presented this year and contains a lot of food for thought. So much that any review has to be fairly selective. I’ve concentrated here on a few of the topics of particular interest to LINE and its clients.
As a longitudinal study, the benchmark research grows richer with time. This year we learn that over the last 30 months, compared with traditional learning approaches, the 700 organisations in the study are achieving:
- 29% improvement in study time
- 25% improvement in delivery time
- 25% improvement in learning reach
- 22% improvement in time to competency and
- 22% reduction in cost of training through their use of learning technologies
Once again it is emphasised that the top performing organisations in the study achieve far higher levels of benefits – benefits which the study does much to quantify. The top quartile gets an average over four times higher than those in the bottom quartile for improvement in customer satisfaction and increase in revenue.
How significant is the survey?
So useful and interesting is all this data that it feels slightly churlish to cavil at the methods by which it is gathered – but the study does have its naysayers, and it is important to be clear about the limits to how the information it contains can be used, so there is no accusations of over-claiming from those quarters.
In essence, the study has a strong statistical foundation, with a sizeable base and a good spread of company sizes and sectors. But there is an acknowledged element of self-selection in the sample – ‘those already with an interest in using learning technologies – which makes it dangerous to extrapolate results too widely. For instance, the fact that 63% of the organisations surveyed allow their employees to access to Facebook on work premises does not mean that such is the case in 63% of UK workplaces. Statistics about adoption of particular technologies should likewise be treated with a bit of care. Of course, Towards Maturity themselves never make false claims, but it is the way of statistics to go far and wide, and often to be taken out of context.
Another criticism sometimes levelled at the study can be dealt with more easily. It has been said in the past that the criteria used to define maturity are argumentative, in that they assume the use of technology for learning to be an inherently good thing, and the more modish the technology being used, the better organisations score. However, it should be borne in mind that these criteria come not from some backroom assembly of habitual conference platform speaker, but from ‘industry subject experts and practitioners’, and that they are changed and refined each year informed by what is happening ‘on the ground’ rather than what is popular in the blogosphere. Who could argue, anyway, with a behaviour such as ‘our learning initiatives support the skills the business needs’, whatever your perspective or background.
It also helps that the behaviours are closely tied in to tangible business benefits, validating the criteria through results. This year it is especially impressive to see that the top quartile did especially well at delivering ‘difficult to achieve benefits’, such as integrating learning into workflow, that often elude less mature participants.
Lastly, there can seem – on the face of it – to be an inherent weakness in any study which, like this one this, relies on participants to rate their own organisations’ performance. You might just get the press release. Conversely, you could happen on a downtrodden and thwarted L&D manager who takes the survey as an opportunity to unload. Due to the thoroughness of the study, however, and its duration over time, subjective effects like the gap between aspiration and performance are highlighted in the data and actually become part of the story. We wanted to achieve these benefits, but actually we achieved those instead. We thought social would be massive this year but it only turned out to be ‘encouraging’. (Interestingly, predictions seem most accurate when it comes to budget levels.)
There is more about evaluation of learning this year, and it is interesting, again, to see how people who guess or estimate their level of improvement tend to put it lower than those who actually measure – though exactly what this correlation proves would take more digging to understand. Does having a measurement culture make you perform better, or are estimators/guessers just more circumspect than those who have figures to prove their case?
With our ongoing interest at LINE in the move from a course-based to a resource-based picture of the world, it is heartening to see pieces of evidence for this shift studded thoughout the report. Growth, for instance, in user generated content, more nugget-shaped learning and enterprise-wide information systems (eg Sharepoint) all point in this direction.
Social learning is at last making inroads, it seems, though use of third-party networks is more popular than in-house systems.
There is a lot to think about in the ‘difficult to achieve’ benefits highlighted here. Adapting programmes to individual needs (or personalisation, for shorthand) is one of these. Personalisation of learning is not only a critical element in thinking architecturally, it is also an important part of the fundamental value proposition of e-learning – and yet organisations struggle with realising this benefit. Clearly we’re in an early stage of development, though top quartile organisations point the way to an achievable future.
Other tough-nuts-to-crack, such as ‘sharing of good practice’, indicate a culture change underway. This is perhaps not so surprising – it’s a big turnaround from passively receiving training interventions to actively participating in sharing your learning. This also shows up elsewhere in the lack of skills amongst employees to manage their own learning being recognised as a barrier to implementation.
Growth of mobile learning
The interest in, and adoption of, mobile learning grows steadily, we can see, while expectations tend to obey their own laws (see chart).
This steady growth, with elements of the hockey stick beginning to emerge, would be very much with what LINE, as a major supplier of mobile learning products and services is seeing in the market.
Helpfully, the report gives more detail this year about what mobile devices are being used for in organisational learning:
- 62% access learning content to reinforce formal learning
- 54% support communication and collaboration
- 53% provide an alternative mode of delivery to PC-based learning content
- 43% support application of learning back in the workplace
- 37% access performance support at the point of need
- 27% support generation and sharing of user generated content
This post was written by John Helmer and first appeared on the LINE blog on 3rd December 2012.