E-assessment harder to deliver than e-learning?

I have another reflection on the learning technologies conference which is about the creation of e-learning in comparison to developing e-assessment. In this context the e-assessment I am referring to is not simple testing but rather the more complex process, of capturing evidence of competence and then having multiple people interacting with that evidence to confirm its validity.

On the eve of the conference City & Guilds/Kineo announced the launch of the Adapt Learning Open Source Framework. At face value it appears to be another ‘open source kit’ on the market and a generous offer from Kineo because in effect it empowers people to build their own e-learning.

At the conference itself I watched a presentation from Epic, like Kineo another successful e-learning company which showed how they use Moodle; Sharepoint and Drupal to create e-learning content for a range of customers. All this software is open source and therefore Epic’s skill is in adapting the software and in creating plug-ins to create solutions that meet the specific requirements of their customers. Their income comes from maintenance contracts and the adaptations of the software and whilst their customers are not tied into licensing specific software they are ‘tied’ to those capable of adapting the open source software.

In both cases I doubt that any of the e-learning is underpinned by the sort of comprehensive and complex database that is required to deliver e-assessment particularly where this a highly complex rule base for the assessment.

To illustrate the difference there are two ways to ensure compliance using electronic methods. First take one of the examples of e-learning used by Mark the presenter from Epic. If for example you are a member of the customer services team working on Jet 2, one of the ways your understanding of the duties you are required to perform can be tested, is by asking questions about a number of scenarios and then tracking your responses. Another way is for someone to actually record how they saw you perform, to cross reference this against specific standards and then to have their judgements checked twice to make sure it was fair and equitable.

I am not implying that either of these two processes are better or worse. What however should be self-evident is that one of the processes is more complex.

The failure to understand the complexities involved in assessing competence electronically, is in part down to the industry itself which still sometimes places the ‘recommendations’ provided by Linked in, on the same level as thorough assessment that required rigorous evaluation and cross checking.

Nevertheless those who have tried to build an e-portfolio for genuine and rigorous assessment know how complex the process is and the type of sophisticated database that is required. It is not something that is easy to build from a ‘do-it-yourself’ kit.

E-assessment; a work in progress?

Just been again to the Learning Technologies conference at Olympia. I wrote a blog about the event last year. This year’s reflection on the event is different from last year’s, where the overarching impression was of every supplier offering everything and as a result it was really hard to distinguish, what were the particularly special offers being made by each provider.

At this year’s conference there was certainly more activity both in terms of participants and exhibitors, however it would again be hard to identify any supplier who was offering anything that was outstanding.

Indeed in terms of two of the presentations I attended by suppliers we know well, namely City & Guilds/Kineo and Atticmedia they both promised something radical, but with respect to both organisations they were launching products that were to come.

City & Guilds/Kineo new product was interesting in that it combines e-learning materials alongside an e-portfolio with a comprehensive reporting tool. A concept we have consistently argued has the potential to transform the market. Indeed it was a point I personally made to the executive team at City & Guilds about five years ago.

We were informed that this new product was being used from this Monday, in which case it was regrettable that the only screen shot provided was of the opening page. It was perhaps because they have not yet quite finished the e-portfolio element, which we know from experience is often far more difficult to develop than any e-learning materials.

It was clearly also strange that City & Guilds/Kineo had opted not to use their own e-portfolio company, Learning Assistant to deliver this solution but rather relied on the Totara LMS system to provide this functionality. To my knowledge the Totara e-portfolio is still a ‘work in progress’ and it is not something that can be created in months!

Atticmedia another company we have also worked with, were announcing a new project called ‘Mojo Central’. It is a new mobile and social learning platform. Again it was not really a launch not least because this development was announced back on October 31st. What they were actually appealing was for volunteers to try out their Beta version in the summer. Um? I really like Attic but they have a bit of a track record of talking about systems that never quite get finished.

Maybe this is a trick I have missed, which is that you no longer have to wait until you have a real site but rather it is sufficient to tell people about some great ideas you have….so here goes. We are working on a tool that enables staff development to be planned and its impact tracked and one that will enable work experience placements to be created and closely matched against the needs of individual students. We are also continuing to work on our upgraded e-portfolio for vocational qualifications and exploring whether we can generate the reports necessary to claim funding.

All due to be delivered in the Spring. The difference from some of the launches above is that we will deliver.