E-assessment; now what does that mean?

I am preparing for a speaking slot at the e-Portfolio conference in Melbourne, Australia and as part of my research discovered a recent report by the Government called, ‘E-assessment and the ATQF (Australian Training and Quality Framework). It took me a while to understand the report until I discovered that the term e-assessment in the Australian context, describes a very limited range of activities, ‘the predominant example of e-assessment is the online quiz.’



There are a few references to e-portfolios other than recognition that there is a ‘slow but persistent growth in the use of e-portfolios in training organisations’ but only a superficial recognition of the benefits that e-portfolios can bring.


What however is ‘common’ with the UK experience is the ‘stifling’ role often played by what in Australia are called ‘auditors’ and in the UK what are called External Verifiers. In fairness to the auditors in Australia they recognise some of their own limitations in terms of their understanding of technology but they raise the same ‘serious questions around the authenticity of evidence gathered by learners and the validity of the assessment tasks given for units of competency.’


It is time that ‘auditors’ recognised, in whatever country they work, that technology provides far more stringent auditing than the most advanced paper system because every action is date stamped in real time and linked to the person undertaking it. Indeed the major opportunity/fear that UK assessment centres recognise is how visible their assessments are now if they use an e-portfolio. It is an opportunity if they are confident in the quality of their assessment procedures or a fear if they are not.

A 'proper' e-portfolio???

I recently found an interesting entry in Derrin Kent's blog http://derrin.biz/?p=88 What is Mahara (versus Paperfree, eNVQ, et.al.)? NVQnow is presumably in the list of et.als.

It is thoughtful contribution designed to help people faced with the decision about whether they should 'buy' one of the current solutions that are in the market, or alternatively use on of the 'Build your own kits' available through open source software like Mahara.

Clearly I need to declare an interest at this point as a provider of one of the solutions however I would also want to assert that we embrace many of the principles that underpin the Open Source movement. The users of our software all become members of the NOW.net community which enables them to have access to any developments of and enhancement to our core platform usually at little or no cost to themselves.

We therefore come to the debate as interested contributors rather than people who just wish to defend our position.

In his blog Derrin makes the bold declaration that TDM is 'now running an adaptation of Mahara (using Moodle and a special patch....which allows training providers (colleges or independent) of Evidence-Based Qualifications to create NVQ/QCF (or similar) template "pages" into which candidates can submit evidence for QCF units (of NVQs).'

There are other features like to the ability to add more content to enable the learner to 'show-off' their extended and reflective learning; submit pages for recording outcomes in Moodle's gradebook; make those pages available for assessment and allow learners to form '"walled garden" communities of practice'.

The blog finishes with an important reflection on who owns the e-portfolio. Clearly in Derrin's view it is key that the e-portfolio is owned by the individual. Whilst we recognise that is desirable, we have had to grapple with the notion of how can that work fully in an environment, such as the one created by QCA and implemented by awarding bodies, where quite rightly there are clear rules about who is allowed to do what and when in relation to the assessing and verification of evidence. In our experience awarding body endorsement of e-portfolios is not a random exercise, it is one that is based on their recognition that an e-portfolio delivers a secure and appropriate audit trail.

There are also some other real challenges to the open source approach that Derrin alludes to by implication if not explicably. Firstly although it is open source this does not mean it is free, something he tacitly acknowledges 'it is free of cost for anyone who has the skill to implement it...and we offer very reasonably-priced, zero vendor lock-in support services (hosting, configuration, end-user training, consultancy, etc.) for those who don't.'

Anyone who has been involved in assessment knows it is a movable feast and that any system that is designed to reflect best assessment practice has to be constantly evolved and changed. It is good that special patches have now been created but whose responsibility is to maintain and change these? Furthermore what are the guarantees that the core code will not be changed in a way that affects it's capacity to deal with any new changes in assessment.

It is indeed arguable that the principles that underpin the development of open source code are counter intuitive to the regulatory environment that is required for assessment that can be fully quality assured. When we have been involved in co-operative ventures with company's that use open source software, their reluctance to produce the sort of admin tool that enables us to control who does what using groups, roles and permissions is notable. The same tools that we use to keep the user experience as simple as possible.

This is where we again we agree with Derrin for we are as focused on the individual as he is. What matters however is that the user experiences an e-portfolio that is both fully accessible and meets their needs.

E-portfolios; bureaucratic?

Welcome to the Team Axia blog. The purpose of the blog is to be a forum where we can share our experiences of creating an e-portfolio to support the gathering,assessment and verification of evidence whether it be for Professional bodies or Assessment Centres. We hope you will share your experiences too. To kick start the blog we share our thoughts on a number of key issues.

There is now a general consensus that e-portfolios do reduce bureaucracy and as a result reduce the cost of delivering an NVQ. They are still a few who do not hold this view. ENTO the awarding body; not to be confused with ENTO Aberaman the Welsh football team are currently holding a series of workshops entitled 'Utilising technology in the Qualifications Framework'. One of the purposes of the workshop is to show the benefits of technology without 'the expense or bureaucracy of an 'electronic' portfolio system.'

The view of ENTO as a major influencer on the practice of Assessor and Verifiers is clearly important but it stands in direct opposition to most other awarding bodies and a growing number of assessment centres.

Rather than creating barriers it would surely be better if ENTO joined the lobby for LLUK to create a new e-assessment unit within the standards, thereby offering practitioners who are working effectively with e-portfolios a chance to have their skills recognised.