E-assessment harder to deliver than e-learning?

I have another reflection on the learning technologies conference which is about the creation of e-learning in comparison to developing e-assessment. In this context the e-assessment I am referring to is not simple testing but rather the more complex process, of capturing evidence of competence and then having multiple people interacting with that evidence to confirm its validity.

On the eve of the conference City & Guilds/Kineo announced the launch of the Adapt Learning Open Source Framework. At face value it appears to be another ‘open source kit’ on the market and a generous offer from Kineo because in effect it empowers people to build their own e-learning.

At the conference itself I watched a presentation from Epic, like Kineo another successful e-learning company which showed how they use Moodle; Sharepoint and Drupal to create e-learning content for a range of customers. All this software is open source and therefore Epic’s skill is in adapting the software and in creating plug-ins to create solutions that meet the specific requirements of their customers. Their income comes from maintenance contracts and the adaptations of the software and whilst their customers are not tied into licensing specific software they are ‘tied’ to those capable of adapting the open source software.

In both cases I doubt that any of the e-learning is underpinned by the sort of comprehensive and complex database that is required to deliver e-assessment particularly where this a highly complex rule base for the assessment.

To illustrate the difference there are two ways to ensure compliance using electronic methods. First take one of the examples of e-learning used by Mark the presenter from Epic. If for example you are a member of the customer services team working on Jet 2, one of the ways your understanding of the duties you are required to perform can be tested, is by asking questions about a number of scenarios and then tracking your responses. Another way is for someone to actually record how they saw you perform, to cross reference this against specific standards and then to have their judgements checked twice to make sure it was fair and equitable.

I am not implying that either of these two processes are better or worse. What however should be self-evident is that one of the processes is more complex.

The failure to understand the complexities involved in assessing competence electronically, is in part down to the industry itself which still sometimes places the ‘recommendations’ provided by Linked in, on the same level as thorough assessment that required rigorous evaluation and cross checking.

Nevertheless those who have tried to build an e-portfolio for genuine and rigorous assessment know how complex the process is and the type of sophisticated database that is required. It is not something that is easy to build from a ‘do-it-yourself’ kit.

E-assessment; a work in progress?

Just been again to the Learning Technologies conference at Olympia. I wrote a blog about the event last year. This year’s reflection on the event is different from last year’s, where the overarching impression was of every supplier offering everything and as a result it was really hard to distinguish, what were the particularly special offers being made by each provider.

At this year’s conference there was certainly more activity both in terms of participants and exhibitors, however it would again be hard to identify any supplier who was offering anything that was outstanding.

Indeed in terms of two of the presentations I attended by suppliers we know well, namely City & Guilds/Kineo and Atticmedia they both promised something radical, but with respect to both organisations they were launching products that were to come.

City & Guilds/Kineo new product was interesting in that it combines e-learning materials alongside an e-portfolio with a comprehensive reporting tool. A concept we have consistently argued has the potential to transform the market. Indeed it was a point I personally made to the executive team at City & Guilds about five years ago.

We were informed that this new product was being used from this Monday, in which case it was regrettable that the only screen shot provided was of the opening page. It was perhaps because they have not yet quite finished the e-portfolio element, which we know from experience is often far more difficult to develop than any e-learning materials.

It was clearly also strange that City & Guilds/Kineo had opted not to use their own e-portfolio company, Learning Assistant to deliver this solution but rather relied on the Totara LMS system to provide this functionality. To my knowledge the Totara e-portfolio is still a ‘work in progress’ and it is not something that can be created in months!

Atticmedia another company we have also worked with, were announcing a new project called ‘Mojo Central’. It is a new mobile and social learning platform. Again it was not really a launch not least because this development was announced back on October 31st. What they were actually appealing was for volunteers to try out their Beta version in the summer. Um? I really like Attic but they have a bit of a track record of talking about systems that never quite get finished.

Maybe this is a trick I have missed, which is that you no longer have to wait until you have a real site but rather it is sufficient to tell people about some great ideas you have….so here goes. We are working on a tool that enables staff development to be planned and its impact tracked and one that will enable work experience placements to be created and closely matched against the needs of individual students. We are also continuing to work on our upgraded e-portfolio for vocational qualifications and exploring whether we can generate the reports necessary to claim funding.

All due to be delivered in the Spring. The difference from some of the launches above is that we will deliver.

Online CPD continuing to make progress in Ireland

It is a year almost to the day since we first launched our first CPD site in Ireland with the Irish Institute of Radiographers and Radiation Therapists. I was back last Saturday in the Gibson Hotel where the launch took place at their annual conference.

The Gibson Hotel is itself an interesting place. It is located at the Point which is at the end of the Red Luas line next to the O2 arena. The location of both buildings is a little incongruous. They face onto the docks on one side and on the other the development that was meant to link these buildings with the main financial centre in Dublin has stopped, hopefully temporarily.

This structural picture in some senses reflects the challenges that we and the professional bodies we work with in Ireland, have faced over the past few years.

Health professional bodies in Ireland have been raising their game over the past few years. They want to be at the forefront of best practice. However the economic conditions in Ireland which have led to reductions in public spending, have made this harder to achieve.

However there have been factors that have mitigated the impact of the economy on the development of effective health professional bodies. The HSE, the Irish equivalent to the NHS clearly views them as partners in driving up professional standards. As a result they have encouraged and supported professional bodies to play a leading role in the development and delivery of CPD by directly funding CPD officers and programmes.

Their regulator CORU has proven to be equally helpful. Initially I sensed that professional bodies were nervous of CORU. They thought that they would take over responsibility for CPD. There were also concerned about other matters. Those professional bodies who have the title ‘Registered Member’ for members who have earned a higher status within their profession, recognise that this might need to change when in effect all members become registered with CORU.

However although these concerns remain the overall impression I have is that CORU is keen to forge a strong relationship with all professional bodies. Indeed arguably the professional bodies offer them the means to drive up engagement with CPD, which is so key to the effective operation of the Registers and improving patient safety.

At this year’s IIRRT conference I spoke to Margaret Murphy who is a Patient Advocate for the World Health Organisation. She received a standing ovation for her presentation. We both agreed that because Ireland is a small place, surrounded by water, it is a great place to drive change. It. There are relatively few universities that need to be influenced and relatively few ‘top people’ who need to be persuaded to bring forward change. Her primary concern is patient safety which ultimately is the objective of CPD.

At the conference it was heartening to hear practitioners sharing their best practice; it was great to watch the ‘coalface’ work of practitioners being shared and celebrated and it was encouraging to hear a profession welcoming the challenges of regulation. Over the last year it has been great to have been a part in this albeit even if it is a small way.

Completing the Circle

The NET2013 conference for heath professionals from across the world was a fitting event at which I was able to talk about for the first time, the recently announced acquisition of a majority share holding in this company by the VIA partnership.

At the conference it was clear that there was substantial interest in the presentation of the e-AoPP e-portfolio we have developed with Southampton University, for the assessment of nurses, midwives and other health care professionals when they are on placement. This interest was generated by the early results from the roll out of the system which indicates that users are finding it easy to use and difficulties accessing the system have been overstated not least because of the range of devices being used to access it. The quality of assessments are being enhanced because practice educators have more time to reflect on their judgements and tutors are alerted when target dates in the placement are not being met.

The e-AoPP also has an innovative way of capturing feedback from other professionals and patients. They are invited by student nurse/midwife to make a comment by e-mail and this is then seamlessly added to the correct section in the portfolio.

The delivery of the project comes at a time when there is a national interest in creating records of competence and linking these to the introduction of national standards. There has been an announcement from the Nursing and Midwifery Council today, confirming that nurses and midwives will be required to gather and submit their evidence of their professional development online.

As a result we find ourselves back at the place where the company first started, which is to be engaged with a project that has the potential to make a significant impact on the care delivered by nurses and midwives.

Over 10 years ago we were contracted to deliver what became recognised then, as the largest e-portfolio project in the world, which was the Learning Zone created for the Royal College of Nursing. It was a visionary project designed to enable nurses to meet their regulatory requirement. For various external and internal reasons its impact was not as great as first hoped. Being involved in the work with Southampton and seeing its potential has brought us back to what has often felt like unfinished business.

At the time we created the Learning Zone we had access to the resources of our then parent company Axia Netmedia Corporation. It is therefore fitting, as we stand on the brink of developing similar work, we have again formed a partnership with a larger company and we now finally have the opportunity to close the circle.

Why in e-learning/e-assessment does it matter so much what it is, rather than what it does?

I am subscriber to many e-learning forums. In my experience too often the discussion is about systems rather than learning. The discussion that is most likely to generate responses is one that starts ‘I about to purchase a Learning Management System, Any recommendations’. What that follows is a mixture of name checking of various LMS systems interspersed with the suppliers recommending their wares. It is a rare for someone to ask the killer question which is what do you want it for?


Today we had an e-learning guru declare how impressed he was that someone had found a way of using Word Press as an LMS. There was little discussion about what transformation in learning had happened as a result of this discovery. The point he was trying to make was that wasn’t it good that someone had got hold of some open source software and come up with a ‘free alternative’ LMS. Aside from the making the obvious point that nothing is free i.e. there was a cost for working out how to use word press in this way, what was the actual impact on learning.

I know of one learning institution that is making creative use of share point to create what someone might want to call an LMS. They would actually reject such a suggestion and rather focus on what it has enabled them to do. This is to create a learning environment that places the learner at the centre of the process. It is not about managing the learner but enabling the learner to manage their own learning, the system is built around a learner not the course they are on. Furthermore in creating such a solution they are not wedded to one approach; they are quite happy to use proprietary software if this fits the bill.

I have written in this blog before how I find it difficult to understand this desire in e-learning to create solutions that do everything. It is hard to imagine in what other field of ‘engineering’ is this approach taken. It seems perfectly OK for someone who is manufacturing a car or constructing a building to source the expertise and the materials from different places. So what’s so different about designing learning systems?

Just because you are using sales force to manage your customers/members of your organisation what is the logic to extending sales force to create an e-portfolio? Why is there such a need to have an open source portfolio if your LMS is open source?

Having an integrated offer is a right and proper objective but if it is all sourced from one place are we sure that we are making the best offer to our learners/users? Maybe there is real value to be had from ‘shopping around’ and filling a shopping basket full of items that have been brought not because of the packaging but because they actually deliver what your learner’s need.

We need to stay out of our silos

Every Friday night for the past two decades I have gone out with the same group of mates who have often delighted at pouring scorn on e-learning and e-portfolios. This week was different one of them has just joined Doosan, a South Korean company and he confessed to be enjoying his e-learning course that was inducting him into the history of the company and also showing him how to interact Koreans. Quite important given they are now his boss. He was not just enjoying it he could see its validity.


Early on in the week I was with the Group Director for E- learning for a large college and he was describing how e-learning was having a transformational effect in his organisation. As a result he was being enabled to harness expertise from a number of sources, including the private sector to create a full range of learning opportunities. Incrementally his team have been able to encourage each Department, to see the benefits of e-learning and they are now transforming the way that learning is delivered and tracked.

What had been clearly critical to this success was that e-learning was not seen as a separate activity from the development of the technical capacity and resources of the College, all this activity is being co-ordinated from one place, by someone who had the vision to see the links and with support from the executive to bring all the links together.

In one sense that is an obvious development but if you were to go around any learning institution in this country and track e-learning projects you are likely to find lots of ‘pet projects’ happening; a number of different e-learning products procured to do the same thing and little evidence of a positive benefit on the whole institution. You might also find that these initiatives were taking place with at best minimal support and at worst outright opposition, from those responsible for technology in the institution.

As someone going in from the outside into these institutions, the one person I am least happy to see at any demonstration is the representative from the IT department. As one of ‘their number’ was clearly pleased to remark recently ‘these e-portfolios were meant to be the big new thing 10 years ago and they have not gone anywhere’ It them became clear that any notion of the transformational effect of technology on learning was clearly not on his agenda.

Just over a decade ago I was manager of the East Leeds Family Learning Centre which was the venue for one of the first job guarantee programmes with Tesco. This was where unemployed people were offered a job, on the basis of their potential to be a good employee and provided they completed a training programme. The success of the programme was lauded because it got lots of unemployed people back into work. That was the end objective and it was spectacularly delivered.

However it was the easy and natural access into other services that ‘seamlessly’ delivered the result. What made the difference, was that a number of the successfully applicants, motivated by the guarantee of getting a job were able to quietly access the centre’s literacy and numeracy programmes, so that they were able to address the key issue that had stopped them making progress before. The ability to read and write. What mattered was that when the store opening was delayed, so that it opened after the summer holidays, we were able to harness the work of the Council’s Early Years’ service to create over 60 child care places. A process helped by the fact they were already in the building. It was the full integration of service delivery that made the difference.

E-learning and e-assessment will only be successfully delivered when it becomes a fully integrated part of curriculum delivery not something that is an occasionally added on and made available. The full extent of what it can offer will only be realised, when it is seen as more than just making learning environments electronic but where there is a recognition that it can transform the way that those learning environments are structured and delivered.

One would have hoped that the organisation like JISC that have asked to make this transformation would have done so by now, but for all the reasons described in this blog this is never likely to be the case. Like most things it is probably down to individual centres to pick up the e-learning ball and run with it in the hope that organisation like JISC might eventually catch up and others might look outside their silos.

It is OK to make a mistake!

The proposal from the Francis Report to introduce a duty of candour into the NHS is a challenging one. My only experience of working in the NHS was one summer placement. It was generally a happy experience but I remember one morning when cold tea was served to the residents on the ward. One of them let us know what he thought and I pacified him with an apology. I was then asked to come into the charge nurse’s (as they were called then) office and I was told that we should never admit to a resident that we had made a mistake.


This is a flippant example in comparison to the serious issues faced by the Stafford enquiry. However I use it a symptomatic of a culture where a lack of openness and honesty was encouraged and expected. Something that appears to have continued.

When I am talking about e-portfolios one of the benefits that I often highlight is how transparent they are. Not because they are ‘open’ documents, because ours are certainly not that, but because those with permission to do so are able to review the whole process. It is not easy to ‘hide’ what you don’t want those involved in the assessment to see.

For some this can become a significant barrier to an adopting an electronic system. They fear that somehow their ‘mistakes’ will be published over the web and even if this is not the case the fact is that there is now evidence that I have not got it always right. For professionals the can fear revealing their mistakes.

However when I am internal verifying vocational qualifications, evidence of students getting ‘it wrong’ and assessors highlighting this, is exactly what I want to see because each occasion provides an opportunity for deeper learning. In the same way the most powerful CPD records are often those where someone has reflected on their existing practice, perhaps as a result of a learning intervention or sometimes not, recognised a ‘mistake’ or improvement and implemented it.

Even when this happens there is a reluctance to share that learning experience as if in some senses it erodes your reputation as a practitioner, whereas in many cases, it is the person who has made a mistake and has reflected and learnt from it, is the one that you most want to see.

It is time we started seeing transparency/candour as an asset not as a threat.

Dear Serge are you finally losing the plot?

I received the annual e-mail(s) from the European Institute for e-Learning to announce their 11th conference in the UK and the fact they had secured some European funding to bring together e-portfolio expertise however I felt that I needed to write the following back to them.

Dear Serge,

I have a lot of time for the team at Eifel and particularly you, because you have been at the forefront of pioneering e-portfolios for over a decade and it is therefore with some regret that I write the following.

I have just received your invitation to submit contributions to the 11th ePortfolio and Identity Conference, Open Me! with the following topic areas.

• open ePortfolio and open badges

• open identity and open data

• open learning and open educational resources

• open assessment and open accreditation

• open employment and open business

• open architecture and open infrastructure

In fairness to you all this is totally consistent with the direction of travel for your understanding of e-portfolios over the past decade. The giveaway is in the title of the conference ePortfolio and Identity.

Somehow you have allowed these two concepts to become intertwined even though you have previously recognised, that there are other tools that are arguably more ubiquitous and appropriate to capture identity than any e-portfolio. On many occasions I have wanted to congratulate you and say well done you have achieved the goal that you set yourself and that everyone does have the same sort of e-portfolio you have wanted them to have, it is called facebook or i-google or linked in

However you can’t but help yourself and move into other areas like assessment still desperately trying to cling onto those first principles, that the portfolio is owned by the individual and that it is open and available to be shared. Can you therefore just try and explain what open assessment and open accreditation looks like. Is this where I award myself a certificate or perhaps my learning group award me a certificate based on assessment criteria we have developed together.

So what I would really like to say to you is, acknowledge what you have achieved and that the e-portfolio world has moved on. However what we are likely to have instead is a conference attended only by the presenters, in the main presenting projects that have not and will never go anywhere.

Lots of buzz but tell me what is actually happening

I called in at the Learning Technologies conference and for those of us in the e-learning/e-assessment industry it was an encouraging experience. There appeared to lots of companies many of which had bigger stands. I particularly enjoyed the fact that the EPIC and KINEO were positioned directly opposite each other, which was ironic since many of the original KINEO team were formerly employed by EPIC. If you are interested my impression was that both stands were the same size and were attracting a similar number of people.


If I were a buyer and apparently there were plenty I wonder what you would make of it all. It was hard to distinguish between the different offers because there was much of a muchness.

I have just read Steve Rayson’s from Kineo’s entry in the e-Professionals linked in discussion forum after I have written the above and he makes exactly the same point.

‘A few years ago suppliers might position themselves as mobile experts or rapid development experts. Now everyone appears to offer everything. Need a tool, we have got one. Need content, we have some nice shiny content. Need a learning platform, we have an award winning platform. Suppliers that previously only did content now offer platforms and vice versa. Increasingly many suppliers are aiming to be one stop learning technology companies. This makes it very difficult for visitors to differentiate and to know where to start when faced with literally hundreds of company stands.’

I too noticed that of direct interest to us, many of these companies also claimed to supply an e-portfolio as part of their offer but not one as we would know it. I stopped and picked up the leaflet that described one and it is at best a crude tracking system that has limited interactivity.

There were also lots of sales representatives making bold speeches about the future of e-learning. They were often full of brash claims about the way social media will transform the way people engage in learning. I am not sold on this not least because there is little evidence. Does the statement made by one presenter that more people use social media then e-mail, imply that we should radically transform the way we should think about how learning is delivered. I have had the discussion with a number of young people about Facebook and learning and it is the last place they would think about using to develop their learning, primarily because it is too open.

I heard another statement that Learners want control of their learning journey. In one sense it would be difficult to argue with that one but do learners really want to be totally self-directing and moreover do those that employ/regulate them really want them to do what they want. It reminds me of the ‘old days’ of CPD where all learning activities could be counted. I can see where social media would bring real benefits for example alerting people to relevant courses and linking them to peers interested in their subject. However does it provide a total solution? I think not.

I therefore remain really pleased that we are now developing e-portfolios that really meet a clearly defined purpose rather than ones that are like the learning platforms at the event, try to be all things to all people.

It's time e-assessment grew up

When I had the opportunity to speak to another group of University senior staff last week, I heard another set of stories about how they had been let down by technology that claimed to be able to support rigorous assessment, yet when it was used it either proved to be too inflexible or provided the student with too much control.


It is not a surprise that this was their experience given that an organisation like JISC on the one hand acknowledge that for ‘for technology-enhanced assessment to be effective, pedagogically sound developments need to be supported by robust and appropriate technology’ ‘(Effective assessment in a digital age; JISC) yet at the same time they ‘showcase’ an e-portfolio that proudly claims that it provides ‘a Personal Learning Space owned and controlled by the user.’

How can these two statements co-exist together? If learners own and control their learning space this surely means they can chose what external feedback can be included in their portfolio and what can be discarded. It might also mean they can make their own assessments? Of course it is good practice that they self-assess their work but an external person must have the final say.

A strong pedagogy is clearly based on the notion that through it learners will gain skills and knowledge, however for quality assurance of that process to be credible it is important that all comments are captured good and bad. Indeed when I act as an Internal Verifier I state my expectation that I will see assessors deferring evidence because that is one indicator of effective assessment taking place. Furthermore electronic assessment makes it very easy to see that this behaviour is in place.

It is important that a student feels strong ownership of their portfolio but that ownership needs to extend to including comments on their work from external people whether they be good and bad and furthermore the learning institution has to be totally assured these comments are captured. E-assessment systems need to be sophisticated enough to provide this quality assurance.

More than just a tick box?

There seems to be suddenly a great amount of discussion amongst doctors about e-portfolios and how they can be used in training and in capturing the impact of professional development. This is in part because all GPs are now required to create an electronic record of their competence, indeed I understand that regrettably this requirement is being viewed as so onerous that a number of GP’s have brought forward their retirement plans!


The debate raises some interesting questions about the educational value of what is characterised as the e-portfolio approach. It is of course the case that the judgements being made are in relation to the two e-portfolios that are predominantly being used. One is the e-portfolio developed by NHS Scotland, which is being used by a number of other health professions and the other is one was specifically developed for GPs.

The questions being asked are relevant not only to e-portfolio systems themselves also to the competency assessment systems in which they operate.

• Is this just a tick box exercise in which no qualitative judgements are made?

• How is high quality feedback encouraged and enabled?

• To what extent does the portfolio encourage real development and progression?

• Is it good enough just to be competent?

• Can a holistic approach to professional development and practice be encouraged by easily linking together related learning and practice?

• Does it pass the ‘three click’ rule and is it accessible from mobile devices?

All these are issues that have been raised by users of the existing systems and I do not know the e-portfolios well enough, to make a judgement on the validity of the comments that have been made. What I do know is that we have addressed some of these questions in solutions we have developed.

Our sites positively encourage progression for example radiographers can become recognised as consultant and advanced practitioners based on the quality of evidence they supply; there are effective ways for feedback to be requested and then seamlessly attached to the relevant evidence that is created; all sites pass the three click rule and are accessible from all mobile devices and related evidence can easily be linked together.

What I also know is that effective competency assessment cannot be delivered solely by an e-portfolio. It has to be part of a culture of effective and meaningful professional development. One of the big advantages of an e-portfolio is that it makes it easier to make judgements about the quality of the assessments that are taking place because there is greater transparency. In other words it is possible, for those with permissions to do so, to check that it is more than a tick box exercise.

Learning is the Best Insurance Policy

I recently spoke at the Irish Institute of Radiography and Radiation Therapy’s annual conference when their new online CPD and membership system was launched. Before demonstrating the site I shared a few observations with the conference about CPD.


I first congratulated the IIRRT on embracing an output based approach to CPD. This was wholly in line with the trend amongst professional bodies. Whilst this is most pronounced amongst other health professional bodies other professions are following suit. Simply put what matters more than the time you spend on CPD is its impact, for example, on patient care. At this conference whilst the certificate of attendance is important, what arguably matters more is what delegates discovered at the conference that they could put into practice.

There is a clear change in the role of professional bodies. It is not only about protecting members when problems occur, although that remains important, it is also about proactively encouraging members, to commit to continuous development so they have the skills required to maintain high quality professional practice.

In other words ongoing learning is the best insurance policy.

In this context CPD should be seen as less an encumbrance and more as an opportunity to demonstrate commitment to development. In making that statement at the conference there was already evidence that I was ‘preaching to the converted’. A visit I made on the previous day to the ‘Breast check’ study day in Dublin confirmed that Radiographers and Radiation Therapists were indeed fully engaged in CPD.

I listened to a number of examples of high quality CPD practice. One radiographer/radiation therapist talked about attending a course; bringing the learning back to the workplace and discussing with the peers which led to a positive change in practice, that was evidenced by the research presented at the study day. There were clearly lots of other examples of this process also being presented at the annual conference.

It is perhaps time that we recognise how much quality CPD is taking place rather than listen too much to those that choose to discourage it, often hiding behind the excuse that their employers have not provided them with the time to ‘do CPD’, as if CPD were somehow divorced from practice.

What is critical is that the best practice is captured, valued, acknowledged and shared. Electronic portfolios provide a great way for doing this.

E-portfolios; Reaching the Tipping Point?

I was at a Sixth Form College recently presenting our e-portfolio for NVQ assessment and was greeted by the college’s Head of Technical Support, who delighted in telling me that e-portfolios were meant to be the big technological advance of the last decade and this has never materialised. I first asked him whether it was worth my while continuing with the demonstration and then asked him what he meant by an e-portfolio and suggested that the technology that underpinned them, may have advanced over the last decade.


I might also have said that in part I shared his delight to see the ‘old concept’ of what was considered to be an e-portfolio being sufficiently challenged, that it was reaching the tipping point where it was significantly under threat.

This challenge is coming from two sources. One is from the ‘So What’ camp which I have highlighted before. If an e-portfolio is just a collection of artefacts that capture someone’s personal development, with the user being in total control of the data, a) what use it is and b) aren’t there better tools that do this like Facebook and Linked In.

The other source is relatively new but arguably even more significant. Various public bodies, notably JISC have promoted e-portfolios that are developed on open source software. This is fine if these portfolios do not contain confidential data. However if they are being used to support the assessment of competence, where that assessment potentially involves confidential information, there are growing concerns about whether these portfolios are sufficiently secure to hold this sort of data. This challenge cannot be a surprise given the origin of the software.

Our approach is that the correct position of e-portfolios needs to be somewhere in between the ‘two camps’. We believe that it is important that the learner’s feels in control of their data and we take great steps to deliver this, however to be able to support effective assessment there does need to be the ability for assessors to interact with that data in a transparent way. Similarly we also think there are great benefits to be gained from sharing technological developments between our customers, but this has to take place in an environment where the software is secure not open.

As for the demonstration it went ahead and the College is now using the e-portfolio although I remain unconvinced that the lessons from one Department’s hopefully successful implementation will be shared across the College.

Setting the Gold Standard

This year the debate about the GCSE results has extended beyond the day of the results. Usually the morning of celebrations is followed by the afternoon debate as to whether real standards have dropped. The debate then ends and it is business as usual. However the events of last week open up a debate about functional literacy that reaches far beyond the impact on individual schools.


This is not to minimise the impact on individual schools and their students. I have personally witnessed this however the significance of the events this week goes well beyond schools.

In the world of adult learning the focus for many years has been on the number of adults who are not functionally literate and numerate and this has been extended into whether we have the skilled workforce required for the 21st century. There are numerous reports that have addressed these issues.

All these reports assume that there is a standard that could be used to measure functionally literacy and numeracy and a minimum skill level. In respect of skills in Adult Learning world the standard that has been adopted is NVQ Level 2 and it is assumed that you have achieved the standard required to become a skilled worker, albeit with further training, if you have an NVQ Level 2 or you have 5 GCSEs A*-C. If you have not reached this standard then there is lots of funding available you to do so, once you have reached this standard then that funding tends to disappear.

Now I have long argued that adopting this approach is like comparing apples and pears. It is just about a reasonable argument that to demonstrate skills at NVQ Level 2 you do need a significant amount of knowledge, but it is not the ‘type of knowledge’ that is measured by a GCSE.

What however this current debate about the English GCSE exposes is that it is no longer possible to use the ‘exam system’ to determine whether someone has reached the level, which determines whether they are functionally literate. If the decision was made to increase the marked required to achieve a level C by 10 points, as appears to be the case, then that decision appears to be based on maintaining the ‘rigour’ of the exam system rather than on providing a gold standard for literacy.

We wait to see what Ofqual decides to do over the coming week. It will be interesting to see whether they factor into that decision these wider implications.

If they do not then when the next person gets up to say that x millions of adults in this country are functionally illiterate and lack ‘basic skills’, then someone should ask what do they actually mean and how do they know.

Is the e-portfolio movement running out of steam?

It was announced recently that the national e-portfolio in Australia had been cancelled and replaced by a ‘connecting and sharing’ forum. Is this a sign that ePortfolios in Australia is losing momentum?


I attended the national event in Australia a couple of years ago and came away from it with a suspicion that the e-portfolio movement in Australia is being ‘bogged down, with the same issues that affect the discussions about e-portfolios in Europe. There was still too much talk about what is an e-portfolio rather than what they can actually do. The discussion was also dominated by academics without any real attempt to engage the commercial sector.

The announcement about the new Forum in Sydney in September is also accompanied about details of two events. The first event examines ‘ePortfolios as Personal Learning Spaces’. There should be a question mark at the end because there is little evidence that anyone wants to use e-Portfolios as a personal learning space primarily because, in my view; they don’t see the point. This sort of solution has been offered across many Higher Education institutions in the UK but there is little evidence of widespread use.

The second event is about the ‘Using eportfolios for online & Recognition of Prior Learning (RPL) assessment’ I am unsure what will be presented but they perhaps should have a look at the Recognise project recogniseme.ncl-coll.ac.uk/ which is already using an e-portfolio for RPL and it works.

I would suggest that the key questions that this forum in September needs to address, based on the lessons learnt from the wider e-portfolio community are;

• Why is the role of e-portfolio in relation to digital identity even being considered when there are far better and more ubiquitous applications that do this like Facebook etc?

• Is the e-portfolio community really being responsible in supporting ‘Do-it-yourself’ e-portfolio solutions when there are already e-portfolios solutions developed that work? I understand that there are clever people who can build a car from a kit but is that a sustainable model for the future production of cars!

The forum also needs to think about what is the point of re-inventing the wheel continually when there are already solutions that are developed that could effectively work in Australia. I am mindful there is a concern that many of these are developed by ‘commercial companies’ and therefore a dependency on these companies could develop. However by opting for internally developed solutions, all that is happening is that dependency on a company is being swopped with a dependency on a group of individuals in an IT department.

Why the development of e-portfolio needs to continue to be pursued is that is tantamount to the ‘bleeding obvious,’ that technology should be able to bring real benefits to assessment and specifically the assessment of vocational skills. The current paper based systems are indefensible in how they divert people from real teaching and learning to bureaucratic activities; their cost and their impact on the environment. However the e-portfolio community should start taking a more responsible approach to moving this whole debate forward or else it will run out of steam.

What to make of the 'new' Ofsted

The education press is awash with stories about the new Ofsted regime. These include claims, that for FE College, 61% of inspections have received a lower grade than the one previously awarded and stories that some institutions are taking legal action, with respect to the actions of the inspection team.



I can only speak from my experience and I have recently been in contact with two of the centres who use our e-portfolio, one fully and the other partially, who have both been awarded outstanding grades in virtually all aspects of their provision. This means that two of the three training providers in the South West, awarded outstanding grades, both use our e-portfolio.


It would be easy to say, especially in this blog, that it is the e-portfolio that makes the difference but that would seriously distort the truth. What would be true is their use of the e-portfolio, demonstrates their openness to embrace innovation particularly through the application of technology.


However what really makes the difference is the quality and commitment that runs throughout both organisation and their level of ambition. One of them Didac took the bold step of purchasing their own training centre two years ago and investing in high quality equipment, for a relatively small skill area. At the heart of Didac though is not a building but a leadership team that really cares for their staff and students. As one of their team said recently it is rare to have a boss, who when he has not seen you for a few days, texts you and wishes you well for the weekend.


The other organisation is Reflections Training academy that rather than resting on their laurels following a good Ofsted, pushed ahead with embracing technology; improving their facilities and driving up achievement rates.


In terms of the Ofsted process both organisations speak of very rigorous and intense experiences with lots of contact with students and observations of delivery. You get the sense that the inspectors really got to know the organisation which I imagine is virtually impossible with a large college. The best the inspectors there can do there is to hope that their examination of the performance figures, highlight an issue that they can follow and that this leads them to practice, that is indicative of the organisation. It is a tall order.


I am in no position to comment on the validity of the inspections that are being contested, other than to say that in the ongoing debate about Ofsted we do not lose sight of the organisations that fully deserve their grades good and bad.

Qualifications Free(d) for All

There has always appeared to be a hidden ‘scandal’ about the way that qualifications are developed and then marketed in the UK. Registration and examination fees have grown incrementally each year, to the point where some High schools are spending the same amount on exam entries than they do on the salaries of five teaching staff.



This would be just about legitimate if the money from the fees, was ploughed back into the education system, however the truth is that there are cases where the money has disappeared directly into the pockets of individuals.


There was a point where this looked like it was all getting sorted out. The previous government had introduced the Qualification Credit Framework. Underpinning this approach was the notion that units of learning were to be shared and therefore the ability of awarding bodies to ‘own’ and market qualifications was starting to be eroded. However one of the first actions was to remove the quango responsible for the implementing the QCF and declare that the business of developing and designing qualifications should again directly be the responsibility of awarding bodies.


So cue for big celebrations amongst all the large awarding bodies because they can again sit back and watch their profits again start to steeply rise. However there are dangers to the market economy and there is news that there are new players starting to enter it, who are beginning to challenge some of the ‘big players’. Losing the ability to award qualifications for a small sector like for example furniture skills, may not fundamentally shake their foundations but like a small hole in the dam it has the potential to have a significant impact, not least in demonstrating that all Goliaths have their David.

Hunt for the missing JISC funded e-portfolio initiative.

Ever heard of LIPID; PortisHEead; ePISTLE; FILE-PASS; ELP; Flouish; Reflect 2.0; PDP4XL2; PC3; SRC; Co-genT; EPPSME; SAMSON; TELSTAR; FASTTECH; MABLE; and PIOP Phase 2 and 3?


Perhaps more importantly do you know where any of them that are currently been used and have had a significant impact on the use of e-portfolios to support the delivery of HE and FE programmes? By significant impact I mean being used across an institution or even a whole faculty or department?

The common feature of all these projects is that according to JISC, all these projects use Pebble pad as their e-portfolio and the subject of investigation or development. In some cases the investigation is wide ranging such as examining the use of e-portfolios by SMEs. In other cases the focus is very narrow e.g. LIPID is about exploring how to take data from the student management system used by Wolverhampton University and make it available via a web service to the Pebble Pad e-portfolio system.

Just for information Pebble Pad was developed by Wolverhampton University but became owned and marketed by a commercial company called Pebble Learning.

I am sure the majority of the money that JISC awarded all these projects went directly to the HE and FE institutions involved and not to a commercial company and I am sure all the money was used and accounted for properly, however it seems right for JISC consider, as is the case with all educational institutions whether they pass the value for money test?

I welcome the fact that all this money has been invested in e-portfolios but to return to the theme of the last blog ‘So What?’ The real question for JISC, amongst others to answer, is have the projects they have been funding been seeking to answer the right questions?

The E-Assessment or the So What question?

This week sees the 10th anniversary of the international ePortfolio & Identity Conference and welcome and the second ePortfolio World Summit. It should be a time for celebration however if you read reports like the one below on the LEAP 2A specification for e-portfolio data exchange http://www.jisc.ac.uk/media/documents/programmes/elearning/Leap2AReviewReport_Final_1.3.p then you would have to conclude that the e-portfolio movement is in real danger of losing its way if it has not done so already. The report is littered with examples of interesting initiatives but the authors find it difficult to come to any conclusions because of the lack of users.



It is worth asking why this is the case when intuitively the development of e-portfolios is an idea that makes such apparent sense. There are potentially huge savings in people’s time and in resources however that is if you consider that the primary purpose of e-portfolios is to support assessment.


The reality will be that at this 10th conference there will be little attention given to this subject and instead there will be the normal focus on e-portfolios and their role, as providing people with their digital identity and then the inter-operability and ownership questions that flow from this.


I have always been baffled by why interoperability is such a big issue for the e-portfolio community. It does not appear to be such an issue for the Apple and Microsofts of this world, who particularly in the case of Apple delight in ‘forcing’ software developers to create applications that work on their devices only.


Is interoperability really such a fundamental issue when individuals are happy to put their content in different places, provided that it can be easily exported to other places and where appropriate links can be made to enable their data, so that it flows between different systems using API links.


With respect of ownership, is it so critically important that the owner is always in total control of their data, particularly when it is being used as part of an assessment. Surely one of the big benefits that e-portfolios bring to assessment is the way that they make the whole process more transparent, to those that have the rights to view evidence because of their role in the assessment process.


So it is not surprising that if all e-portfolios are about is creating digital identities, then users are saying So What and not using them. From our perspective if you do create e-portfolios that have a real purpose, often in the context of assessment and where that e-portfolio is supported by an admin tool that enables you to control who does what and when, then they are used.

Can an e-portfolio really help?

I have been Chair of Governors at a large High School for the past five years. One of the initiatives we have introduced is for Governors to have direct access with departments in the school. This week I had the priviledge of listening to an revealing discussion between two of the more experienced teachers in one department. They have been responsible for helping develop study and thinking skills across the school. Their reflection was that having done this for a couple of years, they prehaps had disadvantaged their students because whilst they had really become engaged in learning, they may have unwittingly walked out the classroom without the information they needed for their exam.

There were real echoes here with the e-portfolio debate, around the question of just how useful they really are. Is an e-portfolio something that encourages individuals to discover and capture using multi media a variety of learning experiences or is it a means for individual's learning to be assessed for a specific purpose.

Of course there had can be elements of both however if an e-portfolio is being used for assessment there has to be a real sense that records should be opened to others for external scrutiny. There are of course questions about how this is done, but in all the systems we have designed there is always the fundamental pedagogical question that needs to be addressed, which is who owns the portfolio. If it is fully owned by the individual then how reliable is the assessment?

I can imagine the cries for those that espouse the benefits of livelong learning portfolios who will argue the importance of holding your learning documents in one place. However how many of us really need this and furthermore when do we actiually look at work we created in our various historical learning episodes.

If you don't face this fundamental question of ownership then e-portfolios creators will be constantly haunted by the So What question. Is this all about just creating a more interesting storing space than an attic.

We are very clear e-portfolios are there for a purpose. To actively support people with their learning journeys through the provision of interactive learning plans; an easy means to gather evidence and where required to put permissions in place for others to interact with the portfolio.

If they are not we might be reflecting like the teachers about whether we have actually missed the point. Like it or not examinations and assessments exist and well designed e-portfolios can help.