“Life can only be understood backwards; but it must be lived forwards.”
Søren Kierkegaard (1813-1855)
Evaluations are in and it's time for reflection
H817 OU UK: Activity 21
It was with anticipation that we awaited an evaluation of our prototype. Examples of what we wanted to include in our assessment design toolkit were presented as Prezis (a presentation tool) and accompanied by word documents and explanatory texts. The site was polished up and we edited the texts to enable a smoother flow throughout the project site. When I read through it a final time, I felt that the language was clear and all the terminology that we thought might be problematic, was now included in a brief glossary.
The first evaluation arrived promptly, followed by a second. It was evident to us as a team that it wasn't obvious to the audience what it was that we were trying to achieve, despite our efforts to simplify language and provide texts explaining what we were presenting. Although our future aims were outlined, this seemed to have been overlooked.
I promptly drafted a document in Google Drive, shared it with Dave and Priya and notified then via Twitter that I felt we should make a few changes to the introduction before other students take to evaluating our prototype.
Priya and I met online the following day via Skype and edited the draft together. Dave reviewed it and also felt it should resolve any misunderstandings.
Points to consider
A third evaluation was completed after we'd made these changes and I felt relieved that our objectives were now clearer. There were relevant points provided from the three evaluations which gave us something to reflect on and consider for the future, such as:
- enabling the evaluators to view resources for the assessment design toolkit in varying forms. With more time we could have achieved this.
- references and further reading would have been appreciated. As a group we researched intensively for this and could have uploaded references from our research section, our blogs and our extra documents which would have brought a comprehensive list together. This is something to keep in mind.
- rubrics for different assessment approaches have been suggested. This is something that we would be doing in reality.
- it was suggested that the elements be linked by a common story for coherence. This was an interesting and valuable point. The texts which introduced the resources and linked them to each other should have resolved this, but obviously didn't. I feel this was a relevant point and something that I would implement if I was presenting a prototype of this kind again.
We were aware from the commencement of our project that our particular resource would present problems when it came to the evaluation stage because it differs from the other groups work. It is not a class activity, or a guide through an historic site. These projects can be outlined in a step by step process and are perhaps easier to understand and walk through as an evaluator. You can get a real sense of these kinds of projects and visualize how it would be to undertake these tasks. In comparison, with a toolkit, it's something that you dip into. You take out an appropriate tool that is going to be suitable for the task at hand. It will not be the same tool each time and different users will have their preferences.
This is where our presentation could have benefited from presenting our prototype from the perspective of a teacher in a specific context. We could have staged this as a narrative, illustrating the needs of the teacher and learners and the choices that the teacher could make and the effects that this might have on the learners and the learning process. We had these resources at hand but opted for another approach. Had time permitted we could have implemented this idea.
Evaluating other projects
I gave one team a full evaluation and commented on several other completed prototypes. It was evident across the board how much time and effort had been invested in projects over the past six weeks. I felt that during my evaluation it was important to read through the entire site to build up the background before doing a walk through. The heuristic evaluation sheet gave me something to focus on and was quite comprehensive. Although what made it difficult, is that I had no comparison and I'm not an expert in the field of providing reflection courses to business mentors. However, this task was part of the learning design process and needed to be thought of as such. It was a valuable part of the process as a whole and made me reflect more on our own project and how we had approached it as a group.
This evaluation phase could have benefited from being allocated more time. This task was designed as a six week project with one week dedicated to the completion of a reflective assignment. This enabled only a couple of days for evaluations and reflection which I felt was too short to experience the process properly. Considering how much work was invested in these projects, an appropriate amount of time should have been allocated to read through them, to offer qualitative feedback and to give teams time to implement some of these suggestions.