Assessment & Feedback Study

Introduction.

Our learning set is constructed from a wide range of disciplines with each member having significantly differing amounts of lecturing experience.  Several meetings were timetabled and scheduled to discuss the different methods within our disciplines used for assessment and feedback, with the intention to explore collectively similarities and distinctions among approaches against a pedagogic rationale.

The summary below was written collectively partly informed by our background reading associated with the PGCAP course and partly informed by our own practice and experiences.  Given the comparatively short word count for this assignment it was felt best that rather than present a multitude of opinions, we have chosen to present the outcome of our various discussions in attempt to be concise.

Overview of pedagogic theory.

Assessment methods and requirements probably have a greater influence on how and what students learn than any other single factor (Boud, 1995) and evidence shows that students spend the vast majority of their ‘out of class’ time on assessment-related tasks possibly to the detriment of their actual learning.  Ramsden (1992) goes so far as to suggest that from the students’ point of view, assessment defines the curriculum.

Assessment methods can be linked to two fundamental over-arching schools of thought developed by Professor Dweck who is a professor of psychology at Stanford, the first camp, known as entity theorists, consider intelligence to be finite and cannot be expanded (such as a persons height, it cannot be altered); the second camp, known as incremental theorists, view intelligence as something that can be improved and expanded (as an analogy a persons strength can be improved by exercise and ‘working’ out (Pink 2010, p121).  This can be thought of with the entity theorists considering that assessments are something that you can prepare for and respond to, incremental theorists perceive an assessment as something integral to the learning experience and not the final measurement. This is comparable to summative and formative assessment.  With summative assessment being a measure of someone’s knowledge or progress to that point and typically being a measurable score, through to summative feedback which is a more akin to coaching or mentoring where more often than not qualitative assessment is undertaken and feedback given to the student.

Whilst it may be true that student motivation towards their studies can be enhanced through a focus on assessment and clarification and sharing of the development of assessment criteria (Iphofen, 1998).  Biggs (2003;158) suggests that in order  to assess learning outcomes holistically, a conceptual framework should be created in order to see clearly the relationship between the parts and the whole. This could be described as assessment FOR learning, rather than assessment OF learning (Leahy 2005), and is compatible with the inclusivity agenda.  Research supports the statement that attention to assessment for learning improves student achievement. (Black & Wiliam, 1998) found that teachers who used assessment for learning would achieve in six or seven months what would otherwise take a year.

Assessment in higher education has long been recognised as an essential aspect of its provision. It has offered a benchmark of achievement for students and institutions alike. In the past it has focused on assessing a set of achievements based on knowledge of a particular subject matter, it often did not assess much else. The QAA (2003) reported a need within Higher Education (HE) for assessment to be improved, and was in most need of improvement above many other aspects of Higher Education provision. Assessment is considered as essential for measurement of quality and needs to be developed so that it can incorporate some longer-term benefits for students. There is now an expectation that Higher Education provides not only knowledge in the subject matter studied, but also to develop and facilitate learning beyond a student’s current experience towards long term learning and skills for employability (Boud & Falchikov,2006).

Institutions are now judged on their employability status with students and this is where assessment strategies can become an integral part of the learning process. Carless et al (2006) in developing ‘learning orientated assessment’ suggest that there are three main purposes of assessment that of – certification, assessment as part of the learning process and assessment that enables learning in the long term. For the purpose of this particular piece of work our focus is that of exploring assessment as part of the learning process and how this enables learning in the longer term and develops ‘real world’ skills and applications for future employability.

Assessment for learning involves the development of assessment processes that are embedded in Higher Education provision, it not only assesses what has been learnt within a specific context but is a process of learning in itself. By this it means incorporating differing aspects of assessment that enable students to self assess and peer assess equipping them with abilities for professional life. It also incorporates a more long term view to learning above and beyond the HE institution (Boud,1995;Boud and Falchikov,2006).  There are many ways that assessment for learning can be implemented and there are many models to consider (Falchikov 2005).

An example of such an assessment for learning model has been formulated in the University of Northumbria. ( www.northumbria.ac.uk/cetl_afl/ (2011)). They suggest that the assessment process needs to incorporate a variety of assessment tools and processes; its focus is on student involvement in the assessment process through collaborative working, peer and self-assessment. The model supports the research on assessment with the suggestion that it enhances the approach to learning, acts as a motivator for students to engage in deeper learning and enables feedback to be offered as learning occurs so providing immediacy which summative feedback alone cannot offer (Boud 1995;Weurlander etal,2011).  Assessment for learning as a concept provides many opportunities to be innovative whilst ensuring quality of provision in HE is maintained.

Indeed the UKPSF has listed within one of its key aims within the framework that “Acknowledges the variety and quality of teaching, learning and assessment practices that support and underpin student learning” (HEA 2011) which reflects the diversity of assessment techniques.

One of the group members is undertaking a collaborative approach by asking her students to write and present regularly in a talent search that is integrated with the BBC.  The area of assessment to be explored in this work is aligned with assessment for learning in that it involves using the contact time to work on a creative writing piece, which is reviewed and feedback is offered both from peers and from the lecturer. It is also an innovative project as it incorporates the development of employability or ‘real world’ skills in that it is not only assessed by the lecturer as an academic piece but also reviewed by an industry expert that of the BBC. The process of assessment is deeply embedded within the module and acts as a learning tool for students.

Student Feedback

In module evaluations forms, 100% of students on the targeted BBC modules highlighted “BBC workshops” as one of the three areas of the module they were most satisfied with.

Other impact of the BBC talent search has been to see 100% of students submit their assessments, due to the extra motivation offered by the prospect of their work being produced by the BBC.

Student feedback has also included the comments from “do you think I stand a good chance of winning?” evidencing extra investment in the work produced to “can I bribe you?” evidencing extra motivation on the part of the students due to the real world impact of the module.  The project has been extremely successful, with Writing for Performance and Playwriting being two of the three highest recruiting creative writing option modules this year.  Some of the key measures of success have been in student recruitment, student attendance which has been high, student submissions which have been on time and of a very high standard, and industry appraisal and response.

Further anecdotal Feedback was obtained from students who had been involved in an Assessment for Learning session. It was a small-scale intervention, which enabled students to engage in set exercises and tasks during a module that would help them work towards a personal reflective assignment. The structure of the exercises was such that whilst they were part of the assignment, the key focus was enabling the students to learn as they worked through the assignments, rather than just focusing on the assignment itself.  They were asked to feedback how they had experienced the exercises and their contribution to learning with regard to fulfilling the assignment requirements.

30 students responded and offered feedback on post it notes. Presented below are selected and representative samples offered by the students:

• The work and discussion has really given me some great points to use in my life story essay and I feel more confident about writing it
• It helps to get my mind focused and calms my nerves to know that I am on target after the support classes.
• I found it helpful and helped me understand more and clarify any questions.
• I found the reflection exercise extremely useful and feel it will be helpful for my next assignment.
• It was really useful as it helped me to reflect and answer some questions relating to the assignment. It enabled me to help another member of the group, which was useful to confirm my experience and knowledge.
• It gave the opportunity to understand the assignment. You can get into it and find out what you’re going to need in a way that you might not think about on your own
• The exercises helped with reflection.
• It helps as main points are summarised in the seminar and it recharges our brain and it makes the work easier as the group has discussed it.
• It helped me gather my thoughts and was useful to put it on paper.

The students found this to be a positive experience, there were no negative aspects offered but this may be as a result of how the feedback was elicited rather than there being no negative experiences of the process. The comments did express how the collaborative and group work process was useful in developing ideas, thoughts and reflection in preparation for the assignment. This supports the literature relating to Assessment for Learning where by deeper learning is facilitated through the collaborative process.

Indeed this collaborative process can present opportunities for students to absorb and rephrase their learning in their own words, a critical stage in the learning process as identified by (Kolb 1984; Ramsden 2003; Race 2006).

The MEng in Civil Engineering course has a similar industrial collaborative element to its work, with the group design project encouraging each team to respond to a brief set by the Technical Director of a local engineering consultancy.  This year the brief has been aligned to the co-operative site in central Manchester, with each team asked to design a sustainable energy centre for the entire site.  The integration of ‘real’ engineers into the design process and access to a real world project has been a positive experience for the students, with weekly meetings scheduled between the groups, lecturer and occasionally the industrial link.  The students then discuss their progress and challenges that they are facing and take minutes which are circulated.  This process is a form of reflection for the students and they’ve found that actually by critically evaluating their own progress as a team they have been able to identify and mitigate several last minute panics on the project.  This weekly meeting and fortnightly reporting is fundamentally a form of formative feedback, with the summative feedback happening at the end of each semester following the submission of their final reports.  The formative feedback takes several shapes ranging from informal presentations, discussions, question and answer sessions, sketches, models, and reports and whilst there are no quizzes or buzzers to integrate into some of the more unusual forms of formative feedback, the mechanics offered are similar to a real engineering practice.

Similar to the BBC case study above, it has been found that by linking the project to an industrial partner, this has increased the credibility of the project and made the work appear more relevant and worthwhile to the students, thus having a substantial increase in participation and involvement.  This is the 5th year or so that this project has been running and every year sees the process become smoother and more refined.

Action plan.

[NC] Having reviewed the different feedback and assessment techniques not only for this report but on the PGCAP course in general I think that historically engineering degrees have been highly driven by the summative feedback model.  Whilst I feel that this model still has its place and is needed to give the comfort to external bodies such as the professional institutions with regards abilities, the inclusion of different formative feedback and assessment techniques would benefit my teaching and more importantly the students learning.  To this end I try to include even more open questions within my lectures and encourage open discussion with the students.  I try to use photographs of my own buildings and allow the students to ask me anything that they wish about the building to help expand their understanding and knowledge as they progress but this could be further improved with site visits where appropriate.  With the MEng Civil Engineering module, I feel that this is a positive experience for those involved, but the attachment to an industrial partner really does increase the motivation and drive of the students and this is something that I would like to introduce to all years of the degrees I teach as best as I am able.  Ideally I think this could work well as a problem based learning exercise for the students, with an industrial partner for the day who can help be a facilitator.  I will make great effort to introduce as many opportunities for formative feedback to support learning with the modules that I am responsible for next semester and for the new modules I will be planning as part of my PGCAP optional module.

[SB]  This module and in particular the action learning set has highlighted for me the value of assessment for learning. I will embed this approach into my Semester two modules, but more importantly will be encouraging my colleagues to adopt this approach as we engage in our Programme review. This model of assessment and feedback is particularly relevant to students of occupational therapy because of the commitment to continuing professional development and it’s requirement for re-registration with the Health Professions Council.

[HM]  The action learning set activity of exploring assessment for learning has enabled me to develop greater awareness of the different approaches to assessment. The pedagogic theory relating to assessment for learning has highlighted areas in which we already offer this approach to assessment but also how it can be developed and embedded more deeply within the programmes offerred. The Counselling and Psychotherapy undergraduate provision at Salford is currently in a process of change with its development of a qualifying degree programme in which the intention is to review assessment strategies and incorporate assessment for learning within this.

[JT] This action learning set activity has allowed me to evaluate the impact of this year’s targeted modules I am running with the BBC and to learn from others experiences of assessment for learning. By initially completing the PGCAP case study form for the BBC activity as part of our action learning set activity, this project has allowed me to reflect upon the aims and intended learning outcomes of the activity and the rationale behind assessment for learning.

Summary.

Whilst the benefits of using a varied and diverse form of assessment is broadly recognised by academia and encouraged within the UKPSF and the aims of their framework it is often viewed as a means to an end. By only viewing the assessment as a final measurement or outcome from learning (summative feedback), then this can result in the student engaging in shallow or strategic learning that whilst initially may be gratifying for the student may lead to long term disappointment. By strategically integrating the assessment process into the learning experience (formative feedback) then this can offer a deeper learning experience for the student and become ultimately more rewarding.

The integration of regular, targeted and constructive feedback has been shown to receive both positive feedback from the students in the ‘all important’ National Student Survey (NSS) but also to encourage and motivate students in their learning.  All of the action learning set noted that if the assessments or activities are carefully considered then students will still engage even if no marks are attached.  This sometimes may rely on novel gimmicks such as buzzers or electronic voting which have been shown to be very effective, but there are still a wide variety of techniques available that we have highlighted above in our individual action plans that can make a positive difference.  Formative feedback doesn’t have to add to the lecturers workload either, this formative feedback can be provided by the student testing and critically reflecting on their understanding to that point.

The key to quality feedback is naturally context and the all important feedback sandwich.  Li (2011) noted that the students preferred feedback when phrased as “presenting your idea in this fashion could improve your grade” rather than “this was poor”.  Personally I feel that whilst targeted feedback on work is important, rather than referencing an improvement to a grade, I prefer to ask a question that encourages the student to think and perhaps expand their learning a little further.

Whilst timely, structured, formative feedback and assessment would appear to be the Holy Grail in higher education, there is still a time and a place for summative feedback.  Employers and other external bodies like to deal with finite quantities and measurable quantities and this will always drive the demand for summative assessment.  Perhaps the solution is a carefully crafted blended amalgamation of the both types of assessment and feedback, allowing them to compliment each other rather than conflict.  This clearly is where the skilled academic can excel through the elegant crafted course design, lecture notes and tutorial plans.

References

Biggs J (2003). Teaching for Quality Learning at University. Buckingham: The Society for Research into Higher Education
and Open University Press

Black PJ and Wiliam D (1998), ‘Assessment and Classroom Learning’, Assessment in Education, p7-74

Boud D (1995) Enhancing learning though self assessment London: Kogan Page

Boud,D & Falchikov,N  (2006) Aligning assessment with long-term learning. Assessment & Evaluation in Higher Education. Vol. 31, No. 4, August 2006, pp. 399–413

Carless ,D (2006) Learning –oriented assessment: principles and practice. . Assessment & Evaluation in Higher Education. Vol. 31, No. 4, August 2006,395 -398.

Falchikov,N (2005) Improving Assessment through student involvement. London. Routledge Falmer.

HEA (2011). The UK Professional Standards Framework for teaching and supporting learning in higher education. York, The Higher Education Academy.

Iphofen R (1998) ‘Understanding motives in learning: mature students and learner
responsibility’, in Brown S, Armstrong S, Thompson G (eds) Motivating students, Kogan
Page, London

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. New Jersey, Prentice-Hall.

Leahy S,  Lyon C,  Thompson M, and  Wiliam D (2005)  Classroom assessment, minuite by minuite, day by day. Educational Leadership 63 (3)

Li, J. and R. Barnard (2011). “Academic tutors ‘beliefs about and practices of giving feedback on students’ written assignments: A New Zealand case study.” Assessing Writing 16(2): 137-148.

Oxford English Dictionary (2011) available online at http://oxforddictionaries.com/definition/culture

Pink, D. H. (2010). Drive: The surprising truth about what motivates us. Edinburgh, Canongate Books Ltd.

Quality Assurance Agency(QAA) (2003)Learning from subject review 1993-2001 sharing good practice (Gloucester, Quality Assurance Agency for Higher Education)

Ramsden P (1992) Learning how to teach in higher education. London: Routledge

Race, P. (2010) Making Learning Happen (2nd edition).  London: Sage Publications

Race, P. (2006). Learning – A natural human process: . The Lecturer’s Toolkit: A practical guide to assessment, learning and teaching. Abingdon, Taylor & Francis e-Library: 1-26.

Ramsden, P. (2003). The nature of good teaching in higher education. Learning to Teach in Higher Education. London, RoutledgeFalmer: 84-105.

Smalley, R.L. & Hank, M.R. (1992).  College teachers.  In A.K. Koshi, Ed., Discoveries: Reading, Thinking, Writing (pp. 125-127).  Boston: Heinle & He

Weurlandera,M; Söderbergb,d,M, Schejac,M, Hultb,H & Wernersonb,H (2011) Exploring formative assessment as a tool for learning: students’ experiences of different methods of formative assessment. Assessment & Evaluation in Higher Education 2011, 1–14, iFirst Article

Edited 08.01.2012 to include JT Action Plan…