Evaluations
“On behalf of the whole PEER team we would like to thank you for the amazing job you did as our external evaluator. All aspects of the project have benefited from your insightful and constructive feedback. Your efforts on the International Symposium deserves special praise – the event’s success owes a great deal to your vision, experience and leadership.” - Mitch Parsell, Macquarie University
Mick and Ruth have considerable experience acting as an evaluator / critical friend to teaching and learning project teams.
It is a common experience that at the beginning of a change process or a project that team members have different views on what is being attempted to achieve and the best way of doing so. Discussion of how the team will know if they have achieved their goals is often left til late on and is often implicit rather than explicit.
The framework we like to explore with project teams is a ‘theory of change’ (ToC) approach (Hart et al., 2009), which may be used to explain how and why a project realizes the results it achieves. It attempts to develop an understanding of the relationships between outcomes and the activities and contextual factors which may influence the outcomes. One of the attractions of the ToC approach is that it may be used to extend our understanding of a project, rather than audit it. Hence the key question might be, for example: “What have we learned about …. ?”. It is essentially a narrative approach, which tells the story of the project.
The process of developing the framework also encourages a conversation between the team and hence promotes a greater shared understanding of what the project is trying to achieve and how you will know if it has done so.
Components of Theory of Change
Current situation:
Enabling Factors / Resources:
Processes / Activities:
Desired Outcomes:
Longer-term impact:
Mick has used this framework for our NTFS project on ‘Rethinking Final Year Dissertations and Capstone Projects’. And went on touse it with two other NTFS projects at Chester and Lincoln; a teaching and learning project at Southampton, three ALTC project teams at Macquarie, Griffith and ANU, and two OLT Senior Fellow programmes at Victoria and Queensland. It has become the standard evaluation tool used at the Change Institutes of the International Students as Partners Institute, based at McMaster.
Hart, D., Diercks-O’Brien, A. G. and Powell, A. (2009) Exploring stakeholder engagement in impact evaluation planning in educational development work, Evaluation, 15: 285-306.
Recent evaluations
International Evaluator on ALTC project on ‘Communicative and Interpersonal Leadership in the Context of Peer Review (PELT)’ (2010-12), Principal Investigator: Judyth Sachs, Macquarie University, Sydney
International Advisor on ALTC project on ‘Teaching Standards Framework’ (2010) Principal Investigator: Judyth Sachs, Macquarie University, Sydney
International Evaluator on ALTC project on ‘Capstone Courses in Undergraduate Business Degrees: Better Course Design, Better Learning Activities, Better Assessment’ (2010-12) Principal Investigator: Liz van Acker, Griffith University, Brisbane
External evaluator on NTFS Project ‘Student as Producer: Research Engaged Teaching and Learning – An Institutional Strategy’ (2010-13) Principal Investigator: Mike Neary, University of Lincolnshire
External evaluator on NTFS Project ‘Personal Learning Environments in Active Field Science Education (PLEASE)’ (2010-13) Principal Investigator: Derek France, University of Chester
Evaluator of ‘Welsh Universities Learning and Teaching Strategies’ on behalf of HE Academy Wales (2011-12)
Evaluator/Advisor to ‘Students as co-producers of the curriculum: the case of an interdisciplinary Global Health course’ (2011-12) Principal Investigator: Julie Wintrup, University of Southampton
International Evaluator on ALTC project on Teaching Research – Evaluation and Assessment Strategies for Undergraduate Research Experiences (TREASURE) (2011-13). Principal Investigator: Anna Wilson, Australian National University / Stirling University
International Advisor and Evaluator to OLT National Senior Teaching Fellowship programme ‘Capstone curriculum across the disciplines’ (2013-14) Nicolette Lee, Victoria University, Australia
International advisor and evaluator McMaster Institute for Innovation and Excellence in Teaching and Learning. Director Arshad Ahmad, McMaster University (2014-18)
Advisor and evaluator to Centre for Advancement of Teaching and Learning, University College London, ‘Connected Curriculum’ initiative. Director: Dilly Fung (2014-17)
International advisor and evaluator University of Queensland, ‘Teaching@UQ’ project. Chair: Julie Duck (2015-16)
Advisor and evaluator Exeter Law School ‘The Impact of Revising the Undergraduate Law Curriculum’, Greta Bosch (2015-16)
International Advisor and Evaluator to OLT National Teaching Fellowship programme ‘Students as partners’ (2015-17) Kelly Matthews, University of Queensland
International Advisor and Evaluator to OLT National Teaching Fellowship programme ‘Engaging students as partners in global learning’ (2016-17) Wendy Green, University of Tasmania
External consultant to Anglia Ruskin University for Active Curriculum Working Group on Research-Informed Learning & Teaching (2017-18) Adam Longcroft
External consultant to Centre for Education and Teaching Innovation, University of Westminster (2021- ) Andy Pitchford