A Comprehensive Framework for Evaluating a Scholarship of Teaching and Learning Initiative in Higher Education: Successes and Challenges
Author(s):
Cheryl Amundsen (presenting / submitting) Gregory Hum (presenting)
Conference:
ECER 2012
Format:
Paper

Session Information

22 SES 04 C, Academic Work and Professional Development

Parallel Paper Session

Time:
2012-09-19
09:00-10:30
Room:
FFL - Aula 27
Chair:
Elinor Edvardsson Stiwne

Contribution

Boyer (1990) conceptualized academic work as having “four separate, yet overlapping, functions” (p. 16): the scholarships of discovery, integration, application and teaching. The first three functions are consistent with the commonly held idea of doing research in one's field (discovery), understanding it in a larger intellectual context (integration) and applying it to problems that matter (application). Boyer argued that the scholarship of teaching should not be separate from these three functions. In other words, scholarly teaching should be founded on an understanding of knowledge development in one’s discipline, knowledge of broader pedagogical principles and practices and the use of systematic inquiry into student learning. Boyer’s notion of the scholarship of teaching (SoTL) has been the catalyst for many institutions, in many countries, to develop initiatives to foster SoTL amongst academic staff (Chalmers, 2011; Hutching et al, 2002). So too has the literature relevant to SoTL grown in size and international authorship with contributions providing further definition (e.g., Kreber, 2002) and models for practice (e.g., Connolly et al, 2007; Healey, 2002; Trigwell et al, 2000).

Policy efforts to promote teaching and learning tend to work at the level of the individual or the institution, with little connection between levels, thus inhibiting change, as Trowler, Fanghanel, and Wareham (2005) have noted in the UK context. Reflective of this situation, evaluations of initiatives to foster and support SoTL tend to focus either on the individual academic (Chalmers, 2011; Kember 2002) or on the departmental/institutional level (Gray et al, 2007; Waterman et al, 2010). What is lacking is a comprehensive assessment across multiple levels of an institution, making it possible to better understand impact more holistically and conclusively. There is also a lack of consistency in points of assessment used in different studies, making it difficult to compare studies and to derive clear ‘next steps’ in the promotion of SoTL.   

This paper is grounded in the implementation of a multi-level assessment framework to evaluate the impact of a program designed to support academic staff in systematically investigating questions about teaching and learning, and to promote conversations and collaborations about teaching across the university, as supported by small ($3K or less) and larger ($10K or less) grants. The program, at Simon Fraser University a mid-sized university located in Vancouver, Canada, is one aspect of a broader institutional process to rethink how best to support and enhance teaching and learning.

In this program, academic staff identify questions they wish to investigate. Questions are focused on student learning, and findings are used to revise or redesign teaching practice. Findings are shared with the purpose of informing the teaching practice of others, and promoting further SoTL work. Experienced educational researchers work with staff to develop project proposals, addressing the critique that SoTL often results in educational research of poor quality, as staff trained in research methodologies suited to their disciplines often are less knowledgeable about, and less experienced with, methods suited to researching teaching and learning (Colet, McAlpine, Fanghanel, & Weston (in press); Gray et al, 2007). 

Method

Our assessment framework focuses on both outcome and process across three levels: individual (academic and student), program/department and institutional (Norton, 2009). To integrate these levels of assessment into a single framework, we chose to follow a cycle of inquiry (Maki, 2004), that deepens as we come to better understand the scope of what we are assessing, providing the opportunity for depth and emergent findings. Individual: We are documenting the thinking of academic staff over time, focusing on their emergent conceptions of teaching as systematic inquiry, and collecting evidence of their application of project findings to teaching practice (analysis of project proposals, final project reports, field notes of interim meetings, and interviews). Student: We are documenting student learning through the analysis of final project reports that include a variety of measures of student learning (grades, classroom observations, questionnaires, focus groups, interviews). Department/Faculty: Local academic contexts can facilitate or hinder research efforts. We are documenting intra- and inter-departmental collaborations on projects, recognition of this work by promotion committees, and dissemination of project findings. Institution: We are documenting, the number of projects completed, the number of departments represented by these projects, and the number of project collaborations across academic units.

Expected Outcomes

Preliminary findings across completed (n=15) and ongoing (n=19) projects include: Individual: Interviews (n=7) indicate some staff have moved from regarding projects as unfamiliar ‘educational’ research, to regarding such inquiry as personal practice and an important aspect of teaching. There is also some evidence of the emergence of a more sophisticated conceptualization of different types of desired student learning. Students: Evidence of improved student learning is clear, with various student learning outcomes across projects reflecting deeper, and more engaged learning. Department/Faculty: There is strong evidence for increasing collaboration. Many projects involve multiple faculty members from within the same academic unit, and nearly all projects involve graduate student research assistants. Two completed projects have been given ongoing funding by the department to apply findings more broadly. Institution: Institutional impact is strong. Attendance at semesterly progress meetings has grown, as have applications for grants. On a policy level, investment in the grants program by central administration has increased substantially, and there is recognition that research conducted in grants must be considered for career progression decisions. Overall: Evidence supports the notion that faculty led initiatives to promote SoTL can impact multiple levels, promoting useful teaching research, and raising its prominence and status.

References

Boyer, E. (1990). Scholarship Reconsidered: Priorities of the Professoriate. Princeton, NJ: Carnegie Foundation for the Advancement of Teaching. Chalmers, D. (2011). Progress and challenges to the recognition and reward of the scholarship of teaching in higher education. Higher Education Research & Development, 30(1), 25-38. Connolly, M., Bouwma-Gearhart, J., & Clifford, M., (2007). The birth of a notion: The windfalls and pitfalls of tailoring a SoTL-like concept to scientists, mathematicians, and engineers. Innovative Higher Education, 32(1), 19-34. Gray, K., Chang, R., & Radloff, A. (2007). Enhancing the scholarship of teaching and learning: Evaluation of a scheme to improve teaching and learning through action research. International Journal of Teaching and Learning in Higher Education, 19(1), 21-32. Kember, D. (2002). Long-term outcomes of educational action research projects. Educational Action Research, 10(1), 83-104. Kreber, C. (2002). Controversy and consensus on the scholarship of teaching. Studies in Higher Education, 27(2), 151-67. Maki, P. (2010). Assessing for Learning: Building a Sustainable Commitment across the University. Sterling, Virginia: Stylus. Norton, L. (2009). Action Research in Teaching and Learning. A Practical Guide to Conducting Pedagogical Research in Universities. Abingdon, UK: Routledge. Rege Colet, N., McAlpine, L., Fanghanel, J., & Weston, C. (in press). La recherche liée à l’enseignement au supérieur et la formalisation des pratiques enseignantes : le concept du de SoTL. N° 65 revue Recherche et Formation. Trowler, P., Fanghanel, J., & Wareham, T. (2005). Freeing the chi of change: the Higher Education Academy and enhancing teaching and learning in higher education. Studies in Higher Education, 30(4), 427-444. Waterman, M., Weber, J., Pracht, C., Conway, K., Kunz, D., Evans, B. (2010). Preparing scholars of teaching and learning using a model of collaborative peer consulting and action research. International Journal of Teaching and Learning in Higher Education, 22(2), 140-151.

Author Information

Cheryl Amundsen (presenting / submitting)
Simon Fraser University
Faculty of Education
Burnaby
Gregory Hum (presenting)
Simon Fraser University
Educational Technology and Learning Design
Burnaby

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.