The Influence of Competences and Support on School Performance Feedback Use: A Path Analysis

Author(s):

Jan Vanhoof(presenting / submitting)Goedele Verhaeghe Peter Van Petegem

Conference:

ECER 2011

Network:

11. Educational Effectiveness and Quality Assurance

Format:

Paper

Session Information

11 SES 09 B, Educational Effectiveness Based on Competences and Learning Styles

Paper Session

Time:

2011-09-15

10:30-12:00

Room:

J 24/14,G, 20

Chair:

Andra Fernate

Contribution

In many educational contexts, information-rich environments are created in order to promote data use in schools for the purpose of self-evaluation and quality assurance. However, providing (feedback) information does not guarantee that schools will actually put it to use. One of the main stumbling blocks relates to the interpretation and diagnosis of the information. This study examines the relationship between the data literacy competences of data users, the support given in interpreting the information, the actual use of school performance feedback (SPF), and the potential school improvement effects.

A competence is the ability to take satisfactory action through the integration of knowledge, skills and attitudes. These three elements are operationalized below in the context of school feedback use. An attitude reveals how positively or negatively a person views a particular matter (Petty & Wegener, 1998). A negative attitude toward SPF is – according to Bosker, Branderhorst and Visscher (2007) – one of the main obstacles in the use of feedback information. The importance of knowledge and skills is evidenced by the impact of data literacy on the process of SPF use (Webber & Johnston, 2000). Data literacy encompasses the strategies, skills and knowledge needed to define information needs, and to locate, evaluate, synthesize, organize, present and/or communicate information as needed (Williams & Coles, 2007, p. 188). Data literacy is a condition for being able to convert data into valuable and usable information (Earl & Fullan, 2003).

This study is aimed at testing insights emerging from the current knowledge base against empirical information. Answers are sought to the following research questions: (1) How do schools use SPF (in terms of processes and results)? What are the effects of this use? (2) To what extent are variations in SPF use explained by data literacy competences? (3) To what extent does specific SPF support has an impact on SPF competences, use and effects?

Support is essential because school principals are unsure of their ability to interpret information relating to their school. An explanation of the support referred to in this study is given in the description of the research methodology. Kirkpatrick’s (1998) four levels of evaluation have been used to create an inventory of the possible effects of this support and to integrate it into the broader theoretical framework.

Method

A between-groups field experiment with post-test was used. The experiment was set up in the context of a project, whereby Flemish primary schools receive feedback based on the comparison of school results with a reference group. 23 - chosen at random - of the 188 schools involved in the project participated in a professional development activity (a half-day workshop). Five months after receiving the school feedback – and after the experimental group had participated in the support – the school principals of the SIBO schools were asked to fill in a written questionnaire. A total of 116 schools completed the questionnaire, which equates to a general response of 62%. The response for the control condition amounted to 60% (n = 99) and the amount for the experimental condition amounted to 74% (n = 17). Path analyses were used to analyse whether theory-based relationship expectations corresponded with the empirical findings.

Expected Outcomes

The results revealed that a minority of schools invested significantly in the interpretation and diagnosis of the school performance feedback (SPF). At a general level, the respondents reported a rather low level of perceived impact of SPF. The paper demonstrates that differences in SPF use correspond with differences in SPF competences. With regard to the attitude toward SPF, we found that the effect does not only run over the process of SPF use and, therefore, that a direct link exists with the results and perceived effects of SPF use. Another relevant finding is that the process of SPF use related more closely to the perception of one's own knowledge and skills (or academic self-efficacy) than to the knowledge and skills measured using an objective test. We learn from this that faith in one’s own knowledge and skills is very important in making the transition to action (Bandura, 1977). Experimental schools score higher on the test score and estimate their own knowledge and skills to be higher. The description of these results leads to concrete suggestions and tangible ways of setting up support activities or support routes, aimed at improving data literacy in schools in different international contexts.

References

Bandura, A. 1977. Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review 84, no. 2: 191-215. Bosker, R.J., E.M. Branderhorst, and A.J. Visscher. 2007. Improving the utilisation of management information systems in secondary schools. School Effectiveness and School Improvement 18, no. 4: 451-67. Earl, L., and M. Fullan. 2003. Using data in leadership for learning. Cambridge Journal of Education 33, no. 3: 383-394. Kirkpatrick, D.L. 1998. Evaluating training programs: The four levels. San Francisco: Berrett-Koehler. Petty, R.E., and D.T. Wegener. 1998. Attitude change: Multiple roles for persuasion variables. In The handbook of social psychology, ed. D. Gilbert, S. Fiske and G. Lindzey, 323-90. New York: McGraw-Hill. Webber, S. and B. Johnston. 2000. Conceptions of information literacy: New perspectives and implications. Journal of Information Science 26, no. 6: 381-97. Williams, D. and L. Coles. 2007. Teachers’ approaches to finding and using research evidence: An information literacy perspective. Educational Research 49, no. 2: 185-206.

Author Information

Jan Vanhoof (presenting / submitting)

University of Antwerp

IOIW

Antwerpen

Goedele Verhaeghe

Ghent University, Belgium

Peter Van Petegem

University of Antwerp, Belgium

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.