Indicators and explanations of student achievement in higher education Comparison of data from two countries

Author(s):

Gabriella Pusztai(presenting / submitting)Klara Kovacs

Conference:

ECER 2014

Network:

22. Research in Higher Education

Format:

Paper

Session Information

22 SES 06 A, Teaching, Learning and Assessment in Higher Education

Paper Session

Time:

2014-09-03

15:30-17:00

Room:

B020 Anfiteatro

Chair:

Dina Soeiro

Contribution

In HE research, which was launched in the 1960s, the first outcome comparisons among institutions sought for the criteria of the excellence of universities and colleges. A remarkable - and still influential - contribution to the project was the creation of the expanded notion of student outcome (Astin 1993), which threw new light on the achievements of both students and institutions. Another essential step forward was a new concept of institutional outcome that took into account the makeup of the entering student population and their attributes at entrance. Realising the importance of student (self-)selection and the institutions' individual impact mechanisms, it was created the IEO (Input-Environment-Output) model of the impact of HE institutions, which gives a comprehensive model of students' characteristics at entrance, the impacts of the environment and student outcomes (Astin 1993). With this, he has made a remarkable contribution to the research of added value in HE (Klein et al. 2005, Banta–Pike 2007, Rodgers 2007). The concept and methodology of that research are becoming subtler all the time, but it is no agreement in indicators of student achievement. In our previous research we emphasized the necessity of a broader interpretation of student outcome in tertiary education, which is undergoing a transformation and adjusting itself to the school system. Factors to be considered are the changes in students’ knowledge, skills, value preferences and attitudes to study and work. We found the so called internal indicators of student outcome within the world of higher education. Indicators of success are successful entrance to various stages of higher education in accordance with one’s career plans, persistence, (fulfilled) aspiration to move one stage forward, exam result averages (taken individually or in comparison to others), advancement (presupposing input and output measurement), and even commitment to one’s studies and doing one’s work in compliance with academic norms (Pascarella-Terenzini 2005, Tinto 2006, Koucky et al 2010). According to our previous research it is not possible to measure the efficiency of higher educational institutions located in peripheral areas through the direct success of their graduates in the labour market. If the so-called „third mission of higher education” is to be taken seriously, more reliable efficiency indicators must be found. In the course of our research we came to the conclusion that it is now inevitable for higher education to develop and implement its own efficiency indicators that are independent of the transitory and temporary processes going on in the economic environment. The indicators should take into account long-term effects and added educational values, regardless of the specific subject major the student has graduated in. As part of our research program, we studied several models described in the related literature and drafted an apparently well-functioning model that meets the requirements set up preliminarily (Pusztai 2011, Pusztai et al. 2012). As we seek a more precise explanation for the students' achievement differences we examined the direct effect of institutional integration on academic career. The individual's academic career is decisively influenced by the student community living together in the various institutional units with their views on the purpose of studies, the norms to adhere to during one's studies, non-obligatory activities and the circle of reliable persons (Pusztai 2014).

Method

The main elements of our achievement indicator system are indirect proxies and predictors of further achievement: the commitment of students to do curricular and extra-curricular work, their devotion to improve their knowledge in formal and informal ways and their readiness to undertake work – work also for public good, and not exclusively personally profitable activity. We try to test whether the new achievement indicator is not biased against different student groups. We carried out a series of student surveys during the last decade in the peripheral area of the European Union, in a higher education region with international attraction from Hungary and Romania. For the empirical test of the recent questions we have used the database of HERD (N=2619). We used a large-scale coordinated survey, conducted in the spring of 2012, among the students of 9 HEIs. The subsamples were chosen on a random multistage basis, chosen groups of students in years 1, 2 or 3 in MA and BA programs or 1, 3 or 4 in undivided programs. Some smaller institutions were overrepresented, but we weighted the data at the end.

Expected Outcomes

We assume that due to the increasing diversity of the student composition the one-dimensional achievement indicators favor certain groups, while a complex achievement indicator is much less biased, if we examined the effects of components separately in each student group. After the comparison of each achievement component and complex indicator of it with institutional, demographical and social background variables, the multidimensional achievement concept seems to form an instrument, which would be useful for international comparison. The comnplex student achievement indicator is not biased at least along the nominal differences, although in terms of social (cultural, economic, municipal) background behave the same way as previously used outcome indicators, and differences between countries were experienced. The conceptualization of students’ achievement of different levels of the multi-cycle HE structure is a new challenge for conceptualizing student success. It is likely that both the same and different components would be elements of the academic achievement concept. It is assumed that influences of HE institutional factors on the achievement could be seen more obviously with the help of the complex student achievement indicator.

References

Astin, Alexander W. (1993): What Matters in College: Four Critical Years Revisited. San Francisco: Jossey-Bass Banta T. W. & Pike G. R. 2007. Revisiting the blind alley of value-added. Assessment Update 19. 1-15. Klein S.P. & Kuh G.D. & Chun M. & Hamilton L. & Shavelson R..J. 2005. An approach to measuring cognitive outcomes across higher-education institutions. Journal of Higher Education 46. 3. 251-276. Rodgers, T. 2007. Measuring Value Added in Higher Education: A Proposed Methodology for Developing a Performance Indicator Based on the Economic Value Added to Graduates. Education Economics 15. 1. 55–74. Koucky, J., Bartusek, A. & Kovarovic, J. (2010): Who gets a degree? Prague: Education Policy Centre Pascarella, E. T. & Terenzini, P. T. (2005): How College Affects Students. San Francisco: Jossey-Bass Pusztai G. (2014) The Effects of Institutional Social Capital on Students’ Success in Higher Education. International Journal of Educational Devepoment (forthcoming) Pusztai G., Baltatescu S., Kovács K. & Barta S. (2012): Institutional social capital and student well-being in higher education – A theoretical framework. In Gabriella Pusztai, Adrian Hatos (eds.): Higher Education for Regional Social Cohesion, Hungarian Educational Research Journal Special Issue 2 (1). 54–73. Tinto, V. (1993). Leaving college. Rethinking the causes and cures of student attrition. Chicago: The University of Chicago Press

Author Information

Gabriella Pusztai (presenting / submitting)

University of Debrecen

Center for Higher Education Research and Development

Debrecen

Klara Kovacs