Does quality make a difference for higher education graduates in Colombia?
Author(s):
Conference:
ECER 2016
Format:
Paper

Session Information

11 SES 06 B, Paper Session

Paper Session

Time:
2016-08-24
15:30-17:00
Room:
OB-E0.32
Chair:
Irina Maslo

Contribution

In this paper, we explore the difference in quality between accredited and non-accredited higher education institutions (HEIs) in Colombia. For that purpose, we test if the difference in the national exam of student performance (Saber Pro) between accredited and non-accredited institutions is statistically significant by employing a propensity matching score approach based on common financial characteristics in order to avoid issues of selection bias. Our results indicate that the difference in performance between accredited and non-accredited institutions is positive and statistically significant. This performance difference can be attributed to the substantial differences in the patterns of expenditure and asset investment between accredited and non-accredited HEIs.  

Although national quality systems for ensuring the quality of higher education are quite heterogeneous, there seems to be consensus on what the common determinants of a successful quality system are. Williams et al. (2013), in their multi-country survey of higher education quality systems, argue that the most common determinants are resources, environment, connectivity, and output. Resources refers to the extent to which teaching and research is either financed by the government or by the private sector, adjusted according to the type of HEI (public or private) that predominates in a given country. Environment refers to the role of government regulation in the higher education sector, the conditions of academic employment, and the extent of the mix in the supply of higher education between public and private HEIs. Connectivity refers to the extent of internationalization of the student body and in the number of research articles or activities conducted with other HEIs at both the national and international level. Finally, output refers to the impact of research, teaching, and training. In the case of research, output is measured by the impact factor of all HEIs in a given country, and similarly, teaching and training are measured by the numbers of graduates and their reported employability at the national level.

In Latin America during the 1990s and 2000s, the process of globalization led to an exponential growth in the offer of higher education services. This growth was led mainly by private-sector universities with different levels of quality. As a common denominator in the region, much of the growth was achieved by offering programs of dubious quality. Therefore, in order to control for quality, many of the national governments created institutions for accrediting the quality of the programs offered at the national level (Lamarra, 2003). However, it is important to note that although there were problems with quality at the low end of the spectrum, at the high end of the spectrum, many private-sector universities in Latin America have been responsible for expanding participation rates among students. Many authors agree that a national system benefits from having both private and public institutions, and that the most effective form of government regulation is the kind that sets the rules in terms of quality and effectively evaluates performance, but allows for some degree of autonomy in the implementation of quality systems by the institutions that conform to the national system (Altbach and Salmi, 2011; Jamil, 2007; Martin et al., 2011; Patrick and Stanley, 1998).

Method

For this study, we merged three different datasets. The first dataset is the Colombian Grand Report of HEIs 2013 (the first of its kind available to the public), which was launched by a local economics magazine called La Nota. The report gives financial information for the fiscal year of 2013 for 95 private and 50 public HEIs. The report gives detailed information about the revenue, operating expenses, earnings before interest expenses, net income, assets, equity, liabilities, number of students, and the number of academic personnel and their type of contract (full time, part time, or adjunct lecturer). Additionally, the report gives other operational financial indicators such as teacher expense per student, assets per student, operating expenses per student, and tuition payments per student (Nota, 2015). The second dataset consists of the results from the national exam of student performance (Saber Pro) for the years 2012 and 2013. This dataset contains information on the Saber Pro results for all 205 public and private HEIs in Colombia that grant undergraduate degrees. The dataset contains categorized information about the average results per program and the number of students that took the test in a particular year. Our procedure for testing differences in values is based on the Average Treatment Effect on the Treated (ATET) framework. This procedure makes a selection of counterfactual values based on propensity score matching. This procedure has certain advantages over traditional sampling or predicted values difference testing since it effectively addresses the problem of selection bias of the comparable sample groups drawn from the non-accredited HEIs. One key advantage of this method is that we can compare the actual values of our descriptive data without forgoing the richness contained in the observable characteristics of a regression model. Moreover, with ATET it is possible to determine which non-accredited institutions are more closely related in terms of common financial characteristics to those that are accredited, which can have important implications regarding policy making.

Expected Outcomes

By using a propensity-matching estimator approach, we tested for significant statistical differences between accredited and non-accredited HEIs in Colombia. The counterfactuals among non-accredited HEIs were selected randomly based on similar financial characteristics in order to avoid selection bias. Our results show that indeed there is a positive statistically significant difference in performance in the national student performance exam (Saber Pro) between students in accredited and non-accredited institutions. The difference can be attributed to other statistically significant differences in quality indicators such as the pattern of expenditure and HEI investment per student. There is conclusive evidence that students from Colombian HEIs that have obtained the institutional accreditation perform consistently better than students from non-accredited HEIs do. It is important to remember that institutional accreditation does not generate significant differences in the growth of revenue, which can also be considered as a warning system for public policy as to why quality is not generating a differential in gross revenue growth. In the case of Colombia, these findings can serve as the basis for a more in-depth discussion as to how public resources are being distributed to strengthen the quality assurance system. These resources should be geared toward those HEIs that can demonstrate a major social impact, transparency in the management of financial resources, and better results in graduate performance. This new policy should be focused on those HEIs that are still non- accredited in order to assure the quality of education imparted.

References

Abadie, A., et al. (2004). Implementing matching estimators for average treatment effects in Stata. Stata journal, 4, 290-311. Alexander, F. K. (2000). The Changing Face of Accountability. Journal of Higher Education, 71(4), 411-431. Altbach, P. G., and Salmi, J. (2011). The road to academic excellence: The making of world-class research universities: World Bank Publications. Breneman, D. W. (1993). Higher Education: On a Collision Course with New Realities. Canton, E., and Blom, A. (2010). Student support and academic performance: experiences at private universities in Mexico. Education Economics, 18(1), 49-65. Jamil, J. S. S. (2007). Autonomy from the State vs Responsiveness to Markets. Higher Education Policy, 20(3), 223-242. Lamarra, N. F. (2003). Higher Education, Quality Evaluation and Accreditation in Latin America and MERCOSUR. European Journal of Education, 38(3), 253. Patrick, W. J., and Stanley, E. C. (1998). Teaching and research quality indicators and the shaping of higher education. Research in Higher Education, 39(1), 19-41. Pedrosa, R. H. L., et al. (2013). Assessing Higher Education Learning Outcomes in Brazil. Higher Education Management and Policy, 24(2), 55-71.

Author Information

Edgardo Cayon (presenting / submitting)
CESA Business School
Bogota
CESA Business School
University Secretary
Bogota
Pontificia Universidad Javeriana

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.