The paper attempts to provide an impact evaluation of the Bologna Process implementation at Master’s level in Romania, focusing on graduates’ employability. The paper is part of an ongoing research for a PhD thesis, and for the limited purpose of this proposal, the main objectives refer to identifying the relevant outputs from the implementation of the Bologna Process and to defining the indicators used to measure its outcomes, shaping an answer to whether the Bologna Process did, in fact, live up to its expectations.

Thus far, most analyses on the Bologna Process focused on the convergence aspect, relying mostly on the description of the Bologna Process and underlining its benefits (Wihlborg & Teelken, 2014) and less on the impact of implementation – not only at the national level, but at the level of higher education institutions (HEIs). Also, the existing research tends to focus mostly on the higher education system as a whole, with the Master’s cycle being particularly underrepresented.

Starting from the main objectives defining the Bologna Process, there has been a macro-focus on the convergence and alignment of the structural features – the implementation of the three-cycle system, the quality assurance and the qualifications’ frameworks, as well as the recognition of comparable academic degrees (European Commission, 2013). At the national level, the Bologna Process legitimized top-down reform measures, with HEIs adapting to changes at their own pace, leading to varied adoption rhythms (Karran, 2004) and depth of implementation, while aiming at the same time for standardization, in order to ensure ‘measurability’ (Landri, 2016). One particular concern refers to the discrepancy between surface (bureaucratic and structural) and more profound reforms and convergence, as it appears to be more of a technical exercise, with little feedback from some of the countries involved (Gleeson, 2013). This stance was also reflected by the European Student Union: ‘the Bologna Process is in grave danger of being revealed as a superficial redesign of higher education structures in Europe rather than a transformation of the whole academic and learning paradigm’ (European Student Union, 2009). Among the objectives and priorities of the Bologna Process, the one regarding employability has particular long-term implications regarding both the educational offer and professional mobility – also responding to the `third mission of higher education institutions`, their relevance and social impact (Gibbons et al. 1994; Benneworth and Jongbloed 2010; Nowotny, Scott, and Gibbons 2002).

The Leuven Declaration stated for the first time that ‘higher education should equip students with the advanced knowledge, skills and competences they need throughout their professional lives` (Leuven Declaration, 2009). An analysis of the European Commission recommended the inclusion of employability in the European Framework for Quality Assurance, once again stressing its importance and relevance as a measurable outcome of higher education (European Commission, 2009). At the 2012 Ministerial Conference in Bucharest, employability was among the three priorities identified, a fact later reaffirmed at the 2015 Ministerial Conference in Yerevan. However, studies in the field reveal that the Bologna follow-up reports provide an unrealistic image of higher education institutions (Neave & Amaral, 2008; Huisman, 2009; Teelken & Wihlborg, 2010, 2014). Thus, in preparation for the 20^th anniversary of the 1999 Bologna Declaration, it is perhaps the time to reassess the definition of success and focus more on the measurable outcomes of the Bologna Process implementation.

The methodological approach is built around a theory-based impact evaluation, engaging a mixed methodology relying on secondary data analysis and a systematic review of public and institutional policy documents. The research uses the approach of the Independent Evaluation Group of the World Bank and an adapted model of the World Bank logical framework (White, 2009). The sources of secondary data are data sets provided by existing national surveys and studies in the field, statistical data at national level (particularly from the National Institute of Statistics, the National Council for Higher Education Funding, Executive Unit for Financing Higher Education, Research, Development and Innovation, the Integrated Educational Registry, the Single Enrollment Registry and the `Students, graduates and the Labour Market` Platform) and previous projects reflecting the pre- and post-implementation stages of the Bologna Process in Romania. The research covers the timeframe between 1990 and 2015, 56 public HEIs (State-funded) and 45 private institutions in Romania, with a total of 583 faculties (National Institute of Statistics, 2015). The main indicators included in the analysis refer to: • Outputs: total student population and their graduation rates at high-school and Bachelor’s levels (including specific fields of study), percentage of students entering a Master’s degree and graduating (in a similar/different field of study as the Bachelor degree). • Outcomes: employment rates of Bachelor and Master graduates (before and after graduation), number of graduates employed in their field of study (Bachelor and/or Master), average age for first employment. The research also includes other indicators, which are useful for the impact evaluation: the internal efficiency of the higher education system and the public financing of the Bachelor and Master’s programs, as well as economic (e.g. GDP/year, minimum wage/year) and demographic data. The theory-based approach will then contribute to a better understanding of what and how different inputs generated by the Bologna Process influenced the evaluated impact.

The implementation of the Bologna Process appears to have led to structural challenges at national and institutional levels, in the context of a decreasing student population and increasing competition among HEIs. The three-cycle system does does not appear to have created a faster-track to the labor market for Bachelor graduates, as the number of Master students has increased. At the same time, while students and graduates still perceive the Master’s degree as a competitive advantage on the labor market (while the majority of Master’s students are already in full-time employment), the employers place no significant value on the Master’s certification, but rather on the graduates’ practical experience. Also, there is a certain degree of overlapping between the Bachelor and Master’s educational offer, while the latter is still highly theoretical and lacks specialization, leading to an imbalance between the dominantly academic and research-oriented structure of the Master’s program and its professional relevance. One main issue still to be addressed both at policy level and in practice is the mismatched correlation between the learning outcomes and the National Qualifications Framework, as well as between the institutional educational offer and the European Higher Education Area Qualification Framework and the European Competency Framework. Another issue refers to the national external evaluation methodology, which only includes two performance indicators regarding employability – one in relation to learning outcomes and the qualifications framework, and the other to graduate tracking. For the latter, there is seldom accurate reporting, as there is no standardized mechanism at the national level and most HEIs do not have any procedures in place for gathering data. Even though the merits of the Bologna Process cannot be denied, it is high time to acknowledge its shortcomings and better understand how to mitigate the unexpected outcomes.

1.Benneworth, P., and B. Jongbloed. (2010). `Who Matters to Universities? A Stakeholder Perspective on Humanities, Arts and Social Sciences Valorisation`. Higher Education 59 (5): 567–588. doi:10.1007/s10734-009-9265-2. 2.Gibbons, M., C. Limoges, H. Nowotny, S. Schwartzman, P. Scott, and M. Trow. (1994). The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies. London: Sage. 3.Gleeson, J. (2013). The European Credit Transfer System and curriculum design: Product before process? Studies in Higher Education, 38, 921–938. 10.1080/03075079.2011.610101 4.Karran, T. (2004). Achieving Bologna convergence: Is ECTS failing to make the grade? Higher Education in Europe, 29, 411–421. doi:10.1080/0379772042000331688 5.Huisman, J. (2009). The Bologna Process towards 2020: Institutional diversification or convergence? The European Higher Education Area: Perspectives on a moving target. 6.Iucu, R. (2010). Ten Years after Bologna: Towards a European Teacher Education Area. In O. Gassner, L. Kerger & M. Schratz (Ed.), Entep - The First Ten Years after Bologna, (pp. 53-95). Bucharest, RO: Editura Universităţii din Bucureşti. 7.Landri, P. (2016). Standards and standardisation in European politics of education. In R. Normand & J.-L. Derouet (Eds.), A European politics of education. Perspectives from sociology, policy studies and poltics (pp. 13–30). Abingdon: Routledge. 8.Neave, G. and Amaral, A. 2008. ‘On process, progress, success and methodology or the unfolding of the Bologna Process as it appears to two reasonably benign observers’. Higher Education Quarterly, 62(1–2): 40–62. 9.Nowotny, H., P. Scott, and M. Gibbons. 2002. Re-thinking Science: Knowledge and the Public in an Age of Uncertainty. Cambridge: Polity Press. 10.Teelken, C. & Wihlborg, M. (2010) Reflecting on the Bologna Outcome Space: some pitfalls to avoid? Exploring universities in Sweden and the Netherlands, European Educational Research Journal, 9(1), 105-115. http://dx.doi.org/10.2304/eerj.2010.9.1.105. 11.White, H. (2009). Impact Evaluation – the Experience of the Independent Evaluation Group of the World Bank. The World Bank. 12.Wihlborg, M., Teelken, C. (2014), Striving for Uniformity, Hoping for Innovation and Diversification: A Critical Review concerning the Bologna Process – Providing an Overview and Reflecting on the Criticism. Policy Futures in Education, Volume 12 Number 8. 13.* European Commission. (2009). Communiqué of the Conference of European Ministers Responsible for Higher Education, Leuvenand Louvain-la-Neuve, 28-29 April 2009 (available at http://www.ehea.info/Uploads/Declarations/Leuven_Louvain-la Neuve_Communiqu%C3%A9_April_2009.pdf).