The paper focuses on the development of a basic eligibility test for admission into tertiary education in Sweden, a test aimed to provide opportunities for a wider group of applicants than today, thereby increasing inclusion and diversity in higher education. This is an emerging type of test, reflecting increasing national and international needs for documented competences required for access to higher education (e.g., the HiSET exam; ETS 2021).

To be allowed to apply for tertiary education in Sweden, basic entry requirements must be met. This corresponds to a leaving certificate from upper secondary level, comprising a combination of core subjects, in particular Swedish/Swedish as a second language, English and mathematics, but also a number of generic/key competences embedded in the national curricula, as well as in corresponding documents, e.g., in the European context (European Commission, 2018). Examples of such generic competences are problem solving, critical thinking and inferencing.

Traditionally, individual students who lack the formal requirements for tertiary education may have their competences evaluated by the university to which they apply. However, this is normally done in relation to a single course and is not generalizable, neither to other courses nor to other universities. These local evaluations are based on a central ordinance (UHRFS 2021:4) that defines the basic components required but not the extent or methods of the validation procedures. Hence, certain variability is self-evident.

To create a standardized alternative to local validation, a suggestion for a test of basic eligibility was made in a governmental investigation concerning entrance into higher education (SOU 2017:20). This resulted in a political decision to conduct a three-year trial round for a basic eligibility test (SFS 2018:1510). It was made clear that the test was intended for people from 24 years of age lacking formal, basic qualifications. In addition, it was decided that the result of the test should not be used for competition but for eligibility purposes only.

In 2020, the Swedish Council for Higher Education (UHR) formed a group of experts within different educational fields to discuss the development of a basic eligibility test. Three members of this group were given the task to develop a tentative framework for the activity. This work was conducted in close collaboration with the larger group, which lead to gradual revisions of the text. The final document was officially approved in January 2021 (UHR, dnr. 00012-2020). It consisted of some 40 pages and comprised subsections focusing on aims and background, including references to different international studies, e.g., PISA (OECD, 2018), test components, quality measures and control, as well as guidelines for use and future changes and developments.

The assignment to conduct the trial was given by the National Council/UHR to [university], which has a long tradition of large-scale educational assessment, regarding test development as well as analyses of results at national and international levels (Author 2 et al.; Author 2 et al.; Author 1 et al., Author 3 et al.) A three-year contract was signed in March 2021, after which the operative developmental phase started. The first, large-scale trial test was administered in October 2022, comprising a wide range of tasks: dichotomous, selected response items within different domains, English listening comprehension, integrated tasks also including graphs, tables etc., and two tasks targeting written production in Swedish and English.

The aim of the current presentation is to briefly

• describe and discuss the rationale and methodology of the test development process,
• present and reflect on some results of the process, and to
• look forward into, and discuss, possible future uses of the test being developed.

Initially, a decision was made to form a project steering group of four people, the intention being twofold: to broaden competences and to decrease vulnerability. It was also essential to establish the basic character of the envisaged product, namely a unified test, where only one standard was to be set, namely the pass level. In addition, the cut-off point was to be determined based on a holistic assessment, not on partial requirements for individual components of the test. It was also emphasized that the generic competences were seen as overarching concepts, however with working groups linked to different competence domains. These groups were directed towards Swedish L1/ L2, English, mathematics, natural and social sciences, and methodology. Closely linked to this structure were also so-called go-betweens (gb:s), i.e., people with competence in two or more of the domains. There were basic requirements for each working group, namely three types of, often overlapping, competences and experiences: subject matter knowledge including pedagogical/teaching experience, experience of large-scale testing, and research competence. Furthermore, a specification matrix was developed for the different groups, on the basis of which they documented gradually information for the material developed, e.g., intended construct/competence, amount and type of input, format, estimated time, estimated/intended difficulty, etc. The working groups followed internal plans when developing items and tasks. The members of the steering group, several of them with a domain-related background, kept in regular touch. Roughly once a month there were meetings for the whole group, approx. 25 people, in which different issues were discussed. Piloting of material, item-based as well as productive writing tasks for Swedish and English, was conducted in a four-step, anonymous process: (1) mini trials per domain in smaller groups; (2) trials including material from two domains; (3) trials of a mix of tasks from different domains; and (4) pre-testing rounds aiming for a large number of participants, representing as wide a range as possible of individuals. In the latter case, tasks were always accompanied by a set of background questions and anchor items. In addition, extensive collection of test-taker feedback was made, using Likert-scales and open comments. Standard setting is a self-evident aspect of test development, also in this trial project. This is always a challenge, not least for tests comprising both dichotomous item data and ratings of productive tasks. A short account of this will be given, including reactions to the first large-scale trial round.

The test development process focused upon in the current proposal is indeed work in progress, with continuous documentation undertaken. Hence, the conclusions drawn at this stage are tentative, and should partly be regarded as observations, which, however, will be combined into final conclusions at the end of the trial. Furthermore, they emanate from distinctly different data. One such observation is related to pre-testing and clearly shows the difficulty of finding the intended, large groups of test takers. Another aspect that needs constant attention is the nature of the test as unified, not consisting of independent parts. This is even more emphasized by the fact that two of the subtests, in particular, have a general character, including and integrating different competences in a way that makes them closely related to the overarching, generic purposes. Also, the rating of productive tasks is a multi-facetted activity that needs lots of consideration and re-consideration, especially when combined with other types of data and analyses. In addition, it can be concluded that anchor items are crucial for standard-setting purposes in making comparisons at different levels possible. Finally, it is obvious that test-taker feedback adds considerably to the quality of the process, as such, by giving the necessary perspective of the users, but also as a complement to analyses of performance. We sometimes characterize the assignment to carry out the trial of this test as a task entailing building a boat while sailing it. This is obviously far from uncomplicated but still works quite well, much thanks to the distinct and positive collaboration that takes place in the process, between different actors: the national council and the university department, disciplines and groups within the project, and with potential users of what we hope will be a future national entrance test for tertiary education.

Author 2 et al. (year 1) Author 1 et al. (year 3) Author 2 et al. (year 2) Author 3 et al. (year 4) Educational Testing Service (2021). 2020 Annual Statistical Report. https://hiset.org/s/pdf/HiSET_2021_Annual_Statistical_Report.pdf European Commission (2018). Council Recommendation of 22 May 2018 on key competences for lifelong learning. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.C_.2018.189.01.0001.01.ENG&toc=OJ:C:2018:189 OECD (2018). PISA 2018 Assessment and Analytical Framework. https://doi.org/10.1787/b25efab8-en Regeringskansliet (2017). SOU 2017:20. Tillträde för nybörjare – ett öppnare och enklare system för tillträde till högskoleutbildning [Admission for beginners – a more open and accessible system for entrance into tertiary education]. https://www.regeringen.se/rattsliga-dokument/statens-offentliga-utredningar/2017/03/sou-201720/ Sveriges Riksdag (2018). Förordning om försöksverksamhet med behörighetsprov för tillträde till högskoleutbildning [Ordinance on trial activities with eligibility tests for admission to higher education]. (SFS 2018:1510). https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/forordning-20181510-om-forsoksverksamhet-med_sfs-2018-1510 Universitets- och högskolerådets författningssamling (UHRFS 2021:4). Föreskrifter om ändring i Universitets- och högskolerådets föreskrifter (UHRFS 2013:1) om grundläggande behörighet och urval [Regulations regarding change in UHR’s regulations on basic eligibility and selection] . https://www.uhr.se/globalassets/_uhr.se/publikationer/lagar-och-regler2/uhrfs/2021/uhrfs-2021-4.pdf Universitets- och Högskolerådet (2021). Ramverk för det nationella behörighetsprovet [Framework for the national eligibility test]. (UHR, dnr. 00012-2020)