Session Information
11 SES 07 A, Quality Assurance at Institutional, District or Country Level
Paper Session
Contribution
The current study identifies areas and provides recommendations for the improvement of the national learning assessment system, informs the development and implementation of enhancement policies as part of a wider education development strategy. The study contributes to the existing knowledge on the practices of large-scale assessments and provides proper and actionable recommendations for improvement. Considering the fact that the Armenia' Education Development State Program 2030 is now being drafted, the importance of such a study is crucial.
The primary purpose of the study is to support the improvement of Armenia’s national learning assessment system through comprehensive analysis and policy recommendations that are systemic and would support the implementation of fair and equitable assessment of all students. By aiming to contribute to the improvement of Armenia’s national learning assessment system the study will utilize qualitative analysis with a focus on three questions based on ANLAS three dimensions of a quality assessment systems.
Research Questions
- To what extent is the context of the national learning assessment system supportive of large-scale assessment programs?
- To what extent does the quality of large-scale assessment programs implemented in Armenia fit the key quality concepts for learning assessments?
- To what extent is the assessment system consistent with important aspects of the broader education system and other aspects within the assessment system?
Objectives
- To delineate the level of supportiveness of the context of the national learning assessment system towards large-scale assessment programs
- To investigate the consistence of the quality of large-scale assessment programs with the key quality concepts for learning assessments in Armenia
- To examine the correspondence of the assessment system with the significant features of the broader education system and other aspects within the assessment system
Method
The study was carry out according to ANLAS Manual. ANLAS provides a toolkit for countries to systematically gather and analyze information about their national learning assessment systems with the aim to inform the development and implementation of improvement strategies as part of the wider education sector planning process. The ANLAS approach includes the following key dimensions: Context of the assessment system, Quality of large-scale assessment and examination, Coherence of the assessment system, ANLAS is conceptualized as a country-led, participative process that is implemented by a national team, led by a team leader, and guided by a Steering Committee and follows a comprehensive two-stage approach. The two main methods for completing the qualitative analysis in ANLAS are document review and consultations with key stakeholders in the education system and in the assessment system. Document review is assuming identification of a list of documents and conducting a consequent document analysis against ANLAS quality criteria. For this purpose a document mapping exercise was conducted by the national team in order to identify the relevant documents to be reviewed as part of the analysis. For each of the ANLAS dimensions a list of legal and policy frameworks related to overall national assessment programs, project-related documents, including the legal package of the establishment of particular assessment programs and their implementation was created. A review of these documents was conducted in order to assess each key area against the defined quality objective using the following tools: Analytical table dimension 1: Context of the assessment system Analytical table dimension 2: Quality of large-scale assessment and examination Analytical table dimension 3: Coherence of the assessment system Primary data for the analysis was collected through stakeholder consultations with the purpose to discuss the initial description of each key area in order to evaluate those against the defined quality objective. For this purpose a stakeholder mapping exercise was conducted by the national team. The primary data for the study was collected through: Key Informant Interviews (KII); Expert Interviews (EI); Focus Group Discussions (FGD). All the data collected within the study will be analyzed in a holistic way to provide comprehensive understanding and at the same time measurable and specific results. Also, triangulation used in data collection will ensure that the conclusions are justified, measured and evidence-based. The dimension-specific analysis approach will be applied to analyze the data based on ANLAS key areas and quality objectives.
Expected Outcomes
Once data from various sources is collected, the results will be synthesized for the evaluation of key areas and aspects using the Synthesis tables provided by the Manual which will be followed by the recommendations for improvement for all dimensions. To report the ANLAS findings two documents will be prepared, namely, the Key Findings and the ANLAS report. The Key Findings document will inform about the main findings and recommendations resulting from ANLAS, including a brief overview of the motivation, purpose and goals for undertaking ANLAS in the country, and the most important aspects of the ANLAS implementation. The Key Findings document is based on the ANLAS report.
References
10. Claire Scoular & Esther Care. (2017). "Teaching of Twenty-First Century Skills: Implications at System Levels in Australia". Springer Science and Business Media, 145-162. 11. Countries Participating in TIMSS 2003. (2003). Retrieved from https://timssandpirls.bc.edu/: https://timssandpirls.bc.edu/timss2003i/countries.html 12. https://saber.worldbank.org/. (2012). Retrieved from Systems Approach for Better Education Results (Armenia): https://saber.worldbank.org/index.cfm?indx=2&ctrn=AM 13. Daniel Caro & Jia He. (2018). Equity in Education in Armenia: Evidence from TIMSS 2003-2015 14. Esther Care & Rebekah Luo. (2016). Assessment of transversal competencies: policy and practice in the Asia-Pacific region. 15. Inter-Agency and Expert Group on Sustainable Development Goal Indicators, "Report of the Inter-Agency and Expert Group on Sustainable Development Goal Indicators," (2016). 16. Marguerite Clarke and Diego Luna-Bazaldua. (2021). Primer on Large-Scale Assessments of Educational Achievement. Washington: World Bank. 17. R. Vivekanandan & M. Pierre-Louis. (2019). Analysis of National Learning Assessment System (ANLAS) Manual. Australian Council for Educational Research (ACER). 18. Thomas Kellaghan and Vincent Greaney. (2019). Public Examinations Examined. Washington: World Bank. 19. Thomas Kellaghan, Vincent Greaney & T. Scott Murray. (2009). Using the Results of a National Assessment of Educational Achievement. World Bank. 20. Toolkit for analysis of national learning assessment systems - ANLAS. (2019, October 25). Retrieved from https://www.globalpartnership.org/: https://www.globalpartnership.org/content/toolkit-analysis-national-learning-assessment-systems-anlas 21. Turner, R. (2014). Described proficiency scales and learning metrics. Assessment GEMs no.4. Melbourne: Australian Council for Educational Research. 22. UNICEF. (2002, January 1). CHILDREN PARTICIPATING IN RESEARCH, MONITORING AND EVALUATION. Retrieved from https://www.alnap.org/help-library/children-participating-in-research-monitoring-and-evaluation-ethics-and-your 23. UNICEF. (2015, April 1). UNICEF PROCEDURE FOR ETHICAL STANDARDS IN RESEARCH, EVALUATION, DATA COLLECTION AND ANALYSIS. Retrieved from https://www.unicef.org/: https://www.unicef.org/media/54796/file 24. UNICEF. (2015, April 1). UNICEF PROCEDURE FOR QUALITY ASSURANCE IN RESEARCH. Retrieved from https://www.unicef-irc.org: https://www.unicef-irc.org/files/upload/documents/UNICEF-%20Quality-Assurance-Research.pdf 25. UNICEF. (2020, July 15). UNICEF POLICY ON PERSONAL DATA PROTECTION. Retrieved from https://www.unicef.org/: https://www.unicef.org/supply/media/5356/file/Policy-on-personal-data-protection-July2020.pdf 26. United Nations Sustainable Development Cooperation Framework (2019).
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.