The measurement in human science has been under tremendous development during last years and also the testing methods have been improved along the development, although the basic methods still are the classical item analysis and logistic models. Depending on the nature of the measured items there are different programs or their alternative ways to analyze data. Based on my 20 years’ experiment it has pointed evident that the Rasch model (Baker, 2001; Rasch, 1960; Wright & Stone, 1979; Wu & al., 1998) is the basic method to analyze an item test. It has been pointed out that understanding of the Rasch model is not very easy and it opens up after a long experience. Along with the Rasch model also the classical item analysis (Allen, 1979, Lord, 1980; Kim, 1999; Törmäkangas, 2007) has its own advantages in evaluating test data, especially distractors of multiple choice items. Multiparametric methods (Du Toit (ed.), 2003) offer another angle to the results, but the problem with those models is that fitting enough parameters, almost all items can be fitted to the item characteristic curve and, as a result from this, it is difficult to combine student’s score based on individual probability distribution of each item, which is solved quite differently in Rasch model (Perline & al,, 1979) How to understand results of the Rasch model and classical item analysis is described by using the ICCS[1] pilot study. Pilot testing of this International Civic Education Study was carried out in autumn 2007. There were 98 items in the main test and 20 items in the European Test designed for European participants. The test consisted of 6 different booklets, in which there were both common and different items in the test. The sample matrix design is not a problem for Rasch analysis or multiparametric models, but there are limitations in the classical item analysis. Only the main test for Finnish data is used as experiment. The main reason for this experiment is to find poorest items in the pilot study, which should be removed from the test or which clearly weakens the test. The reason for using multiparametric models is to compare different fit measures. The standardized mean square error in Rasch model gives sometimes different significance figures than the Chi square test in the multiparametric models.

The basic question for Rasch analysis is: Which items are acceptable for the test and how reliable is the whole test? Classical item analysis gives answer to the question: what is wrong with the distractors or correct answers in problematic items. There are some different measures used in the multiparametric models and the question arises: Could we get more information from these models, when we already have results from the Rasch model and classical item analysis?

In case of Finland, there were no very poor items based on the Rasch analysis (Bond & Fox, 2001), but 14 items were not in the recommended fitting area (11 plus, 3 minus infit z-test values). This phenomenon is very familiar for the international researchers, because some of the items are problematic in one country but can be acceptable in some other countries. Multiple choice items, which do not fit very well in Rasch model, can be further analyzed with classical item analysis to find out, which alternatives: correct or some of the incorrect choices did not work as expected. These items can be reconstructed for the final test, but it is not recommended to use them without additional pretesting.