Archives
Current issue
About
About us
Aims and Scope
Abstracting and Indexing
Editorial Office
Open Access Policy
Publication Ethics
Journal History
Publisher
Follow us: Twitter
Contact
For Authors
Editorial Policy
Peer Review Policy
Manuscript Preparation Guidelines
Copyright & Licencing
Publication Fees
Fee Waiver Policy for Doctoral Students
EJMSTE Language Editing Service
More
Statistics
Special Issues
Special Issue Announcements
Published Special Issues
RESEARCH PAPER
A Calibrated Item Bank for Computerized Adaptive Testing in Measuring Science TIMSS Performance
1
School of Educational Studies, Universiti Sains Malaysia, MALAYSIA
2
Faculty of Technical and Vocational Education, Universiti Tun Hussein Onn MALAYSIA
Publication date: 2020-05-11
EURASIA J. Math., Sci Tech. Ed 2020;16(7):em1863
KEYWORDS
ABSTRACT
The current assessment is demanding for a more personalised and less-time consuming testing
environment. Computer Adaptive Testing (CAT) is seemed as a more effective alternative testing
method in comparison to conventional test in meeting the current standard of assessment. This
research reports on the calibration of the released Grade 8 Science objective items in Trends in
International Mathematics and Science Study (TIMSS) 2003-2015 based on Rasch model
framework to be used in CAT as an alternative testing tool in a low stakes test. Concurrent
common item equating method was used in linking and equating sets of items. Five test sets were
produced consisting of 20 unique items and 10 common items in each set. The unique items were
available only in a single set while common items were available in the two-consecutive set of
items. The sets were administered through Paper and Pencil test to Form 2 (Grade 8) students
who had been selected through a purposive sampling method from secondary schools in the
northern part of Malaysia. The fit analysis, polarity analysis, unidimensionality analysis, item
measure and Person-Map-Item were conducted. The analysis produced 122 calibrated items
which meet the Rasch’s requirements and were suitable to be used in CAT.
REFERENCES (51)
1.
Aleksander, I., & Morton, H. (1995). An introduction to neural computing: Information systems. International Thomson Computer Press.
2.
Aziz, A. A., Masodi, M. S., & Zaharim, A. (2013). Asas model pengukuran Rasch: Pembentukan skala & struktur pengukuran (Fundamental of Rasch ‘s measurement model: Scale formation & measurement structure). Universiti Kebangsaan Malaysia.
3.
Baghaei, P. (2008). The Rasch model as a construct validity tool. Rasch Measurement Transactions, 22, 1145-1146. Retrieved from https://www.researchgate.net/p....
4.
Barker, T. (2008). Computer-adaptive testing in higher education: The validity and reliability of the approach. In F. Khandia (Ed.), 12th CAA International Computer Assisted Assessment Conference (pp. 25-40). Lougborough University. Retrieved from http://caaconference.co.uk/pas....
5.
Bichi, A. A., Embong, R., Mamat, M., & Maiwada, D. A. (2015). Comparison of classical test theory and item response theory: A review of empirical studies. Australian Journal of Basic and Applied Sciences, 9(7), 549-556.
6.
Bjorner, J. B., Chang, C.-H., Thissen, D., & Reeve, B. B. (2007). Developing tailored instruments: Item banking and computerized adaptive assessment. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 16(1), 95-108. http://doi.org/10.1007/s11136-....
7.
Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model fundamental measurement in the human sciences. Lawrence Erlbaum Associates. https://doi.org/10.4324/978141....
8.
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model fundamental measurement in the human sciences (2nd ed.). Lawrence Erlbaum Associates.
9.
Boone, J. W., Staver, R. J., & Yale, S. M. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94....
10.
Choppin, B. (1976). Developments in item banking. Monitoring national standard of attainment in schools, 216-234. Retrieved from http://www.rasch.org/memo76.pd....
11.
Chuesathuchon, C., & Waugh, R. F. (2010). Item banking and computerized adaptive testing with Rasch measurement: An example for primary mathematics in Thailand. In R. F. Waugh, (Ed.), Applications of Rasch Measurement in Education (pp. 1-36). Nova Science.
12.
Conole, G., & Warburton, B. (2005). A review of computer-assisted assessment. Research in Learning Technology, 13(1), 17-31. http://doi.org/10.1080/0968776....
13.
Culbertson, M. J. (2015). Bayesian networks in educational assessment: The state of the field. Applied Psychological Measurement, 40(1), 3-21. https://doi.org/10.1177/014662....
14.
Davey, T. (2011). A guide to computer adaptive testing systems. Council of Chief School Officers. Retrieved from https://www.semanticscholar.or....
15.
Eggen, T. (2007). Choices in CAT models in the context of educational testing. In J. E. Hartig, E. Klieme, & D. Leutner (Eds.), Proceeding of the GMAC Conference on Computerized Adaptive Testing (pp. 199-217). Hogrefe & Huber Publisher. Retrieved from www.psych.umn.edu/psylabs/CATCentral/.
16.
Fensham, P. J. (1998). Student response to the TIMSS test. Research in Science Education, 28(4), 481-489. https://doi.org/10.1007/BF0246....
17.
Fisher, G. H., & Molenaar, I. W. (Eds.). (1995). Rasch models foundations, recent developments and applications. Springer-Verlag New York Incorporation.
18.
Fraenkel, J. R., & Wallen, N. E. (2009). How to design and evaluate research in education (7th Ed.). McGraw-Hill.
19.
Gershon, R. C. (2005). Computer adaptive testing. Journal of Applied Measurement, 6(1), 109-127. Retrieved from https://www.scholars.northwest....
20.
Idris, N. (2013). Penyelidikan dalam pendidikan (Research in education) (2nd Ed.). McGraw-Hill Education.
21.
Linacre, J. M. (2000). Computer-adaptive testing: A methodology whose time has come. In S. Chea, U. Kang, & J. M. Linacre (Eds.), Development of Computerized Middle School Achievement Test. Komesa Press. Retrieved from https://www.rasch.org/memo69.p....
22.
Linacre, J. M. (2012a, June). Winsteps Rasch tutorial 3. Retrieved from http://www.winsteps.com/a/wins....
23.
Linacre, J. M. (2012b, December). Some question on linking and equating method. Old Rasch Forum- Rasch on the Run: 2012. Retrieved from https://www.rasch.org/forum201....
24.
Linacre, J. M. (2013, March). Unidimensionality with dichotomous data. Retrieved from https://www.rasch.org/forum201....
25.
Linacre, J. M. (2019a). Correlations: Point-biserial, point-measure, residual. Retrieved from http://www.winsteps.com/winman....
26.
Linacre, J. M. (2019b). Dimensionality investigation- an example. Retrieved from https://www.winsteps.com/winma....
27.
Ling, S. S., Lan, O. L., Suah, S. L., & Ong, S. L. (2012). Investigating Assessment Practices of In-service Teachers. International Online Journal of Educational Science, 4(1), 91–106. Retrieved from http://www.iojes.net/index.jsp....
28.
López-cuadrado, J., Armendariz, A., Pérez, T. A. & Arruabarrena, R. (2008). Helping tools for item bank calibration and development of computerized adaptive test. In E. L. G. Chova, D. M. Belenguer, & I. C. Torres (Eds.), Proceeding of International Technology, Education and Development Conference (INTED’08), valensia (pp. 1-9). International Association of Technology, Education and Development. Retrieved from http://www.sc.ehu.es/jiwarsar/....
29.
Magyar, A. (2015). Comparing measurement effectiveness of computer-based linear and adaptive test (Doctoral Dissertation). Retrieved from http://doktori.bibl.uszeged.hu....
30.
Mansoor Al-A’ali. (2007). Implementation of an improved adaptive testing theory. Journal of Educational Technology and Society, 10(4), 80-94. Retrieved from https://www.researchgate.net/p....
31.
Masrom, S., & Abd. Rahman, A. S. (2009). An adaptation of agent-based computer-assissted assessment into e-learning environment. International Journal of Education and Information Technologies, 3(3), 163-170. Retrieved from http://www.naun.org/multimedia....
32.
McMillan, J. H., & Lawson, S. R. (2001, January). Secondary science teachers’ classroom assessment and grading practices. Metropolitan Education Research Consortium. https://pdfs.semanticscholar.o....
33.
Md. Desa, Z. N. D., & Abdul Latif, A. (2007). Computerized adaptive testing: An alternative assessment method. In M. Z. Kamsah, M. N. Hassan, K. I. Abdullah, & J. H. Harun (Eds.), Simposium Pengajaran dan Pembelajaran Universiti Teknologi Malaysia (Symposium Proceeding) (pp. 78-85). Centre for Teaching and Learning.
34.
Md. Noor, N., & Atan, N. A. (2008, November 5-7). Tahap kesediaan dan keyakinan pelajar terhadap penggunaan ujian adaptif dalam mempelajari konsep pengaturcaraan komputer (Students’ level of readiness and confidence in the use of adaptive tests in learning computer programming concepts)[Paper presentation]. 2nd International Malaysian Educational Technology Convention, Pahang, Malaysia. Retrieved from https://www.academia.edu/25363....
35.
Morphew, W. J., Mestre, P. J., Kang, H. A., Chang, H.-H., & Fabry, G. (2018). Using computer adaptive testing to assess physics proficiency and improve exam performance in an introductory physics course. Physical Review Physics Education Research, 14(2), 1-16. https://doi.org/10.1103/PhysRe....
36.
Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 international results in Science. Boston College, TIMSS & PIRLS International Study Center. Retrieved from http://timssandpirls.bc.edu/ti....
37.
Mullis, I. V. S. & Martin, M. O. (Eds.). (2017). TIMSS 2019 assessment framework. Boston College, TIMSS & PIRLS International Study Center. Retrieved from http://timssandpirls.bc.edu/ti....
38.
O’Malley, K. J., Murphy, S., McClarty, K. L., Murphy, D., & McBride, Y. (2011). Overview of student growth models [White Paper]. Pearson. Retrieved from http://images.pearsonassessmen....
39.
Oppl, S., Reisinger, F., Eckmaier, A., & Helm, C. (2017). A flexible online platform for computerized adaptive testing. International Journal of Educational Technology in Higher Education, 14(2), 2. https://doi.org/10.1186/s41239....
40.
Özdemir, B. (2016). Comparison of different unidimensional-CAT algorithms measuring students’ language abilities: Post-hoc simulation study. The European Proceedings of Social & Behavioural Sciences. https://doi.org/10.15405/epsbs....
41.
Raman, K., & Yamat, H. (2014). Barriers teachers face in intergrating ICT during English lesson: A case study. The Malaysia Online Journal of Educational Technology. 2(3), 11-19. Retrieved from https://eric.ed.gov/?id=EJ1086....
42.
Ryan, J., & Brockmann, F. (2018). A practitioner’s introduction to equating with primers on Classical Test Theory and Item Response Theory (Rev. ed.). Council of Chief State School Officers.
43.
Suah, S. L., & Ong, S. L. (2012). Investigating Assessment Practices of In-service Teachers. International Online Journal of Educational Sciences, 4(1).
44.
Sumintono, B. (2016, September 3). Penilaian keterampilan berpikir tingkat tinggi: Aplikasi pemodelan Rasch pada asesmen pendidikan (Assessment of higher-order thinking skills: Application of Rasch modeling in educational assessments) [Paper presentation]. Seminar Nasional Pendidikan IPA, FKIP Jurusan PMIPA, Universitas Lambun Mangkurat, Banjarmasin. Retrieved from https://drive.google.com/file/....
45.
Thompson, N. A., & Prometric, T. (2007). A practitioner’s guide for variable-length computerized classification testing. Practical Assessment Research and Evaluation, 12(1), 1-13. Retrieved from https://pareonline.net/getvn.a....
46.
Umar, N. I., & Hassan, S. A. (2015). Malaysia teachers levels of ICT integration and its perceived impact on teaching and learning. Procedia-Social and Behavioral Sciences, 197, 2015-2021. https://doi.org/10.1016/j.sbsp....
47.
van der Linden, W. J., & Glas, C. A. W. (2010). Elements of adaptive testing. Springer. https://doi.org/10.1007/978-0-....
48.
Wang, H. (2010). Comparability of computerized adaptive and paper-pencil tests. Test, Measurements and Research Services Bulletin, 13(1), 1-7. Retrieved from https://pdfs.semanticscholar.o....
49.
Way, W. D., Twing, J. S., Camara, W., Sweeney, K., Lazer, S., & Maeo, J. (2010). Some considerations related to the use of adaptive testing for the common core assessments. Educational Testing Service. Retrieved from http://www.ets.org/s/commonass....
50.
Weiss, D. J. (2011). Better data from better measurements using computerized adaptive testing. Journal of Methods and Measurement in the Social Sciences, 2(1), 1-27. Retrieved from https://journals.uair.arizona.....
51.
Wise, S. L., & Kingsbury, G. G. (2000). Practical issues in developing and maintaining a computerized adaptive testing program. Psicologica, 21, 135-155. Retrieved from https://www.uv.es/psicologica/....
Share
RELATED ARTICLE
| eISSN: | 1305-8223 |
| ISSN: | 1305-8215 |
We process personal data collected when visiting the website. The function of obtaining information about users and their behavior is carried out by voluntarily entered information in forms and saving cookies in end devices. Data, including cookies, are used to provide services, improve the user experience and to analyze the traffic in accordance with the Privacy policy. Data are also collected and processed by Google Analytics tool (more).
You can change cookies settings in your browser. Restricted use of cookies in the browser configuration may affect some functionalities of the website.
You can change cookies settings in your browser. Restricted use of cookies in the browser configuration may affect some functionalities of the website.
