Technology Assisted Student Assessment Institute NAVIGATION » Home » News & Views » About TASA » Research » Publications » Media Releases » Contact THIRD ANNUAL RESEARCH SYMPOSIUM: Innovations in Testing Technology Click here for information. An initiative of the Society for the Advancement of Excellence in Education

Comparisons Between Paper and Computer-based Tests: Foundation Skills Assessment 2001-2006 Data Since 2004, the use of electronic administration of the Grade 7 Foundation Skills Assessment (FSA) in British Columbia has grown. Only one school was offering electronic exams in 2004 compared to 28 schools two years later. This report is based on 15 schools that administered the FSA electronically in at least one year during that period and had a fair proportion of students involved. Because the electronic interface has different characteristics for each of the FSA components and because males and females may approach computer applications differently, separate analyses with gender as a factor were carried out for Numeracy Multiple-Choice, Reading Multiple-Choice, Reading constructed-response, Writing Focused Response and Writing Extended Response. Additionally, students were placed in three achievement categories based on their results on the Grade 4 FSA and the analyses were carried out using Achievement Category as a factor. Student performance appeared to be impacted by certain features unique to the electronic and paper formats resulting in some advantages and some disadvantages for students. The detailed statistical analyses provided in this report are followed by discussion and recommendations for future research on electronic FSA examinations. This research was commissioned by the Technology Assisted Student Assessment Institute (TASA), an assessment research arm of the Society for the Advancement of Excellence in Education (SAEE). The research was supported through funding from the Max Bell Foundation. Published May 2007. ISBN 978-0-9783018-1-1 You can download the complete report by clicking here. It's in Acrobat PDF format for ease of reading and printing. Automated Essay Scoring: A Literature Review Automated Essay Scoring (AES) is an emerging field that has strong supporters and equally strong detractors. This literature review covers the disciplinary approaches of writing instruction, computational linguistics, and computer science, and the spectrum of perspectives that derive from them. Readers will be introduced to a variety of AES models, practical issues, recent research and future directions. Analytical tools and classification systems such as Bayesian text classification, latent semantic analysis and natural language processing are presented as well as a variety of AES engines currently on the market that offer both summative and formative evaluation programs. Susan M. Phillips provides an analysis of pedagogical issues and controversies that arise from the adoption of AES with a focus on best practices to enhance student achievement. Key findings, implications and recommendations for researchers, educators and policy makers are based on a thorough understanding of the current research and lay the foundation for the thoughtful use of AES in K-12 schools in Canada. Published February 2007. ISBN 978-0-9737755-7-0 You can download the complete report by clicking here. It's in Acrobat PDF format for ease of reading and printing. Comparisons Between Paper- and Computer Based Tests: A Literature Review This review of the differential effects of paper- and computer-based tests is of particular importance for those situations where tests are administered in both modes and where results are subject to similar interpretations and examinees are subject to similar consequences. The studies show that there are few generalizations that can be made. There are studies that show no differences between modes in a given subject and others that show differences favouring one mode and others favouring the other mode. The implication is that each testing initiative should demonstrate equivalence. The reports show differential effects related to ability, gender, computer familiarity, keyboarding skill, and race. Some of these effects can be non-linear. Differences emerge at the item level but little qualitative analysis has been carried out on individual items to determine what item characteristics may contribute to the differences. One of the studies cited reports on the cognitive equivalence of a paper-and-pencil test compared to an equivalent computer test and another on the effects of the instructional methodology preceding the testing. The overall implication of the research is that it cannot be taken for granted that computer-based or paper-based tests are valid. In particular, studies of complex design or a series of studies are required to confirm the validity of the instruments for all intended students. You can download the complete report by clicking here. It's in Acrobat PDF format for ease of reading and printing. A Future in the Process of Arrival: Using Computer Technologies for the Assessment of Student Learning One of Canada’s foremost assessment experts examines the potential of computerized assessment programs to revolutionize learning processes in Canadian schools. Based on his January 2005 survey of provincial and territorial assessment branches and cutting-edge developments in other systems, Taylor presents an informative and provocative description of the evolving assessment landscape. The research points to significant savings in costs and time that are made feasible by testing technology. For teachers and students, the potential benefits include assessment ‘on demand’, instant scoring feedback, and the capacity to adjust questions to student ability and measure skills impossible to capture with pencil and paper. The implications and recommendations in the final chapter of this 118 page report offer important direction for educators and system leaders. Dr. Alan Taylor is Director of the TASA Institute, established in 2004 to study technology assisted assessment in the K-12 domain. The research was commissioned in 2004 by the Society for the Advancement of Excellence in Education (SAEE) with a grant from Max Bell Foundation. SAEE Research Series 29, Published May 2005, ISBN 0-97337755-1-3 You can download the complete report by clicking here. It's in Acrobat PDF format for ease of reading and printing. Please note that it's a large file - 2.5 megabytes - so if you're on a slow connection it will take a while to download. You can also purchase a print copy of the complete report for $30 Cdn. by clicking here. Student Assessment in Canada: Improving the Learning Environment through Effective Evaluation How do we measure student learning in Canadian classrooms? This report examines many important facets of student evaluation. It analyzes and compares our various provincial and national assessment systems through interviews and surveys with education officials across Canada. The researchers review the broad international context for testing, the role of testing and measurement in the teaching and learning environment, and emerging assessment technologies, including value-added measurement and on-line testing. Common criticisms of testing and measurement and the shortcomings of current practice are also appraised. The report concludes with recommendations for a balanced model of assessment designed to improve teaching and learning in Canadian classrooms. A number of illustrative Canadian case studies are included in the appendix. The study was conducted by Dr. Alan Taylor and Dr. Teresita-Salve Tubianosa. An internationally recognized expert in the field of testing and measurement, Taylor is the National Research Coordinator for the TIMSS assessments and an adjunct professor at the University of British Columbia. The research was commissioned by SAEE through a research grant from Max Bell Foundation for this purpose. SAEE Research Series 7, Published May 2001, ISBN # 0965144-7-2 We have included the following sections from the Student Assessment in Canada report on this website: » Introduction to the Study of Student Assessment » The Role of Assessment and Evaluation in the Teaching-Learning Environment » International Practices in Evaluation » Trends in Evaluation and the Use of Emerging Technologies » Resources and Bibliography You can download the complete report by clicking here. It's in Acrobat PDF format for ease of reading and printing. Please note that it's a large file - 4.5 megabytes - so if you're on a slow connection it will take a while to download. You can also purchase a print copy of the complete report for $20 Cdn. by clicking here. » Return to Top   » Next: Media Releases