Technology Assisted Student Assessment Institute NAVIGATION » Home » News & Views » About TASA Research » » TASA Agenda » » Phase 2 » » Clearinghouse » » Potential » Publications » Media Releases » Contact THIRD ANNUAL RESEARCH SYMPOSIUM: Innovations in Testing Technology Click here for information. An initiative of the Society for the Advancement of Excellence in Education

Research Clearinghouse » A. General Information/ Overviews » B. Classroom/ School Level Assessment » C. Large-Scale/ System Assessment » D. Policy and Future Directions » E. Links to Other Sources » F. What's New A. General Information/Overviews A Future in the Process of Arrival: Using Computer Technologies for the Assessment of Student Learning A. R. Taylor, SAEE Research Series 29, pp. 1-114, May 2006. Canada Taylor examines the potential of computerized assessment programs to revolutionize learning processes in Canadian schools. Based on his January 2005 survey of provincial and territorial assessment branches and cutting-edge developments in other systems, Taylor presents an informative and provocative description of the evolving assessment landscape. The research points to significant savings in costs and time that are made feasible by testing technology. For teachers and students, the potential benefits include assessment ‘on demand’, instant scoring feedback, and the capacity to adjust questions to student ability and measure skills impossible to capture with pencil and paper. The implications and recommendations in the final chapter of this report offer important direction for educators and system leaders. Literature Review of E-Assessment J. Ridgway, S. McCusker, and D. Pead, Futurelab Report # 10, pp.1-47, 2004. United Kingdom Presentation of key findings on research in assessment, current UK government plans, links to interesting e-assessment examples, and speculations on possible future developments in e-assessment stimulate the reader to consider the role of e-assessment in assessment, teaching and learning. This optimistic and practical comprehensive report has five major sections: (1) Assessment Drives Education; (2) How and Where Might Assessment Be Driven; (3) Current Developments in E-Assessment; (4) Opportunities and Challenges for E-Assessment: and (5) Fundamentals of Assessment Appendix. While focused on the United Kingdom, the information presented has applicability to Canada and elsewhere. Computer-Based Assessment: Can It Deliver on Its Promise? Stanley Rabinowitz and Tara Brandt, WestED®, pp.1-8, 2001. USA This knowledge brief describes the potential advantages of a fully implemented computer-based assessment system which has the potential of radically improving the implementation of assessments and the quality of the resulting information. The authors then describe a series of issues that states must address before that potential can be realized. Serious concerns include the fairness of the new assessment systems and the readiness of states (and their districts and schools) to support them. While focusing on implementation in the USA, the issues are applicable to other jurisdictions. Comparisons Between Paper- and Computer-Based Tests: A Literature Review Jim L. Gaskill, TASA Institute, p. 1-11, June 2006 Canada This review of the differential effects of paper- and computer-based tests is of particular importance for those situations where tests are administered in both modes and where results are subject to similar interpretations and examinees are subject to similar consequences. The studies show that there are few generalizations that can be made. The implication is that each testing initiative should demonstrate equivalence. The reports show differential effects related to ability, gender, computer familiarity, keyboarding skill, and race. The overall implication of the research is that it cannot be taken for granted that computer-based or paper-based tests are valid. In particular, studies of complex design or a series of studies are required to confirm the validity of the instruments for all intended students. Inexorable and Inevitable: The Continuing Story of Technology and Assessment R.E. Bennett, Journal for Technology, Learning and Assessment, pp. 1-24, February 2002. USA The inexorable advance of technology is and will continue to fundamentally change the format and content of educational assessment. In education technology is gaining importance because of workplace requirements, the exponential growth of technological capability, the provision of information resources important for learning, and the ability to deliver instruction to individuals both within and outside the traditional classroom. As technology becomes more embedded in educational practice, assessing students only in a paper and pencil medium instead of in the technological contexts in which they are learning will become increasingly untenable. Charts outline how several states and school districts are at different stages of implementing technology-based tests for low- and high-stakes decisions in elementary and secondary schools. Bennett outlines some of the difficulties such as cost, measurement, technological-dependability, and security issues. He believes K-12 agencies have educational responsibilities not only to incorporate technology in assessment but to enhance the process to create computerized assessments that support learning and instruction in ways that paper tests cannot. A related article by Bennett is How the Internet Will Help Large-Scale Assessment Reinvent Itself . Access to Computer-Based Testing for Students With Disabilities Sandra J. Thompson, Martha L. Thurlow, Rachel F. Quenemoen, and Camilla A. Lehr, National Center on Educational Outcomes, NCEO Synthesis Report 45, pp.1-59, June 2002. USA States are viewing computer-based testing as a way to address assessing all students in a state at nearly all grades, but according to the authors most states and testing companies have not specifically considered the needs of students with disabilities. Simply computerizing paper and pencil tests is not appropriate and may reduce the validity of the assessment results and possibly exclude some groups from participation in the assessment. This paper recognizes both the opportunities and the challenges of computer-based assessments for students with disabilities and recommends a five step process for a good transformation of a paper and pencil test to computer-based testing assuming that the principles of universally designed assessments already have been followed. Detailed initial considerations for common accommodations within the categories of timing/scheduling, presentation, response, and setting are also presented. Except for the US legal section, the concepts presented in this report are transferable to any educational jurisdiction. Computer-Based Assessment in E-Learning: A Framework for Constructing "Intermediate Constraint" Questions and Tasks for Technology Platforms K. Scalise and B. Gifford, Journal of Technology, Learning, and Assessment 4(6), pp.1-44, June 2006. USA Scalise and Gifford state that technology today offers many new opportunities for innovation in educational assessment through rich new assessment tasks and potentially powerful scoring, reporting and real-time feedback mechanisms. Designing assessment items for effective computer scoring and score reporting while still gathering meaningful measurement evidence is one potential limitation for realizing the benefits of computer-based assessment in both formative assessment and large scale testing. This paper presents a very useful taxonomy of 28 innovative item types that may be used in computer-based assessment. These item types are organized along the degree of constraint on the respondent’s options for answering or interacting with the assessment item or task. The categorization fulfills the authors’ stated purpose to provide a practical resource for assessment developers as well as a useful framework for the discussion of innovative assessment formats and uses in computer-based settings. Mixing Technology and Testing A. Russo, American Association of School Administrators, School Administrator 4(59), pp.6-12, April 2002. USA Computer-based testing is the choice of some school districts that are trying to improve cumbersome paper-and-pencil testing programs and integrate computers more completely into classroom curriculum and instruction. Like many other still-unfolding technology initiatives, computer-based testing takes several forms and falls under several different names, including online assessment, computerized testing, electronic testing and computer adaptive testing. Computer-based testing can provide flexibility, instant feedback, individualized assessment and eventually lower costs than traditional paper examinations. Computerized results create opportunities for teaching and assessment to be integrated more than ever before and allow for retesting students, measuring growth and linking assessment to instruction. Russo provides examples of different district initiatives to illustrate his points. The Effects of Online Formative and Summative Assessment on Test Anxiety and Performance J. C. Cassady and B.E. Gridley, Journal of Technology, Learning, and Assessment, 4(1), pp. 1-30, October 2005. USA This 31 page report discusses the effects of online formative and summative assessment on undergraduates’ testing behaviors (e.g., performance, study habits) and beliefs (e.g., test anxiety, perceived test threat). When comparing online to paper and pencil summative assessments there was no detrimental effect on students’ perceptions of tests or test performance and there were lower levels of perceived test threat reported by students who took tests online. For formative assessment Cassady and Gridley indicate a small benefit for using online practice tests prior to the graded exam. This appears to be partially a result of the reduction of the deleterious effects of negative test perceptions when practice tests were available. The results support online practice tests to help students prepare for course exams and also demonstrate that secure web-based testing can help undergraduate learning through improved student confidence and increased instructional time. An Overview of Automated Scoring of Essays Semire Dikli, Journal of Technology, Learning, and Assessment, 5 (1), August 2006. USA This 35 page overview of automated essay scoring systems (the computer technology that evaluates and scores written prose) is a must-read for anyone wanting specific information about the current state of the AES field. Dikli provides an overview of current approaches to AES: Project Essay Grader™ (PEG), Intelligent Essay Assessor™ (IEA), e-rater™ and Criterion™, IntelliMetric™ and MY Access! ™, and Bayesian Essay Test Scoring System (BETSY). The main characteristics of these systems are discussed as are current issues regarding the use of AES both in classrooms (formative) and large-scale assessments (summative). Possible future directions for AES are discussed. This is recent and timely and cites information from a great variety of sources. An Overview of Current Research on Automated Essay Grading S. Valenti, A.J. Nitko, and A. Cucchiarelli, Journal of Information Technology Education 2 (Information Technology for Assessing Student Learning Special Issue), pp.319-30, 2003. Italy This overview of approaches to the automated assessment of essays and free text answers describes ten systems as they existed in 2003: Project Essay Grader™ (PEG), Intelligent Essay Assessor™ (IEA), Educational Testing Service I, Electronic Essay Rater (E-Rater), C-Rater, BETSY, Intelligent Essay Marking System, SEAR, Paperless School free text Marking Engine and Automark. The authors note that in automated essay grading there is the absence both of a good standard to calibrate human marks and of a clear set of rules for selecting master texts. While the authors attempt to compare the performances of different AES systems they conclude that in order to really compare the performance of various systems some sort of unified measure should be defined. They also identify the lack of standard data collection in AES research and state that both these problems represent interesting issues for further research in the AES field. Knowing What All Students Know: Procedures for Developing Universal Design for Assessment L.R. Ketterlin-Geller, Journal of Technology, Learning, and Assessment 4(2), pp.1-22, November 2005. USA Universal design for assessment (UDA) is intended to increase participation of students with disabilities and English-language learners in assessments by addressing individualized student needs through customized testing platforms. As little practical information is available to assist test developers in creating and implementing universally designed tests, this article outlines the steps involved in conceptualizing, constructing, and implementing a universally designed test of grade 3 mathematics. Recommendations for future research and development efforts to create accessible computer-based learning environments for all students are presented. Publications - Pearson Educational Measurement Pearson Educational Measurement, pp. 1-3, accessed January 2007. USA This web page is a bibliography of some of the publications of staff of Pearson Educational Measurement that address diverse aspects of test development and measurement best practice as well as statistical research and evaluation. Topics will vary over time but at time of access some papers that may interest educators in the K-12 system might include: a study of social factors and mathematics achievement among high-scoring students; the impact of the California formative assessment and support system for teachers; large-scale assessment; to name a few. Toward a Growth-Centric Assessment Model Thomas H. Fisher and Jon S. Twing, A white paper from Pearson Educational Measurement, pp. 1-10, April 2006. USA Fisher and Twing state ìvalue-addedî assessment is a confusing model due to varying definitions and that ìgrowth-centricî assessment is a clearer concept, i.e., where all available data are used to drive student growth. A growth-centric model allows educators to use the standard units of measurement available from todayís assessment tools and other available data to manage student progress toward achievement goals. It is important to carefully develop and evaluate a growth-centric assessment model especially when communicating the results to different levels of government who are placing increasing demands for accountability on schools. This paper outlines preconditions for growth-centric assessment, growth-centric analysis strategies and criteria for evaluating growth-centric assessment systems. A Meta-Analysis of the Effectiveness of Teaching and Learning With Technology on Student Outcomes Hersh C. Waxman, Meng-Fen Lin, Georgette M. Michko, NCREL (North Central Regional Educational Laboratory), pp. 1-33, December 2003. USA This meta-analysis was commissioned by NCREL and includes data from 42 studies, representing a combined total of approximately 7000 students, to study the effects of teaching and learning with technology on student outcomes. It is interesting to note that originally over 200 studies were considered but that many were not included in the final synthesis either because they did not provide the relevant statistics for calculating effect sizes or because students in the control groups had access to or used technology. Appendix A provides a comprehensive listing of study characteristics to consider. Table 1 Summary of Recent Meta-Analyses in Educational Technology outlines recent meta-analyses and their effect size. The paper states that one area in which there have not been many meta-analyses and systematic reviews of the research is how teaching and learning with technology impacts student outcomes. This area is important to study because some research has found that technology can change teachersí pedagogic practices which in turn affect student learning. Technology Solutions for Testing Allan Olson, The School Administrator, pp. 1-5, April 2002. USA This article focuses on selecting a computerized test specific to your needs. When you focus on what you need and how quickly you need it instead of trying to fit the latest technology into your assessment program, the author states that the decision process is simplified. Factors such as curriculum alignment, growth measurement, on-demand testing, computerized adaptive testing, local disaggregation of data; student satisfaction; and data quality are examined. When we examine a wide array of student growth and learning questions, the possibilities for improvement will be virtually endless. The Effects of Technology on Reading Performance in the Middle - School Grades: A Meta-Analysis With Recommendations for Policy P. David Pearson, Richard E. Ferdig, Robert L. Blomeyer, Juan Moran, Learning Points Associates, pp. 1-57, November 2005. USA This meta-analysis of 20 research articles contains 89 effect sizes related to the use of digital tools and learning environments to enhance literacy acquisition. The effect size demonstrated that technology can have a positive effect on reading comprehension. The authors conclude that there is reason to encourage the research community to focus its efforts on digital learning environments for students and is in the middle school age range and to broaden the scope of interventions and outcomes they study. This extensive (over 200 entries) bibliography, listing of educational technology and reading journals, keywords used for web searches and other appendixes are useful additions to this document. There is mention that we need more assessment instruments that exhibit greater instructional sensitivity, i.e., sensitivity to changes in performance due to instruction. The researchers also believe that the research on digital tools for middle school literacy acquisition should include a focus on developing measures to evaluate outcomes that are generalizable, comparable and replicable. Students Struggle with Information Literacy: Many students know how to use technology, but fewer know how to apply it to find what they're looking for Justin Appel, Assistant Editor, eSchool News, pp. 1-2, Nov 26, 2006. USA This report comes from an evaluation of the responses of 6,300 students from 63 institutions around the United States to Educational Testing Serviceís (ETS) new ICT (Information and Communications Technology) Literacy Assessment. Students were given scenario-based items that were presented to them in 75-minute web-based test environments. These information literacy tests included extracting information from a database, developing a spreadsheet, or composing email summaries of research findings. The tests are meant to measure students' abilities to overcome three challenges they typically have: the ability to identify trustworthy and useful information; the ability to manage overabundant information; and the ability to communicate information effectively. ETS found that the majority of high school and college students lack the proper critical thinking skills when it comes to researching online and using sources. Online Testing Research: Informing and Guiding Transitions to Computerized Assessment A white paper from Pearson Educational Measurement, Walter (Denny) Way, Vice President, Psychometric and Research Services, pp. 1-7, April 2006. USA This white paper written by Walter (Denny) Way, Vice President, Psychometric and Research Services, Pearson Educational Measurement, highlights the emphasis Pearson Educational Measurement Solutions (PEMSolutionsô) places on research in support of their assessment services. Several research studies are mentioned and links are provided so that the reader can easily access research supported by Pearson educational measurement. Many of these studies or conference proceedings are listed as separate entries in this TASA research clearinghouse. On-line Mathematics Assessment: The Impact of Mode on Performance and Question Answering Strategies Martin Johnson and Sylvia Green, Journal of Technology, Learning and Assessment, 4(5), pp 1-34, March 2006. USA The transition from paper-based to computer-based assessment raises a number of important issues about how mode might affect childrenís performance and question answering strategies. 104 eleven-year-olds completed two sets of matched mathematics questions, one set on-line and the other on paper. The results were analyzed to explore the impact of the delivery mode on student performance. Observation and interviewing of some of the children provided insight into the children's affective responses. Johnson and Green found that although there were no statistically significant differences between overall performances on paper and computer, there were enough differences at the individual question-level to warrant further investigation. Close analysis of the data suggests that it is possible that the question type, the way it is asked, and the numbers involved, might interact with mode to affect studentsí willingness to show working methods. More research is indicated to investigate further any links that may exist between childrenís thinking, behavior and assessment mode in order to satisfy concerns about the relative reliability and validity of computer-based and paper-based testing. Examining the Effect of Computer-Based Passage Presentation of Reading Test Performance Jennifer Higgins, Michael Russell, and Thomas Hoffmann, Journal of Technology, Learning and Assessment, 3(4), pp 1-35, January 2005. USA In a study to examine whether delivery mode affected student test scores, 219 fourth graders were randomly assigned to take a one-hour reading comprehension in one of three ways: (1) on paper; (2) on a computer using scrolling text; or (3) on a computer using paging text. Students also completed a computer skills performance assessment, a paper based computer literacy assessment, and a computer use survey. Analysis found: (1) no significant differences in scores across testing modes but this finding may be due in part to the unusually high computer access and higher socio-economic status of the sample; (2) no statistically significant differences in scores based on computer fluidity and literacy, however students may be disadvantaged by the scrolling text mode; (3) the majority of students who took the reading test on a computer preferred to take the test on computer; (4) providing highlighters and review markers is useful for some students; and (5) further research is needed to understand differences in scores when reading comprehension assessments are administered via computer to a larger and more diverse group of students. Recent Trends in Comparability Studies Pamela Peak, Pearson Educational Measurement, p. 1-29, August 2005. USA This literature review presents the research, particularly the research since 1993, addressing the comparability of computer-delivered tests and pencil-and-paper tests. The state of online testing technology and the different methods used in the comparability studies are discussed first and then the results from the studies, specifically in K-12 testing, are discussed. Finally the potential of online assessments is presented. There are numerous references presented in the text as well as a concise chart- Table 1: ‚Results of Comparability Studies in K-12 which sorts comparability research studies by subject area into three columns: Computer test more difficult; Paper Test More Difficult; and Comparable. This paper is a good resource to find research citations related to the comparability of different delivery modes of student assessment. Developing Computerized Versions of Paper-and-Pencil Tests: Mode Effects for Passage-Based Tests Mary Pommerich, Journal of Technology, Learning and Assessment, 2(6), pp 1-44, February 2004. USA As tests are administered on a computer instead of paper-and-pencil delivery mode, how test information is displayed on a computer screen is important. This is especially true for those items that contain necessary information that cannot be displayed on screen all at once for one item as this has the potential of interfering with the student's performance on the test. This paper discusses the development of a computer interface for passage-based, multiple-choice tests. Findings from two studies that compared performance across computer and paper administrations of several fixed-form tests is presented and the effect of computer interface changes made between the two studies is discussed. Both studies showed some performance differences across modes. Evaluations of individual items suggested a variety of factors that could have contributed to mode effects. Overall the findings suggest that it would be beneficial to develop an understanding of factors that can influence examinee behavior and to design a computer interface accordingly, to ensure that examinees are responding to test content rather than features inherent in presenting the test on computer. Examining the Relationship Between Home and School Computer Use and Studentsí English/Language Arts Test Scores Laura O'Dwyer, Michael Russell, Damian Bebell, and Kevon R. Tucker-Seeley, Journal of Technology, Learning and Assessment, 3(3), pp 1-45, January 2005. USA This paper examines research on the relationship between student achievement and technology use. They state there have been hundreds of studies but many did not provide the relevant statistics for calculating effect sizes or there were not appropriate control groups. The methodological and psychometric issues that arise when investigating such issues are outlined. Then a multilevel regression analysis of the relationship between a variety of student and teacher technology uses and fourth grade test scores on the Massachusetts Comprehensive Assessment System (MCAS) English/Language Arts test is examined. This study found that, while controlling for both prior achievement and socioeconomic status, students who reported greater frequency of technology use at school to edit papers were likely to have higher total English/language arts test scores and higher writing scores; however, use of technology at school to prepare multimedia presentations was associated with lower English/language arts outcome measures. Teachers' use of technology for a variety of purposes were not significant predictors of student achievement, and studentsí recreational use of technology at home was negatively associated with the learning outcomes. Computerized Adaptive Testing: A Comparison of Three Content Balancing Methods Chi-Keung Leung , Hua-Hua Chang, and Kit-Tai Hau, Journal of Technology, Learning and Assessment, 2(5), pp 1-16, December 2003. USA Content balancing is often a practical consideration in the design of computerized adaptive testing (CAT). This study compared three content balancing methods, namely, the constrained CAT (CCAT), the modified constrained CAT (MCCAT), and the modified multinomial model (MMM), under various conditions of test length and target maximum exposure rate. Results of a series of simulation studies indicate that there is no systematic effect of content balancing method in measurement efficiency and pool utilization. However, among the three methods, the MMM appears to consistently over-expose fewer items. This paper would be of interest to test developers and psychometricians. Other readers would only need to know that there is an issue of content balancing for which content balancing methods can be very effective in controlling over-exposure of test items in computerized adaptive testing. Inclusive Design for Maximum Accessibility: A Practical Approach to Universal Design Elizabeth I. Hanna, Pearson Educational Measurement, pp. 1-35, August 2005. USA This article briefly reviews the literature related to Universal Design for Learning (UDL) and Universal Design for Assessment (UDA), and outlines an approach for combining these two philosophies in evaluating large-scale assessment programs. After a brief history of universal design, a discussion of a planning approach to UDL and UDA in assessment programs is presented in three categories: (1) the construct of the assessment; (2) the use of the assessment; and (3) the accommodations provided for the assessment. Some of the psychometric implications of UDL and UDA, specifically related to test scaling and comparability, are discussed. The author cautions that in the rush to be inclusive the greater goal—accurate and comparable measurement for everyone—must not be jeopardized; assessment data needs to be valid and reliable, and assessments should comply with legislation and meet the Standards for Educational and Psychological Testing. The concepts discussed have applicability in all jurisdictions even if the legislation may be different. Constructing Innovative Computer-Administered Tasks and Items According to Universal Design: Establishing Guidelines for Test Developers Michael Harms, Kelly Burling, Walter Way, Elizabeth Hanna and Bob Dolan, Paper presented at the April 2006 Meeting of the National Council on Measurement in Education (NCME), pp. 1-26, April 2006. USA While there are increased capabilities to deliver new assistive technologies to meet studentsí diverse needs these new technologies also increase the range of factors that must be considered. Test developers cannot predict the ways factors might interact with the potential to affect test validity. Therefore, this paper proposes the Universal Design for Computer-Based Testing (UD-CBT) framework for establishing guidelines that may be applied in designing non-standard computer delivered assessments consistent with the goals of Universal Design. Three of the tables (Table 1: Description of Disability Categories, Table 2: CBT Item Components, Table 3: UD-CBT Content Considerations) as well as Figure 1: UD-CBT Phases of Item Interaction Model provide useful information for the reader in understanding UD-CBT.UD-CBT represents a comprehensive framework for analyzing and understanding a wide range of factors that influence construct validity. An On-line, Interactive, Computer Adaptive Testing Tutorial Lawrence Rudner, November, 1998. USA This site provides readers the opportunity to learn and experience the logic of CAT (Computer Adaptive Testing) through an online interactive tutorial. Also on this page terms (true score, ability estimate, standard error) are defined and concepts (probability of a correct response, item response function and item response theory) are explained. Rudner lists the potential and limitations of CAT as well as key technical and procedural issues such as: book balancing content, administering item belongs to sets, examinee considerations, item exposure, item pool characteristics, item pool size, shifting parameter estimates, and stopping rules. This is a site worth visiting to learn or review the basics of CAT. Strategies for Controlling Item Exposure in Computerized Adaptive Testing with the Partial Credit Model Laurie Laughlin Davis and Barbara G. Dodd, pp. 1-32, March 2005. USA This paper discusses randomization procedures, determined by exposure control research with polytomous item pools, to be very effective for controlling test security in computerized adaptive testing (CAT). The study investigated the performance of four procedures for controlling item exposure in a CAT under the partial credit model. This paper would be of interest to test developers and psychometricians. Other readers would only need to know that there is an issue of exposure control for which randomization procedures can be very effective in controlling test security in computerized adaptive testing. Software Enabling School Improvement Through Analysis of Student Data Jeffrey C. Wayman, Sam Stringfield, and Mary Yakimowski, CRESPAR (Center for Research on the Education of Students Placed At Risk) Report # 67, pp. 1-63, January 2004. USA The authors consider issues surrounding the use of student data and data-based decision making, describing the state of the field and possible future directions, present reviews of a range of commercially available software for analyzing student data including an appendix of screen shots, and provide and maintain a website that will contain ongoing updates of software reviews. This web site is http://edadmin.edb.utexas.edu/datause/ (please note this is a different web site than the one originally cited in the paper). The concepts in this paper and the website with software reviews would be of use to readers whether they want an overview of this area or are beginning the process to determine what products to consider. Involving Teachers in Data-Driven Decision Making: Using Computer Data Systems to Support Teacher Inquiry and Reflection Jeffrey C. Wayman, Journal of Education for Students Placed at Risk, 10(3), 295–308, 2005. USA Student data has often been stored in forms that are difficult to access, manipulate, and interpret. Such access barriers additionally preclude the use of data at the classroom level to inform and impact instruction. Wayman discusses teacher use of new computerized data-warehouse and reporting technologies that allow efficient organization and access to student data. He provides insight into the function of these tools and discusses the conditions that make these tools of the most service to teachers. In addition to allowing easier accountability reporting, he states these tools allow user-friendly data access at all educational levels, meaning that teachers can use these tools to engage in the informed reflection necessary to improve classroom practice. There are many references to the related literature. Individualizing Learning Using Intelligent Technology and Universally Designed Curriculum Michael Abell, Journal of Teaching, Learning and Assessment, 5(3), pp. 1-20, November 2006. USA Rigorous accountability and performance standards continually cause educators to explore new ways to increase student achievement. Improvements in computer technology and intelligent computing systems may offer new tools for student learning and increase academic achievement for all if universally designed curricula and assessments are implemented. This paper presents a conceptual framework that addresses the potential of intelligent learning systems, machine learning models, and universal design for learning principles to help develop next generation instructional materials. Abell believes that by using intelligent and interactive curricula, educators could begin to move away from the role of information disseminator into a facilitator of the learning experience. Table 2: Intelligent Student Learning Environment Interface Levels details access levels, components and a description for district, school, classroom and student levels. Teaching One Way and Testing Another: An Interview with Scott Howell James L. Morrison and Scott Howell, Innovate Journal of Online Education, 3(3), pp. 1-14, February/March 2007. USA Scott Howell is co-editor of the three-volume book series Online Assessment and Measurement (2005) published by IDEA group and is this year's chair of the University Continuing Education Association's (UCEA) Distance Learning Community of Practice (DLCoP). Howell focuses on the alignment of testing and assessment methodology with the presentation of instructional content. He perceives a gap between high-tech online teaching practices and current testing practices and believes that assessment strategies should use the opportunities that multimedia technology offers. He also discusses some of the pedagogical considerations that impact students and educators. This is a good overview of this topic and includes some links and citations of related sources. Bridging the Gap Between Testing and Technology in Schools Michael Russell and Walt Haney, Technology and Assessment Study Collaborative, CSTEEP, Boston College, Rereleased by inTASC July 2002 p. 1-9. Originally published April 2000 in Education Policy Analysis Archives 8(19). USA The authors cite the widening gap between the increased use of technology in the schools in the absence of computers in many state level testing programs. They summarize the developments and use of technology in schools and the state level testing programs as of 2000 and then describe how two studies that indicate written tests administered on paper can underestimate the achievement of students accustomed to working on computers. They conclude by discussing four approaches to bridging the gap between technology and testing in US schools. In the short term they recommend the fourth option which is to recognize the limitations of current testing programs, i.e. until students can take tests in the same medium which they generally work and learn, we must recognize that scores from high-stakes tests do not necessarily accurately reflect some studentsí capabilities. It's Time To Upgrade: Tests and Administration Procedures for the New Millennium Michael Russell, Eric ED452822, The Secretary's Conference on Educational Technology, 2000: Measuring Impacts and Shaping the Future, Alexandria, VA, pp. 1-10, September 11-12, 2000. USA Addressing problems with performance on open-ended test questions administered on paper for students accustomed to writing with computers, the paper summarizes the results of three studies that examined differences between computer and paper-and-pencil test results and then identifies why the mismeasurement of students accustomed to writing with computers is problematic. The reasons why state testing programs have resisted a short-term solution of allowing students to select the mode in which open-ended responses are composed are discussed. The authors list other areas of learning that computers can help students develop (e.g., problem solving, research, non-linear thinking, understanding of scientific concepts, collaboration, spatial reasoning, statistics, media, music theory, and modeling/simulation). They conclude describing the steps necessary for testing and accountability programs to develop and apply instruments and testing procedures that capture the types of learning impacted by computer use. Improving Accountability Models by Using Technology-Enabled Knowledge Systems Eva Baker, CSE Report No. 656, CRESST, pp. 1-35, July 2005 USA Baker addresses how technology can improve accountability and simultaneously improve the efficiency and quality of educational outcomes and supporting processes. She states that the reader should learn or update their understanding in the following areas: basics of accountability; functions of technology systems—efficiency and quality; functions of technology in education and accountability systems; criteria for decision making; criteria applied to technology in support of assessment and testing; criteria applied to technology in support of information exchange; and ways to manage system, infrastructure, and capacity demands. This paper provides readers a comprehensive overview of accountability and the role emerging technologies can play in improving education. Cheating in Online Student Assessment: Beyond Plagiarism Neil Rowe, Online Journal of Distance Learning Administration, 7(2), Summer 2004. USA This study looks at online student assessment features common in many distance learning programs. The author believes that the prevention of plagiarism has been the subject of much attention but that insufficient attention has been given to other problems of dishonesty related to online assessment. The types of problems that can occur and what can be done about them are discussed. He believes that many educators are unaware of these issues and that most countermeasures are insufficient to combat cheating in online student assessment and urges caution until there is greater progress is made on the technical development of countermeasures. Evidence for the Interpretation and Use of Scores from an Automated Essay Scorer Paul Nichols, Pearson Educational Measurement, pp. 1-26, March 2005. USA Using the validity framework described by Yang, et al. (2002), the scores from 3244 grade 8and 10 students using the Intelligent Essay Assessor (IEA), an automated essay-scoring engine developed by Pearson Knowledge Technologies, were examined. Yangís framework delineates three approaches to validation studies: (1) examine the relationship among scores given to the same essays by different scorers; (2) examine the relationship between essay scores and external measures; and, (3) examine the scoring processes used by the IEA. Results indicated that, although relationships among scores given to the same essays by different scorers (percent agreement, Spearmen rank-order correlation, kappa statistic and Pearson correlation) indicated a stronger relationship between two human readers than between the IEA and a human reader, stronger relationships were found between the IEA and experts than between readers and experts. Also the scoring processes employed by IEA were similar to those of a human scorer; and more similar to processes used by proficient human scorers than to processes used by non-proficient human scorers. Further research in other assessments and at other grade levels will be helpful in generalizing the results and further strengthening the validity of IEA for scoring writing assessments. Leverage Points for Improving Educational Assessment Robert J. Mislevy, Linda S. Steinberg, and Russell G. Almond, Geneva Haertel and William R. Penuel, CSE Technical Report No. 534, CRESST, pp. 1-41, February 2001. USA The authors state that advances in cognitive psychology deepen our understanding of how students gain and use knowledge while advances in technology make it possible to capture more complex performances in assessment settings, by including, for example, simulation, interactivity, collaboration, and constructed response. They believe the challenge is knowing just how to put this new knowledge to work to produce assessments that may indeed gather complex data to ground inferences about complex student models, to gauge complex learning or evaluate complex programs. An evidence-centered framework for designing and analyzing assessments is reviewed and then this framework is used to discuss and illustrate how advances in technology, education and psychology can be harnessed to improve educational assessment. Software Enabling School Improvement through Analysis of Student Data Jeffrey C.Wayman, Sam Stringfield, and Mary Yakimowski, CRESPAR, Johns Hopkins University's Center for Research on the Education of Students Placed At Risk, No. 67, pp. 1-63, January 2004. USA The report, entitled "Software Enabling School Improvement through Analysis of Student Data," considers the issues surrounding the use of student data to improve instruction, describes the state of the field and its possible future directions, and reviews a wide range of commercially available software for analyzing student information. Student Assessment in Canada: Improving the Learning Environment through Effective Evaluation Alan Taylor, SAEE Research Series 7, May 2001. Canada This report examines many important facets of student evaluation or how we measure student learning in Canadian classrooms. It analyzes and compares the various provincial and national assessment systems through interviews and surveys with education officials across Canada. From Vision to Action: How School Districts Use Data to Improve Performance CoSN, June 2005. USA This report is part of the initiative by the Consortium for School Networking (CoSN) on data-driven decision making to help educators use data to improve instruction. The report identifies best practices employed by school districts in using data on student achievement to effect change in schools and to more effectively meet state and national standards for student achievement. It includes examples of school districts that are successfully implementing improvement strategies based on the data their schools have collected and studied. Three key elements to successful data management to improve performance are identified in the report: the lynchpin is leadership and teamwork; the collection of data should be a cyclical process; and there needs to be technology to support the process. E-Inclusion: Learning Difficulties and Digital Technologies Chris Abbott, Futurelab Series Report No. 15, pp. 1- 36, May 2007. UK This review focuses specifically on the use of digital technologies to enable children with learning difficulties to learn effectively. Its goal is to ask nuanced questions about the evidence that exists of the role of digital technologies in this area. It foregrounds the learning practices and communities that might be enabled with digital technologies to create rich and empowering learning environments for children with learning difficulties. Abbott offers a new taxonomy of the use of digital technologies in this field, providing a historical and philosophical overview of three key approaches to using technology either to train or rehearse; to assist learning; or to enable learning. It concludes by offering a set of challenges to industry and educators to create more collaborative, holistic and inclusive learning communities through digital technologies. Mobile Technologies and Learning Laura Naismith, Peter Lonsdale, Giasemi Vavoula and Mike Sharples, Futurelab Series Report No. 11, pp. 1- 48, December 2004. UK The challenge for educators and designers is one of understanding and exploring how best to use mobile technology resources to support learning. There is a need to include in schools the powerful technologies that are a normal part of everyday life, however, the models for using and developing mobile applications for learning are somewhat lacking. This review provides a vision of the current and potential future developments in this area. It moves away from the dominant view of mobile learning as an isolated activity to explore mobile learning as a rich, collaborative and conversational experience, whether in classrooms, homes or elsewhere. It asks how educators might draw on existing theories of learning to help evaluate the most relevant applications of mobile technologies in education. Descriptions of outstanding projects currently under development in the UK and around the world are presented as is an exploration of what the future might hold for learning with mobile technologies. Learning with Digital Technologies in Museums, Science Centres and Galleries Roy Hawkey, Futurelab Series Report No. 9, pp. 1- 48, September 2004. UK There has been a massive growth in the use of digital technologies for learning in museums, science centres and galleries – both onsite in the form of digital interactives, and online via the creation of ever-more popular websites. These developments provide a number of challenges for educators and curators, at the heart of which lie the questions – what is distinctive about learning in museums, science centres and galleries, and how might this change or evolve through the increasing use of digital technologies? How does learning in museums differ from or complement learning in schools? How can museums fulfill their potential to support lifelong learning? What is the role of objects in the process of learning with digital technologies? How does the relationship between museum educator and learner change as technologies are developed? How can museums and educators carefully evaluate the significance of the formal school curriculum and its assessment process? Tablet PCs in Schools: A Review of Literature and Selected Projects K. Sheehy, A. Kukulska-Hulme, P. Twining, D. Evans, D. Cook and A. Jelfs and others, Open University, Becta, pp. 1-12, June 2005. UK A literature search occurred in late 2004 at which time all 154 LEAs in England were contacted by phone. LEAs where Tablet PCs were in use were asked to provide information via an online questionnaire. Relevant online discussion groups were also invited to provide details of schools or projects that were using Tablet PCs via this questionnaire. The literature and project search identified 90 schools or school-based projects that were using Tablet PCs in England. All 90 were contacted by phone in order to obtain details of their Tablet PC use. This paper analyzes the data in areas such as range of uses, Tablet PC management, lessons learned, and benefits and provides a bibliography, further reading and also lists other available project reports in England and two in the United States. This is a companion publication to Tablet PCs in Schools: Case Study Report . Tablet PCs in Schools: Case Study Report P. Twining, D. Evans, D. Cook, J. Ralston, I. Selwood, A. Jones, J. Underwood, G. Dillon, E. Scanlon, and others, Open University, Becta, pp. 1-58, June 2005. UK This review provides an analysis of Tablet PC use based on case studies from twelve English schools. The analysis is complemented by brief individual reports describing aspects of how each of these schools was using Tablet PCs and key lessons learned. It is a companion publication to Tablet PCs in Schools: A Review of Literature and Selected Projects. SETDA's Technical Assistance Partnership Program (TAPP) SETDA, as accessed July 2007. In October, 2003, the U.S. Department of Education awarded $15 million to conduct scientifically-based evaluations of how technology impacts student achievement in elementary and secondary education. This site reflects the work of nine of the ten states who are ESETP grantees. SETDA's TAPP is designed to offer dissemination and networking services to the state-wide scientifically based research projects, provide interim information regarding best practices in the area of scientifically based research and success stories and progress reportsto the broader ed-tech community via this website and other presentations, and develop a strong, effective Handbook that outlines TAPP participants work throughout the course of the grant process. SETDA will provide updates and interim reports as lessons are learned. These nine projects are researching issues faced by any school, district, state or province such as the impact of professional development on student achievement and the impact of increasing in-home access to computers and the Internet, particularly for ëhigh needí families, in an effort to bridge the ëdigital divideí. Also on the site are links to each stateís project and conference presentations to date. This will be a site to monitor as the projects near completion. Interim Report on the Texas Technology Immersion Pilot (TIP) Texas Education Authority, pp.1-24, December 2006. USA In the Texas Technology Immersion model, schools implement six critical technology resources: a wireless mobile computing device for each student to use at home and school; productivity and communication tools; ongoing professional development for teachers and administrators; online formative and diagnostic assessment tools; and technical support. Therefore teachers have the tools they need to assess student proficiency and develop multimedia lesson plans targeted at student skill strength and weakness areas. Equally as important, students are given the same tools as teachers. The TIP project includes 44 middle schools serving grades six, seven, and eight; a whole district; a vertical team of an elementary school, middle school and high school in a feeder pattern; a secondary campus serving grades seven and eight; over 1300 teachers; and more than 19,000 students. The eTxTIP research team is analyzing the effects of technology immersion in grade 6-8 middle schools as compared to a control group of non immersed grade 6-8 middle school campuses and is assessing factors such as campus progress towards immersion, effects of immersion on campus culture, teacher innovation and productivity, and student learning. Technology Counts 2005 Education Week, May 2005. USA Education Week has released its 2005 comprehensive annual overview of technology in the education system, Electronic Transfer: Moving Technology Dollars in New Directions. The 2005 report notes that the technology landscape is rapidly changing. The growing emphasis on accountability is shifting spending and energy away from instructional technology toward administrative technology uses like data management. States and districts are spending millions of dollars to build online student-data systems that will offer teachers what policymakers hope will be the information needed to craft clear-cut strategies for raising achievement. Designing Adaptive, Diagnostic Math Assessments for Individuals With and Without Visual Disabilities Valerie J. Shute, Edith Aurora Graf and Eric G. Hansen, ETS, RR-06-01, pp.1- 46, January 2006. USA This report summarizes the design and development of an adaptive e-learning prototype for middle school mathematics for use with both sighted and visually disabled students (adaptation refers to the systemís ability to adjust itself to suit particular characteristics of the learner. The report describes the systemís theoretical foundation, architecture, models, and adaptive algorithm as well as reviewing approaches for making assessment systems accessible to students with visual disabilities. A summary of upcoming studies in relation to important research questions concerning micro- and macroadaptation is included. The authors believe using their design approach may set a new precedent for environments that adapt to support student learning based on larger sets of incoming abilities and disabilities than have been considered previously. Rethinking and Redesigning Education Assessment: Preschool through Postsecondary James W. Pellegrino, Education Commission of the States, MetLife Foundation Change in Education Initiative, Document Number: 2488, pp. 1-10, April 2001. USA This short overview, one of nine essays in the MetLife Foundation Change in Education Initiative, states that assessment is playing a greater role in decision making than ever before. In turn, education stakeholders are questioning whether current large-scale assessment practices are yielding the most useful kinds of information for informing and improving education. Pellegrino states that a more realistic view is to address our ongoing concerns about education quality and improvement not by stepping up the amount of testing or abandoning assessments entirely, but rather by refashioning assessments in principled ways to meet current and future needs for quality information about what students know. Using Technology to Improve Accountability Systems Eva Baker, ECS Next Generation Models of Education Accountability: Cross-Cutting Issues: Technology, pp. 1- 15, July 2003. USA Baker states that technology provides ways of designing, collecting and sharing information that can help improve accountability systems and outcomes. This paper gives a set of criteria to educators and policymakers to help them make better technology choices that improve learning and make teachers more effective. It reviews issues that must be considered such as security, compatibility, resilience and transparency. It provides examples of effective use of technology available now and it shows elements of design and technology in practice in two very different school district settings. Baker believes that technologies can provide greater efficiency in managing information or serve to improve the quality of the information used to guide the system. At this point, these functions are frequently mutually exclusive, but in the future, wise selection of technology should include consideration of how the technology adds value by increasing both quality and efficiency. Six Key Uses of Longitudinal Data Chrys Dougherty, Lynn Mellor, and Nancy Smith, Data Quality Campaign, pp.1-3, USA The authors state that the distinctive feature of longitudinal data systems is their ability to follow the same students over time. These systems can create a rich picture of student progress and school, district, and program performance. This brief outlines the six uses of longitudinal data: progress monitoring; diagnosis and prescription; internal benchmarking; external benchmarking; predictive analysis; and evaluation. Also included is what states or other jurisdictions can do to promote these six uses of longitudinal student data. Making Sense of Data-Driven Decision Making in Education: Evidence from Recent RAND Research Julie A. Marsh, John F. Pane, and Laura S. Hamilton, Rand Education, Occasional Paper OP-170-EDU, pp. 1- 18, 2006. USA This occasional paper seeks to clarify the ways in which multiple types of data are being used in schools and districts by synthesizing findings from recent research systematically gathered from large, representative samples of educators at the district, school, and classroom levels in a variety of contexts conducted by the RAND Corporation. The authors state that this paper further provides a comprehensive examination of the many facets of current DDDM (Data-Driven Decision Making) policies and practices and suggests a research agenda to advance the field by addressing four fundamental questions: What types of data are administrators and teachers using? How are administrators and teachers using these data? What kinds of support are available to help with data use? What factors influence the use of data for decision making? A Review of Item Exposure Control Strategies for Computerized Adaptive Testing Developed from 1983 to 2005 Elissavet G. Georgiadou, Evangelos Triantafillou, and Anastasios A. Economides, Journal of Technology, Learning and Assessment, 5(8), pp.1-38, May 2007. Greece The issue of item exposure control in computerized adaptive testing (CAT) is important. Particular items in the item pool may be presented too often and become overexposed, while other items are rarely selected by the CAT algorithm and thus become underexposed. This paper reviews several item exposure control strategies that aim to prevent overexposure of some items and to increase the use rate of rarely or never selected items. This paper does not present new research but focuses on studies from the literature from 1983 to 2005 to evaluate the effectiveness of item exposure control strategies for dichotomous scoring, polytomous scoring and testlet-based CAT systems. The authors discuss the strengths and weaknesses of each strategy group using examples from simulation studies. Their overall objective is to provide researchers of this field, especially newcomers, a wide view of item exposure control strategies Interactive Assessment and Course Transformation Using Web-Based Tools Celina Byers, The Technology Source, pp.1-6, May/June 2002. USA This article, while based on Beyerís experience in a post-secondary setting, is a clear and concise case study of interactive course assessment applicable to all levels of learning. Interactive assessment implies a dynamic process that is both formative and summative based on examination of the three dimensions: the instructor's perception, the student's perception, and the student's performance. Students are provided with instruments to let the instructor know what went well and what did not, both in terms of their performance and in terms of their perceptions of the teacher's performance. The search for excellence in teaching normally involves a large amount of complex work. Familiarity with and application of instructional design principles in the preparation and delivery of course material does not guarantee success; course assessment must also allow for the evaluation and, if necessary, modification of instructional design. Web-based tools can be invaluable in this process. More news from selected NECC exhibitors--listed alphabetically eSchool News staff, eSchool News, pp. 1-11, July 5, 2007. USA This is a roundup of news from the 2007 National Educational Computing Conference exhibit hall, listed alphabetically by company name. Readers can also find news and information grouped according to product categories. Products for education from approximately 70 of the 400 exhibiters are profiled with links to further information about their products or services. A special listing of Assessment and data management exhibitors is also available. Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education, pp. 1- 140, March 2007. USA This major study of the effectiveness of education technology, Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort was mandated by Congress. Using scientifically based research methods and control groups to focus on the impact of technology on student academic achievement the main findings of the study are: (1) test scores were not significantly higher in classrooms using the reading and mathematics software products than those in control classrooms, and (2) there was substantial variation between schools regarding the effects on student achievement. Thirty-three districts, 132 schools, 9, 424 students and 439 teachers participated in the study. Sixteen products were selected for the study based on public submissions and ratings by a study team and expert review panels. According to other media reports(another example) critics of this study contributed many of the problems to poor training and execution in the classroom and not the products themselves. The Effect of Using Item Parameters Calibrated from Paper Administrations in Computer Adaptive Test Administrations Mary Pommerich, Journal of Technology, Learning and Assessment, 5 (7), pp. 1-29, March 2007. USA For testing programs that utilize both computer and paper administrations, mode effects are problematic in that they can result in artificial inflation or deflation of student scores. The research generally seems to indicate that the more complicated it is to present or take a test on computer, the greater the possibility of mode effects. In a computer adaptive test, mode effects may be a particular concern. Pommerich studies the suitability of using parameters calibrated from a paper administration for item selection and scoring in a computer adaptive administration, for two tests with lengthy passages that required navigation in the computer administration. The results showed that the use of paper calibrated parameters versus computer calibrated parameters in computer adaptive administrations had small to moderate effects on the reliability of examinee scores, at fairly short test lengths. This effect was generally diminished for longer test lengths. However, the results suggest that in some cases, some loss in reliability might be inevitable if paper-calibrated parameters are used in computer adaptive administrations. Literature Review in Creativity: New Technologies and Learning Avril M. Loveless, Futurelab, Report 4, pp 1- 40, September 2002. UK The author states that this review is intended to provide: a sound theoretical and empirically informed basis for prototype development of digital learning resources to support the teaching and learning of creativity; a sound theoretical and empirically informed basis for informing policy on the teaching and learning of creativity, a basis for communication between the educational research community and the commercial sector on the subject of the teaching and learning of creativity with ICT. A short section focuses on assessment. CoSN Data-Driven Decision Making Presentations, 2003-2006 CoSN, 2003-2006. USA Over 20 PowerPoint presentations on topics such as :Vision to Know and Do: A Data-Driven Decision Making Partnership, Using Data for Continuous School Improvement, Using Data for Targeted Interventions, and Data Intelligence: Empowering Teachers, Administrators, Students & Parents are available on this site. Technology in Schools: What the Research Says Metri Group commissioned by Cisco Systems, pp.1 -18, 2006. USA The report states among its findings that extracting the full learning return from a technology investment requires much more than the mere introduction of technology with software and web resources aligned with the curriculum. It requires the triangulation of content, sound principles of learning, and high-quality teaching—all of which must be aligned with assessment and accountability. A useful feature of this report is the Trends in Research section which examine the impact of eight specific types or configurations of technologies (TECHtypes) on learning. A Summary of Research on the Effects of Test Accommodations: 1999 through 2001 Sandra Thompson, Amanda Blount, and Martha Thurlow, National Center on Educational Outcomes, Technical Report 34, pp.1-68, 2002. USA This report summarizes 46 empirical research studies on accommodations published from 1999 through 2001 whose primary purpose was to determine the effects of accommodations use on the large-scale test scores of students with disabilities. The tables at the end of this summary provide concise information about each of the studies. Cheating Online Student Assessment: Beyond Plagiarism Neil Rowe, Online Journal of Distance Learning Administration, 7 (2), Summer 2005. USA Though written for post-secondary this article is also applicable to K-12. Rowe identifies serious problems involving cheating in online assessment and suggests countermeasures. Some issues include: getting assessment answers in advance; students supplying questions and answers to those who take the test later, unfair retaking of assessments, unauthorized help during the assessment; and not being able to confirm the identity of the person actually taking the online test. Using Technology to Improve Student Achievement North Central Regional Educational Laboratory, pp. 1-23, updated 2005. USA This critical issues paper in examining the issue looks at the context of using technology including high stakes testing demands, and different types of technology; the research results, implications, factors to consider, action options and implementation pitfalls. The report states that the effectiveness of educational technology on student learning depends not only on what outcomes are targeted and how the technology is integrated into instruction, but also on how teachers assess student performance in classrooms and adjust instruction accordingly. Technology offers teachers a broad range of tools to collect and analyze data, and richer sets of student data to guide instructional decisions. High-Stakes Testing: Can Rapid Assessment Reduce the Pressure? Stuart Yeh, TC Record, pp. 1- 72, 2006. USA This article presents findings about the implementation of a system for rapidly assessing student progress in math and reading in grades K–12. Interviews with 49 teachers and administrators in one Texas school district suggest that the assessments allowed teachers to individualize and target instruction; provide more tutoring; reduce drill and practice; and improve student readiness for, and spend more time on, critical thinking activities, resulting in a more balanced curriculum. Teachers reported that the assessments provided a common point for discussion, increased collaboration among teachers to improve instruction and resolve instructional problems, and supported both new and experienced teachers in implementing sound teaching practices. The individualized curriculum and rapid feedback on progress reportedly gave students the feeling that they were successful and in control of their own learning, engaging students who previously disliked reading and math, reducing stress, and improving student achievement. These findings are interpreted through Corbett and Wilson's framework for understanding why high-stakes testing often has negative effects and why the implementation of rapid assessment systems could reduce unintended negative consequences of testing. Demystifying Assessment Leadership Brian Noonan and Patrick Renihan, Canadian Journal of Educational Administration and Policy, 56, November 4, 2006. Canada Several critical implications of assessment reform for the instructional leadership role of the principal are examined. From the research about the meanings and dynamics of effective assessment and instruction some of the related, hitherto unanticipated, expectations for leadership are identified. The authors frame these expectations in terms of sets of knowledge, appreciations and skills, which may serve as a focus for principal self-reflection and professional planning. The issue of the supports necessary for principals to be successful in what, for many, represents a formidable and ambiguous aspect of their instructional leadership role is also detailed. Technology and Student Achievement Nancy Protheroe, NAESP (National Association of Elementary School Principals), Principal- Effective Intervention- Research Report, 85 (2), pp. 46-48, November/December 2005. USA This paper addresses the question - Is there evidence that using technology leads to higher levels of student learning?î Protheroe believes that the need for an answer has become more urgent in recent years due to two key factors—an emphasis on standards-based accountability and the substantial financial, instructional, and organizational costs—involved in purchasing and implementing technology. An overview of the research is presented as are a section on what other questions should be asked. » Return to Top   » Next: B. Classroom/ School Level Assessment