315 Ramseyer Hall, 29 West Woodruff, Columbus, OH 43210, (614) 292-3239

This project is designed to help teachers use proficiency data to inform instructional decisions. The learning activities in the proposal are specifically directed at the teachers in the Springfield City Schools district. However these workshops and experiences are designed to benefit all the schools in the Urban Initiative. The workshops and learning activities are designed to be incorporated in the Springfield City Schools district CIP plan.

Attachments:

State mandated testing programs in the United States have blossomed over time from only one program in the state of New York decades ago to over forty state mandated testing programs across the country (Jaeger, 1991; Stake, 1991). The state imposed programs differ in content areas tested, formats, grade levels tested, and testing consequences, but are similar in that the data are utilized by policy makers for accountability purposes as well as resource allocation. Primarily, the legislatures behind state testing programs have asserted that they serve the surrounding society by acting as an accountability mechanism (Mehrens and Kaminski, 1989; Darling-Hammond, 1991; Stake, 1991; Mehrens, 1992). However, historically, state testing programs for primary and secondary students across the United States were designed to sort students based upon proficiency in core subject areas and then used to classify students into specific classroom level groups based upon those levels (Stiggins, 1994). As the accountability movement grew, the proficient classifications and labels were later carried beyond school building walls to placing a person in a certain level of society. Many students without high school diploma’s were left in menial labor work force categories when more than half of the country’s testing programs began utilizing student pass scores as a graduation requirement (Cattrel, 1991; Cooley, 1991; Darling-Hammond, 1991; Jaeger, 1991; Robinson and Wronkovich; 1991; Stake, 1991; Lanese, 1992; Birkmire, 1993;). Finally, personnel at the state level have used the test data to identify student needs. Student needs have then been used to develop educational goals, guidelines, and remediation activities based upon those needs. Nevertheless, the latter use of the testing data, while more fundamental to good education, has proved more illusive in the classroom reality of the testing programs than classification or accountability (Darling-Hammond, 1991; Jaeger, 1991; Robinson and Wronkovich, 1991; Birkmire, 1993).

Whether standardized test data has been utilized appropriately or not, over time, teacher professional judgements concerning student achievement have been pushed aside by society and the media in favor of standardized test results. Educator professional judgements have become anecdotal data rather than "hard data" when considering standardized test results (Bracey, 1987; Darling-Hammond, 1991; Smith, 1991; Stake, 1991; Birkmire, 1993). Research suggests that educator perceptions as to the validity of standardized examination outcomes has come into question as a result of the minimization of their professional judgements (Dorr-Bremme and Herman, 1986; Bracey, 1987; Haney and Madaus, 1989; Darling-Hammond, 1991; Herman and Golan, 1991; Jaeger, 1991; Smith, 1991; Smith and Rottenberg, 1991; Stake, 1991; Lanese, 1992; Birkmire, 1993). Specifically, Stake (1991) found that teacher perceptions concerning standardized test data are primarily negative. Nonetheless, he did discuss that teachers conceded that the standardized tests and their results could be useful for "orienting teachers and students to a common curriculum" (Stake, 1991, p.245).

Regardless of data utilization or teacher perceptions, the policy decision-making power and subsequent direct and indirect consequences of these examinations have turned state imposed testing across the United States into what is commonly referred to as "high stakes" testing (Mehrens and Kaminski, 1989; Darling-Hammond, 1991; Cattrel, 1991; Cooley, 1991; Jaeger, 1991; Stake, 1991; Mehrens, 1992; Popham, 1992; Birkmire, 1993). While the stakes or consequences of these examinations and their test results are high for all students, teachers, and schools, they are exacerbated in the urban setting. These school settings have had a history of being considered low-achieving. Contributing factors include the majority of children coming from low socioeconomic settings, from a multitude of ethnic backgrounds, and displaying general at-risk behaviors (Freel, 1998; Scheurich, 1998). Current research has identified wide gaps between non-urban and urban school reading, math, and science standardized test scores (Olson, 1997; Borelli &Maxfield, 1998; Freel, 1998; Hendrie, 1998; Jerald & Olson, 1998). Further, the trend nationally, has shown an increasing disparity, in student achievement scores in non-urban versus urban settings (Freel, 1998). "High stakes" testing for all of Ohio’s students, urban and non-urban, has held a place in Ohio's educational culture since 1993 when the State Board of Education adopted a resolution recommending that the General Assembly enact legislation to require students to pass proficiency tests in order to graduate from high school. The initial legislation, House Bill 231, and subsequent legislation, House Bill 55 required that 4^th, 9^th, and 12^th graders, from 1994 on, take proficiency examinations in reading, writing, math, and citizenship. Since 1994, the stakes for Ohio districts, administrators, teachers, parents, and students alike have increased as grade levels (6^th) and subject areas (science) have been added, the consequences of failure have grown (i.e., graduation requirement; retention in the 4^th grade), and the introduction of the state report card in 1997 with sixteen out of the eighteen indicators being proficiency results. Beginning with the report card to be issued in the Winter of 2000, proficiency results will be included in twenty-five out of the twenty-seven indicators.

The Ohio urban district of Springfield City School District has witnessed the pressure and the increasing stakes of the proficiency tests in all five areas and across all four tested grade levels. The district which is responsible for educating almost 11,000 students, met only one of the state performance standards in the 1997-98 academic year (12^th grade writing) and showed lower proficiency performance in every subject area at all four grade levels than similar districts in the state. The district reacted by sharing with teachers and administrators the test data and explaining their meaning in detail. However, as Stake (1991) suggested, the educators may not respect the data (p.245). Subsequently, the intended use of data as an information tool for improving achievement has resulted in upward student performance overtime, but there is yet a substantial difference. The proficiency data obtained from Springfield City Schools district during the 1998-1999 will be used to create the report card that will be issued in the Winter of 2000. It is very likely the Winter 2000 "results" would place the district in a state of academic emergency. However, there is reason for some optimism as the district has demonstrated a sizable jump with a 10% increase in the mathematics’ passing rate during the past year. Nevertheless, given that the district would need to improve both quickly and incrementally, there is reason to be skeptical that in the near future the district could raise itself to the level of academic watch unless significant professional practices are changed. The goal of this proposal is to aid Springfield City Schools in beginning to make those changes and increase student performance. Furthermore, the ideas presented in this proposal could easily be incorporated into the district’s Continuous Improvement Plan (CIP) Plan.

The challenge for the Springfield City School district is twofold. First, the teachers and administrators must come to value the information that the test data embodies. The second challenge lies in the ability to take the information from the proficiency examinations and turn them into both instructional and curricular changes that will ultimately improve student learning and achievement. The remedy for these barriers is identified in educating teachers in data and assessment utilization, in order for them to inform their instruction. The Springfield teachers can learn to understand, value, and use student data as measures to set goals, lobby for curricular changes, and make daily instructional modifications and enhancements that respond to the goals and data.

The consequences of poor academic achievement must be avoided as effects are debilitating and harm not only the student but the surrounding society and economy (Darling-Hammond, 1991; National Commission on Excellence in Education, 1983; National Education Goals Panel; 1997). Standardized, tests such as Ohio’s Proficiency examinations were developed to describe student academic achievement levels in terms of specific subject areas and educational goals. Achievement can continue to be increased by using student performance data to inform instruction through identifying student strengths and weaknesses and developing educational goals and remediation activities (Darling-Hammond, 1991; Jaeger, 1991; Robinson and Wronkovich, 1991; Birkmire, 1993). A teacher’s ability to use assessment data for this purpose is primary to effective, efficient, productive instruction (Stiggins, 1994). Only through effective, informed instruction and curriculum development can student achievement continue to improve in the Springfield district. Finally, the overarching goal of teaching teachers how to use data to inform instruction is primary as the literature suggests that teachers are challenged with assessing students effectively (Walberg, 1984; Skinner, Wellborn & Connell, 1990; Stiggins, 1994; Thomas, 1998).

This project is designed to help the Springfield teachers to better understand, value and use assessment data to improve instructional decisions and positively impact student achievement. More specifically, the project objectives are designed:

1. To analyze, at more refined levels, the Springfield City Schools Proficiency and Performance Standard Data in order to identify specific opportunities for improvement;

2. To design and engage in multi-contact assessment and data usage workshops for teachers and administrators at 4 Springfield City Middle Schools and 2 Springfield City High Schools;

3. To ensure that Springfield City School educators better understand and continue to use the basic concepts of assessment, large-scale and classroom, and their impact on curriculum and instruction;

4. To have educators better understand and describe the relationship between achievement level diagnosis, classroom assessment, instructional evaluation, and curricular change;

5. To ensure that educators gain increased skills in interpreting standardized assessments;

6. To ensure that educators gain additional skills in using assessment results to make decisions about individual students, planning instruction and classroom activities, developing the curriculum and choosing instructional texts and aids, and for general school improvement;

7. To have educators examine the 1998-99 Springfield City School Proficiency data, at a refined level, for implications in designing, planning, and implementing lessons;

8. To have the Principal Investigator provide a final report on the activities with feedback and recommendations for future professional development activities and procedures to facilitate ongoing improvement in educator competence in the educational assessment of students.

9. To demonstrate a process for using data to inform instructional plans as a part of the districts development of a CIP plan.

 

The purpose of this improvement project is to ultimately improve student performance, specifically in terms of proficiency results, in the Ohio urban district of Springfield City Schools. The first procedure will be to review refined levels of building level proficiency and performance indicator data for the four middle and two high schools. The proficiency subject areas in relation to district and state level peers and will be analyzed for growth over time. Second, the building specific administrators and district personnel will be convened to determine perceived challenges or barriers to educator appreciation of proficiency data. The building administrators will then be asked to provide one to two page qualitative accounts of how they currently use the proficiency data. Specific building level opportunities will be identified and incorporated into the professional development workshop scenarios, examples, and activities. A planning team will be assembled for each participating school to serve as the first point of contact between the improvement project initiative and the rest of the staff in district school buildings.

The workshops will be conducted in multiple sessions in order to form both a relationship with educators as well as provide an opportunity for sustained learning and impact. These sessions will be conducted in sessions on school specific calendar release times. The introductory session will provide educators the opportunity to continue to improve their understanding of refined levels of proficiency results for individual students and in the aggregate and how lesson plans, instructional activities, and curricular improvements can result from such understanding and analysis. School specific scenarios gained from the initial data gathering as well guidelines and procedures from Stiggins (1994) and Standards for Teacher Competence in Educational Assessment of Students (1990) developed by the American Federation of Teachers, National Council on Measurement in Education and the National Education Association will provide session learning materials. Additionally, the sessions will utilize some materials for a graduate level distance learning course on collecting and analyzing data to inform instruction that the Principal Investigator designed and taught. These workshops could be utilized to provide participants with graduate level credit if the district supports this as a method for increased participation. Further, the possibility of increasing participation through distance learning has also been raised with Dr. Lambert.

The educators will be asked to do instructional activities. The instructional activities will be used to analyze the previous academic year’s proficiency results in conjunction with other subject specific student performance measures to make curricular recommendations as well as an outline of lesson plan or instructional activity changes or improvements that they might make based upon the data. These assignments will be reviewed and analyzed prior to closing sessions.

The closing sessions will focus on long term professional behavioral changes for the educators. The professional practice improvements will include data based goal setting and quarterly review and revision of instructional plans based upon those goals. The closing session will include a professional reflection by the educators. The session will ask the educators to reflect upon how they will include daily classroom level data based decision making and instructional planning and ask what building, district and community level support will be necessary to promote these practices.

Finally, the information from the instructional plans, the reflections, course feedback, and instructor observation will be organized, analyzed , and presented to the participants. A report will be prepared that offers building level specific and more generic district level information concerning the future of educator data based decision making and subsequent improvements in student achievement as measured and tracked by the Proficiency examinations.

Quarterly OSU/USI Committee status reports

Minutes of meetings with the Springfield district and building level administrators

Outlines of workshop session curricula

Analysis of Performance Standard Data in relation to educator activity data

A final report with analyses, discussion, and recommendations for future professional development activities

An Executive Summary of the final report

Disk(s) of the Executive Summary and Final Report

In an effort to manage and maintain the project, the Principal Investigator has developed a year long timetable. The specific objectives of the project and anticipated deliverable products are organized around specific completion dates. These objectives and products and their corresponding target dates are subject to changes and modification based upon the needs of the OSU/USI Committee, the Consultation Service research team, and Springfield leadership. The objectives and their target completion dates follow:

We, the research and analysis team, will conduct the project in accordance with the canons of sound and ethical educational research as specified in the Standards of Educational Evaluation and as codified through the Standard Research Practices of the American Psychological Association. Following completion of the project, we will provide the OSU/USI Committee with a complete report, ready for reproduction on a computer readable disk(s). The data will become the property of the OSU/USI Committee. However, we retain the right to publish the results in mutually agreeable professional outlet(s). We are agreeable to negotiate and revise this contract in ways that are mutually agreeable to the OSU/USI Committee and the Principal Investigator.

The experiences of the Springfield teachers must be shared. First, subsequent academic years should bring these workshops with Springfield feedback on opportunities for improvement incorporated to the other Central Ohio urban districts of Columbus-The University Districts, Mansfield City Schools, and South-Western City Schools. All three of these districts would benefit from such intense and directed workshop experiences. Further, as districts were added to those with the knowledge and skills to turn data into instructional changes, they could share experiences by e-mail or in small work groups for purposes of professional development and skill enhancement.

Nevertheless, the real value of these workshop activities may be realized in subsequent academic years in increased student performance and rich research opportunities. The first year after these learning activities have taken place should be a time of teacher reflection and administrator observation. This research team through proficiency data analysis, journal analysis and administrator surveys and interviews could capture these reflections and observations. Teams of teachers who share and analyze achievement data and make subsequent instructional changes and improvements should chronicle the teacher reflections. The journal observations should tell a story of instructional and procedural changes that the teachers made based upon data. Additionally, the chronicles should provide the readers with a sense of how the teachers feel about their changes in terms of impact. The administrators should be observing the entire academic year. The building and district administrators should be able to convey whether or not they see positive changes both large and small. These multiple data sources could then be analyzed, reported on, and presented to the OSU/USI Committee in order to further district learning and skill development in using data to improve student achievement.

Rank or Title

Professor; Coordinator of Measurement, Evaluation, and Research in Teacher Education; Quantitative Research, Evaluation, and Measurement in Education Section; Director of Research, Evaluation, and Data Analysis Consultation Service in College of Education

School of Educational Policy and Leadership

College of Education

The Ohio State University

Quantitative Research, Evaluation and Measurement in Education

Office:

315D Ramseyer Hall

29 W. Woodruff Avenue

Columbus, Ohio 43210-1177

E-Mail: loadman.1@osu.edu

Office:

(614) 292-8055

FAX:

(614) 292-7020

Applied Research and Evaluation Methodology with Content Focus on Assessment of Students and Programs to Achieve Significant Educational Reform. Instrument Development and Analysis. Alternative Assessment Strategies. Survey Research Methodology. Data Analysis and Reporting

1971 Michigan State University, Ph.D., Measurement, Evaluation, Statistics, and Research Design in Educational Psychology

1966 Miami University (Ohio), School Psychology

1965 A.B. Thiel College, Psychology, Mathematics

 

1997-present The Ohio State University, Professor, Quantitative Research, Evaluation, and Measurement in Education

1988-1997 The Ohio State University, Professor, Educational Services and Research

1982-1987 The Ohio State University, Associate Professor, Educational Theory and Practice

1974-1982 The Ohio State University, Associate Professor, Educational Foundations and Research; Director of Research and Development at Nisonger Center for Mental Retardation and Developmental Disabilities

1972-1974 The Ohio State University, Assistant Professor; Director of Evaluation at Nisonger Center for Mental Retardation and Developmental Disabilities

1970-1972 Indiana University, Assistant Professor, Member of Graduate Faculty; Taught advanced quantitative courses in College of Education

Numerous national, state and local consultancies dealing with applied research and evaluation methodology in systemic educational reform.

Member of Ohio Venture Capital* Assessment Team; Site-Visited more than 60 Schools over the last three years to Assess Progress and make Recommendations for Improvement; Responsible for all Data Collection and Analyses of the Venture Capital Data including Instrument Development, Survey Sampling Design, Data Analyses and Program Reporting

Conducted Training for Readers of Ohio Venture Capital Proposal Readers and Site Visitors for Past Four Years and Analyzed Scored Applications

Conducted Training for Readers of Ohio School-to-Work Proposal Applications for Past Three Years and Conducted Analyses of Scored Applications

Conducted Training for Readers of Ohio SchoolNet Proposal Applications and Analyzed Scored Applications for Past Three Years

Conducted Training for Readers of Ohio BEST Practices Proposal Applications and Analyzed Scored Applications for the Past Four years

Primary Consultant on Continuous Learning Planning (CLP) Activities and Continuous Improvement Planning (CIP) Activities with Ohio Department of Education

Primary Consultant and Trainer for Network for Systemic Initiative (NSI) Activities for the Ohio Department of Education

Primary Presenter for State-Wide Training on the Collection and Use of Data for Educational Decision-Making for Ohio Safe and Drug-Free School Program (five regional sessions)

Primary Presenter at State-Wide Data Based Decision Making Workshops for Participants from Venture Capital Schools (13 regional workshops over two years)

Member of ODE Advisory Board on Cleveland Voucher Program

Survey Research consultant with SchoolNet on state-wide survey of all school districts, buildings and teachers on computer access and use in public school education

*Venture Capital is a state supported program to develop and implement systemic educational reform at the building level. The program began in Ohio in 1994 and the state has funded 600 school buildings in the state to engage in systemic educational reform. Each building receives $25,000 per year for five years to engage in this initiative.

Funded Proposals

Director or co-director of 35 state, federal or private foundation grants with total value in excess of $1,000,000. Project research design, statistician or project evaluator responsibilities held in more than 100 projects.

 

More than 50 publications in professional journals; 6 chapters in books; more than 100 professional papers, and more than 100 regional, national and international professional presentations. References available upon request.

 

REFERENCES

Andrew, M.D. (1990). Differences between graduates of 4-year and 5-year teacher education programs. Journal of Teacher Education. 41 (2), 45-51.

Birkmire, K.J. (1993). Ninth grade proficiency, is it the best measure of learning: A survey of teacher attitudes. A U.S. Department of Education Document.

Borelli, J.G. & Maxfield, R.M. Improving academic performance on a shoestring budget. Schools in the Middle. 7 (5), 36-37.

Bracey, G. (1992). Measurement-driven instruction: catchy phrase, dangerous practice. Phi Delta Kappan.68 (9), 683-686.

Catteral, J. (1991). A reform cooled out: competency testing required for high school graduation. The High School Journal. 75 (1), 7-16.

Comer, J. & Maholmes,V. (1999). Creating schools of child development and education in the USA: Teacher preparation for urban schools. Journal of Education for Teaching. 25 (1), 3-15.

Cooley, J.P. (1991). State-wide student assessment. Educational Measurement: Issues and Practice. 10 (4), 3-6, 15.

Darling-Hammond, L. (1991). The implications of testing policy for quality and equality. Phi Delta Kappan. 73 (3), 220-225.

Dorr-Bremme, D. & Herman, J. (1986). Assessing student achievement: A profile of classroom practices. A Center for the Study of Evaluation Document. Los Angeles, CA.

Freel, A. (1998). Achievement in urban schools: What makes the difference? The Education Digest. 64 (1), 17-22.

Haney, W. & Madaus, G. (1989). Searching for alternatives to standardized tests: whys, whats and whithers. Phi Delta Kappan. 70 (9), 683-687.

Hendrie, C. (1998). Study links course-taking with urban achievement. Education Week. 17 (Jan 21), p.5.

Herman, J.L. & Golan, S. (1993). The effects of standardized testing on teaching and schools. Educational Measurement: Issues and Practice. 12 (2), 20-26.

Jaeger, R.M. (1991). Legislative perspectives on statewide testing: goals, hopes and desires. Phi Delta Kappan. 73 (3), 239-242.

Jerald. C.D., Olson, L. (1998). The achievement gap. Education Week. 17 (Jan., 8), 10-13.

Lanese, J.F. (1992). Statewide proficiency testing: establishing standards or barriers. ERIC Document 347 196.

Mehrens, W.A. & Kaminski, J. (1989). Methods for improving standardized test scores: fruitful, fruitless, or fraudulent? Educational Measurement: Issues and Practice. 8 (1), 14-22.

Mehrens, W.A. (1992). Using performance assessment for accountability purposes. Educational Measurement: Issues and Practice.11 (1), 3-9.

National Commission on Excellence in Education. (1983). A Nation at Risk: The Imperative for Educational Reform. Washington, D.C.: Government Printing Office.

National Education Goals Panel. (1997). The National Education Goals Report: Building a Nation of Learners. Washington, D.C.: Government Printing Office.

Olson, L. (1997). New study finds low achievement in city schools. Education Week. 17 (Dec., 10),

p.5.

Popham, W.J. (1987). The merits of measurement-driven instruction. Phi Delta Kappan. 69 (9), 679-682.

Popham, W.J. (1991). Appropriateness of teachers' test preparation practices. Educational Measurement: Issues and Practice. 8 (1), 14-22.

Popham, W.J. (1992). The perils of responsibility sharing. Educational Measurement: Issues and Practice. 8 (1), 14-22.

Robinson, J. & Wronkovich, M. (1991). Proficiency testing in Ohio. American Secondary Education. 20 (2), 10-15.

Scheaffer, R.L., Mendenhall, W., & Ott, R.L. (1996). Elementary Survey Sampling. Belmont CA: Wadsworth Publishing Co.

Scheurich, J.J. (1998). Highly successful and loving public elementary schools populated mainly by low-SES children of color: Core beliefs and cultural characteristics. Urban Education. 33 (4), 451-491.

Skinner, E.A., Wellborn, J.G. & Connell, J.P. (1990). What it takes to do well in school and whether I’ve got it: A process model of perceived control and children’s engagement and achievement in school. Journal of Educational Psychology. 82, 22-32.

Smith, M.L. (1991). Put to the test: The effects of external testing on teachers. Educational Researcher. 20 (5), 8-11.

Smith, M.L. & Rottenberg, C. (1991). Unintended consequences of external testing in elementary schools. Educational Measurement: Issues and Practice. 10 (4), 7-11.

Stake, R. (1991). The teacher, the standardized test and prospects of a revolution. Phi Delta Kappan. 72 (11), 243-247.

Stiggins, R. (1994). Student-Centered Classroom Assessment. Ontario: Macmillan College Publishing Co.

Strauss, A. & Corbin, J. (1990). Basics of Qualitative Research: Grounded Theory, Procedures and Techniques. Newbury Park, CA: Sage Publications.

Thomas, A.M. (1998). A study of teacher preparation program efficacy at the Ohio State University.

Dissertation. Columbus, OH: The Ohio State University.

Walberg, H.J. (1984). Improving the productivity of America’s schools. Educational Leadership. 41 (8), 19-30.

-------------

Letter Re: Continuation of the Project

-----------

Share Ideas with Discipline Partner (Revise As Necessary)

Local Team Discussion/Sharing/Revision

Team I & II Share with Each Other

-----------

Selecting Targeted Outcome (s)

Identify Instructional Strategy

Work on Selected Outcome(s) in Teams

Identify Desired Outcome (s)

Identify Instructional Strategy

Create Assessment Linking All Three Elements

Share Ideas with Discipline Partner (Revise As Necessary)

Local Team Discussion/Sharing/Revision

Team I & II Share with Each Other

ODE Website - Practice Access

Analysis of Springfield Proficiency Data

Item Analyses

Longitudinal and Subscale Passing Rates

3:30 CONCLUDING COMMENTS

-----------------

------------------

Attachment D

----------------

------------

Attachment F

-------------

Attachment G

June 20 and 21 for High Schools

June 27 and 28 for Middle Schools

Combined Results