Regents Exams: Mathematics – Item Criteria Checklist | EngageNY

Regents Exams: Mathematics – Item Criteria Checklist

Students taking exam

Review Criteria Checklist for Potential Math Items

With the transition to the New York P-12 Common Core State Standards (CCSS), significant changes have been made to the New York State Testing Program (NYSTP).  The 2014 Regents Examination in Algebra I (Common Core) will measure the CCSS for Mathematics. This test will approach Mathematics differently from past assessments. To help ensure consistent and rigorous interpretation and measurement of the CCSS for Mathematics, the state has articulated criteria for reviewing test questions (items).

Item Review Criteria
The Item Review Criteria framework helps assessment developers and reviewers ensure that the Common Core Regents Exam in mathematics measures the CCSS for Mathematics with high-quality questions.  The Item Review Criteria articulate the four major item characteristics NYSED looks for in developing quality questions.  NYSED uses the Item Review Criteria to ensure:

  1. Language and Graphical Appropriateness
  2. Sensitivity/Bias
  3. Fidelity of Measurement to CCSS (Item Alignment)
  4. Conformity to the Expectations for the Specific Item Types and Formats (i.e., multiple-choice questions, 2-point constructed-response questions, 4-point constructed-response questions, and 6-point constructed-response questions).

The Item Review Criteria are organized into these four categories.  Each section includes pertinent questions that help reviewers determine whether or not an item is of sufficient quality.  Within the first two categories, the headings Language Appropriateness, Sensitivity/Bias, and Math Art identify the basic components of quality assessment items. The criteria for Language Appropriateness are used to help ensure that students understand what is asked in each question and that the language in the question does not adversely affect a student’s ability to perform the required task.  Likewise, the Sensitivity/Bias criteria are used to evaluate whether questions are unbiased, non-offensive, and not disadvantageous to any given subgroup(s). The Math Art criteria assess the appropriateness and clarity when graphics are used within questions.

The third category of the Item Review Criteria framework, Item Alignment, addresses how each item measures a given Mathematics standard. This criterion asks the reviewer to comment on key aspects of how the item addresses and calls for the skills demanded by the standards.  Additionally, these criteria prompt reviewers to comment on how more than one standard is addressed by a given question.

The fourth category of the Item Review Criteria framework addresses the specific demands for different item types and formats.  Reviewers evaluate each item to ensure that it conforms to the given requirements.  For example, multiple-choice items must have, among other characteristics, one unambiguous correct answer and plausible but incorrect answer choices.

How the Item Review Criteria Are Used
Using these criteria to review draft questions is one of many steps employed to help ensure a valid, fair, and quality assessment. Draft questions that meet these criteria are allowed to move forward in the development process. The next step is for the items to be reviewed, and edited when necessary, by a Committee of certified New York State educators [1].  The educator panel evaluates each question for fairness, valid measurement of the intended standard(s), and language clarity.  Only items that are approved by the educator panel are allowed to be field-tested.

[1] To volunteer to participate in NYSED Educator Item Reviews: http://www.p12.nysed.gov/assessment/teacher/home.html#teacher-op

How Educators Can Use the Item Review Criteria to Design Assessments or Modify Instruction
The Item Review Criteria can also serve as helpful guidelines as educators develop assessment items and instructional activities.  Teachers will want to consider:

  1. Language and Graphical Appropriateness
  2. Sensitivity/Bias
  3. Fidelity to the precise language and structure of the CCSS
    1. In order to ensure the assessment items and instructional activities align with the CCSS, they must adhere to the following:
      1. The learning targets for a lesson are simply the standards themselves or portions of the standards themselves.
      2. The assessment question prompts/instructional materials use language from the actual standard(s) or the alignment is easily discernible.
      3. The learning experience directly informs students' understanding/the item requires students to show understanding of aspects of the particular standard(s).
      4. The item/learning experience does not require students to attend to all aspects of the standard(s).
      5. The item/learning experience is dependent on the students' ability to access skills referenced in the primary standard as well as the secondary and additional standards.
      6. The item/learning experience measures/teaches students to show understanding of the skills in the standard. The item/learning experience includes nonstandard numbers (e.g., students are asked to solve questions using non-whole numbers).
  4. Mirroring in item/lesson design to the "Expectations for the Specific Item Types and Formats" (i.e., multiple-choice questions, 2-point constructed-response questions, 4-point constructed-response questions, and 6-point constructed-response questions).

Downloadable Resources

Resources may contain links to sites external to the EngageNY.org website. These sites may not be within the jurisdiction of NYSED and in such cases NYSED is not responsible for its content.