Development Process

Each item bank will be designed by program area to align with the curriculum by cluster, unit/course, competency and DOK level. Item banks already validated with Mississippi students will result in cost efficiency for the state.

Each operational MS-CPAS2 form will have field-test items embedded so that they cannot be distinguished from the operational items. For programs with 40 or more completers statewide, tests will consist of 80 operational items and 20 field-test items. All other assessments will have 40 operational items and 10 field-test items. Postsecondary tests that are denoted with a Year 1 and Year 2 test will also fall into this category. The purpose of field-testing items with each test administration is to ensure that there are enough items to add the capability to offer formative assessments that can be used during the school year. This is also a cost-efficiency measure to ensure that additional cost is not accrued to field-test items.

The lead content-area Instructional Design Specialists and development team members develop test specifications or revise existing specifications as required:

• The units into which the assessment will be divided are determined by examining the curriculum course outline.
• The total number of technical core hours in the curriculum is calculated using the course outline from the curriculum.
• The percentage for each unit is calculated by adding the unit/course hours in each curriculum to get the total hours for that curriculum. The total number of unit hours for each unit iis divided by the total number of technical core hours in the curriculum to get the percentage of items needed for the test from each unit.

A study will be conducted to compare actual student scores with a teacher-reported rating scale. The teachers will rate students by current classroom performance or grades. A biserial correlation will be used to analyze the data that will be reported by program area and district and will be included in the state report. These data will be used to determine if there is a problem with an item on the assessment or in the curriculum or if there is a problem with the grading practices in the schools.

Reliability is an indication of the consistency of scores internally or over time. An assessment is considered reliable when consistent results occur regardless of when the assessment is given or who does the scoring. To show that results are consistent internally and across scoring occasions, the MAC will determine reliability using the Kuder-Richardson Formula 20 or Cronbach’s Alpha in SPSS. These statistics measure test reliability of inter-item consistency. A higher value indicates a strong relationship between items on the test. A lower value indicates a weaker relationship between test items. Values range from 0 to 1. The better tests are within the .80 to .85 range.

All test items are designed at approximately an eighth-grade reading level. The tests are designed to assess the appropriate occupational skill attainment rather than the student’s ability to read. Appropriate accommodations may be used to ensure that the occupational skill is measured.