Development Process

The Mississippi Career Planning and Assessment System requires item banks and assessment blueprints to be developed and redesigned when necessary. Blueprints and/or item banks are redeveloped when the needs of the stakeholders on the postsecondary or secondary level change. This change could be due to a change in curriculum, national standards or diploma option.

Item Bank
Blueprint
Criterion-Referenced Validity
Reliability
Reading Level

Item bank

Each item bank will be designed by the program area to align with the curriculum by cluster, unit/course, competency and DOK level. Item banks already validated with Mississippi students will result in cost efficiency for the state.

Each operational MS-CPAS2 form will have field-test items embedded so that they cannot be distinguished from the operational items. Secondary career pathways with curricula developed after 2007 have two assessments consisting of 80 operational items and 20 field-test items. Students in these pathways will receive an assessment after they complete each year of the career pathway. Secondary career pathways with curricula developed prior to 2007 have assessments consisisting of 70 operational items and 30 field-test items. Students in these pathways will receive an assessment after they complete both years of the career pathway. Please review the program's blueprints for more information.

Postsecondary programs either have an assessment consisting of 80 operational items and 20 field-test items delivered at the end of the technical certficate/AAS completion or two assessments with one assessment consisting of 40 operational items and 10 field-test items delivered after the career certificate completion and the other assessment delivered after technical certificate completion. Postsecondary assessments that are denoted with a Year 1 and Year 2 test will also fall into this latter category. Please review the program's blueprints for more information.

The purpose of field-testing items with each test administration is to ensure that there are enough items to eventually add the capability to offer formative assessments that can be used during the school year. This is also a cost-efficiency measure to ensure that additional cost is not accrued to field-test items.

Blue Print

The lead content-area Instructional Design Specialists and development team members develop test specifications or revise existing specifications as required:

  • The units into which the assessment will be divided are determined by examining the curriculum course outline.
  • The total number of technical core hours in the curriculum is calculated using the course outline from the curriculum.
  • The percentage for each unit is calculated by adding the unit/course hours in each curriculum to get the total hours for that curriculum. The total number of unit hours for each unit is divided by the total number of technical core hours in the curriculum to get the percentage of items needed for the test from each unit.

Criterion-Referenced Validity

A study will be conducted to compare actual student scores with a teacher-reported rating scale. The teachers will rate students by current classroom performance or grades. A biserial correlation will be used to analyze the data that will be reported by program area and district and will be included in the state report. These data will be used to determine if there is a problem with an item on the assessment or in the curriculum or if there is a problem with the grading practices in the schools.

Reliability

Reliability is an indication of the consistency of scores internally or over time. An assessment is considered reliable when consistent results occur regardless of when the assessment is given or who does the scoring. To show that results are consistent internally and across scoring occasions, the MAC will determine reliability using Cronbach’s Alpha in SPSS. These statistics measure test reliability of inter-item consistency. A higher value indicates a strong relationship between items on the test. A lower value indicates a weaker relationship between test items. Values range from 0 to 1. The better assessments are within the 0.80 to 0.85 range.

Reading Level

All test items are designed at approximately an eighth-grade reading level. The tests are designed to assess the appropriate occupational skill attainment rather than the student’s ability to read. Appropriate accommodations may be used to ensure that the occupational skill is measured.