National Center for Education Statistics/National Assessment of Educational Progress (NCES/NAEP)

The National Assessment for Educational Progress (NAEP) is a critical national instrument in assessment and educational reform. In support of NCES/NAEP, CRP manages a comprehensive program of work in collaboration with senior NCES staff members and NCES/NAEP contractors to include ETS, AIR, HumRRO, Fulcrum, and Westat. CRP also manages logistical requirements for NCES/NAEP meetings and expert panels to include providing an array of national expert consultants who offer editorial reviews, and comments on NCES reports, and who serve as technical reviewers, writing briefings on educational trends and topics.

NAEP - EARLY CHILDHOOD LONGITUDINAL STUDY (ECLS) COGNITIVE INTERVIEWS AND REPORT

CRP conducted cognitive interviews (often referred to as a cognitive laboratory study or cog lab) for the NAEP-ECLS (Early Childhood Longitudinal Study). The survey questionnaire was designed to validate student responses to socioeconomic-status-related questions by comparing them to parent responses from NAEP-ECLS. The cognitive interviews were conducted with fourth and fifth-grade students recruited from the metropolitan Washington, DC area. CRP engaged students from a range of demographic groups with regard to gender, race/ethnicity, socioeconomic background, and urban and suburban settings.
 

NAEP PRETESTING OF SURVEY QUESTIONS FOR CORE, READING, WRITING, AND MATHEMATICS QUESTIONNAIRES - COGNITIVE INTERVIEWS

CRP recruited and conducted 86 cognitive interviews of principals, teachers, and fourth, eighth, and twelfth-grade students in the Washington, DC metropolitan area to include Alexandria, Arlington, and the counties of Fairfax, Prince George's, and Montgomery. The cognitive interviews supported the development and validity of newly developed survey questions for the NAEP assessment, particularly informing which questionnaire items were used during the NAEP administrations. To meet the timeline for this aggressive schedule, CRP trained approximately 15 doctoral students and retired educators in cognitive interviewing techniques and procedures.
 

NAEP FOCUS GROUPS

CRP conducted 14 focus groups (combination of virtual and face-to-face) composed of 4th and 8th-grade administrators, reading and math teachers, and students. NAEP commissioned the focus group research to better understand the following constructs in the context of reading and mathematics:

(a) familiarity with technology, especially as it relates to mathematics and reading (e.g., e-readers, tablets, software, apps, and Internet resources); 
(b) access to technology relevant for learning at home and at school, including patterns of access across student groups; 
(c) ways in which technology (especially tablets, smartphones, and Internet resources) changes students' experiences with text, numbers, and calculations; 
(d) how the use of technology facilitates instruction and learning, both in and outside of the classroom; and
(e) shifts in the nature and type of skills required of students as they read and/or do math in technology-rich environments.

Eight (8) states were randomly selected for the recruitment of administrators and teachers for the virtual focus groups. CRP chose the largest population center of each state (urban/suburban) and a random rural population area.