Early Childhood Research & Practice is in the process of moving to the early childhood special education program at Loyola University Chicago after 17 years at the University of Illinois at Urbana-Champaign. We are delighted by the opportunity to “pass the torch” to our Loyola early childhood colleagues.

We suggest you visit ECRP’s Facebook page for future updates.

HomeJournal ContentsIssue Contents
Volume 5 Number 2
©The Author(s) 2003

Discriminant Validity of a Community-Level Measure of Children's Readiness for School

David A. Murphey
State of Vermont, Agency of Human Services

Abstract

Discriminant validity of a measure of children's readiness for school was assessed by testing whether it distinguished between groups of children hypothesized to differ, on the basis of demographic characteristics, in their school readiness. The volunteer sample consisted of 3,370 kindergartners in Vermont public schools. Previous group care experience, identified learning-related disabilities, and a community-level measure of poverty were used as independent measures in analyses of children's readiness scores in four conceptual domains. Statistically significant relationships were found between prior group care experience and readiness in each of the domains, and between disability status and readiness in each of the domains. In addition, there were significant negative correlations between community-level poverty and community-level readiness in three of the domains. Results are discussed in terms of their value in establishing credibility for a school readiness measure that can serve to inform a community's efforts to address improvements in the early care and education of children.

Introduction

Assessing young children's readiness to enter school "ready to learn" has become a priority for nearly every state and for many local education jurisdictions. However, the specific forms these assessments take vary widely, from brief "screening" tests to comprehensive, in-depth child studies (Mehaffie & McCall, 2002; Meisels & Atkins-Burnett, 2000). Likewise, the results of assessments may be used to identify individual children as "ready" or "not ready" and thus eligible for particular supports or services; or they may serve as indicators, at an aggregate level, of the success of collaborative community efforts to ensure that all children receive high-quality early care and education (Murphey & Burns, 2002).

In any case, a challenge for those developing and using "readiness" assessments is to demonstrate that such measures are in fact valid indices—that they measure what they purport to measure. Content validity is perhaps of prime importance here, and any "readiness" measure should pass the critical review of early childhood professionals, kindergarten teachers, and parents. In addition, the measure should address the five readiness domains around which there is now broad consensus (National Education Goals Panel, 1992). Establishing other forms of validity can be more problematic. In particular, predictive validity is inappropriate in this context because of the rapid and uneven development typical of young children.

Discriminant validity, however, is a reasonable aim for school readiness instruments. Discriminant validity refers to a measure's ability to distinguish among groups that theory claims ought to be so distinguished. Thus, discriminant validity of a readiness-for-school measure would be supported if the measure distinguished between groups hypothesized to be more or less "ready" than others.

Although the construct of "school readiness" is still fairly new in the literature, there is considerable empirical research to suggest the individual characteristics and the family and community resources and experiences for young children with which it should be associated. All other things being equal:

The validity of school readiness measures that fail to make such distinctions could be considered suspect.

The present study used data from Vermont's "School Readiness Initiative"—specifically, data from its "Ready Kindergartners" component—to examine these relationships. In doing so, its aim was also to provide further validation for this brief, teacher-reported assessment of children's readiness.

Method

Ready Kindergartners Instrument

The "Ready Kindergartners" instrument is a 24-item questionnaire completed by kindergarten teachers 4 to 6 weeks after the beginning of the school year, for each student. The items cover four readiness domains: (1) social and emotional development, (2) approaches to learning, (3) communication, and (4) cognitive development and general knowledge (the fifth domain, physical health and well-being, is only partly assessed by this instrument; school nurses provide additional information). On the basis of their interactions since the start of school, teachers are instructed to rate children on each item as "beginning," "practicing," "performing independently," or "not observed."

Items were selected based on extensive review of the kindergarten readiness literature, including examples of measures in use in other states. They were further refined by an expert group that included public health, education, mental health, and human services policy makers, in addition to providers of early childhood services and researchers from the University of Vermont's Department of Psychology.

Relying on kindergarten teachers as informants on children's school readiness of course has its limitations. Although teachers have a great deal of expert knowledge of, and experience with, young children, their ratings may be unreliable, particularly on a relatively unstructured assessment. To mitigate these concerns, the teacher rating instrument was the subject of extensive pilot testing with kindergarten teachers, along with focus group input from kindergarten teachers. However, the measure is not intended to have validity at an individual child level but to provide an indicator of kindergarten readiness at a community level. More detailed information on the instrument, including its basic psychometric properties and internal reliability, has been previously published (Murphey & Burns, 2002).

For purposes of the present study, children were considered "ready" on a domain if teachers rated their performance on each item within that domain as "practicing" or "performing independently."

Children's experience in child care prior to kindergarten was assessed by asking kindergarten teachers to indicate for each child whether prior to kindergarten this student was in a regulated early childhood program. (Regulated was defined as licensed centers, registered family day care homes, Head Start, or other preschool). Teachers could respond with "yes ," "no ," or "don't know." Teachers were not asked to report on the source of this information.

Children's disability status was assessed by asking teachers to indicate for each child whether he or she qualified for special education or Section 504 services.

No family income information was available. However, community poverty-level was estimated as the percentage of enrolled public elementary school students receiving free or reduced-price school meals, by school supervisory union, as reported by the state department of education.

Sample Characteristics

The sample consisted of all valid records on children (N = 3,730) submitted by kindergarten teachers in the fall of 2001. Two hundred fifty teachers (63% of those contacted) responded, representing each of Vermont's 60 supervisory unions. Table 1 shows detailed characteristics of the responding teachers and of the children on whom they reported.

Table 1
Sample Characteristics
Respondents Characteristics

Kindergarten teachers*

Mean length of experience with kindergarten teaching 10.4 years, sd = 7.3
Mean length experience with teaching (total) 17 years, sd = 8.8
Have elementary education license 1.6%
Have early childhood endorsement 34.4%
Teach half-day program 46.4%
Teach full-day, 5 days/week program 37.1%
Teach full-day, partial-week program 12.9%

Kindergarten children (kindergarten teacher report)**

Qualifies for special education services 11.1%
Qualifies for ESL/bilingual services 1.8%
Qualifies for Sec. 504 services 1.5%

Teacher reports on child's experience prior to kindergarten

Was in regulated early childhood program 69.2%
Was not in regulated early childhood program 16.6%
"Don't know" 11.8%
Missing response 2.3%

*250 Responding out of 396 estimated possible respondents; 63.1%.
**3,730 Responding out of 6,289 estimated possible respondents; 59.3%.

A follow-up study of a random sample of non-respondents (N = 40) showed minor differences between teachers who returned child assessments and those who did not. The two groups of teachers were alike on years of experience in the teaching profession, on years as kindergarten teachers specifically, and on the proportion of their students who had attended pre-kindergarten programs. Non-responders had slightly larger classes (median = 16, vs. 15, p < .05) and a greater proportion of classes where one or more children qualified for special services other than special education (Section 504 services: 47% vs. 15%, p < .05; ESL/bilingual services: 34% vs. 16%, p < .05). Non-responders were less likely to hold an early childhood endorsement (8% vs. 34%, p < .05) and were more likely to teach a class that was something other than a 5-day-per-week half-day or full-day program (37% vs. 13%, p < .05).

Results

Relationship of Kindergarten Readiness to Prior Experience in an Early Childhood Program

Chi-square tests were used to examine, for each of the four readiness domains, any differences between children who were or were not in an early childhood program (records for children where teachers did not know their child care history were excluded from the analysis). As shown in Table 2, there were significant differences in favor of children with child care experience in each domain. The largest difference was in cognitive development and general knowledge, where 70% of children with child care experience were performing at criterion on all items, compared with 56% of children without child care experience. In an additional test (ANOVA), where the number of domains (0-4) in which a child failed to meet criterion was the dependent measure, and child care experience the independent measure, there was also a significant difference: the mean number of domains in which children did not meet criterion was 1.22 for those with child care experience and 1.63 for those without (F = 37.47, p < .01).

Table 2
School Readiness (Four Domains) by Children's Prior Early Education Experience

 

Readiness Domain
Percent "Practicing" or "Performing Independently" on All Items

 Chi-Square

No Early Education Experience Indicated Early Education Experience Indicated
Social and emotional development 54.4 63.5 17.18*
Approaches to learning 51.1 60.5 18.15*
Communication 74.6 83.1 23.95*
Cognitive development and general knowledge 56.4 70.4 44.71*

* p < .01.

Relationship of Kindergarten Readiness to Receipt of Special Education or Section 504 Services

Chi-square tests were used to examine, for each of the four readiness domains, any differences between children who were or were not receiving special education or Section 504 services. As shown in Table 3, there were significant differences in favor of children not receiving services in each domain. The magnitude of differences (all greater than 30 percentage points) was fairly consistent across the four domains. In an additional test (ANOVA), where the number of domains (0-4) in which a child failed to meet criterion was the dependent measure, and service receipt status was the independent measure, there was also a significant difference: the mean number of domains in which children did not meet criterion was 2.54 for those receiving services, and 1.16 for those without such designation (F = 370.20, p < .01).

Table 3
School Readiness (Four Domains) by Children's Receipt of Special Education or Section 504 Services
Readiness Domain Percent "Practicing" or "Performing Independently" on All Items    Chi-Square
Does Not Qualify for Special Education or Section 504 Qualifies for Special Education or Section 504
Social and emotional development 64.8 31.9 184.11*
Approaches to learning 62.3 26.9 208.27*
Communication 85.3 49.8 331.03*
Cognitive development and general knowledge 71.2 37.1 213.84*
* p < .01.

Relationship of Community-Level Kindergarten Readiness to Community-Level Poverty Status

By supervisory union, the percentage of kindergartners reaching criterion on each of the four readiness domains was calculated. Pearson correlations (one-tailed) were computed between these community-level scores and the percentage of elementary school children qualifying for free or reduced-price school meals. As shown in Table 4, correlations were significant in two domains (communication and cognitive development and general knowledge), approached significance in another (approaches to learning), and were nonsignificant for social and emotional development. In all cases of significance or near significance, higher poverty was associated with lower readiness scores.

Table 4
Correlations (One-Tailed) between Community Poverty* and Community-Level School Readiness (Four Domains)
Readiness Domain  
Social and emotional development -.121 (ns)
Approaches to learning -.215**
Communication -.223***
Cognitive development and general knowledge -.330****

*Measured by percent of enrolled public elementary school students eligible for free or reduced-price school meals.
** p < .06.
*** p < .05.
**** p < .01.

Discussion

Assessment of school readiness (like any assessment) carries potential for misuse (Shepard, Kagan, & Wurtz, 1998). Perhaps the most fundamental such threat is failing to demonstrate plausible relationships to other recognized measures of children's experiences. Establishing the credibility of school readiness measures is essential to gaining the engagement of the stakeholders—early child care providers, school personnel, parents, and other community members—recognized as critical to improving the well-being of children in this arena.

The present study shows that a carefully constructed, brief measure of readiness for kindergarten can be related meaningfully (and plausibly) to what we know about some of the individual-level and community-level correlates of early success in school; however, some limitations of the study should be noted.

Although the sampling frame included all Vermont public school kindergarten teachers, there was a degree of self-selection bias in the sample of teachers who responded. In particular, the sample under-represented classrooms with children receiving Section 504 and some other special services. In addition, the sample over-represented teachers with an early childhood endorsement; Chi-square analyses (not shown here) showed that teachers with that credential are more likely than those without to rate children as "ready." However, it is not clear how including the non-responders would have altered these findings.

For information on children's participation in early child care, we relied upon teachers' reports. It is possible that teachers are not reliable providers of this information, and for 12% of students, teachers acknowledged that they did not know the children's child care history. Moreover, no information was available on the timing, duration, frequency, stability, or (perhaps most important) quality of the child care experience. Teachers' knowledge of children's child care experience and of their status with respect to special services introduces the possibility of halo effects; however, it is difficult to imagine an assessment relying on teachers who have no such knowledge of their students.

In the community-level analysis, the relationship between poverty and school readiness is probably attenuated, because individual differences on both constructs are not reflected. However, the social-ecological perspective argues that community-level effects can make important contributions to individual well-being.

The predictive validity of the readiness assessment remains to be established, although as argued previously, there are good reasons to believe such relationships may be weak. Although ruled out in the present study by concerns about privacy, in the future, including student identifiers on the readiness assessment will allow direct tests of association between these scores and later student academic assessments.

Despite certain limitations, however, this study's results provide an important degree of external validity for a brief, teacher-reported measure of children's readiness for kindergarten. In a time when schools are under pressure to administer numerous student-based assessments, use of a brief readiness measure has some advantages, particularly if it can be shown to relate meaningfully to known predictors of early school success and to concurrent and subsequent measures of school achievement. In addition, the assessment strategy described here can function as a "ready for school" accountability measure at a community/systems level, while avoiding the potentially negative implications that may attend "high stakes" child-based measures.

Acknowledgments

The author is grateful to the Vermont Research Partnership (including the State of Vermont Agency of Human Services and Department of Education and the University of Vermont) for supporting this work.

References

Barnett, W. Steven. (1995). Long-term effects of early childhood programs on cognitive and school outcomes. Future of Children, 5(3), 25-50. EJ 523 962.

Mehaffie, Kelly E., & McCall, Robert B. (2002, June). Kindergarten readiness: An overview of issues and assessment (Special Report). Pittsburgh, PA: University of Pittsburgh Office of Child Development.

Meisels, Samuel J., & Atkins-Burnett, Sally. (2000). The elements of early childhood assessment. In Jack P. Shonkoff & Samuel J. Meisels (Eds.), Handbook of early childhood intervention (2nd ed., pp. 231-257). New York: Cambridge University Press.

Murphey, David A., & Burns, Catherine E. (2002). Development of a comprehensive community assessment of school readiness. Early Childhood Research & Practice [Online], 4(2). Available: http://ecrp.illinois.edu/v4n2/murphey.html [2003, July 1].

National Education Goals Panel. (1992). Resolutions of the National Education Goals Panel: Assessing progress: Goal 1. Washington, DC: Author.

Resnick, Michael B.; Gueorguieva, Ralitza V.; Carter, Randy L.; Ariet, Mario; Sun, Yuanshan; Roth, Jeffrey; Bucciarelli, Richard L.; Curran, John S.; & Mahan, Charles S. (1999). The impact of low birth weight, perinatal conditions, and sociodemographic factors on educational outcome in kindergarten. Pediatrics, 104(6), e74-e84.

Shepard, Lorrie; Kagan, Sharon Lynn; & Wurtz, Emily (Eds.). (1998). Principles and recommendations for early childhood assessments. Washington, DC: National Education Goals Panel. ED 416 033.

West, Jerry; Denton, Kristin; & Germino-Hausken, Elvie. (2000). America's kindergartners: Findings from the Early Childhood Longitudinal Study, kindergarten class of 1998-99, fall 1998. (Report No. NCES 2000-070). Washington, DC: National Center for Education Statistics. ED 438 089.

Xiang, Zongping, & Schweinhart, Lawrence J. (2002). Effects five years later: The Michigan School Readiness Program evaluation through age 10. Ypsilanti, MI: High/Scope Educational Research Foundation.

Author Information

David Murphey is senior policy analyst in the Planning Division, Vermont Agency of Human Services. He holds a Ph.D. in developmental psychology and a master's degree in education, both from the University of Michigan. His professional interests led to work in early childhood education, child development and social policy, and lifespan development. Moving to Vermont in 1992, Dr. Murphey worked for the Department of Health before coming to his present position with the Agency. At the Agency, Dr. Murphey is responsible for managing the collection and reporting on social indicators statewide, and for preparing Vermont's Community Profiles-local reports on social indicators for the state's 60 school districts. He has also coordinated production of the Agency's What Works publications, summaries of effective prevention practices. In addition, Dr. Murphey provides data support, analysis, and technical assistance to the Office of the Secretary of the Agency, and to a variety of community partners.

David Murphey, Ph.D.
Senior Policy Analyst
State of Vermont, Agency of Human Services
Planning Division
103 S. Main St.
Waterbury, VT 05671
Telephone: 802-241-2238
Fax: 802-241-4461
Email: davidm@wpgate1.ahs.state.vt.us