Evolutionary Attitudes and Literacy Survey (EALS): Development and Validation of a Short Form
© Springer Science+Business Media, LLC 2012
Published: 28 July 2012
The Evolutionary Attitudes and Literacy Survey (EALS) is a multidimensional scale consisting of 16 lower- and 6 higher-order constructs developed to measure the wide array of factors that influence both an individual’s endorsement of and objection to evolutionary theory. Past research has demonstrated the validity and utility of the EALS (Hawley et al., Evol Educ Outreach 4:117–132, 2011); however, the 104-item long-form scale may be excessive for researchers and educators. The present study sought to reduce the number of items in the EALS while maintaining the validity and structure of the long form. For the present study, and following best practices for short-form construction, we surveyed a new sample of several hundred undergraduates from multiple majors and reduced the long form by 40% while maintaining the scale structure and validity. A multiple-group confirmatory factor analysis supported strong factorial invariance across samples, and therefore verified structure and pattern between the six higher-order constructs of the long-form EALS and the EALS short form (EALS-SF). Regression analysis further demonstrated the short form’s validity (i.e., demographics and openness to experience) and replicated previous findings. In the end, the EALS-SF may be a versatile tool that may be used whole or in part for a variety of research areas, including curricular effectiveness of courses on evolution and/or biology.
KeywordsEvolution Survey Confirmatory factor analysis Attitudes Knowledge Short form
In the last decade, intense interest has been focused on educational issues regarding the teaching of evolutionary theory and topics to which it is related in fields such as biology, anthropology, and psychology. This literature suggests that (a) individuals who seek exposure to evolutionary theory (e.g., in coursework or museum visits) demonstrate greater knowledge and report more positive attitudes toward the theory than those who do not seek exposure (e.g., Lombrozo et al. 2008; McFadden et al. 2007), (b) religious identity and political ideology enhance or impede the seeking of such knowledge (e.g., genetic literacy; Scott 2004; Miller et al. 2006; Paterson and Rossow 1999), and (c) knowledge of scientific epistemology is positively associated with both evolutionary knowledge and attitudes about its relevance (Hawley et al. 2011). Until recently, no suitable comprehensive measure existed to assess curricular effectiveness, attitude change, or descriptions of regional populations.
Our previous work described and validated such a comprehensive measure that was designed to assess political and spiritual leanings, knowledge of evolution, distrust of and knowledge about the scientific enterprise, and attitudes toward and objections against evolutionary theory (Hawley et al. 2011). With this measurement tool, our ultimate goal was to create a standard by which to assess aspects of curricular influences of courses in colleges and universities, and to assess the regional effectiveness of the intelligent design movement. That 104-item measure (henceforth referred to as the long form) comprised 16 subscales which were structured into six higher-order factors. Moreover, we demonstrated that attitudes were less reliant on demographics than they were on personality factors such as openness to experience on a Kansas sample.
For practical purposes, however, the long-form version may be too time consuming for repeated assessment in educational settings. It is not clear that a long form of 104 items yields a significant benefit over a version with 40% fewer items. Thus, the goal of the present work is to significantly reduce the long-form version in a way that fully maintains its validated structure.
The Creation of the Short Form
Short-form creation involves several steps that require close attention to the psychometric properties of the measure. First, the long form is created and validated. This step is completed (Hawley et al. 2011). Second, data are collected with the long-form survey with a new sample to confirm the established long-form structure (i.e., the previously published long-form structure, our standard for comparison). If the long-form survey structure cannot be confirmed on the new sample, then the structure of the survey rests too heavily on individual sample characteristics. This possibility must first be ruled out. Third, the survey is systematically shortened according to pre-established criteria (details below) and compared once again to the published long-form structure. Last, the short form is applied to outcome measures (e.g., demographics and openness to experience; Hawley et al. 2011) to replicate documented patterns.
For the present study, data were collected from 526 undergraduate students at a large Midwestern university. These participants (233 men and 290 women) were surveyed at the start of the semester from an introductory biology course covering the principles of cellular and molecular biology. As such, these students were from over 30 majors. Their average age was 19.15 (SD = 2.64) years, the average high school graduating class size was 342.89 (SD = 338.92), and the most frequent response for both the participants’ father’s (N = 109, 29.38%) and mother’s (N = 176, 33.46%) education was four-year college degree. Last, the average rating for rurality of town of origin is 3.14 (SD = 1.86) on a seven-point scale ranging from one (not at all rural) to seven (very rural). Demographically speaking, the present sample was largely homogeneous with the original sample of 371 on which the published long form is based.1 Group differences based on differing sample demographic characteristics are thus not anticipated.
As before in the initial long-form scale validation (see Hawley et al. 2011), the present students were invited to participate for extra-course credit and were asked to complete the web-based survey outside of class time via an e-mailed Web link.2 The students reported informally that it took them 20–25 minutes to complete the long form of the survey containing 104 items.
The Evolutionary Attitudes and Literacy Survey
The long form Evolutionary Attitudes and Literacy Survey (EALS) consists of 17 pages of Web-presented items where respondents rate the degree to which they agree or disagree with 104 statements on a seven-point Likert scale ranging from one (strongly disagree) to seven (strongly agree) with the midpoint 4 (neither agree nor disagree). The EALS measures 16 theoretically derived constructs including adult exposure to evolution, youth exposure to evolution, religious activity, young earth creationist beliefs, moral objections, political ideology, political activity, attitudes toward life, intelligent design fallacies, knowledge about the scientific enterprise, genetic literacy, evolutionary knowledge, misconceptions about evolution, distrust of the scientific enterprise, relevance of evolutionary theory, and social objections. These 16 constructs or subscales can be further accounted for by six higher-order factors representing Political Activity, Political/Religious Conservatism, Creationist Reasoning, Knowledge/Relevance of Evolution, Evolutionary Misconceptions, and Exposure to Evolution (see Hawley et al. 2011 for details on survey creation and survey structure).
Sample Characteristics and Missing Data Imputation
The newly collected data for the short form construction were screened for univariate and multivariate outliers, item normality (with skewness and kurtosis values all within −2.0 and 2.0) and linear relations between all the items. Like the original sample for the long-form construction, the present sample met the assumptions for latent variable modeling techniques. Less than 5% of the data were missing. Thus, missing data were imputed one time via Markov Chain Monte Carlo estimation using Proc MI in SAS (version 9.1.3; Enders, 2010).
EALS Long-Form Measurement Invariance
The factor structure between the previously published long form and the present long form will be compared using mean and covariance structures modeling (MACS; Little 1997). Each sample (previously published and present) has its own representative model reflecting factor structure. We will be imposing constraints across the two models to see if equating the models for the two samples is defensible. If it is defensible, then one has demonstrated measurement invariance; namely, that the EALS long form is functioning similarly for both samples. Measurement invariance will be evaluated first by examining the equivalence of item factor loadings across the two models (weak measurement invariance) and second, by adding the equality constraint of item (indicator) intercepts (strong measurement invariance) across the two models.
As in Hawley et al. 2011, each subscale (e.g., religious activity, genetic literacy, and evolutionary misconceptions) is represented by an aggregate of items known as a parcel (Little et al. 2002; Little et al. 2012). Parceling has the added benefits of requiring fewer model parameter estimates, reduced sampling error, and decreasing the likelihood of correlated residuals between items (Little et al. 2002). These parceled indicators are computed by calculating the mean response for all items representing a particular subscale. The goal of the present study is to reduce the number of items in a parcel that represents a given subscale in a way that ensures that the subscale has not changed in function.
Accordingly, each of the 104 items from the long-form scale was examined both quantitatively and qualitatively for candidacy for retention. We deemed the items appropriate for retention if they (a) possessed strong factor loadings (i.e., standardized factor loadings greater than 0.60), (b) were sufficiently distinct from other items (i.e., minimized redundancy), (c) qualitatively represented their intended construct, and (d) demonstrated normal distributive properties.
Examining Measurement Invariance Between EALS and EALS-SF
Finally, the previously published structure of the EALS long form and the newly derived EALS-SF were again examined for measurement invariance using MACS modeling using the same logic as above. That is, the published long-form model has parceled indicators based on 104 items and the present short-form model has parceled indicators based on the reduced list of items. The question is, are the models functioning similarly? All factor analyses were conducted using maximum likelihood estimation via the software package Mplus (version 6.0).
Replicating Validity with Regressions
Our initial analyses showed the relationships among Kansas demographics, personality factors, and the higher-order factors (Hawley et al. 2011). These analyses demonstrated that scores on the higher-order factor Knowledge/Relevance, for example, were less related to demographics than they were to openness to experience, a facet of personality. Thus, as a final check in the present study, we will seek to replicate these documented patterns with the newly created short form.
EALS Long-Form Measurement Invariance
As outlined above, before we examined items for retention, the long-form structure (Hawley et al. 2011) of the previously published and present samples must first be evaluated for strong measurement invariance (i.e., equivalent loadings and intercepts). To specify the measurement model for each sample, we created parcels by calculating the mean of all of the items within a particular subscale and using these means as indicators of the higher-order EALS constructs. For example, the construct Political/Religious Conservatism was indicated by the parcels religious activity, conservative self-identity, attitudes toward life, young earth creationism, and relevance of evolution. We repeated this procedure for each of the six higher-order EALS constructs in order to recreate the previously published model and a second identical model for our present sample. We then conducted a multiple group confirmatory factor analysis (CFA) to test for measurement invariance between the two models by comparing the structure on the present sample against the previously published structure.
Fit indices for EALS group invariance testing
RMSEA 90% CI
Items were retained based on quantitative and qualitative criteria, including strong factor loadings, distinctness from other items, and normal distributive properties. For example, the social objections item, “applying the theory of evolution to human affairs implies we are not fully in control of our behavior” was removed because it did not demonstrate a sufficiently high factor loading. Similarly, “theories requiring more untested assumptions are generally better than theories with fewer assumptions” was also removed as its factor loading did not reach criterion. The item “the theory of evolution helps us understand plants” measuring relevance of evolutionary theory was removed because it was redundant with “evolutionary theory is highly relevant for biology.” Finally, “Humans were specially designed” was removed from intelligent design fallacies because it is not an unambiguous representative of the creationist movement despite its high factor loading on the construct. In order to maintain construct stability (see Brown 2006), we retained at least three items for each subscale. In the end, 42 items were removed from the 104-item long-form EALS to result in a 62-item short form (EALS-SF).
Measurement Invariance Between EALS and EALS-SF
As before, the structure of the previously published long from was used as the standard of comparison for the present short-form structure. Parceled indicators were computed for the present sample by calculating the mean response for the shortened list of items representing a particular subscale. For example, in the previously published EALS long form, the lower-order construct political activity was represented by six items, and as a parceled indicator, had a mean of 2.96 (SD = 1.14). The present abbreviated parceled indicator for Political Activity of the EALS-SF consisted of only three items which had a mean of 2.86 (SD = 1.25). This procedure was carried out for each of the 16 subscales of the shortened form. We then used these 16 newly created EALS-SF parcels to specify the same hierarchical model tested and validated in the previously published long-form EALS. We conducted another multiple groups CFA to examine the levels of measurement invariance between a hierarchical model indicated with parcels from the previously published long-form EALS, and a hierarchical model indicated with parcels from the 62 items of the EALS-SF.
Fit indices for EALS and EALS-SF invariance testing
RMSEA 90% CI
Means, standard deviations, alpha coefficients for each construct, and the standardized factor loading for each item of the EALS- SF
Standardized factor loading
To what degree are you political?
To what degree do your political views influence your daily life?
To what degree do your political views influence your decisions?
To what degree are you religious?
To what degree does religion impact your daily life?
To what degree does your religion influence your decisions?
Religion is especially important to me because it answers many of my questions about the meaning of life. A
Conservative Self Identity
To what degree are you conservative?
In general, how do you self identify politically? B
In general how liberal/conservative are you on Economic issues (welfare, taxation, free market policies, etc)? C
Attitudes Toward Life D
Life begins at conception.
After conception, a developing human is only a cluster of cells, and it makes no sense to discuss its moral condition. (R)
All stages of human life- embryo, fetus, child, adult- should have the same legal protections.
Intelligent Design Fallacies
There is scientific evidence that humans were created by a supreme being or intelligent designer.
There is no evidence that humans evolved from other animals.
There are no transitional fossils (remains of life forms that illustrate an evolutionary transition).
Complex biological systems cannot come about by slight successive modifications (i.e., they are irreducibly complex).
Evolution is a theory in crisis.
Natural selection cannot create complex structures; It is like a tornado blowing through a junkyard and creating a 747.
Young Earth Creationist Beliefs
I read the bible literally.
The Earth isn’t old enough for evolution to have taken place.
There was a time when humans and dinosaurs lived on earth together.
Present animal diversity can be explained by the Great Flood.
Adam and Eve of Genesis are our universal ancestors of the entire human race.
All modern species of land vertebrates are descended from those original animals on the ark.
People who accept evolution as fact are immoral.
If you accept evolution, you really can’t believe in God.
Darwinism strips meaning from our lives.
The theory of evolution has contributed to racism.
The theory of evolution has contributed to sexism.
The theory of evolution has contributed to genocide (the deliberate killing of a group based on nationality, race, politics, or culture).
Distrust of the Scientific Enterprise
Contemporary methods of determining the age of fossils and rocks are untrustworthy. E
The data used to support evolution is untrustworthy.
The available data are ambiguous as to whether evolution actually occurs. F
Relevance of Evolutionary Theory
Evolutionary theory is highly relevant for biology.
The theory of evolution helps us understand human origins.
Evolutionary theory is highly relevant for the social sciences (e.g., anthropology, psychology, sociology).
Evolutionary theory is highly relevant for the humanities (e.g., history, literature, philosophy).
Evolutionary theory is relevant to our everyday lives.
Humans share a majority of their genes with chimpanzees. D
Humans share more than half of their genes with mice.D
Humans have somewhat less than half of the DNA in common with chimpanzees. D(R)
Mutations are never beneficial. D(R)
In most populations, more offspring are born than can survive.
Mutations can be passed down to the next generation.
Increased genetic variability makes a population more resistant to extinction.
The more recently species share a common ancestor, the more closely related they are.
Mutations occur all the time.
Misconceptions about Evolution
Characteristics acquired during the lifetime of an organism are passed down to that individual’s offspring.
Evolution means progression towards perfection.
Evolution is a linear progression from primitive to advanced species.
Knowledge about the Scientific Enterprise
For scientific evidence to be deemed adequate, it must be reproducible by others.
Scientific ideas can be tested and supported by feelings and beliefs. (R)
Scientific explanations can be supernatural. (R)
Good theories give rise to testable predictions.
Self Exposure to Evolution
I’ve watched evolution related videos on the web (e.g., Ted.com, YouTube).
I read science magazines featuring evolution (e.g., Discover, National Geographic, Nature).
I’ve watched nature shows that discussed evolution (e.g., PBS/Nova, Discovery, National Geographic)
Youth Exposure to Evolution
I have visited natural history museums on field trips or with family.
As a child, I attended science and nature camps (e.g., Outdoor Ed Lab, local nature centers or zoos).
How much training in evolution did you receive in high school?
Latent correlations between the 16 lower-order constructs (i.e., subscales) of the EALS-SF
1. Political Activity
2. Religious Activity
3. Conservative Self-Identity
4. Attitudes Towards Life
5. Intelligent Design Fallacies
6. Young Earth Creationist Beliefs
7. Moral Objections
8. Social Objections
9. Distrust of the Scientific Enterprise
10. Relevance of Evolutionary Theory
11. Genetic Literacy
12. Evolutionary Knowledge
13. Knowledge about Scientific Enterprise
14. Evolutionary Misconceptions
15. Self Exposure to Evolution
16. Youth Exposure to Evolution
Latent correlations among the six higher-order constructs of the EALS-SF
Exposure to Evolution
Exposure to Evolution
Replication of Validity with Regression
Latent regressions for the six higher-order EALS-SF constructs
Exposure to Evolution
Openness to experience
The present study demonstrated the suitability of a short form of the EALS to stand in for the long form. Using best practice methodology, we reduced the 104-item survey to 62 items (a reduction of 40%), all the while maintaining weak and strong measurement invariance and replicating previously published patterns. For this reason, researchers can confidently use the short form over the long form which should substantially reduce the amount time needed for participants to complete the measure while maintaining survey validity.
With a workable short form of the EALS, researchers and educators can now more easily adopt the EALS-SF as a capable measurement tool for testing theoretically derived predictions about the relationships among political and religious ideation, and knowledge acquisition and attitudes. Additionally, the EALS-SF may benefit educators interested in assessing the curricular effectiveness of their evolutionary and biology themed courses. Future longitudinal research will assess students in a variety of evolutionary themed courses with the EALS and EALS-SF, as well as continue to validate these scales with cross-national samples.
Work with the EALS and EALS-SF has been predominantly (though not solely; O’Brien et al. 2009) used with college students from Kansas. It could be argued that Kansas youth are not representative of youth at large and that this lack of representation affects the survey structure. Several issues stand in the way of this conclusion, however. First, the subscales were all theoretically derived and informed by others’ works based on non-Kansas samples (e.g., Carney et al. 2008; Ingram and Nelson 2006; Miller et al. 2006). Second, preliminary analyses (unpublished) suggest that the structures derived from Kansas and New York data are remarkably similar. Where states’ differences are expected to occur, however, are mean and variances across the higher-order structures. For example, some states may have lower means and be more homogenous (i.e., less variability) in constructs such as creationist reasoning. These latter types of questions, though highly important, stand outside of the present work.
Additionally, we sought concurrent validity in relationships among predictors of our survey subscales, but validity can also be sought in scales’ ability to predict important outcomes (i.e., predictive ability) such as academic achievement (e.g., grades in school). These are important questions for future work.
Last, not all content areas of evolution are represented in our survey. Indeed, the survey is not intended to be used as a comprehensive examination, but rather as a workable measurement tool for assessing knowledge and attitudes writ large. For this reason, we suggest that researchers freely include their own subscales in their assessment protocol in addition to those of interest from the EALS-SF. Questions concerning the relationships between additional subscales and validated EALS-SF subscales would certainly be interesting.
The EALS-SF is a comprehensive survey instrument with a validated structure. We hope it will be useful for college educators to assess curricular effectiveness and attainment of specified learning goals, changes in attitudes about course content, and predominant regional belief systems and their roles in science understanding and attitudes.
The sample on which the published long form is based included 371 undergraduates; 327 from a Child Psychology course and 44 from a Social Psychology course from the same large Midwestern University (see Hawley et al. 2011 for details). In their aggregate, they represented nearly 40 declared majors, 102 were men and 269 were women, and their average age was 20.67 (SD = 2.05) years.
Participants were informed their responses were anonymous and confidential with no identifying information being associated with their survey data, and that they could opt out of the study at any time without penalty. In addition, an alternative extra credit assignment was available for students who did not wish to participate in the current study. Assured anonymity and confidentiality were implemented to reduce the possibility of social desirability influencing survey responses.
- Brown TA. Confirmatory factor analysis for applied research. New York: Guilford; 2006.Google Scholar
- Carney DA, Jost JT, Gosling SD, Potter J. The secret lives of liberals and conservatives: personality profiles, interaction styles, and the things they leave behind. Polit Psychol. 2008;29:807–40.View ArticleGoogle Scholar
- Cheung GW, Rensvold RB. Evaluating goodness-of-fit indexes for testing measurement invariance. Struct Equ Model. 2002;9:233–55.View ArticleGoogle Scholar
- Dudley RL, Cruise RJ. Measuring religious maturity: a proposed scale. Rev Relig Res. 1990;32:97–109.View ArticleGoogle Scholar
- Hawley PH, Short SD, McCune LA, Osman MR, Little TD. What’s the matter with Kansas?: the development and confirmation of the Evolutionary Attitudes and Literacy Survey (EALS). Evol Educ Outreach. 2011;4:117–32.View ArticleGoogle Scholar
- Ingram EL, Nelson CE. Relationship between achievement and students’ acceptance of evolution or creation in an upper-level evolution course. J Res Sci Teach. 2006;43(1):7–24.View ArticleGoogle Scholar
- Little TD. Mean and covariance structures (MACS) analyses of cross-cultural data: practical and theoretical issues. Multivar Behav Res. 1997;32:53–76.View ArticleGoogle Scholar
- Little TD, Cunningham WA, Shahar G, Widaman KF. To parcel or not to parcel: exploring the question, weighing the merits. Struct Equ Model. 2002;9:151–73.View ArticleGoogle Scholar
- Little TD, Rhemtulla M, Gibson K, Schoemann AM. (2012) Why the items versus parcels controversy needn’t be one. Psychol Methods (in press).Google Scholar
- Lombrozo T, Thanukos A, Weisberg M. The importance of understanding the nature of science for accepting evolution. Evol Educ Outreach. 2008;1:290–8.View ArticleGoogle Scholar
- McFadden BJ, Dunckel BA, Ellis S, Dierking LD, Abraham-Silver L, Kisiel J, Koke J. Natural history museum visitors’ understanding of evolution. Bioscience. 2007;57:875–82.View ArticleGoogle Scholar
- Miller JD, Scott EC, Okamoto S. Public acceptance of evolution. Science. 2006;313:765–6.View ArticlePubMedGoogle Scholar
- O’Brien DT, Wilson DS, Hawley PH. “Evolution for Everyone”: a course that expands evolutionary theory beyond the biological sciences. Evol Educ Outreach. 2009;2(3):445–57.View ArticleGoogle Scholar
- Paterson FRA, Rossow LA. “Chained to the devil’s throne”: evolution & creation science as a religio-political issue. Am Biol Teach. 1999;5:358–64.Google Scholar
- Rutledge ML, Sadler KC. Reliability of the Measure of Acceptance of the Theory of Evolution (MATE) instrument with university students. Am Biol Teach. 2007;69:332–5.View ArticleGoogle Scholar
- Scott EC. Evolution vs. Creationism. Westport, CT: Greenwood Press; 2004.Google Scholar