An Avida-ED digital evolution curriculum for undergraduate biology
© The Author(s) 2016
Received: 9 February 2016
Accepted: 13 July 2016
Published: 29 July 2016
We present an inquiry-based curriculum based on the digital evolution platform Avida-ED (http://avida-ed.msu.edu). We designed an instructional sequence and lab book consisting of an introduction to Avida-ED and a set of three lessons focused on specific evolutionary concepts. These served to familiarize students with experimental evolution and Avida-ED. Students then developed independent Avida-ED research projects to test their own questions. Curriculum design and implementation occurred over the course or two semesters, with a pilot implementation in the first semester, followed by curriculum revision and full implementation in the second semester. The curriculum was implemented in an undergraduate Introductory Cell and Molecular Biology course at a major research university. Full implementation of the curriculum in semester two involved the use of Avida-ED mainly in the teaching lab in parallel with a bacterial antibiotic resistance experimental research stream, allowing students to draw connections between Avidian digital evolution and the evolution of antibiotic resistance in microbial populations. After carrying out the introductory exercises, students developed independent Avida-ED projects to test their own research questions, and presented their data to researchers in the NSF-funded BEACON Center for the Study of Evolution in Action. Preliminary results of our studies to assess the impacts of an Avida-ED curriculum indicate a positive effect on student learning of evolutionary concepts, particularly in increasing the level of complexity of student explanations about the random nature of mutation.
KeywordsEvolution education Artificial life Avida Avida-ED Digital evolution Introductory biology
Evolution presents a challenging set of concepts and principles for students to master, with these challenges amplified due to societal resistance (Nelson 2012). Many misconceptions about evolutionary principles have been identified (Alters and Nelson 2002; Gregory 2009), and many resources are available to teachers as they find ways to help students understand evolutionary concepts (e.g., Understanding Evolution website; Goldsmith 2003; Abraham et al. 2009; White et al. 2013). Despite this abundance of teaching resources, teaching evolution through an inquiry approach remains somewhat intractable due to the timeframes typically involved in evolutionary processes and other difficulties of trying to study evolution in the laboratory classroom.
Speth et al. (2009) described the use of digital organisms and the digital evolution platform Avida-ED (http://avida-ed.msu.edu) as a potential way to overcome some of the difficulties in teaching the dynamic aspects of evolutionary processes. In this paper, we extend the previous work by presenting an inquiry-based curriculum, based on Avida-ED, that addresses specific evolution misconceptions held by students. In addition, the curriculum provides a framework for working with students to design and carry out evolution experiments.
What is Avida-ED?
The research tool Avida is an artificial life platform that was developed at Cal Tech in the late 1990s, with continued development at Michigan State University (Ofria and Wilke 2004). The name, “Avida”, is a Spanish abbreviation of artificial life, or “A-life”. Many researchers have used Avida to explore fundamental questions about evolutionary processes, including questions that can be difficult to address using biological organisms (e.g., Lenski et al. 2003; Ofria et al. 2003; Misevic et al. 2006; Elena and Sanjuan 2008; Clune et al. 2011; Zaman et al. 2014).
Avida-ED is an adaptation of Avida that turned the research platform into an education tool (Pennock 2007a). The main objective of this adaptation was to give students an evolutionary lab bench that produces research-grade data, allowing them to learn not only basic evolutionary principles but also the nature and practices of science. Avida-ED is particularly well suited for dynamic, visual representations of evolutionary and scientific processes, providing an easy way for students to carry out experiments in evolution, without the need for laboratory space to do so (Speth et al. 2009).
How does Avida-ED work?
Avidians have a “virtual metabolism” that allows them to gain extra “energy” (processing power) from resources in their environment if they evolve the ability to process those resources. Details of the system have been described elsewhere (Ofria and Wilke 2004), but the basic idea is that through random substitution mutations Avidians evolve the ability to perform Boolean logic operations on numbers they receive as inputs from the environment. Simple operations provide a small energy boost; difficult operations give a larger boost.
Some of these resulting Avidians will have faster metabolic rates (they use data faster) and some of them will take less time to make copies of themselves (they have a shorter gestation time1). In either case, they will produce more offspring per unit time and are thus more fit in the biological sense. What is important to recognize about this model is that it is not a simulation, but an instantiation of the causal elements of the evolutionary mechanism (Pennock 2007b). New variations of organisms arise through random changes in organisms’ genomes; these changes are heritable; and they are the source of differential functionality, leading to natural selection as organisms compete for space in the virtual Petri dish (Fig. 1b). The organisms are not preprogrammed with regard to how they adapt to their environment and they are not rewarded by how they accomplish a task, but just by the results—evolve to perform a rewarded function and the resulting boost lets one replicate faster, thus contributing more offspring to the next generation than other organisms that are not as competitive in that environment. Avida-ED’s graphical representations of populations of organisms, their genomes, and their operations allows students to observe evolution in action and run their own experiments to test evolutionary hypotheses.
Avida-ED curriculum development process
We developed and pilot-tested an Avida-ED curriculum over two semesters in spring semester 2014 in an Introductory Cell and Molecular Biology course (LB145) followed by full implementation in that same course in fall 2014. We took advantage of the summer months to evaluate the results of the spring 2014 implementation and modify the curriculum accordingly, as a part of our teaching and learning cycle (Elwood and Klenowski 2002).
Timeline of activities and assessments associated with implementation of Avida-ED in LB145 S14 and LB145 F14
Lab book distributed; introduction to Avida-ED, including lecture/video/demo and tutorial; Abr question (pre-test)
Introductory exercise quiz due; exercise 1 assigned
Exercise 1 group data explored; exercise 2 assigned; exercise 2 data explored; Avida-ED/clam case tie-in
Exercise 3 assigned; guidelines and expectations for proposal distributed
Demonstration of the proposal presentation; rubric for proposal and presentation distributed
Introduction to Avida-ED, including lecture/video/demo and tutorial
Student proposal presentations
Feedback on proposals; poster guidelines and expectations distributed, with the scoring rubric
Avida-ED research; Abr question (post-test)
In class activity
Poster presentations in the BEACON center; peer evaluation of posters
Activity debrief; Abr question
Avida-ED user’s survey
Avida-ED user’s survey
We reviewed and improved the spring 2014 implementation over the summer. Among the major changes made was pairing Avida-ED with a wet lab curriculum in which student teams isolated antibiotic resistant bacteria from different soil environments and characterized them in the teaching lab (Luckie et al. 2013). This change was motivated by Johnson’s (2012) study showing that pairing Avida-ED with a wet lab in a high school AP Biology course appeared to strengthen students’ abilities to draw connections between the Avidians (digital organisms) and biological organisms, particularly bacteria. In addition, we implemented “best practices” for Avida-ED identified by Lark (2014), including implementing Avida-ED over the entire semester (long duration), having one of us (JS) carry out the actual implementation (high level of instructor expertise), scaffolding the implementation via a set of guided introductory exercises, and implementing Avida-ED in the lab in a way that was tied to the course curriculum.
We developed an Avida-ED Lab Book (Additional file 1) for fall 2014 consisting of an introduction to Avida-ED, three introductory exercises targeting specific student misconceptions about evolutionary processes, and an independent research piece in which students used Avida-ED to answer a research question that they devised. The introductory exercises were drawn from a set individual exercises that had been developed previously and made available on the Avida-ED web site (http://avida-ed.msu.edu).
The Avida-ED curriculum and its implementation in fall 2014
The curriculum was implemented in an Introductory Cell and Molecular Biology course (LB145) in a residential college on the campus of a major Research I University comprised mainly of students majoring in the natural sciences. Students typically take this introductory five-credit course in year 2 of their 4-year program (first-year chemistry is prerequisite). The course consists mainly of suburban and rural Michigan students, with approximately 10 % of the students coming from traditionally underrepresented groups in the sciences. The students enter the class with a range of biology backgrounds and math preparedness. LB145 is taken by science majors (mainly Human Biology and Physiology), and includes many pre-professional students.
In the pilot implementation in spring 2014, there were 100 students in the lecture section, which was divided into five lab sections of 20 students each. The lecture was taught by the instructor (JS), while the labs were taught either by the instructor (one section) or one of two graduate teaching assistants, each assisted by two undergraduate learning assistants. Fall enrollment in the lecture section of LB145 was 39 students, divided again into two lab sections of 19–20 students each. Both lab sections were taught by the instructor, assisted by two undergraduate learning assistants in each case.
The Avida-ED lab book
The Avida-ED Lab Book (Additional file 1) represents the core component of the curriculum. The lab book begins with an introduction to Avida-ED, including Zimmer’s (2005) Discover article, “Testing Darwin”, which describes digital organisms and how they are being used to study fundamental evolutionary processes (note: not included in Additional file 1), and a tutorial that takes students through the fundamentals of the use of the Avida-ED program. The introductory lesson is followed by three exercises (Exercises 1–3), each geared towards addressing particular learning goals with respect to evolutionary principles. These exercises not only serve to provide a scaffold for student learning about evolutionary principles, but also allow students to gain more familiarity with the Avida-ED platform. The Avida-ED Lab Book concludes with a description of and guide to an Independent Research Project, in which students worked in research teams to design and carry out their own experiments using Avida-ED.
Avida-ED Lab Book Exercise 1 and the Independent Investigation Exercise are modifications of exercises of the same name, while Exercise 2 is a modification of the exercise titled “Exploring Selection and Fitness”, all originally produced by two of us (RTP and AL). Complete materials for these original lessons are available under the Curriculum link at http://avida-ed.msu.edu.
Elements of implementation
The full Avida-ED curriculum that was implemented in the fall 2014 Introductory Cell and Molecular Biology course consisted of classroom instruction in both the lecture hall and laboratory classroom, including background about digital evolution in general and Avida-ED in particular as an experimental model system. A summary of the implementation of Avida-ED in LB145 in fall 2014 is shown in Table 1. Activities primarily focused on the set of introductory exercises in The Avida-ED Lab Book that student teams completed, and an independent investigation in which student teams designed and carried out their own experiments using Avida-ED.
Introduction to Avida-ED/introductory lesson
Avida-ED was introduced in a lab session via a mini-lecture (approx. 15–20 min) followed by the assignment of the Introductory Lesson and Tutorial in The Avida-ED Lab Book. Students worked together in lab teams, which were chosen at the beginning of the semester using the Team-Maker program included in Purdue University’s CATME package (http://catme.org). The Introductory Lesson and Tutorial was modified from the “Introduction to Digital Evolution Handout & Tutorial” written by Johnson et al. (2009) for the Teach Engineering Curriculum for K12 Teachers (teachengineering.org). Students used this introductory activity to gain familiarity with Avida-ED and the concept of Avidians, the Avida-ED program, and the user interface. They also read the article by Zimmer (2005), which introduced digital evolution research and helped students think about parallels between digital organisms and biological organisms.
Avida-ED exercises 1–3
Exercises 1–3 each were introduced briefly in a lab session (approx. 10 min) and time was built into the lab schedule to allow students to work on these exercises with members of the teaching team present to provide direction and feedback. Students then worked individually outside of class on these three exercises, each of which required students to contribute their data to the pooled class data via Survey Monkey. The course instructor (JS) monitored and summarized the data for presentation and discussion in both the lecture section and the lab sections of the course. These pooled class data provided a powerful visual representation of the processes playing out in the Avida-ED program. Each exercise also had an associated ungraded quiz consisting of a set of free response-type questions pertaining to the exercise. Students were required to submit individual responses to these questions through the Desire2Learn course management system.
Avida-ED exercise 1: understanding the introduction of genetic variations by random mutation
The primary learning goal for students in Exercise 1 was to be able to explain what it means to say that mutations occur at random. Students often carry the mistaken impression that evolution itself is random (Garvin-Doxas and Klymkowsky 2008; Mead and Scott 2010b). Exercise 1 addressed this point by showing that while mutations occur at random, natural selection itself is not random. We also explored the concept of a mutation rate, which provided the opportunity to discuss central tendencies and dispersion about a mean.
Students began Exercise 1 by replicating the “@ancestor” Avidian in the organism viewer with a 10 % mutation rate applied. Students were asked to predict how many mutations they expected to see, and then recorded how many they actually observed. Students compared their initial results with those obtained by a classmate. Students were also asked to explore how their mutations were distributed as a function of position within the Avidian “genome” and to record the locations of each of the mutations that they observed. In this way, we illustrated the point that, in theory, every location in a genome is subject to mutation, even though not all mutations will survive to appear in later generations.
Avida-ED exercise 2: exploring fitness, functions, and selection
The primary learning goal for students in Exercise 2 was to be able to explain what it means to say that a trait has increased in frequency or has gone to fixation in a population. The difficulty of thinking in terms of populations instead of individuals is a recurring impediment to student understanding of evolutionary principles and processes (Bishop and Anderson 1990; Shtulman 2006; Gregory 2009), and one of the strongest features of Avida-ED is its ability to provide a dynamic visual representation of population-level processes. In Exercise 2, an individual Avidian is saved that can perform a particular function (the logic function, NOT). This organism is then separately put into environments either with or without the resource for that function (notose), which allows students to see how selection affects a trait when it provides a reproductive advantage.
Students began the exercise by evolving an Avidian population starting with “@ancestor”, a simple self-replicator ancestor. After sufficient time passed (typically 100–200 updates), students located and isolated an individual Avidian that had evolved the ability to perform the logic function NOT. This individual was used to seed two subsequent independent Avida-ED runs (i.e., it was the ancestor in these runs). In the first of these two subsequent runs, there was no resource in the environment for the ability to perform NOT, which meant that there was no reward for performing NOT. After 2000 updates, students recorded the frequency in the population of individuals able to perform NOT (phenotype frequency). In the second run, the metabolic resource notose was included in the environment, which provided an energy reward to an organism if it performed NOT. Once again, after 2000 updates, students recorded the frequency in the population of individuals able to perform NOT.
Avida-ED exercise 3—exploring mutations and selection: pre-adaptive or post-adaptive?
One enduring misconception students hold is that evolutionary processes are forward looking and that events occur in anticipation of some future need (Mead and Scott 2010a). Thus, the primary learning objective in Exercise 3 is for students to demonstrate that they understand that mutations are random and do not occur simply because they are needed. Avida-ED provides a unique and direct way to test this evolutionary principle.
Students again begin the exercise by seeding a population with the @ancestor, making sure that all rewards are turned off (and thus, that the ability to perform a logic function does not confer a selective advantage). Students then begin their Avida-ED run, and stop the run when the first individual appears in the population that can perform the logic function NOT. Students then record the update at which this event occurred. Next, students repeat the exercise, but now set the initial environmental conditions such that the ability to perform NOT is rewarded (notose is present in the “medium”). Once again, the run is stopped when the first individual appears in the population that can perform NOT, and students record the update at which this event occurred.
Independent research investigation—experimental evolution project with evolving digital organisms
Lark (2014) showed that students do better when they have substantial time and scaffolding to make sense of what they see in Avida-ED. After the students had completed the three preliminary exercises, and had time to explore the data they had generated as a class, the student teams were challenged to come up with a research question of their own to test using Avida-ED. This independent investigation component of the curriculum provided a way for students to do science and think like biologists, and we incorporated into these investigations all of the elements described by Thornton (1972) as adapted by Sundberg and Moncada (1994). Students were told that the purpose of the exercise was to engage in investigation, students consulted with teaching team members to formulate problems and investigatory procedures, students were provided ample time to repeat and/or modify experiments, and students prepared and presented both written and oral reports describing their work.
The Independent Investigations consisted of an initial exercise in which students worked within their research teams to write a research proposal that included a description of their research question, framed the question as a formal hypothesis to be tested, described the experimental design (including descriptions of relevant variables, what data they would collect, and how many replicates they would carry out), and stated a prediction of their expected outcomes if indeed their hypothesis was true. Teams then presented their proposals to the entire class with each team member explaining one part of it. The Guidelines and Expectations for the Proposal are included as Additional file 2, and a sample presentation slide set from one of the research teams is included as Additional file 3.
Student teams carried out their experiments in the teaching lab over the following 3–4 weeks. During this time, the teaching team met regularly with the student teams to check on progress, go over preliminary data, and suggest directions for next steps, much as would happen in an actual research laboratory. Students were also counseled with respect to how to organize their data within spreadsheets, and summarize their data in figures and tables that could be used in their poster presentations. How to organize the data collected during the experiments proved to be one of the biggest challenges faced by the teams, prompting one student to comment that, “I’ve never had so much data before!” A sample Excel file generated by a student team is included as Additional file 4.
Students put together their research posters using a set of guidelines and expectations that were distributed and discussed in the lab (Additional file 5). Preliminary drafts of the posters were discussed during a lab session and formal feedback provided for each group. During the Poster Presentation Session at the BEACON Center, each research team of four students was divided into two teams of two, and each team of two presented their team’s poster and fielded questions about their research. A sample poster from one of the research teams is included as Additional file 6.
The Avida-ED Independent Research Project allowed students to engage in authentic science practices (“do science”), thus addressing a number of our science process skills objectives. Students worked in teams to brainstorm ideas, ask questions, develop hypotheses, design experimental plans, conduct experiments and collect data, analyze data, and write results for presentation to peers and experts. Students were given time to think about their projects and feedback was provided by the teaching team both informally in conversations in the planning stage and formally in response to their proposal presentations. All of these elements provided practice for our students doing the kinds of things that scientists do when they conduct research.
Initial assessment of student learning
Preliminary assessment of student learning using Avida-ED
Students affective response to the Avida-ED curriculum
In this paper, we have described the development of an Avida-ED course curriculum that was implemented in an Introductory Cell and Molecular Biology course (LB145) taught by one of us (JS) over two semesters (spring 2014 and fall 2014). The pilot implementation in spring 2014 occurred mainly in the lecture hall and required students to work on assignments outside of class, without a lot of guidance or support. This is likely the explanation for the dramatic increase in affective response from spring 2014 to fall 2014 (Fig. 6), when the curriculum was better constructed to incorporate many of the best practices for Avida-ED implementation identified by Lark (2014).
Full implementation of the Avida-ED curriculum as described here may not be possible for others in their own courses, especially if learning objectives are different from our own. However, we encourage instructors to adapt and use whichever parts of the curriculum they might find useful, depending on the learning outcomes desired. The different parts of the curriculum can be adapted for use in isolation, and we encourage those who adopt Avida-ED to start with the Lab Book included here and create something new for their own use. This is the intent of the Creative Commons license under which the Lab Book was produced.
Possible curricular modifications and extensions
One modification that practitioners might find useful would be to reverse the order of Exercises 2 and 3. By doing Exercise 3 first, students will have already evolved an Avidian that can perform the logic function not, which is required for Exercise 2, prior to the beginning of this exercise. The original ordering of Exercises 2 and 3 will be fine for those with adequate time, but the modification will save time for those who need it.
Also, a number of additional curricular materials are available at the Avida-ED website (http://avida-ed.msu.edu), including three additional Avida-ED exercises previously developed in conjunction with the Avida-ED Curriculum Development Group: “Exploring the Effects of Mutation Rate on Individuals”, a lesson based on the Fukushima nuclear disaster (Lark et al. 2014); “From Genotype to Phenotype: Understanding the Introduction of Phenotypic Variations”; and “Evolving TCE Biodegraders”. Student handouts are provided at the website for each of these exercises, as are web links to more complete curricular materials.
Conclusions and future directions
Evolution presents a challenging set of ideas and concepts for students to master, with students regularly incorporating naïve and scientific concepts simultaneously (Nehm and Ha 2011). Our data indicate that using Avida-ED helps students understand the role of mutation, and may minimize student use of naïve concepts. Thus, Avida-ED appears to hold great promise as a tool that students can use to learn evolutionary principles and to carry out scientific investigations of evolutionary processes.
“Gestation time” will be referred to as “generation length” in future iterations of Avida and Avida-ED.
JJS, WRJ, AML, LSM and RTP developed curricular materials and implemented them in the classroom. All authors collected and analyzed data. MJW did the statistical analyses. JJS and RTP wrote the manuscript. All authors provided comments on the final manuscript. All authors read and approved the final manuscript.
We thank all of the LB145 students in spring 2014 and fall 2014 for using Avida-ED as a part of their learning experience. Special thanks go to the team members of the research team “Black Mamba”, who contributed their Proposal Presentation slide set, their data file, and a copy of their final poster for inclusion in this paper. We also thank Dr. Peter J. T. White and his LB145 students for serving as an Avida-ED “control” group in spring semester 2014. This material is based in part upon work supported by grants from the Howard Hughes Medical Institute through the Science Education Program to Michigan State University, Pennock (co-PI), Mead and Smith (Senior Personnel), and from the National Science Foundation IUSE No. 1432563, Pennock (PI), Mead, and Smith (Co-PIs). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding agency. Datasets and analysis scripts are available at the Dryad Digital Repository (doi:10.5061/dryad.7t457 Available upon acceptance).
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Abraham J, Meir E, Perry J, Herron J, Maruca S, Stal D. Addressing undergraduate student misconceptions about natural selection with an interactive simulated laboratory. Evol Educ Outreach. 2009;2:393–404.View ArticleGoogle Scholar
- Alters BJ, Nelson CE. Perspective: teaching evolution in higher education. Evolution. 2002;56:1891–901.View ArticlePubMedGoogle Scholar
- Bishop B, Anderson CW. Student conceptions of natural selection and its role in evolution. J Res Sci Teach. 1990;27:41–57.View ArticleGoogle Scholar
- Clune J, Goldsby HJ, Ofria C, Pennock RT. Selective pressures for accurate altruism targeting: evidence from digital evolution for difficult-to-test aspects of inclusive fitness theory. Proc Biol Sci. 2011;278:666–74. doi:10.1098/rspb.2010.1557.View ArticlePubMedGoogle Scholar
- Elena S, Sanjuan R. The effect of genetic robustness on evolvability in digital organisms. BMC Evol Biol. 2008;8:284.View ArticlePubMedPubMed CentralGoogle Scholar
- Elwood J, Klenowski V. Creating communities of shared practice: the challenges of assessment use in learning and teaching. Assess Eval High Educ. 2002;27:243–56. doi:10.1080/02602930220138606.View ArticleGoogle Scholar
- Garvin-Doxas K, Klymkowsky MW. Understanding randomness and its impact on student learning: lessons learned from building the biology concept inventory (BCI). CBE Life Sci Educ. 2008;7:227–33. doi:10.1187/cbe.07-08-0063.View ArticlePubMedPubMed CentralGoogle Scholar
- Goldsmith DW. The great clade race: presenting cladistic thinking to biology majors & general science students. Am Biol Teach. 2003;2003:679–83.View ArticleGoogle Scholar
- Gregory TR. Understanding natural selection: essential concepts and common misconceptions. Evol Educ Outreach. 2009;2:156–75.View ArticleGoogle Scholar
- Johnson WR. Evolution in action in the classroom: Engaging students in scientific practices to develop a conceptual understanding of natural selection (Master’s thesis). ProQuest Dissertations and Theses database. (UMI No. 1517061). 2012.Google Scholar
- Johnson W, Pennock RT, Mead L. Studying evolution with digital organisms activity—introduction to digital evolution handout & tutorial. Teach Engineering Curriculum for K12 Teachers. 2009. http://teachengineering.org. Accessed 28 Dec 2015.
- Lark AM. Teaching and learning with digital evolution: factors influencing implementation and student outcomes (Doctoral dissertation). ProQuest Dissertations and Theses database. (UMI No. 3645332). 2014.Google Scholar
- Lark A, Richmond G, Pennock RT. Modeling evolution in the classroom: the case of Fukushima’s mutant butterflies. Am Biol Teach. 2014;76:450–4. doi:10.1525/abt.2014.76.7.6.View ArticleGoogle Scholar
- Lenski RL, Ofria C, Pennock RT. The evolutionary origin of complex features. Nature. 2003;423:139–45.View ArticlePubMedGoogle Scholar
- Luckie DL, Smith JJ, Cheruvelil KS, Fata-Hartley C, Murphy CA, Urquhart GR. The “Anti- Cookbook Laboratory”: converting “Canned” introductory biology laboratories to multi-week independent investigations. Proc Assoc Biol Lab Educ. 2013;34:196–213.Google Scholar
- Maher JM, Markey JC, Ebert-May D. The other half of the story: effect size analysis in quantitative research. CBE—Life Sci Educ. 2013;12:345–51. doi:10.1187/cbe.13-04-0082.PubMedPubMed CentralGoogle Scholar
- Mead LS, Scott EC. Problem concepts in evolution part I: purpose and design. Evol Educ Outreach. 2010a;3:78–81.View ArticleGoogle Scholar
- Mead LS, Scott EC. Problem concepts in evolution part II: cause and chance. Evol Educ Outreach. 2010b;3:261–4.View ArticleGoogle Scholar
- Misevic D, Ofria C, Lenski RE. Sexual reproduction reshapes the genetic architecture of digital organisms. Proc Biol Sci. 2006;273:457–64. doi:10.1098/rspb.2005.3338.View ArticlePubMedGoogle Scholar
- Moharreri K, Ha M, Nehm RH. EvoGrader: an online formative assessment tool for automatically evaluating written evolutionary explanations. Evol Educ Outreach. 2014;7:15.View ArticleGoogle Scholar
- Nehm RH, Ha M. Item feature effects in evolution assessment. J Res Sci Teach. 2011;48:237–56.View ArticleGoogle Scholar
- Nelson C. Why don’t undergraduates really “get” evolution? What can faculty do? In: Brem SK, Rosengren KS, Evans EM, Sinatra GM, editors. Evolution challenges: integrating research and practice in teaching and learning about evolution. Oxford: Oxford University Press; 2012.Google Scholar
- Ofria C, Wilke CO. Avida: a software platform for research in computational evolutionary biology. J Artif Life. 2004;10:191–229.View ArticleGoogle Scholar
- Ofria C, Adami C, Collier TC. Selective pressures on genomes in molecular evolution. J Theor Biol. 2003;222:477–83.View ArticlePubMedGoogle Scholar
- Pennock RT. Learning evolution and the nature of science using evolutionary computing and artificial life. McGill J Educ. 2007a;42:211–24.Google Scholar
- Pennock RT. Models, simulations, instantiations and evidence: the case of digital evolution. J Exp Theor Artif Intell. 2007b;19:29–42.View ArticleGoogle Scholar
- R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. 2013. (http://www.R-project.org/).
- Shtulman A. Qualitative differences between naïve and scientific theories of evolution. Cogn Psychol. 2006;52:170–94.View ArticlePubMedGoogle Scholar
- Speth EB, Long T, Pennock RT, Ebert-May D. Using Avida-ED for teaching and learning about evolution in undergraduate introductory biology courses. Evol Educ Outreach. 2009;2:415–28.View ArticleGoogle Scholar
- Sundberg MD, Moncada GJ. Creating effective investigative laboratories for undergraduates. Bioscience. 1994;44:698–704.View ArticleGoogle Scholar
- Thornton J. The laboratory: a place to investigate. Commission on Undergraduate Education in the Biological Sciences. Washington DC: American Institute of Biological Sciences. p. 158. 1972.Google Scholar
- Weber RP. Basic content analysis. 2nd ed. Newbury Park: Sage; 1990.View ArticleGoogle Scholar
- White PJT, Heidemann MJ, Loh M, Smith JJ. Integrative cases for teaching evolution. Evol Educ Outreach. 2013;6:1–7.View ArticleGoogle Scholar
- Zaman L, Meyer JR, Devangam S, Bryson DM, Lenski RE, Ofria C. Coevolution drives the emergence of complex traits and promotes evolvability. PLoS Biol. 2014;12:12. doi:10.1371/journal.pbio.1002023.View ArticleGoogle Scholar
- Zimmer C. Testing Darwin. Discover. 2005;26:28–35.Google Scholar