Dashboard helps teachers understand student progress and performance in genetics game

Our dragon genetics games have engaged thousands of students for many years. In that time, teachers have asked for an easy way to track their students’ progress and performance. Until now, teacher reports have been difficult to pull out of our system and impossible to parse in real time. The GeniGUIDE project, in partnership with North Carolina State University, is developing a teacher dashboard to accompany our new Geniventure software. We are currently piloting the beta version of this dashboard in multiple classrooms in Maine, North Carolina, New Jersey, and Massachusetts.

“A dashboard is a visual display of the most important information needed to achieve one or more objectives that has been consolidated on a single computer screen so it can be monitored at a glance.” – Stephen Few

Our dashboard displays information processed by an Intelligent Tutoring System (ITS) integrated into Geniventure. As students complete challenges in the game, they are rewarded with different color crystals for their accomplishments (Figure 1). Students who complete a challenge efficiently and without mistakes receive a blue-green crystal. Those who make a small number of missteps receive a yellow crystal while those with more mistakes receive a red. A black “try again” crystal is given to a student with too many mistakes to move on. As students level up through the missions, the ITS builds a model of conceptual understanding of specific learning goals. As student performance on these concepts improves over time, evidence that they have a solid understanding grows stronger.

Figure 1. Student view within Geniventure of the colored crystals (bottom of screen).

Our preliminary teacher dashboard design (Figure 2) was guided by three factors. First, we looked back at our many years of classroom observations of teachers who implemented our suite of dragon genetics games—from our most recent Geniverse to GeniGames and BioLogica—and asked: What information could have helped teachers better facilitate student use of the game? Second, we examined recent dashboard designs implemented in prior Concord Consortium projects to help us distinguish between in-class and after-class use. Finally, we looked at other teacher dashboards that are currently available on the market.

Figure 2. Beta version of Geniventure teacher dashboard.

During the pilot testing, we’re closely observing how teachers use the primary view of the dashboard, which provides information on both student progress and performance during class time. We hope to answer the following questions:

  • Can the teacher adequately track student progress through the game?
  • When do teachers intervene and when do they allow students to struggle? (Do teachers first help those students with black or red crystals?)
  • Do teachers look at how many attempts a student made at a challenge?
  • If teachers notice that particular students are ahead of the class, what actions do they take?

The dashboard also displays a graphical representation of student understanding of genetics concepts highlighted in the game. Some concepts are directly related to specific student actions (e.g., two recessive alleles are required to produce a recessive trait) while others are calculated based on performance across certain challenge types (e.g., genotype to phenotype mapping). The teacher can delve deeper into these secondary reports to view not only individual student data (Figure 3), but also aggregated class data (Figure 4). Through classroom observations and interviews with teachers, we hope to determine:

  • Do teachers have the time and bandwidth to make sense of the concept understanding graphs during class?
  • To what extent do the concept graphs help teachers understand where individual students, or the entire class, are having trouble?
  • What action, if any, do teachers take based on the concept graphs?

Figure 3. Display of individual student’s conceptual understanding.

Figure 4. Representation of class average conceptual understanding.

As our ITS becomes more sophisticated, we plan to widen the concepts we track and make better use of student data to inform teachers.

How do you make use of dashboards? Let us know what features you’d like to see as we improve our ITS-enhanced dashboard.

Energy2D used as a simulation tool in astrobiology research

Fig. 1: Frasassi Caves, Italy (credit: Astrobiology)
Deposition of minerals in caves may be affected by microbes. Geochemical analysis of these minerals can reveal biosignatures of subsurface life on a planet such as the Mars. Research in this area can help NASA build subsurface life probes for future planetary missions.

Fig. 2: Energy2D simulations (credit: Astrobiology)
Astrobiology, a peer-reviewed scientific journal covering research on the origin, evolution, distribution and future of life across the universe, just published a research paper titled "Transport-Induced Spatial Patterns of Sulfur Isotopes (δ34S) as Biosignatures" by a group of researchers at Pennsylvania State University, the University of Texas at El Paso, and Rice University. The lead author is Dr. Muammar Mansor. The researchers analyzed sample sites in the Frasassi Caves, Italy (Figure 1) and used Energy2D to simulate the effects of convection and diffusion on the chemical deposition processes (Figure 2). According to the paper, the results of the deposition simulated using Energy2D are consistent with the data collected from the cave sites, suggesting the importance of the effect of natural convection.

This is the second paper that uses Energy2D in astrobiology research (and the 16th published paper that used Energy2D in scientific research to simulate a natural or man-made system). In the first paper, Energy2D was used to simulate the thermal conditions for the origin of life. Once again, the publication of this paper provides fresh evidence for the broader impacts of our work.

Lights, camera, action: A video that introduces the NGSS practice of scientific argumentation

Following the recommendation to incorporate the Next Generation Science Standards (NGSS) science and engineering practices in their classrooms, schools across the country are looking for ways to integrate scientific argumentation into their curriculum. Since 2012 the High-Adventure Science project in collaboration with National Geographic Education has offered free online modules for Earth and space science topics—including climate change, freshwater availability, the future of energy sources, air quality, land management, and the search for life in the universe—that include multiple opportunities for students to engage in argument from evidence.

Over 67,000 teachers and students across the globe have used High-Adventure Science modules. Based on teacher feedback, classroom observations, and analysis of student data, we have learned that when students engage in argumentation from data and model-based evidence, they need a lot of support on how to write a convincing argument.

Last year, we added an introductory activity to each module where students learn about the component parts of a scientific argument before they are asked to write one. In this highly scaffolded task, students see written examples of a claim and explanation and learn about uncertainty in scientific data and how to express this uncertainty. In High-Adventure Science, argumentation takes a special form, including a multiple-choice structured claim, open-ended explanation, five-point Likert scale uncertainty rating, and uncertainty rationale.

In this introductory activity, students learn about the components of a good explanation.

Even with this new activity, some students still struggled, so we recently created an animated video to introduce the scientific practice of developing an argument. We start by helping students identify the difference between a scientific argument and so-called “arguments” they may have with their friends (e.g., arguing about favorite ice cream flavors!), and making the distinction between claims backed by evidence and opinion. The goal is to introduce students to scientific arguments in a fun and relatable way and to make the terminology and process of scientific argumentation less daunting.

We’re piloting the video in our Will there be enough fresh water? module for select students. We’re looking forward to student and teacher feedback and may revise the video based on their comments. We want everyone to be able to engage in the critical practice of arguing from evidence.

We welcome your comments about our video, as well as your challenges and successes with incorporating the NGSS practice of engaging in argument from evidence.

Energy3D uses intelligent agents to create adaptive feedback based on analyzing the "DNA of design"

Fig. 1: A simple case of teaching thermal insulation.
Energy3D is a "smart" CAD tool because it can monitor the designer's behavior in real time, based on which it can generate feedback to the designer to regulate the design behavior. This capacity has tremendous implications to learning and teaching scientific inquiry and engineering design with open-ended nature that requires, ideally, one-to-one tutoring so intense that no teacher can easily provide in real classrooms.

The computational mechanism for generating feedback in Energy3D is based on intelligent agents, which consist of sensors and actuators (in very generic terms). In Energy3D, all the events are logged behind the scenes. The events provide the raw data stream from which various sensors produce signals based on subsets of the raw data. For instance, a sensor can be created to monitor any event related to solar panels of a house. An agent then uses a decision tree model to determine which actuators should be called to provide feedback to the user or direct Energy3D to change its state. For instance, if a solar panel is detected to be placed on the north-facing roof, the agent can remind the designer to rethink about the decision. Just like what a teacher may do, the agent can even suggest a comparative analysis between a solar panel on the north-facing roof and a solar panel on the south-facing or west-facing roof. Although this type of inquiry and design can be also taught using directly scaffolded instruction that guides students to explore step by step, in practice we have found the effect of this approach often diminishes because many students do not read instruction carefully enough and remember them long enough. It is also challenging for teachers to guide the whole class through this kind of long learning process as students often pace differently. Adaptive feedback provides a way to help students only when they need or just when a need is detected, thus providing a better chance to deliver effective instruction.

Let's look at a very simple example. Figure 1 shows a learning activity, the goal of which is to teach how the thermal property of a wall, called the U-value, affects the energy use of a house. Many students may walk away with a shallow understanding that the higher the U-value is, the more energy a house uses. The challenge is to help them deepen their understanding. For example, how can we make sure that students will collect enough data points to discover that the energy a house uses is proportional to the U-value? How can we support them to find out that the relationship is independent of seasonal change, wall orientation, and solar radiation (e.g., a lower U-value is good in both summer and winter, irrespective of whether or not the wall faces the sun). Helping students accomplish this level of understanding through inquiry-based activities is by no means a trivial task, even in this seemingly simple example. Let's explore what we may do in Energy3D now that we have a way to monitor students' interactions with it.

Fig. 2: An event sequence coded like a DNA sequence.
In nearly all software that support learning and teaching, the events during a process can be coded as a string with characters representing the events and ordered by their timestamps, such as Figure 2. In this case, A represents an analysis event in the Energy3D CAD tool, U represents an event of changing the U-value of a wall, C represents an event of changing the date for the energy simulation, a questionmark (?) represents an event of requesting help from the software, an underscore (_) represents an inactive time period longer than a certain threshold, and * is a wildcard that represents any other event "silenced" in this expression in order to reduce the dimensionality of the problem. For those who know a bit about bioinformatics, this resembles a DNA sequence. In the context of Energy3D, we may also call it as the DNA of a design, if that helps your imagination.

Now that we have converted the sequence of events into a string, we can use all sorts of techniques that have been developed to analyze strings to analyze these events, including those developed in bioinformatics such as sequence alignment or those developed in natural language processing. In this article, I am going to show how the widely-supported regular expressions (regex) can be used as a technique to detect whether a certain type of event or a certain combination of events occurred or how many times it occurred. I feel that regex, in our case, may be more accurate than edit distances such as the Levenshtein distance in matching the pattern. For example, a single substitution of event may represent a very different process despite the short edit distance.

Fig. 3: A sequence that shows high usage of feedback
We know that, a fundamental skill of inquiry is to keep everything else fixed but change only one variable at a time and then test how the system's output depends on that variable. Through this process of inquiry, we learn the meaning of that variable, as explained by Bruce Alberts, former president of the National Academy of Sciences and former Editor-in-Chief of the Science Magazine. In the example discussed here, that variable is the U-value of a selected wall of the house and the test is the simulation-based analysis. A pattern that has alternating U and A characters in the event string suggests a high probability of inquiry, which can be captured using a simple regex such as (U[_\\*\\?]*A)+. Between U and A, however, there may be other types of events that may or may not exist to weaken the probability or compromise the rigor. For example, changing the color of the wall between U and A may also result in an additional difference in energy use of the house that originates from the absorption of solar radiation by the external surface of the wall and has nothing to do with its U-value. In this case, changing multiple variables at a time appears to be a violation of the aforementioned inquiry principle that should be called out by the agent using another regex to analyze the substring between U and A.

An interesting feature in Energy3D is that feedback itself is also logged. Figure 3 shows a sequence that has an alternation pattern similar to that of Figure 2, but it records a type of behavior showing that the user may rely overly on feedback from the system to learn (the questionmarks in the string stand for feedback requests made by the user) and avoid deep thinking on their own. This may be a common problem in many intelligent tutors (sometimes this behavior is called "gaming the system").

The development of data mining and intelligent agents in Energy3D is opening interesting opportunities of research that will only grow more important in the era of artificial intelligence (AI). We are excited to be part of this wave of AI innovation.

General Motors funds engineering education based on Energy3D

Designing a parking lot solar canopy at Detroit Airport
General Motors (GM), along with other RE100 companies, has committed to powering its worldwide factories and offices with 100% renewable energy by 2050. Last month, the company furthered its commitment by giving the Engineering Computation Team at the Concord Consortium a $200,000 grant to promote engineering education using renewable energy as a learning context and artificial intelligence as a teaching assistant.

Modeling GM's rooftop solar arrays in Baltimore, MD
Modeling GM's solar arrays in Warren, MI
The project will use our signature Energy3D software, which is a one-stop-shop CAD tool for designing and simulating all kinds of solar power systems including photovoltaic (PV) and concentrated solar power (CSP), both of which have reached a very competitive cost of merely 5¢ per kWh or below in the world market. A unique feature of Energy3D is its ability to collect and analyze "atomically" fine-grained process data while users are designing with it. This capability makes it possible for us to develop machine learning algorithms to understand users' design behaviors, based on which we can develop intelligent agents to help users design better products and even unleash their creativity.

The generous grant from GM will allow us to bring this incredible engineering learning tool and the curriculum materials it supports to more science teachers across New England. It will also help extend our fruitful collaboration with the Virtual High School (VHS) to convert our Solarize Your World curriculum into an online course for sustainable engineering. VHS currently offers more than 200 titles to over 600 member schools. Through their large network, we hope to inspire and support more students and teachers to join the crucial mission that GM and other RE100 companies are already undertaking.

By supporting today's students to learn critical engineering design skills needed to meet the energy and environmental challenges, GM is setting an example of preparing tomorrow's workforce to realize its renewable energy vision.

High Frequency Electronics and Thermtest feature Energy2D

Credit: High Frequency Electronics
High Frequency Electronics is a magazine for engineers. In the cover article titled "Substrate Selection Can Simplify Thermal Management" in its November 2017 issue, author John Ranieri included our Energy2D software as one of the modeling tools recommended to the reader, alongside with mainstream commercial products from industry leaders such as Mentor Graphics and ANSYS. The software is also featured by Thermtest, a UK-based company that focuses on thermophysical instruments. Thermtest supplements the software with a database of standard materials, making it easier for engineers to use.

An Energy2D model of a heat source and a heat sink
According to the article, "heat haunts many RF/microwave and power electronics circuits and can limit performance and reliability. The heat generated by a circuit is a function of many factors, including input power, active device efficiencies, and losses through passive devices and transmission lines. It is often not practical to disperse heat from a circuit by convection fan-driven cooling, and heat must be removed from sensitive components and devices, by creating a thermal path to a metal enclosure or heat sink with good thermal conductivity." As a thermal simulation tool, Energy2D can certainly be very useful in helping engineers conceptualize and design such thermal paths.

More importantly, Energy2D can make your engineering experience as fun as playing a sandbox game! As one of our users recently wrote, "I am working as consulting engineer and we often have to make quick estimations where a steady-state node model is too simplified and setting up a complex FEM model is overkill. Energy2D is a very handy tool for something [like] that and I like the click'n'play sandbox feeling in combination with the physical correctness. I never thought FEM could be that fun."

Energy3D allows users to select brand name solar panels

Fig. 1: 20 brand name solar panels in Energy3D
Fig. 2: The daily outputs of 20 types of solar panels
Previous versions of Energy3D were based on a generic model of solar panel, which users can set its properties such as solar cell type, peak efficiency, panel dimension, color, nominal operating cell temperature, temperature coefficient of power, and so on. While it is essential for users to be able to adjust these parameters and learn what they represent and how they affect the output, it is sometimes inconvenient for a designer to manually set the properties of a solar panel to those of a brand name.

Fig. 3: The Micky Mouse solar farm
From Version 7.4.4, I started to add support of brand name solar panels to Energy3D. Twenty brand names were initially added to this version (Figure 1). These models are: ASP-400M (Advanced Solar Photonics), CS6X-330M-FG (Canadian Solar), CS6X-330P-FG (Canadian Solar), FS-4122-3 (First Solar), HiS-M280MI (Hyundai), HiS-S360RI (Hyundai), JAM6(K)-60-300/PR (JA Solar), JKM300M-60 (Jinko), LG300N1C-B3 (LG), LG350Q1K-A5 (LG), PV-UJ235GA6 (Mitsubishi), Q.PRO-G4 265 (Q-cells), SPR-E20-435-COM (SunPower), SPR-P17-350-COM (SunPower), SPR-X21-335-BLK (SunPower), SPR-X21-345 (SunPower), TSM-325PEG14(II) (Trina Solar), TSM-365DD14A(II) (Trina Solar), VBHN330SA16 (Panasonic), and YL305P-35b (Yingli). Figure 2 shows a comparison of their daily outputs in Boston on June 22 when they are laid flat (i.e., with zero tilt angle). Not surprisingly, a smaller solar panel with a lower cell efficiency produces less electricity.

Note that these models are relatively new. There are hundreds of older and other types of solar panels that will take a long time to add. If your type is not currently supported, you can always fall back to defining it using the "Custom" option, which is the default model for a solar panel.

Adding these brand names helped me figure out that the solar panels deployed in the Micky Mouse Solar Farm in Orlando (Figure 3) are probably from First Solar -- only they make solar panels of such a relatively small size (1200 mm × 600 mm).

Learn about two Concord Consortium projects at EdSurge Fusion Conference

Bill Finzer and Sherry Hsi will both present at the EdSurge Fusion Conference in Burlingame, California, near our Emeryville office.

The Common Online Data Analysis Platform—Getting more students in more classrooms to do more with data

William Finzer
Thursday, November 2
12:00 – 1:00 PM

CODAP is a free web-based data tool designed as a platform for developers and as an application for students in grades 6–14. Designed with learning in mind, CODAP continues the legacy of the award-winning software packages Fathom and TinkerPlots. It builds on a decades-long legacy of research into interactive environments encouraging exploration, play, and puzzlement. CODAP is about exploring and learning from data from any content area—from math and science to social studies or physical education!

The data set in CODAP has information on 27 mammals, including humans! Learn more by examining the tables and graphs.

Computationally-Enhanced Papercrafts for Engineering Education

Sherry Hsi
Thursday, November 2
12:00 – 1:00 PM

Paper Mechatronics is a novel design medium integrating traditional educational papercrafts with mechanical design, electronic engineering, and computational thinking. Paper mechatronics makes possible a craft-oriented approach to engineering and computing education that integrates key concepts from mechanical engineering, electrical engineering, control systems, and computer programming, while using paper as the primary material for learner design, exploration, and inquiry.

Watch how to create your own devices from cardboard – machines, robots, toys, automata, kinetic artwork – that move!

Even Fiction Can Expand Our Understanding of Science

Andy Zucker was a senior scientist at the Concord Consortium who is now enjoying his retirement, including working with the Greater Boston Interfaith Organization (GBIO).  

Many people know Michael Crichton’s novel Jurassic Park, in which he posits that humans used remnants of dinosaur DNA to imprudently create a modern theme park populated with dinosaurs. Crichton often used science as a takeoff point in his novels. But Harvard scientist George Church is currently working to revive woolly mammoths using DNA samples frozen for thousands of years.

A value of Crichton’s works is they remind us of the important role that data play in science. Science is not only an experimental science. It often relies more heavily than standard textbooks suggest on the accumulation of accurate data long before theories explain the data.

In the latest Crichton novel, Dragon Teeth, a newly discovered manuscript posthumously published nine years after his death, fossil hunters work in the American West in 1876. Although fictionalized, Dragon Teeth is based on a real-life rivalry between two remarkable, obsessive men—Edwin Drinker Cope of the University of Pennsylvania and Othniel Charles Marsh of Yale—who were responsible for finding fossils of more than 1,500 species. Their exploits were known as “the Bone Wars.”

The fictional protagonist is based on the real-life fossil hunter Charles Sternberg, who supplied superb fossils to scholars and museums around the world, and who wrote, “I could tell of a hundred narrow escapes from death.” Larger-than-life figures like Cornelius Vanderbilt and Wyatt Earp were alive in 1876 and play roles in the novel. It is easy to see how Crichton used authentic history to create a fast-paced adventure story.

Some ancient people thought dinosaur bones came from dragons, but it was not until the 1840s that the term “dinosaur” was coined. The first dinosaur fossil was discovered in America in 1858. The site where Custer and his troops met their ignominious end in 1876, Montana’s Little Bighorn, is not far from key locations where dinosaur fossils were collected.

Data collectors are the unsung heroes of science. Without the thousands of butterflies collected in the Amazon by Henry Bates, scientists would not have had direct evidence of the creation of a new species—a discovery that Darwin called the “beautiful proof” for natural selection. Johannes Kepler was the first to understand that the planets move in elliptical orbits; his theory relied on the data of others (e.g., Tycho Brahe). The photos of X-ray crystallographer Rosalind Franklin were used by Watson, Crick, and Wilkins to establish that DNA has a helical structure. Our current understanding of dinosaurs and how they were wiped out by a meteor strike depends on data from fossils, but also from ancient pollen, geological finds, and astronomical data.

Dragon Teeth proves that even fiction can broaden our understanding of science and of the data collectors responsible for enlarging human understanding of the world.

Learn about watersheds at MSELA Conference

Carolyn Staudt will present information about the NSF-funded Teaching Environmental Sustainability: Model My Watershed project and share free resources at the Massachusetts Education Leadership Association (MSELA) 2017 conference.

Friday, October 20, 8:00 – 9:15 AM
Courtyard Marriott in Marlborough, MA
Marlborough Salon E

The Teaching Environmental Sustainability: Model My Watershed project is a collaborative research project at the Concord Consortium, Millersville University, and the Stroud Water Research Center.

Together, we’re teaching a systems approach to problem solving through modeling and hands-on activities based on local watershed data and issues. The curricula also integrate low-cost environmental sensors, allowing students to collect and upload their own data and compare them to data visualized on the free Model My Watershed app.

If you’re wondering what a watershed is, you’re not alone. Simply put, a watershed is “all the land area where the rain runs downhill to a certain point,” explains Carolyn Staudt, who directs the Teaching Environmental Sustainability: Model My Watershed project at the Concord Consortium. She continues, “Water is shared—there are people upstream and downstream. What you do with your local watershed impacts everyone.”

Model My Watershed models human impacts on a watershed.

Learn more

MSELA conference
Teaching Environmental Sustainability: Model My Watershed
Part I: What is a Watershed?
Part II: Part II: Students Learn about Water . . .  and Take Action
Monday’s Lesson: Can you filter your water?