Tag Archives: Engineering design

Seeing student learning with visual analytics

Technology allows us to record almost everything happening in the classroom. The fact that students' interactions with learning environments can be logged in every detail raises the interesting question about whether or not there is any significant meaning and value in those data and how we can make use of them to help students and teachers, as pointed out in a report sponsored by the U.S. Department of Education:
New technologies thus bring the potential of transforming education from a data-poor to a data-rich enterprise. Yet while an abundance of data is an advantage, it is not a solution. Data do not interpret themselves and are often confusing — but data can provide evidence for making sound decisions when thoughtfully analyzed.” — Expanding Evidence Approaches for Learning in a Digital World, Office of Educational Technology, U.S. Department of Education, 2013
A digraph of action transition
A histogram of action intensity
Here we are not talking about just analyzing students' answers to some multiple-choice questions, or their scores in quizzes and tests, or their frequencies of logging into a learning management system. We are talking about something much more fundamental, something that runs deep in cognition and learning, such as how students conduct a scientific experiment, solve a problem, or design a product. As learning goes deeper in those directions, data produced by students becomes bigger. It is by no means an easy task to analyze large volumes of learner data, which contain a lot of noisy elements that cast uncertainty to assessment. The validity of an assessment inference rests on  the strength of evidence. Evidence construction often relies on the search for relations, patterns, and trends in student data.With a lot of data, this mandates some sophisticated computation similar to cognitive computing.

Data gathered from highly open-ended inquiry and design activities, key to authentic science and engineering practices that we want students to learn, are often intensive and “messy.” Without analytic tools that can discern systematic learning from random walk, what is provided to researchers and teachers is nothing but a DRIP (“data rich, information poor”) problem.

A polar chart of problem space exploration
Recognizing the difficulty in analyzing the sheer volume of messy student data, we turned to visual analytics, a whole category of techniques extensively used in cutting-edge business intelligence systems such as software developed by SAS and others. We see interactive, visual process analytics key to accelerating the analysis procedures so that researchers can adjust mining rules easily, view results rapidly, and identify patterns clearly. This kind of visual analytics optimally combines the computational power of the computer, the graphical user interface of the software, and the pattern recognition power of the brain to support complex data analyses in data-intensive educational research.

A scatter plot of action timeline
Within a week, I was able to write four interactive graphs and charts that can be used to study four different aspects of the design action data that we collected from our Energy3D CAD software. Recording several weeks of student work on complex engineering design challenges, these datasets are high-dimensional, meaning that it is improper to treat them from a single point of view. For each question we are interested in getting answers from student data, we usually need a different representation to capture the outstanding features specific to the question.

In the long run, our overall objective is to add as many graphic representations as possible as we move along in answering more and more research questions based on our datasets. Given time, this growing library of visual analytics would develop sufficient power to the point that it may also become useful for teachers to monitor their students' work and thereby conduct formative assessment. To guarantee that our visual analytics runs on all devices, this library is written in JavaScript/HTML/CSS. A number of touch gestures are also supported for users to use the library on a multi-touch screen. A neat feature of this library is that multiple graphs and charts can be grouped together so that when you are interacting with one of them, the linked ones also change at the same time. As the datasets are temporal in nature, you can also animate these graphs to reconstruct and track exactly what students do throughout.

The National Science Foundation funds SmartCAD—an intelligent learning system for engineering design

We are pleased to announce that the National Science Foundation has awarded the Concord Consortium, Purdue University, and the University of Virginia a $3 million, four-year collaborative project to conduct research and development on SmartCAD, an intelligent learning system that informs engineering design of students with automatic feedback generated using computational analysis of their work.

Engineering design is one of the most complex learning processes because it builds on top of multiple layers of inquiry, involves creating products that meet multiple criteria and constraints, and requires the orchestration of mathematical thinking, scientific reasoning, systems thinking, and sometimes, computational thinking. Teaching and learning engineering design becomes important as it is now officially part of the Next Generation Science Standards in the United States. These new standards mandate every student to learn and practice engineering design in every science subject at every level of K-12 education.
Figure 1

In typical engineering projects, students are challenged to construct an artifact that performs specified functions under constraints. What makes engineering design different from other design practices such as art design is that engineering design must be guided by scientific principles and the end products must operate predictably based on science. A common problem observed in students' engineering design activities is that their design work is insufficiently informed by science, resulting in the reduction of engineering design to drawing or crafting. To circumvent this problem, engineering design curricula often encourage students to learn or review the related science concepts and practices before they try to put the design elements together to construct a product. After students create a prototype, they then test and evaluate it using the governing scientific principles, which, in turn, gives them a chance to deepen their understanding of the scientific principles. This common approach of learning is illustrated in the upper image of Figure 1.

There is a problem in the common approach, however. Exploring the form-function relationship is a critical inquiry step to understanding the underlying science. To determine whether a change of form can result in a desired function, students have to build and test a physical prototype or rely on the opinions of an instructor. This creates a delay in getting feedback at the most critical stage of the learning process, slowing down the iterative cycle of design and cutting short the exploration in the design space. As a result of this delay, experimenting and evaluating "micro ideas"--very small stepwise ideas such as those that investigate a design parameter at a time--through building, revising, and testing physical prototypes becomes impractical in many cases. From the perspective of learning, however, it is often at this level of granularity that foundational science and engineering design ultimately meet.

Figure 2
All these problems can be addressed by supporting engineering design with a computer-aided design (CAD) platform that embeds powerful science simulations to provide formative feedback to students in a timely manner. Simulations based on solving fundamental equations in science such as Newton’s Laws model the real world accurately and connect many science concepts coherently. Such simulations can computationally generate objective feedback about a design, allowing students to rapidly test a design idea on a scientific basis. Such simulations also allow the connections between design elements and science concepts to be explicitly established through fine-grained feedback, supporting students to make informed design decisions for each design element one at a time, as illustrated by the lower image of Figure 1. These scientific simulations give the CAD software tremendous disciplinary intelligence and instructional power, transforming it into a SmartCAD system that is capable of guiding student design towards a more scientific end.

Despite these advantages, there are very few developmentally appropriate CAD software available to K-12 students—most CAD software used in industry not only are science “black boxes” to students, but also require a cumbersome tool chaining of pre-processors, solvers, and post-processors, making them extremely challenging to use in secondary education. The SmartCAD project will fill in this gap with key educational features centered on guiding student design with feedback composed from simulations. For example, science simulations can be used to analyze student design artifacts and compute their distances to specific goals to detect whether students are zeroing in towards those goals or going astray. The development of these features will also draw upon decades of research on formative assessments of complex learning.

A stock-and-flow model for building thermal analysis

Figure 1. A stock-and-flow model of building energy.
Our Energy3D CAD software has two built-in simulation engines for performing solar energy analysis and building thermal analysis. I have extensively blogged about solar energy analysis using Energy3D. This article introduces building thermal analysis with Energy3D.

Figure 2. A colonial house.
The current version of the building energy simulation engine is based on a simple stock-and-flow model of building energy. Viewed from the perspective of system dynamics—a subject that studies the behavior of complex systems, the total thermal energy of a building is a stock and the energy gains or losses through its various components are flows. These gains or losses usually happen via the energy exchange between the building and the environment through the components. For instance, the solar radiation that shines into a building through its windows are inputs; the heat transfer through its walls may be inputs or outputs depending on the temperature difference between the inside and the outside.

Figure 3. The annual energy graph.
Figure1 illustrates how energy flows into and out of a building in the winter and summer, respectively. In order to maintain the temperature inside a building, the thermal energy it contains must remain constant—any shortage of thermal energy must be compensated and any excessive thermal energy must be removed. These are done through heating and air conditioning systems, which, together with ventilation systems, are commonly known as HVAC systems. Based on the stock-and-flow model, we can predict the energy cost of heating and air conditioning by summing up the energy flows in various processes of heat transfer, solar radiation, and energy generation over all the components of the building such as walls, windows, or roofs and over a certain period of time such as a day, a month, or a year.

Figure 2 shows the solar radiation heat map of a house and the distribution of the heat flux density over its building envelope. Figure 3 shows the results of the annual energy analysis for the house shown in Figure 2.

More information can be found in Chapter 3 of Energy3D's User Guide.

Common architectural styles supported by Energy3D


Energy3D supports the design of some basic architectural styles commonly seen in New England, such as Colonial and Cape Cod. Its simple 3D user interface allows users to quickly sketch up a house with an aesthetically pleasing look -- with only mouse clicks and drags (and, of course, some patience). This makes it easy for middle and high school students to create meaningful, realistic designs and learn science and engineering from these authentic experiences -- who wants to keep doing those cardboard houses that look nothing like a real house for another 100 years?

The true enabler of science learning in Energy3D is its analytic capability that can tell students the energy consequences of their designs while they are working on them. Without this analytical capability, learning would have been cut short at architectural design (which undeniably is the fun part of Energy3D that entices students to explore many different design options that entertain the eyes). With the analytical capability, the relationship between form and function becomes a major driving force for student design. It is at this point that an Energy3D project becomes an engineering design project.

Architectural design, which focuses on designing the form, and engineering design, which focuses on designing the function, are equally important in both educational and professional practices. Students need to learn both. After all, the purpose of design is to meet various people's needs, including their aesthetic needs. This principle of coupling architectural design and engineering design is of generic importance as it can be extended to the broader case of integrating industrial design and engineering design. It is this coupling that marries art, science, and usability.

We are working on providing a list of common architectural styles that can be designed using Energy3D. These styles, four of them are shown in this article, show only the basic form of each style. Each should only take less than an hour to sketch up for beginners. If you want, you can derive more complex and detailed designs for each style.

Simulating cool roofs with Energy3D

Fig. 1: Solar absorption of colors.
Cool roofs represent a simple solution that can save significant air-conditioning cost and help mitigate the urban heat island effect, especially in hot climates. Nobel Prize winner and former Secretary of Energy Steven Chu is a strong advocate of cool roofs. It was estimated that painting all the roofs and pavements around the world with reflective coatings would be "equivalent to getting 300 millions cars off the road!"

With Version 4.0 of Energy3D (BTW, this version supports 200+ worldwide locations -- with 150+ in the US), you can model cool roofs and evaluate how much energy you can save by switching from a dark-colored roof to a light-colored one. All you need to do is to set the colors of your roofs and other building blocks. Energy3D will automatically assign an albedo value to each building block according to the lightness of its color.

Figure 1 shows five rectangles in different gray colors (upper) and their thermal view (lower). In this thermal view, blue represents low energy absorption, red represents high energy absorption, and the colors in-between represents the energy absorption at the level in-between.

Now let's compare the thermal views of a black roof and a white roof of a cape code house, as shown in Figure 2. To produce Figure 2, the date was set to July 1st, the hottest time of the year in northern hemisphere, and the location was set to Boston.

Fig. 2: Compare dark and white roofs.
How much energy can we save if we switch from a perfectly black roof (100% absorption) to a perfectly white roof (0% absorption)? We can run the Annual Energy Analysis Tool of Energy3D to figure this out in a matter of seconds. The results are shown in Figure 3. Overall, the total yearly energy cost is cut from 6876 kWh to 6217 kWh for this small cape code house, about 10% of saving.

Figure 3 shows that the majority of savings comes from the reduction of AC cost. The reason that the color has no effect on heating in the winter is because the passive solar heat gains through the windows in this well-insulated house is enough to keep it warm during the sunshine hours. So the additional heat absorbed by the black roof in the same period doesn't offset the heating cost (it took me quite a while to figure out that this was not a bug in our code but actually the case in the simulation).

Fig. 3: Compare heating and AC costs (blue is white roof).
Of course, this result depends on other factors such as the U-value and thermal mass of the roof. In general, the better the roof is insulated, the less its color impacts the energy cost. With Energy3D, students can easily explore these design variables.

This new feature, along with others such as the heat flux visualization that we have introduced earlier, represents the increased capacity of Energy3D for performing function design using scientific simulations.

Here is a video that shows the heating effect on roofs of different colors.

Visualization of heat flux in Energy3D using vector fields

Fig. 1: Winter in Boston
One of the strengths of our Energy3D CAD software is its 3D visualizations of energy transfer. These visualizations not only allow students to see science concepts in action in engineering design, but also provide informative feedback for students to make their design choices based on scientific analyses of their design artifacts.

Fig. 2: Summer in Boston
A new feature has been added to Energy3D to visualize heat transfer across the building envelope using arrays of arrows. Each arrow represents the heat flux at a point on the surface of the building envelope. Its direction represents the direction of the heat flux and its length represents the magnitude of the heat flux, calculated by using Fourier's Law of Heat Conduction. Quantitatively, the length is proportional to the difference between the temperatures inside and outside the building, as well as the U-value of the material.

Fig. 3: Winter in Miami
The figures in this post show the heat flux visualizations of the same house in the winter and summer in Boston and Miami, respectively. Like the solar radiation heat map shown in the figures, the heat flux is the daily average. The U-value of the windows is greater than those of the walls and roof. Hence, you can see that the heat flux vectors in the winter sticking out of the windows are much longer than those sticking out of the walls or roof. In the summer, the heat flux vectors point into the house but they are much shorter, agreeing with the fact that Boston's summer is not very hot.

Fig. 4: Summer in Miami
Now move the same house to Miami. You can see that even in the winter, the daily average heat flux points inside the house, agreeing with the fact that Miami doesn't really have a winter. In the summer, however, the heat flux into the house becomes significantly large.

These visualizations give students clear ideas about where a house loses or gains energy the most. They can then adjust the insulation values of those weak points and run simulations to check if they have been fixed or not. Compared with just giving students some formulas or numbers to figure out what they actually mean to science and engineering practices, experiential learning like this should help students develop a true understanding of thermal conduction and insulation in the context of building science and technology.

Here is a YouTube video of the heat flux view.

A 16-year-old’s designs with Energy3D

This post needs no explanation. The images say it all.

All these beautiful structures were designed from scratch (NOT imported from other sources) by Cormac Paterson using our Energy3D CAD software.

He is only 16 years old. (We have his parents' permission to reveal his name and his work.)

From conceptual design to detailed design with Energy3D

Figure 1: Empire State Building
An important objective of our Energy3D software is to explore how to create CAD software that support students to practice the full cycle of engineering design from conceptual design to detailed design in a single piece of software. We believe that interactive 3D visualizations and simulations provided by CAD tools are cognitively important for K-12 students who have little prior knowledge about the subject of design or the process of design to develop some sense of them -- through practice. Instantaneous visualizations of the results of their actions within the CAD software can give students some concrete clues to develop, share, and refine their ideas directly within a visual design space.

Although there have been some cautions about the use of CAD software in design education, my take is that, in the very early stage of grassroots design education, the problem is not that students are handicapped by a design tool; the problem is that they lack ideas to start with or skills to put their ideas into actions and should be aided by an intelligent design tool (in addition to a teacher, of course). A good CAD tool will be very instructive in this stage. Only after students rise to a certain expert level will the limitations of the CAD software begin to emerge. Often in the K-12 settings, the time constraint does not allow the majority of students to reach that level through conventional instruction, however. Hence, it is likely that the positive effects of using CAD software in K-12 engineering education will outweigh the negative effects, letting alone that students will learn important computer design and modeling skills that will be extremely useful to their future STEM careers.
Figure 2: Freedom Tower

But not all CAD software were created equal. Many CAD software have been developed for professional engineers and are not appropriate for K-12 applications, even though many software vendors have managed to enter the K-12 market in recent years. Given the rise of engineering in K-12 schools, it is probably the right time to rethink how to develop a CAD platform that supports design, learning, and assessment from the ground up.

Our Energy3D CAD software has provided us a powerful platform to ponder about these questions. From the beginning of this project back in 2010, we had been envisioning a CAD platform that integrates conceptual design, detailed design, collaborative design, numerical analysis, designer modeling, machine learning, and digital fabrication. After four years' continuous work, the software can now do not only conceptual design like a sketch tool but also detailed design like a production CAD program (look at the details in Figure 3). In terms of education, this means that it has a very short learning curve that allows all students to translate their ideas into computer models in a short amount of time and, meanwhile, a very deep design space that allows some of the willing students to advance to an expert level. With these capacities, we are now conducting leading-edge data mining research to investigate how to facilitate the transition. The research will eventually translate into novel software features of the CAD program.
Figure 3: Kendall Square

Cormac Paterson, a brilliant high school student from Arlington, MA, has demonstrated these possibilities. He has created many designs with Energy3D that are showcased in our model repository, including all the designs shown in the figures of this post.

On the instructional sensitivity of computer-aided design logs

Figure 1: Hypothetical student responses to an intervention.
In the fourth issue this year, the International Journal of Engineering Education published our 19-page-long paper on the instructional sensitivity of computer-aided design (CAD) logs. This study was based on our Energy3D software, which supports students to learn science and engineering concepts and skills through creating sustainable buildings using a variety of built-in design and analysis tools related to Earth science, heat transfer, and solar energy. This paper proposed an innovative approach of using response functions -- a concept borrowed from electrical engineering -- to measure instructional sensitivity from data logs (Figure 1).

Many researchers are interested in studying what students learn through complex engineering design projects. CAD logs provide fine-grained empirical data of student activities for assessing learning in engineering design projects. However, the instructional sensitivity of CAD logs, which describes how students respond to interventions with CAD actions, has never been examined, to the best of our knowledge.
Figure 2. An indicator of statistical reliability.

For the logs to be used as reliable data sources for assessments, they must be instructionally sensitive. Our paper reports the results of our systematic research on this important topic. To guide the research, we first propose a theoretical framework for computer-based assessments based on signal processing. This framework views assessments as detecting signals from the noisy background often present in large temporal learner datasets due to many uncontrollable factors and events in learning processes. To measure instructional sensitivity, we analyzed nearly 900 megabytes of process data logged by Energy3D as collections of time series. These time-varying data were gathered from 65 high school students who solved a solar urban design challenge using Energy3D in seven class periods, with an intervention occurred in the middle of their design projects.

Our analyses of these data show that the occurrence of the design actions unrelated to the intervention were not affected by it, whereas the occurrence of the design actions that the intervention targeted reveals a continuum of reactions ranging from no response to strong response (Figure 2). From the temporal patterns of these student responses, persistent effect and temporary effect (with different decay rates) were identified. Students’ electronic notes taken during the design processes were used to validate their learning trajectories. These results show that an intervention occurring outside a CAD tool can leave a detectable trace in the CAD logs, suggesting that the logs can be used to quantitatively determine how effective an intervention has been for each individual student during an engineering design project.

Design replay: Reconstruction of students’ engineering design processes from Energy3D logs

One of the useful features of our Energy3D software is the ability to record the entire design process of a student behind the scenes. We call the reconstruction of a design process from fine-grained process data design replay.


Design replay is not a screencast technology. The main difference is that it records a sequence of CAD models, not in any video format such as MP4. This sequence is played back in the original CAD tool that generated it, not in a video player. As such, every snapshot model is fully functional and editable. For instance, a viewer can pause the replay and click on the user interface of the CAD tool to obtain or visualize more information, if necessary. In this sense, design replay can provide far richer information than screencast (which records as much information as the pixels in the recording screen permit).


Design replay provides a convenient method for researchers and teachers to quickly look into students' design work. It compresses hours of student work into minutes of replay without losing any important information for analyses. Furthermore, the reconstructed sequence of design can be post-processed in many ways to extract additional information that may shed light on student learning, as we can use any model in the recorded sequence to calculate any of its properties.



The three videos embedded in this post show the design replays of three students' work from a classroom study that we just completed yesterday in a Massachusetts high school. Sixty-seven students spent approximately two weeks designing zero-energy houses -- a zero-energy house is a highly energy-efficient house that consumes net zero (or even negative) energy over a year due to its use of passive and active solar technologies to conserve and generate energy. These videos may give you a clue how these three students solved the design challenge.