The deception of unconditionally stable solvers

Unconditionally stable solvers for time-dependent ordinary or partial differential equations are desirable in game development because they are highly resilient to player actions -- they never "blow up." In the entertainment industry, unconditionally stable solvers for creating visual fluid effects (e.g., flow, smoke, or fire) in games and movies were popularized by Jos Stam's 1999 paper "Stable Fluids."

Figure 1: Heat conduction between two objects.
The reason that a solver explodes is because the error generated in a numerical procedure gets amplified in iteration and grows exponentially. This occurs especially when the differential equation is stiff. A stiff equation often contains one or more terms that change very rapidly in space or time. For example, a sudden change of temperature between two touching objects (Figure 1) creates what is known as a singularity in mathematics (a jump discontinuity, to be more specific). Even if the system described by the equation has many other terms that do not behave like this, one such term is enough to crash the whole solver if it is linked to other terms directly or indirectly. To avoid this breakdown, a very small time step must be used, which often makes the simulation look too slow to be useful for games.

The above problem typically occurs in what is known as the explicit method in the family of the finite-difference methods (FDMs) commonly used to solve time-dependent differential equations. There is a magic bullet for solving this problem. This method is known as the implicit method. The secret is that it introduces numerical diffusion, an unphysical mechanism that causes the errors to dissipate before they grow uncontrollably. Many unconditionally stable solvers use the implicit method, allowing the user to use a much larger time step to speed up the simulation.

There ain't no such thing as a free lunch, however. It turns out that we cannot have the advantages of both speed and accuracy at the same time (efficiency and quality are often at odd in reality, as we have all learned from life experiences). Worse, we may even be deceived by the stability of an unconditionally stable solver without questioning the validity of the predicted results. If the error does not drive the solver nuts and the visual looks fine, the result must be good, right?

Figure 2: Predicted final temperature vs. time step.
Not really.

The default FDM solver in Energy2D for simulating thermal conduction uses the implicit method as well. As a result, it never blows up no matter how large the time step is. While this provides good user experiences, you must be cautious if you are using it in serious engineering work that requires not only numerical stability but also numerical reliability (in games we normally do not care about accuracy as long as the visual looks entertaining, but engineering is a precision science). In the following, I will explain the problems using very simple simulations:

1. Inaccurate prediction of steady states

Figure 3. Much longer equilibration with a large time step.
Figure 1 shows a simulation in which two objects at different temperatures come into contact and thermal energy flows from the high-temperature object into the low-temperature one. The two objects have different heat capacities (another jump discontinuity other than the difference in initial temperatures). As expected, the simulation shows that the two objects approach the same temperature, as illustrated by the convergence of the two temperature curves in the graph. If you increase the time step, this overall equilibration behavior does not change. Everything seems good at this point. But if you look at the final temperature after the system reaches the steady state, you will find that there are some deviations from the exact result, as illustrated in Figure 2, when the time step is larger than 0.1 second. The deviation stabilizes at about 24°C -- 4°C higher than the exact result.
Figure 4. Accurate behavior at a small time step.

2. Inaccurate equilibration time

The inaccuracy at large time steps is not limited to steady states. Figure 3 shows that the time it takes the system to reach the steady state is more than 10 times (about 1.5 hours as opposed to roughly 0.1 hours -- if you read the labels of the horizontal time axis of the graph) if we use a time step of 5 seconds as opposed to 0.05 second. The deceiving part of this is that the simulation appears to run equally quickly in both cases, which may fool your eyes until you look at the numerical outputs in the graphs.

3. Incorrect transient behaviors

Figure 5. Incorrect behavior at a very large time step.
With a more complex system, the transient behaviors can be affected more significantly when a large time step is used. Figure 4 shows a case in which the thermal conduction through two materials of different thermal conductivities (wood vs. metal) are compared, with a small time step (1 second). Figure 5 shows that when a time step of 1,000 seconds is used, the wood turns out to be initially more conductive than metal, which, of course, is not correct. If the previous example with two touching objects suggests that the simulation result can be quantitatively inaccurate at large time steps, this example means that the results can also be qualitatively incorrect in some cases (which is worse).

The general advice is to always choose a few smaller time steps to check if your results would change significantly. You can use a large time step to set up and test your model rapidly. But you should run your model at smaller time steps to validate your results.

The purpose of this article is to inform you that there are certain issues with Energy2D simulations that you must be aware if you are using it for engineering purposes. If these issues are taken care of, Energy2D can be highly accurate for conduction simulations, as illustrated by this example that demonstrates the conservation of energy of an isolated conductive system.

Energy2D and Quantum Workbench featured in Springer books


Two recently published Springer books have featured our visual simulation software, indicating perhaps that their broader impacts beyond their originally intended audiences (earlier I have blogged about the publication of the first scientific paper that used Energy2D to simulate geological problems).

A German book "Faszinierende Physik" (Fantastic Physics) includes a series of screenshots from a 2D quantum tunneling simulation from our Quantum Workbench software that shows how wave functions split when they smash into a barrier. The lead author of the book said in the email to us that he found the images generated by the Quantum Workbench "particularly beautiful."

Another book "Simulation and Learning: A Model-Centered Approach" chose our Energy2D software as a showcase that demonstrates how powerful scientific simulations can convey complex science and engineering ideas.

Quantum Workbench and Energy2D are based on solving extremely complex partial differential equations that govern the quantum world and the macroscopic world, respectively. Despite the complexity in the math and computation, both software present intuitive visualizations and support real-time interactions so that anyone can mess around with them and discover rich scientific phenomena on the computer.

European scientists use Energy2D to simulate submarine eruptions

The November issue of the Remote Sensing of Environment published a research article "Magma emission rates from shallow submarine eruptions using airborne thermal imaging" by a team of Spanish scientists in collaboration with Italian and American scientists. The researchers used airborne infrared cameras to monitor the 2011–2012 submarine volcanic eruption at El Hierro, Canary Islands and used our Energy2D software to calculate the heat flux distribution from the sea floor to the sea surface. The two figures in the blog post are from their paper.

According to their paper, "volcanoes are widely spread out over the seabed of our planet, being concentrated mainly along mid-ocean ridges. Due to the depths where this volcanic activity occurs, monitoring submarine volcanic eruptions is a very difficult task." The use of thermal imaging in this research, unfortunately, can only detect temperature distribution on the sea surface. Energy2D simulations turn out to be a complementary tool for understanding the vertical body flow.

Their research was supported by the European Union and assisted by the Spanish Air Force.

Although Energy2D started out as an educational program, we are very pleased to witness that its power has grown to the point that even scientists find it useful in conducting serious scientific research. We are totally thrilled by the publication of the first scientific paper that documents the validity of Energy2D as a research tool and appreciate the efforts of the European scientists in adopting this piece of software in their work.

Common architectural styles supported by Energy3D


Energy3D supports the design of some basic architectural styles commonly seen in New England, such as Colonial and Cape Cod. Its simple 3D user interface allows users to quickly sketch up a house with an aesthetically pleasing look -- with only mouse clicks and drags (and, of course, some patience). This makes it easy for middle and high school students to create meaningful, realistic designs and learn science and engineering from these authentic experiences -- who wants to keep doing those cardboard houses that look nothing like a real house for another 100 years?

The true enabler of science learning in Energy3D is its analytic capability that can tell students the energy consequences of their designs while they are working on them. Without this analytical capability, learning would have been cut short at architectural design (which undeniably is the fun part of Energy3D that entices students to explore many different design options that entertain the eyes). With the analytical capability, the relationship between form and function becomes a major driving force for student design. It is at this point that an Energy3D project becomes an engineering design project.

Architectural design, which focuses on designing the form, and engineering design, which focuses on designing the function, are equally important in both educational and professional practices. Students need to learn both. After all, the purpose of design is to meet various people's needs, including their aesthetic needs. This principle of coupling architectural design and engineering design is of generic importance as it can be extended to the broader case of integrating industrial design and engineering design. It is this coupling that marries art, science, and usability.

We are working on providing a list of common architectural styles that can be designed using Energy3D. These styles, four of them are shown in this article, show only the basic form of each style. Each should only take less than an hour to sketch up for beginners. If you want, you can derive more complex and detailed designs for each style.

Beautiful Chemistry


It is hard for students to associate chemistry with beauty. The image of chemistry in schools is mostly linked to something dangerous, dirty, or smelly. Yet Dr. Yan Liang, a collaborator and a materials scientist with a Ph.D. degree from the University of Minnesota, is launching a campaign to change that image. The result of his work is now online at beautifulchemistry.net.

To bring the beauty of chemistry to the general public, Dr. Liang uses 4K UltraHD cameras and special lenses to capture chemical reactions in astonishing detail and advanced computer graphics to render stunning images of molecular structures.

Using the beauty of science to interest students has rarely been taken seriously by educators. The federal government has invested billions of dollars in instructional materials development. But from a layman's point of view, it is hard to imagine how children can be engaged in science if they do not fall in love with it. Beautiful Chemistry represents an attempt that could inspire a whole new genre of high-quality educational materials based on breathtaking scientific visualizations. How about Beautiful Physics and Beautiful Biology?

Our work is well aligned with this vision. Our interactive, visual Energy2D simulations bring a beautiful world of heat and mass flow to students like never seen before; our Energy3D software creates splendid 3D scenes based on scientific calculations; and our infrared visualization of the real world has uncovered a beautiful hidden universe through an IR lens. These materials demonstrate computational and experimental ways to marry science and beauty and have resulted in great enticements in science classrooms.

BTW, Dr. Liang is the artist who designed the splash panes of Energy2D and Energy3D.

Two “Global Experiments” about Climate Change

In an earlier post on this blog I wrote about the need to increase our knowledge of how people think about climate change and then apply that knowledge to expedite policy changes. Subsequently I discovered that there is an active community of psychologists, experts in communications and other researchers who conduct valuable inquiries in this field.

Some of their findings are sobering, such as data from April 2014 showing that only one in three Americans discusses global warming with family and friends even occasionally. One of the many reasons this is true is that for more than five years, with little variation from year to year, only about one in three Americans have believed people in the U.S. are being harmed “right now” by global warming—despite Superstorm Sandy, Hurricane Katrina, extreme drought in the West and other once-rare climate events. One does not wish for more droughts, extreme heat waves, superstorms or wildfires; however, those are the kinds of events that may, slowly, change opinions about climate change.

People who realize how threatening climate change is believe that we are conducting a risky “experiment” with the sensitivity of Earth systems to increasing levels of carbon dioxide and other heat-trapping gases. For example, we still don’t understand climate “tipping points” well; however, our global experiment is already teaching us about tipping points, like it or not.

Professor Richard Somerville of the Scripps Institution of Oceanography at UC San Diego—who was Coordinating Lead Author of the Intergovernmental Panel on Climate Change’s fourth assessment report (2007)—wrote, in his 2006 paper called “Medical metaphors for climate change,”

“What is still not obvious to many is that all of us are now engaged in a second global experiment [emphasis added], this time an educational and geopolitical one. We are going to find out whether humanity is going to take climate science seriously enough to act meaningfully, rather than just procrastinating until nature ultimately proves that our climate predictions were right.”

This educational experiment—where education is broadly defined and includes more than what is taught in schools—could hardly be more important. It will take a concerted effort, over many years, by formal and informal institutions, political leaders and organizations, TV stations, museums, churches and others, to increase knowledge and a sense of urgency among policymakers and the public. The Concord Consortium’s High Adventure Science (HAS) project is one element in this long-term effort.

The Next Generation Science Standards include climate change as an important topic for instruction, which is significant because more people need some understanding of the science behind climate change. But the “educational experiment” the world is conducting requires policymakers and the public to learn far more than science. Better understanding the economics, political, regulatory, governance, diplomatic and technology issues needed to address climate change will be vital, as will an emphasis on ethics and values. It would be interesting to know how, and how often, climate change is a topic of study in other subjects taught in schools, colleges and universities (including social science classes, like Psychology), or whether it is addressed almost exclusively in “hard science” courses.

Visualizing the "thermal breathing" of a house in 24-hour cycle with Energy3D

The behavior of a house losing or gaining thermal energy from the outside in a 24-hour cycle, when visualized using Energy3D's heat flux view, resembles breathing, especially in the transition between seasons in which the midday can be hot and the midnight can be cold. We call this phenomenon the "thermal breathing" of a house. This embedded YouTube video in this blog post illustrates this effect. For the house shown in the video, the date was set to be May 1st and the location is set to Santa Fe, New Mexico.


This video only shows the daily thermal breathing of a house. Considering the seasonal change of temperature, we may also definite a concept "annual thermal breathing," which describes this behavior on an annual basis.

This breathing metaphor may help students build a more vivid mental picture of the dynamic heat exchange between a house and the environment. Interestingly, it was only after I realized this thermal visualization feature in Energy3D that this metaphor came to my mind. This experience reflects the importance of doing in science and engineering: Ideas often do not emerge until we get something concrete done. This process of externalization of thinking is critically important to the eventual internalization of ideas or concepts.

Using particle feeders in Energy2D for advection simulations

Fig. 1: Particle advection behind two obstacles.
Advection is a transport mechanism in which a substance is carried by the flow of a fluid. An example is the transport of sand in a river or pollen in the air. Advection is different from diffusion, whereas the more commonly known term, convection, is the combination of advection and diffusion.

Our Energy2D can simulate advection as it integrates particle dynamics in the Lagrangian frame and fluid dynamics in the Eulerian frame. Particles in Energy2D do not spontaneously diffuse -- they are driven by gravity or fluid, though we can introduce Brownian particles in the future by incorporating the Langevin Equation into Energy2D.

Fig. 2: Blowing away particles.
Over this weekend, I added a new object, the particle feeder, for creating continuous particle flow in the presence of open mass boundary. A particle feeder can emit a specified type of particle at a specified frequency. All these settings can be adjusted in its property window, which can be opened by right-clicking on it and selecting the relevant menu.

Figure 1 shows a comparison of particle advection behind a turbulent flow and a streamlined flow. Have you ever seen these kinds of patterns in rivers?

Figure 2 shows how particles of different densities separate when you blow them with a fan. There are six particle feeders at the top that continually drop particles. A fan is placed not far below the feeders.

With these new additions to Energy2D, we hope to be able to simulate more complex atmospheric phenomena (such as pollutant transport through jet streams) in the future.

Simulating cool roofs with Energy3D

Fig. 1: Solar absorption of colors.
Cool roofs represent a simple solution that can save significant air-conditioning cost and help mitigate the urban heat island effect, especially in hot climates. Nobel Prize winner and former Secretary of Energy Steven Chu is a strong advocate of cool roofs. It was estimated that painting all the roofs and pavements around the world with reflective coatings would be "equivalent to getting 300 millions cars off the road!"

With Version 4.0 of Energy3D (BTW, this version supports 200+ worldwide locations -- with 150+ in the US), you can model cool roofs and evaluate how much energy you can save by switching from a dark-colored roof to a light-colored one. All you need to do is to set the colors of your roofs and other building blocks. Energy3D will automatically assign an albedo value to each building block according to the lightness of its color.

Figure 1 shows five rectangles in different gray colors (upper) and their thermal view (lower). In this thermal view, blue represents low energy absorption, red represents high energy absorption, and the colors in-between represents the energy absorption at the level in-between.

Now let's compare the thermal views of a black roof and a white roof of a cape code house, as shown in Figure 2. To produce Figure 2, the date was set to July 1st, the hottest time of the year in northern hemisphere, and the location was set to Boston.

Fig. 2: Compare dark and white roofs.
How much energy can we save if we switch from a perfectly black roof (100% absorption) to a perfectly white roof (0% absorption)? We can run the Annual Energy Analysis Tool of Energy3D to figure this out in a matter of seconds. The results are shown in Figure 3. Overall, the total yearly energy cost is cut from 6876 kWh to 6217 kWh for this small cape code house, about 10% of saving.

Figure 3 shows that the majority of savings comes from the reduction of AC cost. The reason that the color has no effect on heating in the winter is because the passive solar heat gains through the windows in this well-insulated house is enough to keep it warm during the sunshine hours. So the additional heat absorbed by the black roof in the same period doesn't offset the heating cost (it took me quite a while to figure out that this was not a bug in our code but actually the case in the simulation).

Fig. 3: Compare heating and AC costs (blue is white roof).
Of course, this result depends on other factors such as the U-value and thermal mass of the roof. In general, the better the roof is insulated, the less its color impacts the energy cost. With Energy3D, students can easily explore these design variables.

This new feature, along with others such as the heat flux visualization that we have introduced earlier, represents the increased capacity of Energy3D for performing function design using scientific simulations.

Here is a video that shows the heating effect on roofs of different colors.