Tag Archives: Sensors

A mixed-reality gas lab

In his Critique of Pure Reason, the Enlightenment philosopher Immanuel Kant asserted that “conception without perception is empty, perception without conception is blind. The understanding can intuit nothing, the senses can think nothing. Only through their unison can knowledge arise.” More than 200 years later, his wisdom is still enlightening our NSF-funded Mixed-Reality Labs project.

Mixed reality (more commonly known as augmented reality) refers to the blending of real and virtual worlds to create new environments where physical and digital objects co-exist and interact in real time to provide user experiences that are impossible in only real or virtual world. Mixed reality provides a perfect technology to promote the unison of perception and conception. Perception happens in the real world, whereas conception can be enhanced by the virtual world. Knitting the real and virtual worlds together, we can build a pathway that leads perceptual experiences to conceptual development.

We have developed and perfected a prototype of mixed reality for teaching the Kinetic Molecular Theory and the gas laws using our Frame technology. This Gas Frame uses three different types of sensors to translate user inputs into changes of variables in a molecular simulation on the computer: A temperature sensor is used to detect thermal changes in the real world and then change the temperature of the gas molecules in the virtual world; a gas pressure sensor is used to detect gas compression or decompression in the real world and then change the density of the gas molecules in the virtual world; a force sensor is used to detect force changes in the real world and then change the force on a piston in the virtual world. Because of this underlying linkage with the real world through the sensors, the simulation appears to be "smart" enough to detect user actions and react in meaningful ways accordingly.

Each sensor is attached to a physical object installed along the edge of the computer screen (see the illustration above). The temperature sensor is attached to a thermal contact area made of highly conductive material, the gas pressure sensor is attached to a syringe, and the force sensor is attached to a spring that provides some kind of force feedback. These three physical objects provide the real-world contextualization of the interactions. In this way, the Gas Frame not only produces an illusion as if students could directly manipulate tiny gas molecules, but also creates a natural association between microscopic concepts and macroscopic perception. Uniting the actions of students in the real world and the reactions of the molecules in the virtual world, the Gas Frame provides an unprecedented way of learning a set of important concepts in physical science.

Pilot tests of the Gas Frame will begin at Concord-Carlisle High School this week and, collaborating with our project partners Drs. Jennie Chiu and Jie Chao at the University of Virginia, unfold at several middle schools in Virginia shortly. Through the planned sequence of studies, we hope to understand the cognitive aspects of mixed reality, especially on whether perceptual changes can lead to conceptual changes in this particular kind of setup.

Acknowledgements: My colleague Ed Hazzard made a beautiful wood prototype of the Frame (in which we can hide the messy wires and sensor parts). The current version of the Gas Frame uses Vernier's sensors and a Java API to their sensors developed primarily by Scott Cytacki. This work is made possible by the National Science Foundation.

Natural learning interfaces

Natural user interfaces (NUIs) are the third generation of user interface for computers, after command line interfaces and graphical user interfaces. A NUI uses natural elements or natural interactions (such as voice or gestures) to control a computer program. Being natural means that the user interface is built upon something that most people are already familiar with. Thus, the learning curve can be significantly shortened. This ease of use allows computer scientists to build more complicated but richer user interfaces that simulate the existing ways people interact with the real world.

Research on NUIs is currently one of the most active areas in computer science and engineering. It is one of the most important directions of Microsoft Research. In line with this future, our NSF-funded Mixed-Reality Labs (MRL) project has proposed a novel concept called the Natural Learning Interfaces (NLIs), which represents our latest ambition to realize the educational promise of cutting-edge technology. In the context of science education, an NLI provides a natural user interface to interact with a scientific simulation on the computer. It maps a natural user action to the change of a variable in the simulation. For example, the user uses a hot or cold source to control a temperature variable in a thermal simulation. The user exerts a force to control the pressure of a gas simulation. NLIs use sensors to acquire real-time data that are then used to drive the simulation in real time. In most cases, it involves a combination of multiple sensors (or multiple types of sensors) to feed more comprehensive data to a simulation and to enrich the user interface.

I have recently invented a technology called the Frame, which may provide a rough idea of what NLIs may look like as an emerging learning technology for science education. The Frame technology is based on the fact that the frame of a computer screen is the natural boundary between the virtual world and the physical world and is, therefore, an intuitive user interface for certain human-computer interactions. Compared with other interfaces such as touch screens or motion trackers, the Frame allows users to interact with the computer from the edges of the screen.

Collaborating with Jennie Chiu's group at the University of Virginia (UVA), we have been working on a few Frame prototypes that will be field tested with several hundred Virginia students in the fall of 2012. These Frame prototypes will be manufactured using UVA's 3D printers. One of the prototypes shown in this blog post is a mixed-reality gas lab, which was designed for eighth graders to learn the particulate nature of temperature and pressure of a gas. With this prototype, students can push or pull a spring to exert a force on a virtual piston, or use a cup of hot water or ice water to adjust the temperature of the virtual molecules. The responsive simulation will immediately show the effect of those natural actions on the state of the virtual system. Besides the conventional gas law behavior, students may discover something interesting. For example, when they exert a large force, the gas molecules can be liquified, simulating gas liquifying under high pressure. When they apply a force rapidly, a high-density layer will be created, simulating the initiation of a sound wave. I can imagine that science centers and museums may be very interested in using this Frame lab as a kiosk for visitors to explore gas molecules in a quick and fun way.

A mixed-reality gas lab (a Frame prototype)
As these actions can happen concurrently, two students can control the simulation using two different mechanisms: changing temperature or changing pressure. This makes it possible for us to design a student competition in which two students use these two different mechanisms to push the piston into each other's side as far as possible. To the best of our knowledge, this is the first collaborative learning of this kind mediated by a scientific simulation.

NLIs are not just the results of some programming fun. NLIs are deeply rooted in cognitive science. Constructivism views learning as a process in which the learner actively constructs or builds new ideas or concepts based upon current and past knowledge or experience. In other words, learning involves constructing one's own knowledge from one's own experiences. NLIs are learning systems built on what learners already know or what they feel natural. The key of a NLI is that it engineers natural interactions that connect prior experiences to what students are supposed to learn, thus building a bridge for stronger mental association and deeper conceptual understanding.

Open source spin-off projects

Developers at the Concord Consortium work on a wide variety of grants, and in the process we create reusable pieces of code. With a little work some of these reusable bits of code can be turned into spin-off projects that have a life of their own. In my opinion these spin-off projects have the best potential for broad long-term impact.

Recently I was reminded about these types of spin-off projects when Richard Klancer relayed a conversation he had with Jeremy Ashkenas. Jeremy has been very successful in this area during his work on DocumentCloud.

We strive to make our individual projects successful, but often their technology is complex and not easy to re-use. The impact of the individual project is the research enabled by the technology, or demonstrating the usefulness of a new concept. However, the collection of technologies used in the project normally becomes a one-off: it is no longer used once the project reaches its 2-5 year end.

Alternatively, within these complex projects are reusable pieces of code that are simple, easy to maintain, and solve a common need. Because of this they have potential to be popular outside of our organization. We do have some partial successes with spin-offs like this.

  • MozSwing – mostly abandoned, though it was used in at least one commercial product
  • Java Sensor Library – collection of JAR files for communicating with a variety of sensors available in schools
  • RaphaelViews – SproutCore 1.x library for creating fully fledged SproutCore 1.x views with Raphael
  • SproutCore TestDriver – ruby gem for running SproutCore Jasmine and QUnit tests on a CI server

None of these has become a successful open source spin-off project. To be successful, such a project needs an active community that includes both developers and users. And the amount of work required to maintain it by Concord Consortium developers needs to be small enough that it doesn’t prevent us from reaching the goals of individual grant projects.

The MozSwing project would require too much maintenance. The Java sensor project is too intertwined in our other Java code. RaphaelViews and CapybaraTestrunner don’t have the above problems, but they have not been polished and announced to the right audience. I don’t think the polishing would take a lot of effort, but making the time and finding the support to do so is hard. We are always working on the next big thing, so it takes discipline to really finish up what is already working internally.

There are more potential open source spin-off projects within the technology at the Concord Consortium that have wider audiences than the ones above. With luck, we can change our culture to encourage this work more and make more of this great stuff accessible.

Do you agree that we should be spinning off more projects?
Do you have experience with spinning off projects like these? Any tips?

Salinity gradient vs. temperature gradient

Figure 1. The salinity gradient and 
temperature gradient observed in an
open cup of saturated saltwater.
This is the fifth follow-up of the blog article: "A perfect storm in a cup of salt water?This investigation focused on the relationship between the salinity gradient and the temperature gradient. Is the temperature gradient caused by the salinity gradient, or the other way around? Both arguments seem to make some sense. On the one hand, one can argue that the salinity gradient stops the convection. On the other hand, warmer water tends to dissolve more salt. So we are in a chicken-egg situation.

Let's do an experiment to explore a bit further. I prepared two cups of saturated saltwater. One open and the other sealed. I let them sit overnight and then checked the salinity and temperature distribution the next day using Vernier's salinity sensor and temperature sensor. I did this by moving the salinity sensor and the temperature sensor together up and down in the saltwater. Figure 1 shows the results for the open cup.

Figure 2. The salinity gradient and
temperature  gradient observed in
a closed cup of saturated saltwater.
Note: The measurement was done
shortly after removing the seal. 
Hence the results can be regarded
as approximately those of the
sealed cup as the gradients will
take a longer while to establ
To measure the data for the closed cup, I first removed the seal and then quickly did the measurement. Since the salinity and temperature gradient would take some time to readjust after the seal was removed, we can pretty much assume that the results I got approximately reflect what would have been measured if the seal had not been removed. Figure 2 shows the results.

 The comparison of the results shows that the salinity gradient is about the same for the open and closed cup--the bottom is about 1.3 ppt saltier than the top, but the temperature gradients are quite different--the open cup measured about three times as large as the closed cup (0.3°C vs. 0.1°C). 

Due to the evaporative cooling effect, the overall temperature of the open cup is at least 0.5°C lower than the closed one.

What do these results suggest? A weak temperature gradient may exist in a closed system that does not have the driving force of evaporative updraft.

Mystery solved?

This is the third followup of the blog article: "A perfect storm in a cup of salt water?"

I woke up last night with a perfect explanation for the mysterious temperature gradient observed in a saturated salt solution. It is the recrystallization of salt at the bottom of the cup that releases the heat.

Since water molecules are constantly evaporating from the surface of the solution, a corresponding amount of ions must return to the crystal form at the same time--because a reduced amount of water in a saturated solution in the cup cannot take them any more. This most likely occurs at the bottom since the surface of the precipitate already provides a perfect ground of crystal growth. When ions adhere to the surface of a crystal, heat is released. The amount of released heat is approximately equal to half of the cohesive energy of the salt crystal (because it is a surface adhesion), which may be quite high because of the strong electrostatic attractions in the ionic crystal. The released heat transfers to the solution near the bottom and, together with the evaporative cooling effect on the surface, creates the temperature gradient we observed. The entire process runs continually across the solution because of the diffusion of water molecules and ions driven by their concentration gradients: the concentration of water/salt becomes lower/higher at the surface when water evaporates. 

There are four evidences that support this theory:

  1. The temperature gradient disappears when we sealed the cup, because that stopped the evaporation at the surface as well as the recrystallization at the bottom.
  2. We observed no temperature gradient in an unsaturated solution because there is no recrystallization process.
  3. The temperature hiked when the sensor touched the salt deposit at the bottom.
  4. This temperature gradient lasts for a long time because this process will continue until all the water molecules evaporate.

Now, how can we make use of this effect to produce clean energy? As we produce sea salt by using solar energy to evaporate the water in brine anyway, might it be possible to harvest the energy released from the crystallization process? This seems like a stone that kills two birds: generating electricity while producing salt.

The diagram above illustrates the energy cycle of a saltpan/ionic power plant combo. This design is based on a chain reaction that involves two phase changes in a salt solution to convert solar energy into electricity through the ionic potential.

PS: I found in Wikipedia the concept of solar pond that uses a large pool of saltwater to collect solar energy. I think its mechanism is different from what I discussed above. I have had no luck reproducing the effect of a solar pond in a cup yet.

The temperature gradient exists only in a saturated solution

This is the second followup of an earlier blog article "A perfect storm in a cup of salt water?"

I did an experiment to investigate the relationship of the salt concentration with the mysterious temperature gradient in a cup of salt water. The experiment was to measure the top-bottom temperature differences in three cups of salt water: low-concentration, medium-concentration, and saturated solution. In the saturated solution, there is a salt precipitate at the bottom of the cup. In all measurements, a fast-response temperature sensor was moved up and down in a cup. And the solutions had existed for over 100 hours to ensure that the salt was completely dissolved and the systems had reached thermal equilibrium with the environment.

The results shown in the graph above clearly indicate that the temperature gradient exists only in the saturated solution. The two unsaturated solutions exhibit no appreciable temperature gradients and measure approximately the same temperature with plain water.

The results were confirmed by an IR image shown above (from left to right: low-concentration, medium-concentration, and saturated).

This experiment suggests that there is probably no ion gradient in an unsaturated solution. An unsaturated salt solution has the same temperature everywhere and that temperature is the same as that of the plain water, whatever its concentration is. I originally expected that an unsaturated solution would have a temperature gradient more or less proportional to the salt concentration, as would a colligative property. This surprising result made me think that the prime suspect is the salt precipitate at the bottom of the cup. We know there is a lot going on on the surface of the precipitate layer. The dissolving and crystallization never cease. It is just that the two processes reach a dynamic equilibrium--the rate of dissolving becomes the same as the rate of crystallization. Sort of like what is shown below:

Let's think a bit more about the meaning of this experiment. Notice that the temperature curve of the saturated solution lies entirely between that of the ambient temperature and that of the pure water temperature in the graph. This means that the existence of the precipitate somehow weakens the evaporative cooling effect, and probably the evaporation process itself. Why would the evaporation of water at the top of the solution slow down in the presence of some precipitate at the bottom? Exactly how does this process contribute to the temperature gradient existing in the solution?

We can plausibly reason that the rate of evaporation decreases because the ions somewhat act as binding agents that hold the water molecules more tightly through the strong electrostatic attractions. This is known as the water shell effect—water molecules are attracted to an ion and form a dynamic shell around it. As a result, it is more difficult for water molecules to leave the solution to evaporate. But this picture cannot explain why there is virtually no difference between the temperature of a cup of pure water and the temperature of a cup of unsaturated salt water.

It seems the mystery is far from being uncovered. While clarifying a few things, this experiment makes the phenomenon more baffling. Stay tuned for our next investigation.

In terms of its other implications, there is one thing that we can rule out now. There is no such effect in the ocean, as sea water is not saturated.