Category Archives: Development Blog

Our blog about development.

Building Learn.Ember.js, part 1: I say App, you say Document

Summary: I created a prototype of Learn.Ember.js, an interactive tutorial application for web developers who want to learn about Ember.js. Along the way I was reminded that one of the most useful things about HTML5 is that it helps us to blur the app vs. document distinction in useful ways.

Oh, and by the way, we’re hiring!

Here at the Concord Consortium we believe that interactive computational simulations are powerful tools for learning about the world in ways that were not previously practical, or even possible. Google seems to agree; their philanthropic arm Google.org recently gave us a substantial grant to make an HTML5 version of our Molecular Workbench molecular simulation environment.

Changing the world

But Google didn’t approach us just because they agree that simulations of molecular behavior are a great way to learn about science. They approached us because we have spent 10 years writing well-regarded content for Molecular Workbench. We don’t just make simulations. We embed them in documents that introduce topics gently, encourage you to play with the simulation in productive ways, and in general encourage you to think.

It turns out there are many other domains that can benefit from open-ended tools embedded in structured “learning activities” available via browser. In particular, web development itself can benefit.

Inspiration from learn.knockoutjs.com

Here at Concord I mostly do client-side web app development, and so recently I found myself surveying the new crop of client-side MVC libraries. I was looking for a lighter-weight alternative to SproutCore (which we have used for a few projects) while we waited to see what would come of on the greatly slimmed-down, SproutCore-inspired library then that was then supposed to become SproutCore 2.0, and is now a separate project called Ember.js.

But there a lot of “maybe” development tools out there — tools which might be useful someday, but which I don’t need urgently, and which aren’t such breakthroughs that they need to be understood for their own sake. One of the “maybe” libraries I came across was Steve Sanderson’s impressive Knockout.js.

Since I wasn’t doing this survey “for real”, there was a chance that I would have read through the Knockout documentation in detail, downloaded the library, and made sample pages to play with its features. A small chance. There are only so many hours in a day.

But Knockout.js has a secret weapon: its companion tutorial site, learn.knockoutjs.com. Without quite intending to, within a few minutes of stumbling onto the tutorials I built and ran working examples that felt like plausible components of a Knockout-powered app, right in the tutorial page itself. After I finished the first tutorial I had a much better idea of what kind of problems Knockout solves, and how it solves them, than I would have gotten from the usual desultory flip through the Knockout homepage. (You should try the tutorials yourself!)

Prototyping Learn.Ember.js

As it turns out, Ember.js (née SproutCore 2.0) is shaping up to be a cleanly designed and powerful library with a solid team behind it, and I am enthusiastic about its future.

And as Scott has previously blogged, we at Concord would like to create more value for the open source ecosystem. So I’ve begun work on a side project I call Learn.Ember.js. You can see the first public prototype here. (Warning: this does not work in some browsers, notably older versions of Firefox and — wait for it — IE.)

Once I had the most basic functionality working — 2 Ace editors for the Javascript code and the view template, and an embedded iframe for the results — I wanted to focus on establishing a clean visual design. That meant I had to stop writing code and stop dreaming up potential features long enough to focus on design. Fortunately I was saved from withdrawal symptoms by all the opportunities which that opened up for obsessive font fiddling and CSS tweaking.

The challenge here was not so much the design of the text content — though I tried to borrow the best from well-designed, readable sites I like, such as the new Boston Globe website, the Nieman Foundation’s Nieman Labs blog, arc90’s Readability tool, and Mark Pilgrim’s Dive Into HTML5. Rather, the challenge proved to be finding a way to keep all the buttons and assorted interactive knobby bits from interfering with the text.

My first attempts weren’t very promising. I couldn’t put my finger on why until I realized that the 4-box layout of learn.knockoutjs.com just wasn’t working for me. Somehow I got the idea that in order to make the tutorial readable, I would have to find a way to “unbox” the design and make it look something like a page of a good technical book that just happened to be able to run code. But that introduced its own problems. Where to put the results of the program the user writes (which is an interactive web app unto itself)? Put that in a box, and, together with the Javascript and Handlebars/HTML input, which seem to need to be in boxes — de facto, you have four little boxes again!

Gradually, it occurred to me that the program output could be in flow with the text, right below whichever paragraph prompted you to try running the updated program. Then, with just a little position: fixed and fluid-layout magic, it would be perfectly reasonable to have the whole page scroll, and the tutorial content with it. That is to say, I rediscovered the basic design of every web page ever.

You say app, I say document. Let’s call the whole thing off.

I mention this particular, uh, discovery because for some reason it seems to be common to design news and learning interactives to have little snippets of text written in large type and stuffed into little boxes. I confess to having cargo-culted this particular design idea not long ago; last year I even fired up an ancient Multimedia Beethoven CD-ROM made some time in the last century to confirm that, yup, instructional text is supposed to be really short and go into a little box on the left!

Microsoft Multimedia Beethoven, circa 1992. Via http://www.uah.edu/music/technology/cai/programs/msbeethoven.html

I wonder if this design habit is an artifact of the days of Flash and native applications built using layout manager APIs and visual UI builders. I get the impression that it’s both difficult and out of the ordinary to try to get text and interactive elements to flow together using those technologies. After all, the designer usually doesn’t know what the text is going to be in advance, and you, the developer, would have to come up with a way to keep track of where in the text the widgets go, then create the appropriate widget objects and break up the text string at the appropriate spots, so that you can feed it all to a layout manager that you would probably have to tweak and fiddle for your somewhat unusual use case. Which suggests a great idea — perhaps we could invent tokens that mean “a widget goes here” and have the author use those to mark up the text somehow…

I kid. But in a serious way, because one of the things I liked least about SproutCore is the way it seems to want to pretend that the web hasn’t been invented yet. It provides widgets that are really meant to be a particular size and at a particular, absolutely-positioned offset specified in Javascript. Until the oddly named StaticContentView was invented, the standard UI widget for displaying text was called a LabelView and wanted, again, to be a particular size (regardless of the size of its content) and at a particular location (regardless of the size of the content surrounding it).

The theory was that SproutCore is for designing “apps” rather than “documents”. But as you might guess, I don’t find that distinction very compelling in late 2011. Yes, clearly, there will always be some apps whose UI is legitimately just a box of buttons or a glorified data entry form. And “everything in its right place, and just where it was last time” is exactly the right motto for such apps.

But much of the interesting stuff in your life happens in some kind of stream of context. Facebook and Gmail (especially the new look) are containers for what are basically documents relevant to your life, yet their designers are not shy about inserting app widgets — stuff that does stuff — right into the middle of that “document”-like flow.

Educational apps likewise should include plenty of text that helps you understand the things they help you to do. At Concord, we’ve been calling for a “deeply digital” curriculum that weaves (among other interactive elements) sensors and simulations tightly into the fabric of textbooks and other media.

You occasionally hear “technology X is for app builders, and web technology Y is really for documents“–but that ignores an important category of innovation that is going on right now: apps that are documents. Or, wait, is that documents that are apps…?

What’s next for Learn.Ember.js

But, back to Learn.Ember.js and what’s next. The single page of tutorial text and the trivial example code I have so far are somewhat lazily inspired by the first page of the Knockout.js tutorial; I just needed some text that isn’t plain lorem ipsum. So I need to write more content. But it’s of equal importance to make it trivial for anyone to clone the Learn.Ember.js repo and submit pull requests with new content — or to simply host their own version, modified as they see fit.

For the time being the tutorial text itself is written as a Handlebars template with embedded expressions that tell where to put the buttons; and the initial example code is a string-valued property of a Javascript object. So far, it’s been pretty painless to edit the tutorial text in Handlebars form, but the need to include view class names into the text is an obvious mixing of unrelated concerns — and, worse, the tutorial text is transported to clients as a compiled Handlebars template that is completely invisible to search engines. (Until Javascript gets to work, the index.html file consists of a blank page.)

I think the solution is to put the actual tutorial content, written in clean, semantic HTML5, into the body of the index.html file. Then we can agree as a convention to identify the “run” buttons by applying a particular CSS class, and to represent the location of the output by inserting an empty div with a particular CSS class. The Learn application can then easily use jQuery to scan the DOM as needed, inserting Ember.js views into the right places using Ember.View’s appendTo method and a little bit of DOM manipulation magic.

A remaining question would be whether and how to specify the initial code and the working “help me” code inside the HTML document. Putting the code in script tags with a fake MIME type (text/x-example-javascript) would make it easy to insert the code without having to HTML-escape it and without it running on page load, but then the code isn’t visible to user agents — like search engines — that don’t execute Javascript. Perhaps that is enough, or perhaps the code should go, properly escaped, into hidden <div> elements.

If that were done, then anyone could write their own interactive Ember tutorial by writing an appropriately-marked up HTML file and inserting a few lines into the head of the document to include the Javascript code of the Learn application, which would take care of translating the tutorial document into a working app. And if they were to publish the HTML file to a server, it would be fully searchable.

Before I get that far, of course, I’ll have to tackle navigation between tutorials and pages of a tutorial — a bit of design I left for later. As fodder for a new blog post, of course!

Updated 1am Monday, December 19 with better information about browser compatibility after I made a quick fix to the Learn.Ember.js prototype itself to make it work in Safari, and with a link to all of our open positions rather than just the developer position.

Open source spin-off projects

Developers at the Concord Consortium work on a wide variety of grants, and in the process we create reusable pieces of code. With a little work some of these reusable bits of code can be turned into spin-off projects that have a life of their own. In my opinion these spin-off projects have the best potential for broad long-term impact.

Recently I was reminded about these types of spin-off projects when Richard Klancer relayed a conversation he had with Jeremy Ashkenas. Jeremy has been very successful in this area during his work on DocumentCloud.

We strive to make our individual projects successful, but often their technology is complex and not easy to re-use. The impact of the individual project is the research enabled by the technology, or demonstrating the usefulness of a new concept. However, the collection of technologies used in the project normally becomes a one-off: it is no longer used once the project reaches its 2-5 year end.

Alternatively, within these complex projects are reusable pieces of code that are simple, easy to maintain, and solve a common need. Because of this they have potential to be popular outside of our organization. We do have some partial successes with spin-offs like this.

  • MozSwing – mostly abandoned, though it was used in at least one commercial product
  • Java Sensor Library – collection of JAR files for communicating with a variety of sensors available in schools
  • RaphaelViews – SproutCore 1.x library for creating fully fledged SproutCore 1.x views with Raphael
  • SproutCore TestDriver – ruby gem for running SproutCore Jasmine and QUnit tests on a CI server

None of these has become a successful open source spin-off project. To be successful, such a project needs an active community that includes both developers and users. And the amount of work required to maintain it by Concord Consortium developers needs to be small enough that it doesn’t prevent us from reaching the goals of individual grant projects.

The MozSwing project would require too much maintenance. The Java sensor project is too intertwined in our other Java code. RaphaelViews and CapybaraTestrunner don’t have the above problems, but they have not been polished and announced to the right audience. I don’t think the polishing would take a lot of effort, but making the time and finding the support to do so is hard. We are always working on the next big thing, so it takes discipline to really finish up what is already working internally.

There are more potential open source spin-off projects within the technology at the Concord Consortium that have wider audiences than the ones above. With luck, we can change our culture to encourage this work more and make more of this great stuff accessible.

Do you agree that we should be spinning off more projects? Do you have experience with spinning off projects like these? Any tips?

Idea for a concise syntax of nested models in cucumber

I started thinking about how to more easily specify some of the deeply nested structures we need during testing.

First we already have a step for doing this.  An example which looks like this:

And the following investigations with multiple choices exist:
| investigation         | activity    | section     | page       | multiple_choices  | image_questions   |
| first investigation   | act 1     | section 1  | page 1   | a, b              |                   |
| first investigation   | act 2     | section 2  | page 2   | c, d              |                   |
| second investigation  | act 3     | section 3  | page 3   | b_a, b_b          | image_q           |
| second investigation  | act 4     | section 4  | page 4   | b_c, b_d          |                   |

That is using some previously defined multiple_choice and image_question objects. I was trying to see if I could write it more like this:

And the following investigations with multiple choices exist:
      investigation "first investigation"
        activity "act 1"
          section "section 1"
            page "page 1"
              multiple_choice :prompt => "Prompt 1", :choices => "a,b,c,d", :correct_answer => "a"
              multiple_choice :prompt => "Prompt 2", :choices => "a,b,c,d", :correct_answer => "a"
            page "page 2"
              image_question :prompt => "Image Prompt"

I didn’t see an easy way to do that but with a little method_missing, haml, and using cucumber’s “”” notation, the following looks pretty straight forward:

And the following investigations with multiple choices exist:
      """
      - investigation "first investigation" do
        - activity "act 1" do
          - section "section 1" do
            - page "page 1" do
              - multiple_choice :prompt => "Prompt 1", :choices => "a,b,c,d", :correct_answer => "a" do
              - multiple_choice :prompt => "Prompt 2", :choices => "a,b,c,d", :correct_answer => "a" do
            - page "page 2" do
              - image_question :prompt => "Image Prompt" do
      """

In a simple implementation a map could be used to define the code to add a child object. So for investigation it would be {activities<<child}, and for page it is {add_element(child)}. The creation of of the objects could use factory girl so the names could be any of the existing factories.

I’m curious if anyone has seen something like this that is already written? And I’m wondering if there is something like haml that uses indentation for blocks but defaults to “ruby mode” so there wouldn’t be a need for the “- ” before each line.

iPad2 HTML5 stats look good

Sencha has the latest on the new iPad’s HTML5 performance, and the verdict looks quite good:

The iPad 2’s Mobile Safari browser is the best implementation of WebKit on a mobile device. In our testing we tried to throw everything we could at the browser and it had no issues keeping up with the most advanced HTML5 and CSS3 sites. For any developer building for the mobile web, the iPad 2 provides an outstanding platform from which you can use modern browser features.

It will be exciting to see how some of our newest Web-based software performs on this device, especially given these advances.

Accessing Sensor Data from Java

For years we have been using several layers of Java, Java Native Interface, and native driver code to support common access to sensors from multiple Probeware interfaces from different vendors. We’ve been calling these layers the org-concord-sensor framework.

Our Java/OTrunk framework which has supported many kinds of interactive educational activities uses the org-concord-sensor framework to route data from sensors to real-time graphs and as input into dynamic models.

The Google Code org-concord-sensor project includes a link to a some older (but still useful) documentation in our Confluence wiki Sensor Device System.

Here’s a general page in our wiki which shows how to (setup a full development environment for OTrunk in Eclipse)[http://confluence.concord.org/display/CSP/Setup+Development+Environment].

Recently we have adapted this framework to support collecting data from Vernier GoIO probes directly in the browser.

If you have a GoMotion probe try the GoMotion Graph Demo page.

Crowd Sourcing JavaScript Performance-testing

jsPerf.com lets you write two equivalent ways to accomplish something in JavaScript and it then measures how fast each method is in every browser you run the test in. Other people’s performance tests can be browsed here Browse test cases.

Taking a look at a specific test could show data where you might want to pick an approach that is 25% slower in Chrome …

For example operator-vs-localecompage shows that FireFox starts out seven times slower than Chrome and the slower method in Chrome runs twice as fast in FireFox as the faster method in Chrome.

Other people who view your test can contribute new results when it is run in their browser also.

Comparison of Ruby 1.8.6 1.9 and JRuby running on Java 1.5 1.6 and 1.7

Ruby is a powerful and dynamic open-source object-oriented language we have been using extensively at CC in the last few years for the web applications that manage and coordinate authoring and deployment of activities based on the SAIL/OTrunk framework .

The standard Ruby VM is written in C and we’ve been using version 1.8.6, the latest stable release on our servers. A beta version of the next major release, version 1.9.1 has recently been released.

Ruby 1.9 looks to be about twice as fast as Ruby 1.8.6.

I’m even more impressed by the recent performance increases in JRuby however. This is a version of Ruby that runs in Java. Programs written in JRuby can easily access code written in Java which makes integration with the rest of our Java codebase much easier.

A year ago programs written in JRuby were often slower than ones written in version 1.8.6 of C Ruby. Now for some benchmarks JRuby is twice as fast as Ruby 1.9.

Ruby Merge Sort Speed Comparison

More details about some of these measurements are here:

Ruby 1.8.6, 1.9, and JRuby running on Java 1.5, 1.6, and 1.7 compared