Monthly Archives: May 2011

New Node-Rhino-Server-Batch-Non-Browser Skeleton

Posted on by
3

We’ve added an experimental new skeleton type to the framework. It addresses applications that target run times outside the browser, namely Rhino and Node.js. Its working title is “basic“, which has nothing to do with the complexity or relevance of the potential applications. It just refers to the basic elements of qooxdoo that are used to build these apps, the class system, qooxdoo events, and data binding. It is the direct continuation of the work we started with qxoo, our shrink-wrapped OO library for those environments. With the skeleton, you can now create custom classes and have them integrated automatically with a loader script by the generator, together with framework and other library code, if you use any.

It is certainly still rough around the edges. source and build generation work, and the resulting scripts should run with both Rhino and Node. Utility jobs like fix, lint or api should work too, only test is currently missing and will be added later when we have established a test runner for the two engines. Now it’s up to you to go ahead and build the next generation web app or network monitoring tool or you-name-it, using qooxdoo.

If you want to give it a try you need a recent trunk check-out,  and then the usual pattern applies for the skeleton:

create-application.py -n foo -t basic
cd foo
generate.py source
node source/script/foo.js

As you can see in the last line, you just don’t load it in the browser through an index.html, but invoke it directly. Use the mailing list to share your experiences, so that we can continue improving it.

Generator: Cache Invalidation – Revisited

Posted on by
2

The compile cache is one of the main means to improve build performance of the generator. Among other things, like optimized versions of qooxdoo classes, it contains information about the dependency relations between them.

Dependency analysis is also one of the most time-consuming activities of the generator. That means: The more you can re-use of this information, the faster the generator will run. To come up with this information, the generator basically parses class code into a syntax tree, then traverses that tree to find symbols that refer to other classes (or to other symbols in general). With each symbol it finds it has to decide about its availability at run time. Built-in JavaScript symbols like Date or RegExp are known to be provided automatically by the JavaScript run time, so there is nothing the generator needs to care about. But e.g. references to other qooxdoo classes are recorded. This information is then used to e.g. derive the complete list of classes that need to go into the current build, so all dependencies are satisfied.

Another important aspect of the dependency information is the fact whether a particular symbol is used by a given class at load time (when the class code is parsed and evaluated in the interpreter) or at run time (when instance methods are executed). This influences the ordering of classes which is done by the generator, and eventually burned into the loader script. Classes that are needed by any particular class at load time need to be loaded ahead of the requiring class (This is usually not true for run time dependencies, which just have to be loaded into the interpreter eventually).

Now, determining whether a given dependency is needed at load time or run time can be a tricky question. There are obvious examples. A class referred to in the extend entry of the class map signify the current class’s super-class; it has to be known to the interpreter ahead to the moment when the the extend key is parsed.

qx.Class.define("foo.Bar", {
  extend : qx.core.Object,
  ...
});

Here, qx.core.Object needs to loaded prior to foo.Bar. A method of another class called in one of its own instance methods needs only be available at run time. It is not necessary at loading time of the class, as the method body is parsed, but references are not yet evaluated.

  baz : function() {
    var a = new foo.Baz();  // <- calling another class' constructor
    ...
  }

Caching Dependencies

As with every cache implementation, a crucial aspect is that of cache invalidation: Knowing when any particular item in the cache is out-of-date and needs to be re-calculated. Using an outdated cache object causes wrong results; invalidating a cache object that is still fresh causes unnecessary re-calculation and hits the performance. This is particular challenging when we look at transitive load dependencies of a class. Some dependencies of a class have to be explored recursively. Here is a simple example:

qx.Class.define("foo.Bar", {
  ...
  statics : {
    bez : new foo.Baz(),
    ...
  }
  ...

A static member of the class is initialized with an instance of another class. So this other class becomes a load time requirement of the current class. Fair enough. But the story doesn’t end here. foo.Baz has to be available when the JS interpreter comes to parsing foo.Bar. As foo.Baz has been loaded before, its constructor method can be called. But what about the dependencies of this constructor method?! Those dependencies have been considered run time dependencies when foo.Baz was analysed. Now for the foo.Bar class, those suddenly become load requirements.

This means that the generator has to follow those recursive dependencies during analysis of foo.Bar. This recursive analysis is particularly expensive. Once completed, the result is cached. The next time an application needs to be build with this class, those dependencies are retrieved from the cache. But wait – are they still fresh? The foo.Bar class might not have changed on disk, but what about the other classes, those discovered analysing the dependencies of the static bez member?! A change in any of them might result in new or altered dependencies coming in for our foo.Bar class. So to validate the cached result, the generator needs to go through the required classes of the recursive dependencies and check if they have changed.

Checking the status of each class is straight forward. It’s last-modified time stamp from the file system is compared to the time stamp of the cached information. In the first implementation this check was done directly going to the file system using an operating system stats() call. But with an increasing number of classes in an app, and an increasing number of detected recursive dependencies, also the number of stats() calls increased. This began to show a significant impact on generator performance.

Checking Freshness

After  some head-scratching I decided to be contempt with a comparison time taken at generator start-up. I reasoned that it is rather unlikely that somebody is changing source files midway through a generator run, and expect the changes to be picked up immediately, and everything remaining consistent :-) . So times taken a start-up seemed good enough for freshness checking. The first solution was to compare against the cumulative last-modified time stamp of any used library (the time stamp of the most recently changed file of any of the files in the library). This has to be calculated anyway, as – you guessed it – the result of library scans (which classes are in it, which resources, …) is also cached.

This had a very nice effect on building the Demobrowser, with its hundreds of demo apps, and a lot of cache checking when building every one of them. But the downside also showed. For iterative development, when you changed just one file in a library, all contained classes where suddenly considered newer, basically invalidating any cached information relating to any of them. That was grossly overshooting, and was like using no cache at all, everything was re-calculated with the next generator run.

So the best solution so far is to record all last-modified time stamps of the classes when scanning a library, and then comparing the cache against each recorded class time stamp individually. This continues the reasoning that generator start-up times are good enough, without penalizing iterative development.

The week in qooxdoo (2011-05-27)

Posted on by
1

Up comes the next activity report.

Mobile Framework: New CheckBox Widget & PhoneGap

One of the important form elements that was missing in the mobile widget set was the CheckBox. We are always trying hard to keep the API of the mobile widgets close to their RIA counterpart, so the usage of the new CheckBox will be immediately familiar to qooxdoo developers. Here is a screen shot of a check box switched on and off.

In the mobile showcase application we show how to add a CheckBox to a form and how to use it with the data binding feature of qooxdoo.

Another thing we are proud of is that we are now listed as an official tool for PhoneGap development. Thanks to Brian LeRoux from the PhoneGap team for adding us.

Maven Integration

Some activity has been going on in the community to integrate the qooxdoo SDK into Maven repositories. Benjamin Dreux and Charles Salmon have been instrumental in putting together a good deal of documentation on that topic. And the story will continue, so expect more to come.

Bugfixes

For a complete list of tasks accomplished during the last working week, use this bugzilla query.

Tutorials: Unit Testing & Virtual List

Tutorials keep coming in. The past week saw the arrival of two of them: Daniel provided an intense introduction into using qooxdoo’s unit testing infrastructure. He takes you through the basic design for test classes and writing test methods, looking into the fundamental test protocol of setting up and tearing down tests, as well as using assertions and requirements. Advanced topics like the use of spies, stubs and mocks (facilitated through qooxdoo’s integration with Sinon) will be treated in a follow-up tutorial.

As another installment of the Twitter tutorial series, Chris provided one covering the Virtual List. It takes you throught the steps of replacing the traditional list in the demo app with a VirtualList, taking advantage of qooxdoo’s virtual infrastructure. To get the most out of this tutorial, you should be familiar with the basic Twitter demo app as introduced in tutorial parts 1, 2 and 3.

Check them out, if you want to dig deeper into these qooxdoo goodies. The tutorial series will be continued with one about basic tool chain usage this week.

That’s it for this time, folks, enjoy!

Tutorial: Virtual List

Posted on by
Reply

Tutorial: Virtual List

This time we will have a look at the virtual widget stuff. The plan is to remove the normal List and use the virtual List. Using the virtual list has a big advantage when we have to render a huge count of items. The virtual list only creates widgets for visible items. This saves memory and execution time.  As a base we use the already known twitter client we built in the former tutorials.

Change the instantiation

First, we have to use the virtual List instead. Open the twitter.MainWindow class and search for the list instantiation:

// list
this.__list = new qx.ui.form.List();
this.add(this.__list, {row: 1, column: 0, colSpan: 2});

And create a virtual List instead:

// list
this.__list = new qx.ui.list.List();
this.add(this.__list, {row: 1, column: 0, colSpan: 2});

Now we use the virtual List instead of the non virtual List. But before we can use the twitter application with the virtual List we have to configure the usage with a delegate.

Configure the virtual List

The current implementation uses the list controller to bind the tweets with the list. This makes it easy to reuse the delegation implementation, because the delegation interface from the virtual List has the same methods for bindItem, createItem, configureItem and filter. We only need to remove the controller stuff and use the virtual list instead. The controller is not needed anymore, because the virtual list has its own controller implementation. Open the twitter.Application and search for the controller instantiation:

// create the controller
var controller = new qx.data.controller.List(null, main.getList());
controller.setDelegate({
  createItem : function() {
    return new twitter.TweetView();
  },
  ...
});

Instead of the controller use the virtual List:

// setup list binding
var list = main.getList();
list.setItemHeight(68);
list.setDelegate({
  createItem : function() {
    return new twitter.TweetView();
  },
  ...
});

Now we have replaced the controller with the virtual List and reused the delegate implementation. We have only added one line to configure the default item hight. This is necessary, because the virtual List has no auto sizing for the item height. This is due to the huge count of model items.

Update list binding

Finally, we have to adapt the binding between the twitter service and the virtual list. The virtual list always needs a model instance so we need to adapt the current binding:

service.bind("tweets", controller, "model");

We only use a converter which returns an empty model when the service returns null:

service.bind("tweets", list, "model", {
  converter : function(value) {
    return value || new qx.data.Array();
  }
});

Now we only need to run the generator to resolve the new dependencies:

The virtual List supports some more features like grouping, for additional details have a look at the virtual demos. As always, the code of the tutorial is on github.