Monday, November 26, 2012

JavaFX: Leveraging Multi-core Performance

Like many other user interface frameworks, JavaFX is single-threaded; but since most modern computers have multi-core CPUs, many JavaFX applications can benefit from introducing multiple threads – as long as it is done correctly, in a manner compatible with JavaFX. This article shows you how.

Benefits of being Single Threaded

It is most likely that the device you are using to read this article is multi-core – most devices are nowadays. Many laptops are already quadcore, and even a tiny table like the Nexus 7 (shipped in the fall 2012) has a total of 16 cores (graphics cores included).
So why would JavaFX be single-threaded, thereby seemingly loosing its ability to leverage the full potential of multi-core computers? There are three answers:
  1. Simplicity: A single-threaded programming model is in many aspects much simpler than its multi-threaded counterpart. Since using a single thread is also sufficient for many applications, forcing the complexity of multi-threaded programming onto every JavaFX application would be unjustified.
  2. Overhead: Although multi-threading has the potential of increasing system performance, improper use actually has the potential of decreasing performance. Deciding when and when not to use multi-threading is rarely trivial, so in many cases, a single-threaded approach might be the better choice.
  3. It is still partially multi-core: Even though JavaFX is single-threaded, much of the pixel pushing actually runs in hardware on your computer's graphics board, which still may benefit from multi-core parallelism when rotating part of your scene graph, even though JavaFX itself is single threaded.

Introducing Multi-threading in JavaFX

So if JavaFX has a single-threaded programming model, how can we ever get away with using multiple threads in a JavaFX application? To answer that question, realize that JavaFX is only responsible for purely user interface-related tasks, such as laying out text fields on screen, styling your application, handling user interactions, and so forth. Everything else is completely up to you.
Consequently, you can run as many additional threads as you prefer – just do not tinker with the user interface in parallel with the JavaFX rendering thread. So while this code is illegal if running in its own thread
int result = performLongRunningCalculation();
textField.setText(Integer.toString(result)); 
the following code will work just fine
int result = performLongRunningCalculation();
Platform.runLater(
    new Runnable() {
        public void run() {
            textField.setText(
                Integer.toString(result)
            );
        }
    }
);              
Here, instead of invoking the setText() method of class TextField directly, we define a job for JavaFX to do on the JavaFX rendering thread, using a brief implementation of the Runnable interface.
Be advised, that the runLater() method gives no guarantee as to when later is. In my experience, responsiveness can be as slow as a few seconds, if – and only if – the CPU load is high when you invoke runLater(). Still, under normal circumstances, it happens instantaneously.
I have come across situations where it would be best to have a guaranteed immediate execution, wherefore I have filed the following proposal, which is currently being consideration by the JavaFX team:

A Practical Example

In the JavaFX application I am building for wefend.com, users will be able to monitor their homes and see an event log of everything that is going on. This event log is presented as a semi-3D landscape which allows users to surf freely through time and reexperience the events. Constructing this landscape for a particular date turned out to be rather CPU demanding, because each event has its own little user interface with pictures, text fields, buttons, and what not. Constructing the user interface for just a single event takes 50 milliseconds, which is not bad – in fact, I am quite satisfied with JavaFX's performance in this regard. The problem, however, is the fact that my application is sometimes constructing more than 100 of these, meaning the user will be waiting for at least
100 × 50 milliseconds = 5 seconds
Needless to say, this has a huge impact on the user interface experience.
But what if the user has a multicore CPU? Assuming, for instance, it is a quad core, and given that JavaFX occupies one of these cores for animating the complex scene graph I have, we would still have three cores to split the construction task, thereby lowering the execution time from 5 seconds to 1.67 seconds. At least, that was my theory, and, of course, a theory that I felt obligated to ascertain.

Divide and Conquer

Last week was such a great week at the office, because indeed, using multi-threading turned out to be the perfect remedy. The JavaFX class Task proved a good starting point. In Java SE terms this class is what is called a RunnableFuture. This just means it is like any other Runnable (as seen in the earlier example), so it can be run by a the Thread class. but it is also a Future, which means it works as a handle to a result that is being computed.
Now, in stead of creating 100 user interfaces on a single thread, I am now creating 100 Tasks that can be executed on an arbitrary number of threads. This is an important point. We have 100 tasks to execute, but we do not want 100 threads competing for the CPU's attention – that would put additional strain on the CPU from having to constantly switch between threads.
So how many threads should we choose? The Java class Runtime has a method called availableProcessors() which – contrary to the name – actually gives you the number of threads your machine can run in true, uninterrupted parallel. Mine has two CPU's with 6 cores each, hence the method returns 12. If you have an Intel quad core CPU supporting hyperthreading, the returned count is 8 (thanks to Knut Arne Vedaa for reporting this).

Putting it All Together

To take care of running the threads and reusing them, Java has a built in thread pooling mechanism, that can be created by invoking the static method newFixedThreadPool() of class Executors. So, all in all, my code looks like this:
/* Determine thread count using number of items
   being added to the scene. */
int itemCount = addedItems.size();
int availableProcessors =
    Runtime.getRuntime().availableProcessors();
/* Avoid competing with JavaFX rendering thread
   – i am still experimenting with this, and it
   might end up being removed. */
if(availableProcessors > 1)
    availableProcessors -= 1;
final int threadCount =
    Math.min(availableProcessors, itemCount);
// Create the tasks
List<Task> tasks = new ArrayList();
for(
    int threadNumber = 0;
    threadNumber < threadCount;
    threadNumber++
) {
    //…
    Task task = new Task() {
        @Override
        protected Object call() {
            // Construct the UI
            // …
        }
    };
    tasks.add(task);
}
// Run the threads
ExecutorService threadPool =
    Executors.newFixedThreadPool(threadCount);
for(final Task task : tasks) {
    task.stateProperty().addListener(
    new ChangeListener<State>() {
        @Override
        public void changed(
            ObservableValue<? extends State>
            observable,
            State oldValue,
            State newValue
        ) {
            // Respond to state changes
            // …
            /* If everything goes well update the
               user interface. */
            Platform.runLater(new Runnable() {
                @Override
                public void run() {
                    /* Add the created nodes
                       to the scene graph. */
                    // …
                }
            });
            //…
        }
    });
    threadPool.submit(task);
}

Dodging the Bullets

In my particular application, I actually ran into a JavaFX bug because of this. The bug may be caused by the fact that I am using my own control skins, so this bug may not affect you. However, if you start seeing IllegalStateExceptions, have a look at my work-around described on the JavaFX Jira, or contact me on Twitter (@r4nd4hl).

Conclusion

Multi-threading has great potential for speeding up your JavaFX based user interfaces. Since JavaFX is single-threaded, only the JavaFX rendering thread can work on the live scene graph. However, other lengthy tasks such as constructing complex user interfaces (before they are attached to the scene), can – and sometimes should – be implemented using a mutli-threaded approach. Most modern computers are multi-core, so following the approach described in this article will in many cases increase your user interface performance significantly.

8 comments:

  1. Thanks for sharing. I'm in a project where will soon start using JavaFX for a complex GUI. I noticed that the link to wefend.com is not working. Is it misspelled? It would also be nice with some screenshots of your app.

    ReplyDelete
  2. You are welcome. The website for wefend.com will go online in late 2013. There will be a free beta at some point, so you will be able to take it for a spin. If you would like to be notified, just hit "like" on the product at https://www.facebook.com/wefend – there has not been many posts yet, but there will be more information in 2013.

    ReplyDelete
  3. Great article!

    availableProcessors() does not seem to return the number of physical cores, though. On my 4-core Intel on Windows 7 it returns 8, probably because of the so-called "hyperthreading" (which I've never understood how works).

    ReplyDelete
  4. Thanks for the tip Knut. I will update the article with that information.

    ReplyDelete
  5. Done. Your information has now been added. Thanks again, Knut.

    ReplyDelete
  6. Thanks a lot for this article.
    I have the following question : imagine you have an application where the user can create multiple windows (for example 30). In that case if you have only one main application javafx thread isn't it going to produce a sort of bottleneck while rendering ? If so is it possible to use different process with each one a javafx main thread for each window ?

    ReplyDelete
  7. Good question Emmanuel. It is my present understanding that all graphics run on a single rendering thread no matter how many windows. However, recently support was added for running the application thread in true parallel with the rendering thread, so at least we are now able to benefit from two cores.
    On the other hand, I believe no user would be able cope with 30 simultaneous windows. Can you think of an application that would benefit from just 5 simultaneously realtime rendering windows? I cannot.

    ReplyDelete
  8. What about a financial application where you could have multiple windows whose content is updated in parallel ?
    When you mention "rendering thread" are you suggesting using the javafx.concurrent package and its service class to allow a task to be executed in parallel of the javafx application thread ? If so my concern is about performance. With only one process is the interaction between the background threads and the JavaFX Application thread (to update the windows) going to be effective ?

    ReplyDelete