Structured concurrency to simplify multithreaded Java programming

JEP 428, Structured Competition (Incubator), has been promoted of Proposed to target at Target status for JDK 19. Under the aegis of Project loom, this JEP proposes to simplify multithreaded programming by introducing a library to treat multiple tasks running on different threads as an atomic operation. As a result, it will streamline error handling and cancellation, improve reliability, and improve observability. This is one more Incubation API.

This allows developers to organize their concurrency code using the StructuredTaskScope to classify. It will treat a family of subtasks as a unit. Subtasks will be created on their own threads by forking them individually, but then joined as a unit and possibly canceled as a unit; their exceptions or positive results will be aggregated and handled by the parent task. Let’s see an example:

Response handle() throws ExecutionException, InterruptedException {
   try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
       Future user = scope.fork(() -> findUser());
       Future order = scope.fork(() -> fetchOrder());

       scope.join();          // Join both forks
       scope.throwIfFailed(); // ... and propagate errors

       // Here, both forks have succeeded, so compose their results
       return new Response(user.resultNow(), order.resultNow());
   }
}

What precedes manipulate() The method represents a task in a server application. It handles an incoming request by creating two subtasks. As ExecutorService.submit(), StructuredTaskScope.fork() take a Callable and returns a Future. Contrary to ExecutorServicethe return Future is not attached via Future.get(). This API runs on top of JEP 425, Virtual Threads (Preview)also targeted for JDK 19.

The examples above use the StructuredTaskScope APIs, so to run them on JDK 19 a developer needs to add the jdk.incubator.concurrent module, as well as enable preview features to use virtual threads:

Compile the above code as shown in the following command:

javac --release 19 --enable-preview --add-modules jdk.incubator.concurrent Main.java

The same flag is also required to run the program:

java --enable-preview --add-modules jdk.incubator.concurrent Main;

However, it can be executed directly using the source code launcher. In this case, the command line would be:

java --source 19 --enable-preview --add-modules jdk.incubator.concurrent Main.java

The jshell is also available, but also requires the preview feature to be enabled:

jshell --enable-preview --add-modules jdk.incubator.concurrent

The benefits of structured competition are numerous. It creates a child-parent relationship between the invoker’s method and its subtasks. For example, from the example above, the handle() the task is a parent and its subtasks, findUser() and fetchOrder(), are children. As a result, the whole block of code becomes atomic. It ensures observability by demonstrating the hierarchy of tasks in the thread dump. It also allows short-circuiting in error handling. If one of the subtasks fails, the other tasks will be canceled if not completed. If the parent task’s thread is interrupted before or during the call to join(), the two forks will be automatically canceled when exiting the oscilloscope. These clarify the structure of concurrent code, and the developer can now reason and follow the code as if it is reading as if it is running in a single-threaded environment.

In the early days of programming, the flow of a program was controlled by the widespread use of the GOTO statement, and it resulted in messy, spaghetti-like code that was hard to read and debug. As the programming paradigm matured, the programming community understood that the GOTO statement was wrong. In 1969, Donald Knutha computer scientist widely known for the bookThe art of computer programming defended that programs can be written efficiently without GOTO. Later, structured programming emerged to address all these shortcomings. Consider the following example:

Response handle() throws IOException {
   String theUser = findUser();
   int theOrder = fetchOrder();
   return new Response(theUser, theOrder);
}

The code above is an example of structured code. In a single-threaded environment, it is executed sequentially when the handle() method is called. The fetchOrder() method does not start before findUser() method. If the findUser() fails, the next method invocation will not start at all and the handle() The method fails implicitly, which in turn guarantees that the atomic operation succeeds or fails. This gives us a parent-child relationship between the handle() method and its child method calls, which tracks error propagation and gives us a call stack at runtime.

However, this approach and reasoning does not work with our current threading programming model. For example, if we want to write the code above with ExecutorServicethe code becomes the following:

Response handle() throws ExecutionException, InterruptedException {
   Future  user  = executorService.submit(() -> findUser());
   Future order = executorService.submit(() -> fetchOrder());
   String theUser  = user.get();   // Join findUser
   int theOrder = order.get();  // Join fetchOrder
   return new Response(theUser, theOrder);
}

The subtasks in ExecutorService operate independently, so they can succeed or fail independently. The interrupt does not propagate to the subtasks even if the parent is interrupted and thus creates a leak scenario. He loses the parental relationship. It also complicates debugging because parent and child tasks appear on the call stack of unrelated threads in the thread dump. Although the code may seem logically structured, it stays in the developer’s mind rather than running; thus, concurrent code becomes unstructured.

Observing all these issues with unstructured concurrent code, the term “Structured Concurrency” was coined by Martin Sustrik in his blog post then popularized by Nathaniel J. Smith in his Notes on structured competition article. About structured competition, Ron PresserConsulting member of Oracle’s technical staff and project manager for Project Loom, in an InfoQ podcast, states:

Structured means that if you generate something, you have to wait for it and join it. And the word structure here is similar to its use in structured programming. And the idea is that the block structure of your code reflects the runtime behavior of the program. So just like structured programming gives you that for sequential control flow, structured concurrency does the same for concurrency.

Developers interested in a deep dive into structured concurrency and learning the backstory can listen to the InfoQ podcast, a Youtube session by Ron Pressler and the Inside Java articles.

Comments are closed.