Investigating the Performance of Reactive Libraries in a Quarkus Microservice

What’s the performance benefit of using reactive libraries in my Java applications? Can I go step-by-step, or do I need to go all-in and refactor the whole code to gain performance benefits of these reactive libraries? If you are dealing with similar questions, this blog is for you! In this blog, we will review different refactoring options for a Quarkus-based Microservice using frameworks such as RESTEasy Reactive, Hibernate reactive and Mutiny from a performance perspective. A typical service architecture of a micro-service is shown in figure 1. In this blog post we will outline which reactive frameworks we have used in this evaluation on the different layers of the architecture. Before going over the details and which combinations of reactive libraries are possible and their performance implications, let us start with the mechanism that is at the core of most reactive libraries.

Figure 1: Service Architecture [1][2]

IO-Threads

Traditionally, Java-Enterprise frameworks assign a thread per request. If one thread interacts with a remote source, then that thread is blocked. In other words, this thread cannot process another request as displayed in figure 2. This system places a limit on achievable concurrency. On the other side, the reactive thread mechanism seen in figure 3, relies on non-blocking I/O and therefore the same thread can manage multiple requests. When communicating with a remote service (API or database) instead of blocking the thread, an I/O operation is scheduled with a continuation attached to it. For instance, if a request comes in that retrieves an entity from the database, instead of blocking, that request schedules an I/O operation and attaches a continuation so that other requests can start their execution.

Figure 2: Worker Threads

Figure 3: I/O-Threads

Reactive Libraries for Java

RESTEasy Reactive

The point of entry in your Java-based microservice applications is usually a REST-based endpoint where the RESTEasy Reactive library supports you in the transition to a reactive request processing shown in figure 4. This is especially convenient if you already use the non-reactive version of RESTEasy as of today.

Figure 4: RESTEasy Reactive in a Microservice [3]

RESTEasy is responsible for modeling your business domain as a resource. With an application protocol such as HTTP over REST it is possible to communicate with the underlying resource. It checks an incoming request and provides a response appropriate to the outcome of the business transaction. RESTEasy Reactive has multiple ways to be utilized. You can start working with RESTEasy Reactive by simply adding the @Blocking annotation as shown in listing 1. In this case the library would use traditional worker threads instead of I/O-threads.

If the processing of the request should happen on the I/O-Thread aswell, then omit the @Blocking annotation and make sure that the underlying code has no blocking interaction as shown in listing 2. With RESTEasy you can also utilize reactive types.

Mutiny

Mutiny is a reactive library that supports asynchronous programming with Observable like types. It is event-driven meaning that you can express what you want to do upon a given event (i.e., failure, success, completion). The use of a reactive programming library will affect the majority of your codebase therefore it is positioned at the core of your application as shown in figure 5 and will of course lead to the increased refactoring effort of all libraries mentioned in this blog post.

Figure 5: Mutiny in a Microservice

You only have two reactive types in the library. A Uni representing a single item or a failure and a Multi representing no items or potentially an infinite amount of items. Let’s compare a traditional implementation of a service method with a version utilizing Mutiny. In listing 3, the service method starts with calls to other services. These calls retrieve the concrete values referenced in the payload of the request. The total value of all included items is calculated before storing and returning the entity.

If Mutiny was utilized, then the results of external services are concurrently combined into a single object as shown in listing 4. That object is then mapped to a new object before it is persisted. The second version also does not reactively interact with external services or the underlying datastore. Instead it simply wrapps the results of those calls inside of reactive types.

Hibernate Reactive

Hibernate is an Object Relational Mapping (ORM) framework which also released a reactive version of its library recently. In a Java-based microservice, Hibernate typically represents the data access layer as shown in figure 6:

Figure 6: Hibernate Reactive in a Microservice [4][5]

The hibernate framework empowers you to more easily write applications with persistence in mind. The core functionality is to simplify the mapping of java objects to a relational data model. To interact with the traditional hibernate in a Java application you can utilize the Java Persistence API (JPA) to inject an EntityManager that provides the necessary methods to communicate with the underlying datastore as shown in listing 5. If you have a solution with RESTEasy Reactive and a blocking Hibernate JPA-implementation, then you would have to set the @Blocking annotation on the api method to make sure that the processing is dispatched to a worker thread and no exception is thrown.

To utilize Hibernate Reactive instead of the traditional Hibernate version, you cannot use JPAs EntityManager but need to use the Hibernate Session objects instead. Hibernate reactive introduces specific session types depending on the Reactive framework which is used in the application. In our example we use a Mutiny.Session was we use Mutiny as reactive framework. This Mutiny.Session can be injected in two ways using a direct Injection as shown in listing 6 or using a SessionFactory as shown in listing 7. The single session essentially provides the same methods as a simple EntityManager but instead of returning commonly known types, every result is wrapped inside a reactive type. When using a single Mutiny Session, you are not allowed to share that session between concurrent reactive streams. If you do, you’ll encounter an exception. To avoid that, you have to inject a Mutiny.SessionFactory.

Performance Test – Implementation Variants

In the following we will outline the performance evaluation we have performed on different implementation variants of an example Microservice. We have refactored the application in several steps until it became fully reactive. For each step we have performed a performance test to see the impact of the reactive refactoring. Overall seven variants have been implemented: and are shown in table 1. If a column is marked, then the corresponding reactive library has been used.

Performance Test Setup

The REST-Service under test provides basic ordering capabilities typical in an e-commerce platform. In particular, the following endpoints are available:

• GET & POST /orders
• GET & PUT & DELETE /orders/{orderId}
• GET /orders/{orderId}/items
• POST & GET & DELETE /orders/{orderId}/items/{itemId}

We are going to take a closer look at the HTTP-Post on /orders. This method out of all the available api methods is the most complex one. It interacts with six external sources while the rest of the api either changes a value from an entity or simply returns that entity.
A suitable test environment was provided to conduct the experiment and is shown in the next figure.

Figure 7: Software Experiment Setup [6][7]

The experiment is orchestrated with Kubernetes in which every container has resource limits defined. The Order-Service is instrumented with the java-auto-instrumentation of OpenTelemetry. The instrumented Order-Service through its agent provides telemetry data to a Jaeger service. Lastly, for load testing JMeter was used with an additional plugin providing arrival thread groups.

During the execution of a test run, the Order-Service is built and a Docker image is pushed to RETITs docker registry. Afterwards, Kubernetes deploys the core containers of the application (i.e., Order-Service, PostgreSQL, Jaeger) and follows that with a deployment of a Maven container which executes a JMeter project with the included test script. Before finishing the test execution, results are archived for later use and the previous deployed containers are deleted.

Performance tests are run with a consistent workload of 1000 requests per minute. This workload was derived from an average customer coversion rate of 3% and the traffic of a randomly chosen e-commerce website. Different test cases exist, as seen in the next table 2, that try to mimic a workflow on an e-commerce website as close as possible.

The Online Shop Order test case simply consists of the user creating an order and after checking out receiving the order information. This workflow will receive the bulk of arriving requests. In particular, every minute test case #1 should be executed 300 times. The experiment also models the user Rescinding an Order with 50 arrivals per minute. In contrast to the first use case, the user rescinds the order after creating it. Changing Payment Information with 50 arrivals per minute and Creating Orders With Multiple Items with 25 arrivals per minute also portray typical use cases in e-commerce platforms. The former simply changes the referenced card in the created order, while the latter adds multiple items after the order creation. Lastly, test case #5 describes the Order Creation With Unwanted Items, meaning that some of the added items will be removed.

With the average conversion rate of 3% for e-commerce applications and the traffic count for a randomly chosen e-commerce site, we have derived roughly 1000 requests per minute for a representative average workload. The request amount is distributed across all previously defined test cases. The arrivals per minute do not equal requests per minute. Every test case consists of at least one request. Multiplying the arrivals per minute and request per iteration results in requests per minute.

Performance Test Results

The test setup was executed for all the seven implementation variants. The resulting response times can be seen in the figure 8. The fully imperative solution averages a response time of 50 milliseconds (ms) while the variant with a blocking RESTEasy Reactive has a mean response time of 49ms. There is another set of implementations that is equally comparable. A variant with RESTEasy Classic, Mutiny and Hibernate Classic has a mean response time of 31ms. Replacing RESTEasy Classic with RESTEasy Reactive and the @Blocking annotation produces a mean response time of 30ms. It can be seen that the overall response time distribution has a lower tendency for the RESTEasy Reactive Blocking version. The performance of every variant utilizing Hibernate Reactive is almost identical. Comparing the three solutions presents only slight changes in the mean response time or the range of data. Additionally, every variant with Hibernate Reactive seems to be outperformed by its counterpart where Hibernate Classic is the difference.

Figure 8: Software Experiments Test Results

The negative impact of Hibernate Reactive in the current environment is most likely caused by how the library obtains reactive sessions. A session is obtained through the withSession method of a Mutiny.SesssionFactory. The session is automatically associated with the current reactive stream and also automatically closed after work has been done. If a nested call to withSession occurs, then that same session is used. During a test run, this causes the performance test to start & stop multiple sessions. With the already low response times the overhead of managing multiple reactive sessions impacts the overall performance in the current testing environment unfavorably. Using single Mutiny sessions in the current setup is not functional, because a single Hibernate Reactive session cannot be shared between concurrent reactive streams. This is especially the case for the createOrder method where the results of different external services are combined. Additionally, because the performance test are executed with concurrent thread groups, a single injected Mutiny session would not work. Given the aforementioned handling of the sessions in Hibernate reactive, a detailed look on the transaction behaviour of the different session handling options will be done in a seperate blog post.

Conclusion

Out of all reactive libraries utilized, the hardest one to get used to is Mutiny. The performance gain with Mutiny can be substantial, especially when combining the results of unrelated external services. Making the switch to RESTEasy Reactive can be simple if no reactive types are returned. Blocking RESTEasy Reactive already provides a minor performance improvement. The transition to Hibernate Reactive can be challenging. The data access layer is usually separated from the service layer. Even if the data access is put into dedicated objects, the service classes which make use of these objects would have to be refactored because Hibernate Reactive either uses reactive types from Mutiny or the asynchronous type CompletionStages. It requires less work and would provide a significant performance gain if Mutiny alone is used on top of existing data access objects. The easiest way to get started with Mutiny is to introduce it at the entry-level of your application.

Useful Links

• Software Performance Meetup Group
• Quarkus Reactive Starting Guide
• RESTEasy Reactive
• Mutiny Guides
• Hibernate Reactive Documentation

References

• [1]: Anatomy of a Microservice
• [2]: Quarkus Logo
• [3]: RESTEasy Logo
• [4]: Hibernate Logo
• [5]: Postgres Logo
• [6]: JMeter Logo
• [7]: OpenTelemetry Logo