Demystifying the Event Loop
The event loop plays an important role in Vert.x for writing highly scalable and performant network applications.
The event loop is inherited from the Netty library on which Vert.x is based.
We often use the expression running on the event loop, it has a very specific meaning: it means that the current Thread is an event loop thread. This article provides an overview of the Vert.x event loop and the concepts related to it.
The golden rule
When using Vert.x there is one Vert.x golden rule to respect:
Never block the event loop!
The code executed on the event loop should never block the event loop, for instance:
using a blocking method directly or, for instance, reading a file with the
java.io.FileInputStreamapi or a JDBC connection.
doing a long and CPU intensive task
When the event loop is blocked:
Vert.x will detect it and log a warn:
The event loop must not be blocked, because it will freeze the parts of the applications using that event loop, with severe consequences on the scalability and the throughput of the application.
Beyond the event loop, Vert.x defines the notion of a context. At a high level, the context can be thought of as controlling the scope and order in which a set of handlers (or tasks created by handlers) are executed.
When the Vert.x API consumes callbacks (for instance setting an
HttpServer request handler), it associates a callback handler
with a context. This context is then used for scheduling the callbacks, when such context is needed:
if the current thread is a Vert.x thread, it reuses the context associated with this thread: the context is propagated.
otherwise a new context is created for this purpose.
However there is one case where context propagation does not apply: deploying a Verticle creates a new context for this Verticle, according to the deployment options of the deployment. Therefore a Verticle is always associated with a context. Any handler registered within a verticle - whether it be an event bus consumer, HTTP server handler, or any other asynchronous operation - will be registered using the verticle’s context.
Vert.x provides three different types of contexts.
Event loop context
Multi-threaded worker context
Event loop context
An event loop context executes handlers on an event loop: handlers are executed directly on the IO threads, as a consequence:
an handler will always be executed with the same thread
an handler must never block the thread, otherwise it will create starvation for all the IO tasks associated with that event loop.
This behavior allows for a greatly simplified threading model by guaranteeing that associated handlers will always be executed on the same thread, thus removing the need for synchronization and other locking mechanisms.
This is the type of context that is the default and most commonly used type of context. Verticles deployed without the worker flag will always be deployed with an event loop context.
When Vert.x creates an event loop context, it chooses an event loop for this context, the event loop is chosen via a round robin algorithm. This can be demonstrated by creating a timer many times:
The result is:
As we can see we obtained different event loop threads for each timer and the threads are obtained with a round robin policy. Note that the number of event loop threads by default depends on your CPU but this can be configured.
An event loop context guarantees to always use the same thread, however the converse is not true: the same thread can be used by different event loop contexts. The previous example shows clearly that a same thread is used for different event loops by the Round Robin policy.
The default number of event loop created by a Vertx instance is twice the number of cores of your CPU. This value can be overriden when creating a Vertx instance:
Worker contexts are assigned to verticles deployed with the worker option enabled. The worker context is differentiated from standard event loop contexts in that workers are executed on a separate worker thread pool.
This separation from event loop threads allows worker contexts to execute the types of blocking operations that will block the event loop: blocking such thread will not impact the application other than blocking one thread.
Just as is the case with the event loop context, worker contexts ensure that handlers are only executed on one thread at any given time. That is, handlers executed on a worker context will always be executed sequentially - one after the other - but different actions may be executed on different threads.
A common pattern is to deploy worker verticles and send them a message and then the worker replies to this message:
The previous example clearly shows that the worker context of the verticle use different worker threads for delivering the messages:
However the same thread can be used by several worker verticles:
The same worker verticle class can be deployed several times by specifying the number of instances. This allows to concurrently process blocking tasks:
Workers can schedule timers:
Again the timer thread is not the same than the thread that created the timer.
With a periodic timer:
we get a different thread for each event:
Since the worker thread may block, the delivery cannot be guaranteed in time:
Just like event loop, the size of the worker thread pool can be configured when creatin a Vertx instance:
Multi-threaded worker context
Multi-threaded contexts are assigned to verticles deployed with the multi-threaded option enabled. Whereas standard worker contexts execute actions in order on a variety of threads, the multi-threaded worker context removes the strong ordering of events to allow the execution of multiple events concurrently. This means that the user is responsible for performing the appropriate concurrency control such as synchronization and locking.
Dealing with contexts
Using a context is usually transparent, Vert.x will manage contexts implicitly when deploying a Verticle, registering an Event Bus handler, etc… However the Vert.x API provides several ways to interact with a Context allowing for manual context switching.
The current context
Vertx.currentContext() methods returns the current context if there is one, it returns null otherwise.
We get obviously
null no matter the Vertx instance we created before:
Current context is null
Now the same from a verticle leads to obtaining the
Creating or reusing a context
vertx.getOrCreateContext() method returns the context associated with the current thread (like
otherwise it creates a new context, associates it to an event loop and returns it:
Note, that creating a context, will not associate the current thread with this context. This will indeed not change the nature of the current thread! However we can now use this context for running an action:
getOrCreateContext from a verticle returns the context associated with the Verticle:
Running on context
io.vertx.core.Context.runOnContext(Handler) method can be used when the thread attached to the context needs
to run a particular task on a context.
For instance, the context thread initiates a non Vert.x action, when this action ends it needs to do update some state and it needs to be done with the context thread to guarantee that the state will be visible by the context thread.
Running with context : io.vertx.core.impl.EventLoopContext@69cdd6d8 Current context : null Runs on the original context : io.vertx.core.impl.EventLoopContext@69cdd6d8
vertx.runOnContext(Handler<Void>) is a shortcut for what we have seen before: it calls the
getOrCreateContext method and schedule a task for execution via the
Before Vert.x 3, using blocking API required to deploy a worker Verticle. Vert.x 3 provides an additional API for using a blocking API:
Calling blocking block from Thread[vert.x-eventloop-thread-0,5,main] Computing with Thread[vert.x-worker-thread-0,5,main] Got result in Thread[vert.x-eventloop-thread-0,5,main]
While the blocking action executes with a worker thread, the result handler is executed with the same event loop context.
The blocking action is provided a
Future argument that is used for signaling when the result is obtained,
usually a result of the blocking API.
When the blocking action fails the result handler will get the failure as cause of the async result object:
Blocking code failed java.lang.RuntimeException at org.vietj.vertx.eventloop.ExecuteBlockingThrowingFailure.lambda$null$0(ExecuteBlockingThrowingFailure.java:19) at org.vietj.vertx.eventloop.ExecuteBlockingThrowingFailure$$Lambda$4/163784093.handle(Unknown Source) at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:217) at io.vertx.core.impl.ContextImpl$$Lambda$6/1645685573.run(Unknown Source) at io.vertx.core.impl.OrderedExecutorFactory$OrderedExecutor.lambda$new$180(OrderedExecutorFactory.java:91) at io.vertx.core.impl.OrderedExecutorFactory$OrderedExecutor$$Lambda$2/1053782781.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
The blocking action can also report the failure on the
Obviously executing a task from the blocking action on the context will use the event loop:
Calling blocking block from Thread[vert.x-eventloop-thread-0,5,main] Computing with Thread[vert.x-worker-thread-0,5,main] Running on context from the worker Thread[vert.x-eventloop-thread-0,5,main]
This API is somewhat similar to deploying a worker Verticle, although its purpose is to execute a single blocking operation from an event loop context.
Execute blocking for any particular verticle instance uses the same context as that instance.
By default, if you call
executeBlocking multiple times in any particular instance they will be executed in the order you
called them. If we didn’t do that you’d get into a mess, e.g. if you did an insertBlocking to insert
some data into a table, followed by another to select from that table, then there’d be no guarantee in which
order they occurred so you might not find your data.
When several blocking tasks are submitted, the current implementation picks an available worker for executing the first task, after its execution, it will execute any pending tasks. After the executions of all the tasks, the worker stops and goes back in the worker pool.
It is possible to execute also unordered blocks, i.e the blocks can be executed in parallel by setting the
ordered argument to
Determining the kind of context
The kind of a context can be determined with the methods:
the nature of the context does not guarantee the nature of the thread, indeed the
Determining the kind of thread
As said earlier, the nature of the context impacts the concurrency. The
executeBlocking method can even change
use a worker thread in an event loop context. The kind of context should be properly determined with the static methods:
When the Vert.x API needs a context, it calls the
vertx.getOrCreateContext() method, when the Vert.x API is used
in a context, for instance when deploying a Verticle. This implies that any service created from this Verticle
will reuse the same context, for instance:
Creating a server
Creating a client
Creating a timer
Registering an event bus handler
Such services will call back the Verticle that created them at some point, how this happens is according to the context: the context remains the same, however its nature has a direct impact on the concurrency as it govers the threading model:
Deployed as a worker, no exclusion is required, however the changes must be visibles between threads, pretty much like this:
When Vert.x is embedded like in a main Java method or a junit test, the thread creating Vert.x can be any kind of thread, but it is certainly not a Vert.x thread. Any action that requires a context will implicitly create an event loop context for executing this action.
When several actions are done, there will use different context and there are high chances they will use a different event loop thread.
Current thread is Thread[vert.x-eventloop-thread-1,5,main] Current thread is Thread[vert.x-eventloop-thread-0,5,main]
Therefore accessing a shared state from both servers should not be done!
When the same context needs to be used then the actions can be grouped with a
Current thread is Thread[vert.x-eventloop-thread-0,5,main] Current thread is Thread[vert.x-eventloop-thread-0,5,main]
Now we can share state between the two servers safely.
Vert.x Core apis
Vert.x API consumes handlers and assign them to context, this section provides a quick overview of the Vert.x Core APIs.
TCP servers (HttpServer and NetServer) can run with both event loop and worker contexts. A TCP server consumes a context for the various handlers it uses.
A worker server uses under the hood an event loop for its IO operations, however the worker context is used for calling the registered handlers. Consequently a worker server can block directly, when it happens, this will not have consequences on the underlying event loop, however it does impact directly the server, as this particular server will be blocked: of course the server can be scaled to many workers to handle multiple blocking requests concurrently, this is the classic multithreaded server model.
TCP clients (HttpClient and NetClient) can run with both event loop and worker contexts. Clients don’t have a particular context assigned. A context is instead assigned every time a connection or a request is done.
Every time a timer or periodic is created, a context is assigned, this context is then used when the timer or periodic fires. Event bus or worker contexts are allowed.
A context is assigned when an handler is registered for consuming a message, it can be a registered consumer or registering a message reply handler. Event bus or worker contexts are allowed.