Java’s ThreadPoolExecutor is great for running multiple tasks in parallel. It’s a thread pool with a task queue and a pool of threads wrapped in ExecutorService interface.
In this blog post, let’s examine some strategies to deal with situations when the thread pool cannot accept new tasks, that is, there are no idle threads to perform the incoming task and the task queue is full. If this happens, the new task is rejected.
Built-in strategies for rejecting tasks
By default, the ThreadPoolExecutor throws RejectedExecutionException to reject a task. This shifts the responsibility back to the task submitter to deal with the task rejection. Developers could modify this behaviour and specify discard policy to drop task that cannot be executed. This makes sense when tasks are non-critical and could be retried later (e.g. request for a web resource). Similarly, discard oldest policy will drop the oldest submitted task from the queue.
Java also provides a caller-runs-policy, which means that if task cannot be executed, it will be run in the thread that invoked the task.
A missing strategy: Block the caller
Often times, it makes sense to make task submitter wait until a thread is available to accept the task. In Java speak: “block” the caller. For example:
“Yaneeve needed it to analyze a huge directory with a
very long list of files, where there was no point in piling on more
and moreFileAnalyzeTaskinstances without a free
thread to handle them. The analyze operation takes some time, while
the speed in which we can pile files for analysis is much higher.
Thus, not controlling for thread availability for the task would
create a huge queue with a possible memory problem, and for no
benefit.
Java doesn’t provide this block-caller-until-thread-becomes-available behaviour out of box. However, Java provides ability to write customer rejection handlers. I wrote a BlockingThreadPoolExecutor which I used in one of our projects. It is available with full source code here:
I attached a custom of RejectedExecutionHandler to ThreadPoolExecutor which keeps retrying task submission forever (until the thread pool shuts down). In addition, on each subsequent rejection or acceptance, handler methods are called giving them a chance to take some application defined action.
Why Caller-Runs Policy isn’t enough?
You might wonder if BlockingThreadPool executor has any real benefits over caller-runs policy. In other words, rather than making task submitters wait idly until a thread becomes available, doesn’t it make more sense to let them run the task instead?
This article highlights a potential drawback of the caller-run policy:
When the producer is working on its task, no one fills the queue.
So if one of the worker threads, or more, finish their tasks while the producer is still working, they will become idle. It requires fine configuration tuning of the queue size in order to minimize it, but you can never guarantee to avoid this situation. It would have been nice if there was a way to set theThreadPoolExecutorin a true Leader-Followers manner
(a design pattern in which the producer gets to run the task while a thread from the pool becomes the new producer), but theCallerRunsPolicystrategy does not work like that.
Summary
In this post, we looked at various built-in strategies to handle task rejections in ThreadPoolExecutor when the pool cannot accept new tasks. In some situations, a blocking strategy, that is, make the task submitter wait until thread pool is ready to accept the task is desirable. Since Java doesn’t provide this by default, a source code of blocking thread pool executor was provided.
10K-LOC 