THREADPOOL(9) Kernel Developer's Manual THREADPOOL(9)

threadpool — shared pools of kthreads

#include <sys/threadpool.h>

typedef void threadpool_job_fn_t(struct threadpool_job *);

int
threadpool_get(struct threadpool **poolp, pri_t pri);

void
threadpool_put(struct threadpool *pool, pri_t pri);

int
threadpool_percpu_get(struct threadpool_percpu **pool_percpup, pri_t pri);

void
threadpool_percpu_put(struct threadpool_percpu *pool_percpu, pri_t pri);

struct threadpool *
threadpool_percpu_ref(struct threadpool_percpu *pool);

struct threadpool *
threadpool_percpu_ref_remote(struct threadpool_percpu *pool, struct cpu_info *ci);

void
threadpool_job_init(struct threadpool_job *job, threadpool_job_fn_t fn, kmutex_t *interlock, const char *fmt, ...);

void
threadpool_job_destroy(struct threadpool_job *job);

void
threadpool_job_done(struct threadpool_job *job);

void
threadpool_schedule_job(struct threadpool *pool, struct threadpool_job *job);

void
threadpool_cancel_job(struct threadpool *pool, struct threadpool_job *job);

bool
threadpool_cancel_job_async(struct threadpool *pool, struct threadpool_job *job);

The threadpool abstraction is provided to share a pool of kthread(9) kernel threads for medium- to long-term actions, called jobs, which can be scheduled from contexts that do not allow sleeping.

For each priority level, there is one unbound thread pool, and one collection of per-CPU thread pools. Access to the unbound thread pools is provided by () and threadpool_put(). Access to the per-CPU thread pools is provided by () and threadpool_percpu_put().

Job state is stored in the threadpool_job structure. Callers of the threadpool abstraction must allocate memory for threadpool_job structures, but should consider them opaque, and should not inspect or copy them. Each job represented by a threadpool_job structure will be run only once at a time, until the action associated with it calls ().

Jobs are run in thread context and may take arbitrarily long to run or sleep arbitrarily long.

(poolp, pri)
Obtain a reference to the unbound thread pool at priority pri and store it in poolp.

May sleep.

(pool, pri)
Release the reference to the unbound thread pool pool at priority pri, which must be the same as the priority that was passed to threadpool_get() to obtain pool.

May sleep.

Do not use () with thread pools obtained from threadpool_percpu_ref() or threadpool_percpu_ref_remote().

threadpool_percpu_get(pool_percpup, pri)
Obtain a reference to the per-CPU thread pool at priority pri and store it in pool_percpup.

Use () or () with it to get at the thread pool for a particular CPU.

May sleep.

(pool_percpu, pri)
Release a reference to the per-CPU thread pool pool_percpu at priority pri.

May sleep.

(pool_percpu)
Return the thread pool in pool_percpu for the current CPU.

The resulting thread pool pointer is stable until pool_percpu is released with (). Using it to schedule or cancel a job does not require being on the same CPU.

Do not use () with thread pools obtained from threadpool_percpu_ref().

threadpool_percpu_ref_remote(pool_percpu, ci)
Return the thread pool in pool_percpu for the CPU whose struct cpu_info is given by ci.

The resulting thread pool pointer is stable until pool_percpu is released with (). Using it to schedule or cancel a job does not require being on the same CPU, but it is faster and friendlier to the cache to use threadpool_percpu_ref() and use the resulting thread pool only on the same CPU.

Do not use () with thread pools obtained from ().

threadpool_job_init(job, fn, interlock, fmt, ...)
Initialize the threadpool job job to run fn when scheduled and to interlock with interlock. The argument fmt is a printf(9) format string for the job's name.

The mutex interlock is used to synchronize job scheduling and completion. The action fn is required to eventually call (), with interlock held. This is so that while the job is running and may be waiting for work to do, scheduling the job has no effect, but as soon as the job is done, scheduling the job will cause it to run again.

To change the action of a job, you must use () first and then call () again.

threadpool_job_destroy(job)
Destroy the threadpool job job. job must not currently be scheduled to run. If it may still be scheduled, you can use threadpool_cancel_job() to cancel it. However, threadpool_cancel_job_async() is not enough.
threadpool_job_done(job)
Notify that job is done, so that subsequent calls to threadpool_schedule_job() will cause it to re-run its action.

() must be called exactly once by a job's action, and may not be called in any other context.

threadpool_schedule_job(pool, job)
Schedule job to run in a thread in pool as soon as possible, creating a new thread if necessary.

Caller must hold the interlock of job.

() may be called in any context, including hard interrupt context, except at interrupt priority levels above IPL_VM.

threadpool_cancel_job(pool, job)
Cancel job if it has been scheduled but has not yet been assigned a thread, or wait for it to complete if it has.

Caller must hold the interlock of job, which may be released in order to wait for completion.

If job has not been scheduled, () returns immediately. If job has been scheduled, it must have been scheduled in pool, not in any other thread pool.

May sleep.

(pool, job)
Try to cancel job like threadpool_cancel_job(), but if it is already running, return false instead of waiting; otherwise, if it was not scheduled, or if it was scheduled and has not yet begun to run, return true.

Caller must hold the interlock of job.

() may be called in any context, including hard interrupt context, except at interrupt priority levels above IPL_VM.

The threadpool abstraction is implemented in sys/kern/kern_threadpool.c.

kthread(9), workqueue(9)

December 26, 2018 NetBSD 11.0