Module Core_kernel.Limiter

Implements a token bucket based throttling rate limiter. This module is useful for limiting network clients to a sensible query rate, or in any case where you have jobs that consume a scarce, but replenishable resource.

In a standard token bucket there is an infinite incoming supply of tokens that fill a single bucket.

This version implements a closed system where tokens move through three possible states:

tokens "drop" from the hopper into the bucket at a set rate, and can be taken from the bucket by clients and put into flight. Once the client is finished with whatever task required tokens they are responsible for moving them from "in flight" back into the hopper.

Most use cases are covered by the Token_bucket, Throttle, and Throttled_rate_limiter modules, but the Expert module provides full access to the module internals.

This interface is the simple, non-concurrent interface, and requires machinery on top to implement a specific strategy. See Async_extra for an async-friendly implementation on top of this module.

Most functions in this interface take an explicit time as an argument. now is expected to be monotonically increasing. now's that are set in the past are effectively moved up to the current time of the bucket.

type t
include sig ... end
val sexp_of_t : t ‑> Sexplib.Sexp.t
type limiter = t
include sig ... end
val sexp_of_limiter : limiter ‑> Sexplib.Sexp.t
module Infinite_or_finite : sig ... end
module Try_take_result : sig ... end
module Try_return_to_bucket_result : sig ... end
module Tokens_may_be_available_result : sig ... end
module Token_bucket : sig ... end

Implements a basic token bucket based rate limiter. Users of the throttle must successfully call try_take before doing work.

module Throttle : sig ... end

Implements a basic throttle. Users of the throttle must successfully call start_job before beginning work and must call finish_job once, and only once, when a job is completed.

module Throttled_rate_limiter : sig ... end

A Throttled_rate_limiter combines a Token_bucket and a Throttle. Unlike a Token_bucket jobs cannot consume variable numbers of tokens, but the number of outstanding jobs is also limited to max_concurrent_jobs. Like a Throttlefinish_job must be called once, and only once when a job is completed.

common read-only operations
val bucket_limit : t ‑> Core_kernel__.Import.int
val in_bucket : t ‑> now:Time_ns.t ‑> Core_kernel__.Import.int

tokens available to immediately take

val in_hopper : t ‑> now:Time_ns.t ‑> Core_kernel__.Import.int Infinite_or_finite.t

tokens waiting to drop at the hopper_to_bucket_rate_per_sec

val in_flight : t ‑> now:Time_ns.t ‑> Core_kernel__.Import.int

tokens that have been taken, but not yet returned

val in_limiter : t ‑> now:Time_ns.t ‑> Core_kernel__.Import.int Infinite_or_finite.t

total number of tokens in the limiter in_hopper + in_bucket

val in_system : t ‑> now:Time_ns.t ‑> Core_kernel__.Import.int Infinite_or_finite.t

total number of tokens in the entire system in_hopper + in_bucket + in_flight

val hopper_to_bucket_rate_per_sec : t ‑> Core_kernel__.Import.float Infinite_or_finite.t

Note that this isn't guaranteed to be equal to the rate_per_sec that was passed in on the constructor, due to floating point error.

module Expert : sig ... end
expert operations
include Core_kernel__.Import.Invariant.S with type t := t
type t