Module Async_smtp__Multispool_intf

Multispool

Multispool allows multiple, separate processes to cooperate via a filesystem-based queue. The design was influenced by various UNIX tools that use typical POSIX-y write/fsync/close/rename semantics to provide atomic filesystem operations (Maildir, in particular--see http://www.qmail.org/man/man5/maildir.html for an overview).

One or more processes may place files in a queue, wait for files to appear in queues (and handle them), or iterate over files in a queue.

Usage Summary

A spool is physically represented by a directory, and logically as a module created by applying the Multispool functor. A spool deals in a data type that implements the Spoolable interface. This interface tells the spool how to encode and decode items for on-disk storage and how to map queue names to directories on disk. See ../test/lib/widget.ml for an example Spoolable implementation.

An existing spool is opened with load and a new one is created with create.

Use enqueue to add items to a queue, optionally reserving a name beforehand with reserve_name.

If you want to wait on entries to appear, use Queue_reader.iter. If you want to make periodic passes over all entries in a queue, use Queue_reader.iter_available.

Lower-level functionality is available in the Expert module.

Implementation Details

Enqueueing attempts to create a file with a unique name within the .registry/ directory by open(2)ing the file with the O_CREAT and O_EXCL flag (which will fail if the name exists). If it fails, a new name is generated via Spoolable.Name_generator and the process repeats. Once a file is created in .registry/, it remains as a name "reservation" and a file (with the same name) is created in the desired queue. Keeping the empty file in .registry/ ensures that no other process can create the same file name while this name is in use within the spool.