A non-moving (in the GC sense) contiguous range of bytes, useful for I/O operations.
An iobuf consists of:
All iobuf operations are restricted to operate within the limits. Initially, the
window of an iobuf is identical to the limits. A phantom type, "seek" permission,
controls whether or not code is allowed to change the limits and window. With seek
permission, the limits can be narrow
ed, but can never be widened, and the window can
be set to an arbitrary subrange of the limits.
A phantom type controls whether code can read and write bytes in the bigstring (within the limits) or can only read them.
To present a restricted view of an iobuf to a client, one can create a sub-iobuf or add a type constraint.
Functions operate on the window unless the documentation or naming indicates otherwise.
include sig ... end
val sexp_of_seek : seek ‑> Sexplib.Sexp.t
include sig ... end
val sexp_of_no_seek : no_seek ‑> Sexplib.Sexp.t
type (-'data_perm_read_write, +'seek_permission) t
The first type parameter controls whether the iobuf can be written to. The second type parameter controls whether the window and limits can be changed.
See the Perms
module for information on how the first type parameter is used.
To allow no_seek
or seek
access, a function's type uses _
rather than no_seek
as the type argument to t
. Using _
allows the function to be directly applied to
either permission. Using a specific permission would require code to use coercion
:>
.
There is no t_of_sexp
. One should use Iobuf.Hexdump.t_of_sexp
or sexp_opaque
as desired.
include Core__.Import.Invariant.S2 with type (rw, seek) t := (rw, seek) t
val invariant : 'a Base__.Invariant_intf.inv ‑> 'b Base__.Invariant_intf.inv ‑> ('a, 'b) t Base__.Invariant_intf.inv
module Window : Core__.Import.Hexdump.S2 with type (rw, seek) t := (rw, seek) t
Provides a Window.Hexdump
submodule that renders the contents of t
's window.
module Limits : Core__.Import.Hexdump.S2 with type (rw, seek) t := (rw, seek) t
Provides a Window.Hexdump
submodule that renders the contents of t
's limits.
Provides a Hexdump
submodule that renders the contents of t
's window and limits
using indices relative to the limits.
include Core.Iobuf_intf.Compound_hexdump with type (rw, seek) t := (rw, seek) t
module Hexdump : sig ... end
module Debug : Core.Iobuf_intf.Compound_hexdump with type (rw, seek) t := (rw, seek) t
Provides a Debug.Hexdump
submodule that renders the contents of t
's window,
limits, and underlying bigstring using indices relative to the bigstring.
val create : len:int ‑> (_, _) t
create ~len
creates a new iobuf, backed by a bigstring of length len
,
with the limits and window set to the entire bigstring.
val of_bigstring : ?pos:int ‑> ?len:int ‑> Core__.Import.Bigstring.t ‑> ([< Core__.Import.read_write ], _) t
of_bigstring bigstring ~pos ~len
returns an iobuf backed by bigstring
, with the
window and limits specified starting at pos
and of length len
.
forbid immutable
to prevent aliasing
val set_bounds_and_buffer : src:([> Core__.Import.write ] as data, _) t ‑> dst:('data, seek) t ‑> unit
set_bounds_and_buffer ~src ~dst
copies bounds metadata (i.e., limits and window) and
shallowly copies the buffer (data pointer) from src
to dst
. It does not access
data, but does allow access through dst
. This makes dst
an alias of src
.
Because set_bounds_and_buffer
creates an alias, we disallow immutable src
and
dst
using [> write]
. Otherwise, one of src
or dst
could be read_write :>
read
and the other immutable :> read
, which would allow to write the immutable
alias's data through the read_write
alias.
set_bounds_and_buffer
is typically used with a frame iobuf that need be allocated
only once. This frame can be updated repeatedly and handed to users, without further
allocation. Allocation-sensitive applications need this.
val set_bounds_and_buffer_sub : pos:int ‑> len:int ‑> src:([> Core__.Import.write ] as data, _) t ‑> dst:('data, seek) t ‑> unit
set_bounds_and_buffer_sub ~pos ~len ~src ~dst
is a more efficient version of:
set_bounds_and_buffer ~src:(Iobuf.sub_shared ~pos ~len src) ~dst
.
set_bounds_and_buffer ~src ~dst
is not the same as set_bounds_and_buffer_sub ~dst
~src ~len:(Iobuf.length src)
because the limits are narrowed in the latter case.
~len
and ~pos
are mandatory for performance reasons, in concert with @@inline
.
If they were optional, allocation would be necessary when passing a non-default,
non-constant value, which is an important use case.
val read_only : ([> Core__.Import.read ], 's) t ‑> (Core__.Import.read, 's) t
One may wonder why you'd want to call no_seek
, given that a cast is already
possible, e.g. t : (_, seek) t :> (_, no_seek) t
. It turns out that if you want to
define some f : (_, _) t -> unit
of your own, which can be conveniently applied to
seek
iobufs without the user having to cast seek
up, you need this no_seek
function.
read_only
is more of an historical convenience now that read_write
is a
polymorphic variant, as one can now explicitly specify the general type for an
argument with something like t : (_ perms, _) t :> (read, _) t
.
val capacity : (_, _) t ‑> int
capacity t
returns the size of t
's limits subrange. The capacity of an iobuf can
be reduced via narrow
.
One can call Lo_bound.window t
to get a snapshot of the lower bound of the
window, and then later restore that snapshot with Lo_bound.restore
. This is
useful for speculatively parsing, and then rewinding when there isn't enough data to
finish.
Similarly for Hi_bound.window
and Lo_bound.restore
.
Using a snapshot with a different iobuf, even a sub iobuf of the snapshotted one, has unspecified results. An exception may be raised, or a silent error may occur. However, the safety guarantees of the iobuf will not be violated, i.e., the attempt will not enlarge the limits of the subject iobuf.
module type Bound : Core.Iobuf_intf.Bound with type (d, w) iobuf := (d, w) t
advance t amount
advances the lower bound of the window by amount
. It is an error
to advance past the upper bound of the window or the lower limit.
unsafe_advance
is like advance
but with no bounds checking, so incorrect usage can
easily cause segfaults.
unsafe_resize
is like resize
but with no bounds checking, so incorrect usage can
easily cause segfaults.
flip_lo t
sets the window to range from the lower limit to the lower bound of the
old window. This is typically called after a series of Fill
s, to reposition the
window in preparation to Consume
the newly written data.
The bounded version narrows the effective limit. This can preserve some data near the
limit, such as an hypothetical packet header, in the case of bounded_flip_lo
or
unfilled suffix of a buffer, in bounded_flip_hi
.
val bounded_flip_lo : (_, seek) t ‑> Lo_bound.t ‑> unit
val compact : (Core__.Import.read_write, seek) t ‑> unit
compact t
copies data from the window to the lower limit of the iobuf and sets the
window to range from the end of the copied data to the upper limit. This is typically
called after a series of Consume
s to save unread data and prepare for the next
series of Fill
s and flip_lo
.
val bounded_compact : (Core__.Import.read_write, seek) t ‑> Lo_bound.t ‑> Hi_bound.t ‑> unit
flip_hi t
sets the window to range from the the upper bound of the current window to
the upper limit. This operation is dual to flip_lo
and is typically called when the
data in the current (narrowed) window has been processed and the window needs to be
positioned over the remaining data in the buffer. For example:
(* ... determine initial_data_len ... *)
Iobuf.resize buf ~len:initial_data_len;
(* ... and process initial data ... *)
Iobuf.flip_hi buf;
Now the window of buf
ranges over the remainder of the data.
val bounded_flip_hi : (_, seek) t ‑> Hi_bound.t ‑> unit
"consume" and "fill" functions access data at the lower bound of the window and advance lower bound of the window. "peek" and "poke" functions access data but do not advance the window.
val to_string : ?len:int ‑> ([> Core__.Import.read ], _) t ‑> string
to_string t
returns the bytes in t
as a string. It does not alter the window.
val to_string_hum : ?max_lines:int ‑> (_, _) t ‑> string
Equivalent to Hexdump.to_string_hum
. Renders t
's windows and limits.
module Consume : sig ... end
Consume.string t ~len
reads len
characters (all, by default) from t
into a new
string and advances the lower bound of the window accordingly.
module Fill : sig ... end
Fill.bin_prot X.bin_write_t t x
writes x
to t
in bin-prot form, advancing past
the bytes written.
module Peek : sig ... end
Peek
and Poke
functions access a value at pos
from the lower bound of the window
and do not advance.
module Poke : sig ... end
Poke.bin_prot X.bin_write_t t x
writes x
to the beginning of t
in binary form
without advancing. You can use X.bin_size_t
to tell how long it was.
X.bin_write_t
is only allowed to write that portion of the buffer to which you have
access.
module Unsafe : sig ... end
Unsafe
has submodules that are like their corresponding module, except with no range
checks. Hence, mistaken uses can cause segfaults. Be careful!
val crc32 : ([> Core__.Import.read ], _) t ‑> Core__.Import.Int63.t
val bin_prot_length_prefix_bytes : int
The number of bytes in the length prefix of consume_bin_prot
and fill_bin_prot
.
val fill_bin_prot : ([> Core__.Import.write ], seek) t ‑> 'a Bin_prot.Type_class.writer ‑> 'a ‑> unit Core__.Import.Or_error.t
fill_bin_prot
writes a bin-prot value to the lower bound of the window, prefixed by
its length, and advances by the amount written. fill_bin_prot
returns an error if
the window is too small to write the value.
consume_bin_prot t reader
reads a bin-prot value from the lower bound of the window,
which should have been written using fill_bin_prot
, and advances the window by the
amount read. consume_bin_prot
returns an error if there is not a complete message
in the window and in that case the window is left unchanged.
Don't use these without a good reason, as they are incompatible with similar functions
in Reader
and Writer
. They use a 4-byte length rather than an 8-byte length.
val consume_bin_prot : ([> Core__.Import.read ], seek) t ‑> 'a Bin_prot.Type_class.reader ‑> 'a Core__.Import.Or_error.t
module Blit_consume : sig ... end
Blit_consume
copies between iobufs and advances src
but does not advance dst
.
module Blit_fill : sig ... end
Blit_fill
copies between iobufs and advances dst
but does not advance src
.
module Blit_consume_and_fill : sig ... end
Blit_consume_and_fill
copies between iobufs and advances both src
and dst
.
include sig ... end
val sexp_of_ok_or_eof : ok_or_eof ‑> Sexplib.Sexp.t
val input : ([> Core__.Import.write ], seek) t ‑> Core__.Import.In_channel.t ‑> ok_or_eof
Iobuf
has analogs of various Bigstring
functions. These analogs advance by the
amount written/read.
val read : ([> Core__.Import.write ], seek) t ‑> Core.Iobuf_intf.Unix.File_descr.t ‑> ok_or_eof
val read_assume_fd_is_nonblocking : ([> Core__.Import.write ], seek) t ‑> Core.Iobuf_intf.Unix.File_descr.t ‑> Core__.Syscall_result.Unit.t
val pread_assume_fd_is_nonblocking : ([> Core__.Import.write ], seek) t ‑> Core.Iobuf_intf.Unix.File_descr.t ‑> offset:int ‑> unit
val recvfrom_assume_fd_is_nonblocking : ([> Core__.Import.write ], seek) t ‑> Core.Iobuf_intf.Unix.File_descr.t ‑> Core.Iobuf_intf.Unix.sockaddr
module Recvmmsg_context : sig ... end with type (rw, seek) iobuf := (rw, seek) t
recvmmsg
's context comprises data needed by the system call. Setup can be
expensive, particularly for many buffers.
val recvmmsg_assume_fd_is_nonblocking : (Core.Iobuf_intf.Unix.File_descr.t ‑> Recvmmsg_context.t ‑> Core.Iobuf_intf.Unix.Syscall_result.Int.t) Core__.Import.Or_error.t
recvmmsg_assume_fd_is_nonblocking fd context
returns the number of context
iobufs
read into (or errno
). fd
must not block. THREAD_IO_CUTOFF
is ignored.
EINVAL
is returned if an Iobuf
passed to Recvmmsg_context.create
has its buf
or limits changed.
val send_nonblocking_no_sigpipe : unit ‑> (([> Core__.Import.read ], seek) t ‑> Core.Iobuf_intf.Unix.File_descr.t ‑> Core.Iobuf_intf.Unix.Syscall_result.Unit.t) Core__.Import.Or_error.t
val sendto_nonblocking_no_sigpipe : unit ‑> (([> Core__.Import.read ], seek) t ‑> Core.Iobuf_intf.Unix.File_descr.t ‑> Core.Iobuf_intf.Unix.sockaddr ‑> Core.Iobuf_intf.Unix.Syscall_result.Unit.t) Core__.Import.Or_error.t
val output : ([> Core__.Import.read ], seek) t ‑> Core__.Import.Out_channel.t ‑> unit
val write : ([> Core__.Import.read ], seek) t ‑> Core.Iobuf_intf.Unix.File_descr.t ‑> unit
val write_assume_fd_is_nonblocking : ([> Core__.Import.read ], seek) t ‑> Core.Iobuf_intf.Unix.File_descr.t ‑> unit
val pwrite_assume_fd_is_nonblocking : ([> Core__.Import.read ], seek) t ‑> Core.Iobuf_intf.Unix.File_descr.t ‑> offset:int ‑> unit
module Expert : sig ... end
The Expert
module is for building efficient out-of-module Iobuf
abstractions.