Module Reader

module Reader: sig .. end

Reader is Async's main API for buffered input from a file descriptor. It is the analog of Core.Std.In_channel.

Each reader has an internal buffer, which is filled via read() system calls when data is needed to satisfy a Reader.read* call.

Each of the read functions returns a deferred that will become determined when the read completes. It is an error to have two simultaneous reads. That is, if one calls a read function, one should not call another read function until the first one completes.

If the file descriptor underlying a reader is closed, the reader will return EOF (after all the buffered bytes have been read).

Any Reader.read* call could, rather than determine its result, send an exception to the monitor in effect when read was called. Such exceptions can be handled in the usual way by using try_with, e.g.:

      try_with (fun () -> Reader.read reader ...)

module Read_result: sig .. end

module Id: Core.Std.Unique_id

type t

include Invariant.S

val io_stats : Io_stats.t

io_stats Overall IO statistics for all readers

val last_read_time : t -> Core.Std.Time.t

last_read_time t returns time of the most recent read system call that returned data.

val stdin : t Core.Std.Lazy.t

stdin is a reader for file descriptor 0. It is lazy because we don't want to create it in all programs that happen to link with async.

val open_file : ?close_on_exec:bool -> ?buf_len:int -> string -> t Import.Deferred.t

open_file file opens file for reading and returns a reader reading from it.

val transfer : t -> string Import.Pipe.Writer.t -> unit Import.Deferred.t

transfer t pipe_w transfers data from t into pipe_w one chunk at a time (whatever is read from the underlying file descriptor without post-processing). The result becomes determined after reaching EOF on t and the final bytes have been transferred.

This function will normally not be needed (see pipe).

val pipe : t -> string Import.Pipe.Reader.t

pipe t returns the reader end of a pipe that will continually be filled with chunks of data from the underlying Reader.t. When the reader reaches EOF or the pipe is closed, pipe closes the the reader, and then after the reader close is finished, closes the pipe.

val create : ?buf_len:int -> Fd.t -> t

create ~buf_len fd creates a new reader that is reading from fd.

val of_in_channel : Pervasives.in_channel -> Fd.Kind.t -> t

val with_file : ?buf_len:int ->
       ?exclusive:bool ->
       string -> f:(t -> 'a Import.Deferred.t) -> 'a Import.Deferred.t

with_file file f opens files, creates a reader with it, and passes the reader to f. It closes the reader when the result of f becomes determined, and returns f's result.

val close : t -> unit Import.Deferred.t

close t prevents further use of t and closes t's underlying file descriptor. The result of close becomes determined once the underlying file descriptor has been closed. It is an error to call other operations on t after close t has been called, except that calls of close subsequent to the original call to close will return the same deferred as the original call.

close_finished t becomes determined after t's underlying file descriptor has been closed, i.e. it is the same as the result of close. close_finished differs from close in that it does not have the side effect of initiating a close.

is_closed t returns true iff close t has been called.

with_close t ~f runs f (), and closes t after f finishes or raises.

val close_finished : t -> unit Import.Deferred.t

val is_closed : t -> bool

val with_close : t -> f:(unit -> 'a Import.Deferred.t) -> 'a Import.Deferred.t

val id : t -> Id.t

id t
Returns a name for this reader that is unique across all instances of the reader module.

val fd : t -> Fd.t

fd t
Returns the Fd.t used to create this reader

val read : t ->
       ?pos:int -> ?len:int -> string -> int Read_result.t Import.Deferred.t

read t ?pos ?len buf reads up to len bytes into buf, blocking until some data is available or end-of-input is reached. The resulting i satisfies 0 < i <= len.

type 'a read_one_chunk_at_a_time_result = [ `Eof | `Eof_with_unconsumed_data of string | `Stopped of 'a ]

read_one_chunk_at_a_time t ~handle_chunk reads into t's internal buffer, and whenever bytes are available, applies handle_chunk to them. It waits to read again until the deferred returned by handle_chunk becomes determined. read_one_chunk_at_a_time continues reading until it reaches `Eof or handle_chunk returns `Stop or `Stop_consumed. In the case of `Stop and `Stop_consumed, one may read from t after read_one_chunk_at_a_time returns.

val read_one_chunk_at_a_time : t ->
       handle_chunk:(Core.Std.Bigstring.t ->
                     pos:int ->
                     len:int ->
                     [ `Consumed of int * [ `Need of int | `Need_unknown ]
                     | `Continue
                     | `Stop of 'a
                     | `Stop_consumed of 'a * int ] Import.Deferred.t) ->
       'a read_one_chunk_at_a_time_result Import.Deferred.t

val read_substring : t ->
       Core.Std.Substring.t -> int Read_result.t Import.Deferred.t

read_substring t ss reads up to Substring.length ss bytes into ss, blocking until some data is available or Eof is reched. The resulting i satisfies 0 < i <= Substring.length ss.

val read_bigsubstring : t ->
       Core.Std.Bigsubstring.t -> int Read_result.t Import.Deferred.t

val read_char : t -> char Read_result.t Import.Deferred.t

val really_read : t ->
       ?pos:int -> ?len:int -> string -> [ `Eof of int | `Ok ] Import.Deferred.t

really_read t buf ?pos ?len reads until it fills len bytes of buf starting at pos or runs out of input. In the former case it returns `Ok. In the latter, it returns `Eof n where n is the number of bytes that were read before end of input, and 0 <= n < String.length ss.

val really_read_substring : t -> Core.Std.Substring.t -> [ `Eof of int | `Ok ] Import.Deferred.t

val really_read_bigsubstring : t ->
       Core.Std.Bigsubstring.t -> [ `Eof of int | `Ok ] Import.Deferred.t

val read_until : t ->
       [ `Char of char | `Pred of char -> bool ] ->
       keep_delim:bool ->
       [ `Eof | `Eof_without_delim of string | `Ok of string ] Import.Deferred.t

read_until t pred ~keep_delim reads until it hits a delimiter c such that:

if pred = `Char c' then c = c'
if pred = `Pred p then p c

`Char c' is equivalent to `Pred (fun c -> c = c') but the underlying implementation is more efficient, in particular it will not call a function on every input character.

read_until returns a freshly-allocated string consisting of all the characters read and optionally including the delimiter as per keep_delim.

val read_until_max : t ->
       [ `Char of char | `Pred of char -> bool ] ->
       keep_delim:bool ->
       max:int ->
       [ `Eof
       | `Eof_without_delim of string
       | `Max_exceeded of string
       | `Ok of string ] Import.Deferred.t

just like read_until, except you have the option of specifiying a maximum number of chars to read.

val read_line : t -> string Read_result.t Import.Deferred.t

read_line t reads up to, and including the next newline (\n) character (or \r\n) and returns a freshly-allocated string containing everything up to but not including the newline character. If read_line encounters EOF before the newline char then everything read up to but not including EOF will be returned as a line.

val really_read_line : wait_time:Core.Std.Time.Span.t -> t -> string option Import.Deferred.t

really_read_line ~wait_time t reads up to, and including the next newline (\n) character and returns an optional, freshly-allocated string containing everything up to but not including the newline character. If really_read_line encounters EOF before the newline char, then a time span of wait_time will be used before the input operation is retried. If the descriptor is closed, None will be returned.

type 'a read = ?parse_pos:Core.Std.Sexp.Parse_pos.t -> 'a

val read_sexp : (t -> Core.Std.Sexp.t Read_result.t Import.Deferred.t)
       read

read_sexp t reads the next sexp.

val read_sexps : (t -> Core.Std.Sexp.t Import.Pipe.Reader.t) read

read_sexps t reads all the sexps and returns them as a pipe. When the reader reaches EOF or the pipe is closed, read_sexps closes the the reader, and then after the reader close is finished, closes the pipe.

val read_bin_prot : ?max_len:int ->
       t ->
       'a Bin_prot.Type_class.reader ->
       'a Read_result.t Import.Deferred.t

read_bin_prot ?max_len t bp_reader reads the next binary protocol message using binary protocol reader bp_reader. The format is the "size-prefixed binary protocol", in which the length of the data is prefixed as a 64-bit integer to the data. This is the format that Writer.write_bin_prot writes.

val read_marshal_raw : t -> string Read_result.t Import.Deferred.t

Read and return a buffer containing one marshaled value, but don't unmarshal it. You can just call Marshal.from_string on the string, and cast it to the desired type (preferrably the actual type). similar to Marshal.from_channel, but suffers from the String-length limitation (16MB) on 32bit platforms.

val read_marshal : t -> 'a Read_result.t Import.Deferred.t

Like read_marshal_raw, but unmarshal the value after reading it

val recv : t -> string Read_result.t Import.Deferred.t

recv t returns a string that was written with Writer.send

val read_all : t ->
       (t -> 'a Read_result.t Import.Deferred.t) ->
       'a Import.Pipe.Reader.t

read_all t read_one returns a pipe that receives all values read from t by repeatedly using read_one t. When the reader reaches EOF, it closes the reader, and then after the reader close is finished, closes the pipe.

val lseek : t -> int64 -> mode:[< `End | `Set ] -> int64 Import.Deferred.t

lseek t offset ~mode clears t's buffer and calls Unix.lseek on t's file descriptor. The `Cur mode is not exposed because seeking relative to the current position of the file descriptor is not the same as seeking to relative to the current position of the reader.

val lines : t -> string Import.Pipe.Reader.t

lines t reads all the lines from t and puts them in the pipe, one line per pipe element. The lines do not contain the trailing newline. When the reader reaches EOF or the pipe is closed, lines closes the the reader, and then after the reader close is finished, closes the pipe.

val contents : t -> string Import.Deferred.t

contents t returns the string corresponding to the full contents (up to EOF) of the reader. contents closes t before returning the string.

val file_contents : string -> string Import.Deferred.t

file_contents file returns the string with the full contents of the file

val file_lines : string -> string list Import.Deferred.t

file_lines file returns a list of the lines in the file. The lines do not contain the trailing newline.

val load_sexp : ?exclusive:bool ->
       string -> (Core.Std.Sexp.t -> 'a) -> 'a Core.Std.Or_error.t Import.Deferred.t

load_sexp ?exclusive file ~f loads and convert the S-expression in a given file using f, and returns the deferred conversion result as a variant of either Ok res or Error exn otherwise. This function provides accurate error locations for failed conversions.

val load_sexp_exn : ?exclusive:bool -> string -> (Core.Std.Sexp.t -> 'a) -> 'a Import.Deferred.t

val load_sexps : ?exclusive:bool ->
       string ->
       (Core.Std.Sexp.t -> 'a) -> 'a list Core.Std.Or_error.t Import.Deferred.t

load_sexps file ~f load and convert the S-expressions in a given file using f, and return the deferred list of conversion results as variants of either Ok res or Error exn otherwise. This function is as efficient as load_sexps followed by conversion if there are no errors, but provides accurate error locations for failed conversions.

val load_sexps_exn : ?exclusive:bool ->
       string -> (Core.Std.Sexp.t -> 'a) -> 'a list Import.Deferred.t

val sexp_of_t : t -> Sexplib.Sexp.t

io_stats Overall IO statistics for all readers

last_read_time t returns time of the most recent read system call that returned data.

stdin is a reader for file descriptor 0. It is lazy because we don't want to create it in all programs that happen to link with async.

open_file file opens file for reading and returns a reader reading from it.

transfer t pipe_w transfers data from t into pipe_w one chunk at a time (whatever is read from the underlying file descriptor without post-processing). The result becomes determined after reaching EOF on t and the final bytes have been transferred.

This function will normally not be needed (see pipe).

pipe t returns the reader end of a pipe that will continually be filled with chunks of data from the underlying Reader.t. When the reader reaches EOF or the pipe is closed, pipe closes the the reader, and then after the reader close is finished, closes the pipe.

create ~buf_len fd creates a new reader that is reading from fd.

with_file file f opens files, creates a reader with it, and passes the reader to f. It closes the reader when the result of f becomes determined, and returns f's result.

NOTE, you need to be careful that all your IO is done when the deferred you return becomes determined. If for example, you use with_file, and call lines, make sure you return a deferred that becomes determined when the EOF is reached on the pipe, not when you get the pipe (because you get it straight away).

close t prevents further use of t and closes t's underlying file descriptor. The result of close becomes determined once the underlying file descriptor has been closed. It is an error to call other operations on t after close t has been called, except that calls of close subsequent to the original call to close will return the same deferred as the original call.

is_closed t returns true iff close t has been called.

with_close t ~f runs f (), and closes t after f finishes or raises.

id t

fd t

read t ?pos ?len buf reads up to len bytes into buf, blocking until some data is available or end-of-input is reached. The resulting i satisfies 0 < i <= len.

val sexp_of_read_one_chunk_at_a_time_result : ('a -> Sexplib.Sexp.t) ->
       'a read_one_chunk_at_a_time_result -> Sexplib.Sexp.t

`Stop a means that handle_chunk consumed all len bytes, and that read_one_chunk_at_a_time should stop reading and return `Stopped a.

`Stop_consumed (a, n) means that handle_chunk consumed n bytes, and that read_one_chunk_at_a_time should stop reading and return `Stopped a.

`Continue means that handle_chunk has consumed all len bytes.

`Consumed (c, need) means that c bytes were consumed and need says how many bytes are needed (including the data remaining in the buffer after the c were already consumed). It is an error if c < 0 || c > len. For `Need n, it is an error if n < 0 || c + n <= len.

read_substring t ss reads up to Substring.length ss bytes into ss, blocking until some data is available or Eof is reched. The resulting i satisfies 0 < i <= Substring.length ss.

really_read t buf ?pos ?len reads until it fills len bytes of buf starting at pos or runs out of input. In the former case it returns `Ok. In the latter, it returns `Eof n where n is the number of bytes that were read before end of input, and 0 <= n < String.length ss.

read_until t pred ~keep_delim reads until it hits a delimiter c such that:

if pred = `Char c' then c = c'
if pred = `Pred p then p c

`Char c' is equivalent to `Pred (fun c -> c = c') but the underlying implementation is more efficient, in particular it will not call a function on every input character.

read_until returns a freshly-allocated string consisting of all the characters read and optionally including the delimiter as per keep_delim.

just like read_until, except you have the option of specifiying a maximum number of chars to read.

read_line t reads up to, and including the next newline (\n) character (or \r\n) and returns a freshly-allocated string containing everything up to but not including the newline character. If read_line encounters EOF before the newline char then everything read up to but not including EOF will be returned as a line.

really_read_line ~wait_time t reads up to, and including the next newline (\n) character and returns an optional, freshly-allocated string containing everything up to but not including the newline character. If really_read_line encounters EOF before the newline char, then a time span of wait_time will be used before the input operation is retried. If the descriptor is closed, None will be returned.

read_sexp t reads the next sexp.

read_sexps t reads all the sexps and returns them as a pipe. When the reader reaches EOF or the pipe is closed, read_sexps closes the the reader, and then after the reader close is finished, closes the pipe.

read_bin_prot ?max_len t bp_reader reads the next binary protocol message using binary protocol reader bp_reader. The format is the "size-prefixed binary protocol", in which the length of the data is prefixed as a 64-bit integer to the data. This is the format that Writer.write_bin_prot writes.

Read and return a buffer containing one marshaled value, but don't unmarshal it. You can just call Marshal.from_string on the string, and cast it to the desired type (preferrably the actual type). similar to Marshal.from_channel, but suffers from the String-length limitation (16MB) on 32bit platforms.

Like read_marshal_raw, but unmarshal the value after reading it

recv t returns a string that was written with Writer.send

read_all t read_one returns a pipe that receives all values read from t by repeatedly using read_one t. When the reader reaches EOF, it closes the reader, and then after the reader close is finished, closes the pipe.

lseek t offset ~mode clears t's buffer and calls Unix.lseek on t's file descriptor. The `Cur mode is not exposed because seeking relative to the current position of the file descriptor is not the same as seeking to relative to the current position of the reader.

lines t reads all the lines from t and puts them in the pipe, one line per pipe element. The lines do not contain the trailing newline. When the reader reaches EOF or the pipe is closed, lines closes the the reader, and then after the reader close is finished, closes the pipe.

contents t returns the string corresponding to the full contents (up to EOF) of the reader. contents closes t before returning the string.

file_contents file returns the string with the full contents of the file

file_lines file returns a list of the lines in the file. The lines do not contain the trailing newline.

load_sexp ?exclusive file ~f loads and convert the S-expression in a given file using f, and returns the deferred conversion result as a variant of either Ok res or Error exn otherwise. This function provides accurate error locations for failed conversions.

load_sexps file ~f load and convert the S-expressions in a given file using f, and return the deferred list of conversion results as variants of either Ok res or Error exn otherwise. This function is as efficient as load_sexps followed by conversion if there are no errors, but provides accurate error locations for failed conversions.