Module Async_extra__.File_tail

File_tail is useful for pulling data from a file that is being appended to by another process. Creating a file tail returns the reader half of a pipe whose writer half is populated by a background process that roughly does the following loop.

      loop:
        stat to find out if data is available
        read data (repeatedly [ open, seek, read, close ] until all data is read)
        wait for some time
module Error : sig ... end
module Warning : sig ... end
module Update : sig ... end
val create : ?⁠read_buf_len:int ‑> ?⁠read_delay:Core.Time.Span.t ‑> ?⁠retry_null_reads:bool ‑> ?⁠break_on_lines:bool ‑> ?⁠ignore_inode_change:bool ‑> ?⁠start_at:[ `Beginning | `End | `Pos of Core.Int64.t ] ‑> ?⁠eof_latency_tolerance:Core.Time.Span.t ‑> ?⁠null_read_tolerance:Core.Time.Span.t ‑> ?⁠throttle:unit Async_extra__.Import.Throttle.t ‑> string ‑> Update.t Async_extra__.Import.Pipe.Reader.t

create file creates a File_tail.t that will immediately begin reading file, and then will start the stat-read loop.

read_buf_len sets the size of the internal buffer used for making read system calls.

read_delay sets how long the stat-read loop waits each time after it reaches eof before stat'ing again. Setting read_delay too low could cause unecessary load.

If retry_null_reads = true, then reads that return data with null ('\000') characters are ignored and cause the system to delay 0.2s and attempt the read again. If retry_null_reads = false, then the file tail will process data with nulls just as it would any other data.

If break_on_lines = true, the file tail will break data into lines on '\n'. If not, the fill tail will return chunks of data from the end of the file as they are available.

If ignore_inode_change = true, the file tail will silently press on when the file's inode changes. If not, an inode change will cause the file tail to report an error and stop. CIFS changes inodes of mounted files few times a day and we need ignore_inode_change = true option to keep tailers watching files on it alive.

start_at determines the file position at which the file tail starts.

eof_latency_tolerance affects the Did_not_reach_eof_for warning.

null_read_tolerance determines how long the tailing must observe null reads before it will report a Delayed_due_to_null_reads_for warning.

The default throttle is a global throttle shared among all file tails that has continue_on_error = true and max_concurrent_jobs = 50.