Changelog

113.24.00

async

Keep up to date with interface changes in Async_kernel, Async_extra and Async_unix.

async_extended

Make LTL predicates comparable by tagging and id to each one. Fixes a functional comparison bug.
Switched to PPX.
Add an mli for async_extended/src/reader_ext.ml and remove a couple of unused functions (notably Reader_ext had its own version of Reader.read_char).
Add async-friendly color print
Ltl.eval should close the pipe after it is done with it.
Deleted Async_extended.Cml.
Remove Async_extended.Std.Gzip and redirect references to Async_gzip.Std.Gzip.
Update Command_rpc.Connection to check the program before exec'ing it. The filename must now be absolute, exist, and be executable. Previously errors with nonexistent or nonexecutable files would only be found out after forking.
Change Command_rpc.Command to use Versioned_rpc.Callee_converts instead of Versioned_rpc.Both_convert so that commands can be constructed without client-side conversions. Clients remain free to use conversions or not, as appropriate.

Removed val rpcs from Callee_converts interfaces because nothing appears to use it, and Both_convert does not provide it. Now Both_convert.S can be supplied to satisfy Callee_converts.S.
Add simple example of Command_rpc
Add Deferred_cache.
Fixing a couple of issues noticed in Command_rpc:
- If propagate_stderr is false, the child's stderr is now drained instead of ignored.
- When connections are closed, stderr is now closed as well, which prevents a file descriptor leak if the child process is unresponsive.

async_extra

N.B. some changes happening for this release are not listed in this changelog since they appear only as a consequence of changes in core or async_kernel.

When Transfer.Writer.send* raises, send an error to the client.
Add a new rpc that enables a "push" rather than a "poll" model.
Switched to PPX.
For connected UDP sockets, expose send in the same fashion as sendto.
Tcp.Server is documented to refuse excess connections beyond max_connections + max_pending_connections, but it treats them as pending connections in our standard OS configuration. In fact, research indicates that the documented behavior is nearly impossible to obtain directly and consistently from listen.

Clarify the name and role of the backlog argument to listen and rename and update documentation for max_pending_connections to clarify what it actually does, in light of some research:
```
`listen` does not generally respect the backlog argument as an
upper limit, but as a lower limit (mod `tcp_max_syn_backlog`) and,

with `tcp_abort_on_overflow=0`, `listen` will ignore excess
connections rather than actively refusing them.

(With `syncookies=1`, this can look like an indefinite backlog.)
```
Existing, working code can substitute max_pending_connections -> backlog and move on. The behavior is not changed.

When possible, consider architecting applications so the server can simply accept and close excess connections, rather than relying on the listen backlog to return an active indication to the client that they won't be serviced. To make sure the client receives an RST rather than an orderly shutdown, you can set the linger time to 0 before closing the socket. (Added to unit tests.)

Direct Tcp.Server support for this paradigm is left for future work.
Make Rpc_low_latency_transport treat disconnections as eof, like Async_unix.Reader does.
Add an implementation of Mvars to Async
Allow custom handling of missed async_rpc heartbeats.
adds a configuration limit on the number of tokens that can be in-flight
Replace an #include <sys/errno.h> by #include <errno.h>.

Fixes janestreet/async_extra#4
Added Tcp.Server.sexp_of_t
Adds Rpc.Pipe_rpc.dispatch_iter, plus a bunch of additional types to support it. The main reason for this is to reduce space usage: Pipe_rpc.dispatch followed by Pipe.iter_without_pushback consumes ~105 words in the steady state (i.e., when no messages are coming in) while dispatch_iter consumes ~15. I'm sure dispatch can be improved a lot, but a pipe by itself is 46 words, so it can't possibly become as small as dispatch_iter.

Both cases can be made smaller by making Connection.response_handler a GADT instead of a closure. I plan to do this later.

One annoying property of the interface is that the only way to cancel a subscription is to use Pipe_rpc.abort, which has a terrible interface. The logical way to improve the interface is to return a record of a Pipe_rpc.t, a Connection.t, and a Query_id.t, which allocates an additional few words. I'd kind of like to do this but it seems counter to the goal of reducing space usage.
Added Tcp.Server.listening_on_address, so that one can get the address a server is listening on, as compared with listening_on, which just returns the port.
Marked Command.async_basic as deprecated using the appropriate ocaml attribute.

@@ocaml.deprecated

(http://caml.inria.fr/pub/docs/manual-ocaml/extn.html#sec241)
Extend the interface of Persistent_rpc_client to make the "address" type - previously fixed as Host_and_port.t - abstract. This is helpful for integrating with libraries that have a different notion of an address, e.g. rpc_discovery_lib.
Typed_tcp mutated a Hashtbl while iterating over it when closing.
Added Async.Bus.first_exn, which takes a bus and a function, and returns a deferred that becomes determined when the first event is published to the bus for which the function returns Some.

This function is useful to reduce boilerplate for dealing with unsubscription.
Reduced the number of threads required by tests in:

async_extra/src/tcp.ml
Added to the error message Bus.subscribe_exn called after first write the source-code position of the caller, in case there isn't a backtrace, to make the source of the problem clearer, and to avoid confusion with other source-code positions of subscribers already in the bus.
Added to Bus.first_exn a Source_code_position.t argument, so that in the event of subscription failure, we can see who caused the subscription to the bus.
Added to Tcp.Server.close an optional argument:

?close_existing_connections : bool

This closes the sockets of all existing connections.
Annotate errors returned by the async-rpc library with the name of the RPC for which the error was returned (if it's an rpc-level error) and a description of the remote side of the connection (the ip:host if connected via a network socket).
Improved Async.Udp.bind's error message when it fails to mcast_join a multicast group.
Change ~callback to ~f throughout the Bus interface

async_find

Initial release.

async_kernel

N.B. some interface change in Core (notably to Hashtbl and Map) implied some interface change in this package as well, although they are not mentionned in this changelog.

Switched to ppx.
Improved the Async scheduler's to allocate a handle_fired function once, rather than every time it calls advance_clock.
Removed configurability of Monitor's try_with-ignored-exception handling, i.e. removed Monitor.try_with_rest_handling and Monitor.try_with_ignored_exn_handling.

The behavior of exceptions raised to a monitor after it returns is unchanged -- i.e. they are logged, as they have been since 112.28.

Changed Monitor.try_with's ?rest argument from:

?rest : Ignore | Raise ?rest : Log | Raise

This naming reflects the fact that subsequent exceptions are logged, not ignored.
In Async_kernel, moved Scheduler.set_execution_context from the Types.Scheduler module to its own file. Because Types is a module rec, set_execution_context hadn't been inlined and was called via caml_apply2. In its own file, it will be inlined.

This release creates a new scheduler0.ml, and moves the old scheduler0.ml to scheduler1.ml.
Fixed a space leak in Pipe due to a pipe holding pointers to its upstream_flusheds after they are closed. The leak shows up in Pipe.transfer and related functions, e.g. with:

Pipe.transfer temporary_pipe long_lived_pipe

called repeatedly, in which long_lived_pipe would accumulate a large number of upstream_flusheds.

The fix is to maintain upstream_flusheds as a Bag.t, and to remove an upstream pipe when it is closed.
Implement Pipe.of_sequence
Improved the error message when an exception is raised to a Monitor.try_with that has returned before Async has initialized Monitor.try_with_log_exn.
Improved the implementation of Monitor.get_next_error, by replacing the monitor's list of handlers:

; mutable handlers_for_next_error : (exn -> unit) list

with a single ivar:

; mutable next_error : exn Ivar.t

I think this wasn't done originally because of a dependency cycle. But now that we have types.ml, we can do the clear thing.
Improved the implementation of Monitor exception handling, i.e. detach_and_iter_errorsto make it clear that Monitor.send_exn does not run user code -- it only schedules jobs.
Fix an error message in Pipe to match the condition that led to it.
Add a new pipe constructor:

val unfold : 'b -> f:('b -> ('a * 'b) option Deferred.t) -> 'a Reader.t

unfold is more powerful than the combination of

Useful for, e.g., creating a pipe of natural numbers:

Pipe.unfold 0 ~f:(fun n -> return (Some (n, n+1)))
Add Deferred.Map.all similar to Deferred.List.all.

This does what you would expect:

val all : ('a, 'b Deferred.t, 'cmp) Map.t -> ('a, 'b, 'cmp) Map.t Deferred.t
Added some simple functions that seem missing from Deferred and Ivar.

val Ivar.peek : 'a t -> 'a option val Ivar.value_exn : 'a t -> 'a val Deferred.value_exn : 'a t -> 'a
Add Monitor.catch_error, which provides error handling for processes/subsystems intended to run forever.
Added to the Async scheduler a configurable:

min_inter_cycle_timeout : Time_ns.Span.t

When scheduler calls epoll(), it uses a timeout of at least min_inter_cycle_timeout.

min_inter_cycle_timeout can be configured via ASYNC_CONFIG or via

val Scheduler.set_min_inter_cycle_timeout : Time_ns.Span.t -> unit

This allows one to tweak the scheduler to be more fair w.r.t. threads, e.g. with:

Scheduler.set_min_inter_cycle_timeout <- Time_ns.Span.of_us 1.;
Optimized Scheduler.schedule' to avoid a closure allocation.
Removed Monitor.kill, which was unused internally. This removes the kill_index field from Execution_context.t, which saves us a word everytime we allocate or store an execution context.
Assert that Deferred.forever never returns statically, rather than dynamically.
Changed Async.Std to not include Deferred.Monad_syntax, so that one must explicitly opt in (via open Deferred.Monad_syntax) to use let%bind syntax with Async.
Add Pipe.to_sequence
Make Stream.closed s return immediately when s is already closed.

Currently the following property holds:

for any s, Deferred.peek (Stream.closed s) = None
For Pipe functions that deal with batches of elements in a queue, added an optional argument:

?max_queue_length : int (** default is Int.max_value *)

This limits the size of the queue that is used, which can improve Async fairness.

Affected functions are:

filter_map filter_map' fold' iter' map' read' read_now' transfer' transfer_id

This also obviates read_at_most and read_now_at_most, which we will deprecate in a later release.

Removed a couple helper types, iter and fold, that had been used to express commonality among functions, but were becoming unwieldy due to differences.
Changed Pipe's default max_queue_length from Int.max_value to 100.
Added to Pipe.iter_without_pushback an optional argument:

?max_iterations_per_job : int (** default is Int.max_value *)

iter_without_pushback will not make more than max_iterations_per_job calls to f in a single Async_job; this can be used to increase Async-scheduling fairness.
Added Pipe.write_if_open which does exactly what it says. This is a common pattern. Also added a pushback-oblivious variant write_without_pushback_if_open.

Call sites for these two new functions were introduced wherever I found that doing so would not introduce any side effects (even counting allocation) in the case of a closed pipe.

async_parallel

Switched to ppx.

async_rpc_kernel

When Transfer.Writer.send* raises, send an error to the client.
Added Pipe_rpc in Versioned_rpc.Both_convert.
Switched to ppx.
Expose the lower-level registration hook in Versioned_rpc.
Allow custom handling of missed async_rpc heartbeats.
Client-side State_rpc dispatch function does not behave well when the reader side of the pipe is closed.

It should gracefully abort the rpc instead of raising exceptions, or whatever it currently does.
Add Rpc.Expert.implement and Rpc.Expert.implement' with a similar interface as One_way.Expert.implement but for 2-way rpcs.

Exceptions raised by an expert implementations are handled as follow:
- if the query has already been answered, stop the server (as for one-way expert)
- if not, send a Rpc_error.Uncaught_exn (as for 2-way rpc)
Adds Rpc.Pipe_rpc.dispatch_iter, plus a bunch of additional types to support it. The main reason for this is to reduce space usage: Pipe_rpc.dispatch followed by Pipe.iter_without_pushback consumes ~105 words in the steady state (i.e., when no messages are coming in) while dispatch_iter consumes ~15. I'm sure dispatch can be improved a lot, but a pipe by itself is 46 words, so it can't possibly become as small as dispatch_iter.

Both cases can be made smaller by making Connection.response_handler a GADT instead of a closure. I plan to do this later.

One annoying property of the interface is that the only way to cancel a subscription is to use Pipe_rpc.abort, which has a terrible interface. The logical way to improve the interface is to return a record of a Pipe_rpc.t, a Connection.t, and a Query_id.t, which allocates an additional few words. I'd kind of like to do this but it seems counter to the goal of reducing space usage.
Adds Rpc.Pipe_rpc.implement_direct, which uses a "direct stream writer" to write results to the other side, rather than using a Pipe.Writer.t. This consumes much less memory, ~15 words per open query as opposed to ~225 for a regular Pipe_rpc implementation.

A large part of the diff in this feature is the addition of a module Implementation_types, which just contains copies of type definitions from Implementation and Implementations. This is necessary to handle some cyclic type definitions (both of these modules now depend on types defined in the other module).

This is the implementation-side dual of Pipe_rpc.dispatch_iter.
Change Command_rpc.Command to use Versioned_rpc.Callee_converts instead of Versioned_rpc.Both_convert so that commands can be constructed without client-side conversions. Clients remain free to use conversions or not, as appropriate.

Removed val rpcs from Callee_converts interfaces because nothing appears to use it, and Both_convert does not provide it. Now Both_convert.S can be supplied to satisfy Callee_converts.S.
Annotate errors returned by the async-rpc library with the name of the RPC for which the error was returned (if it's an rpc-level error) and a description of the remote side of the connection (the ip:host if connected via a network socket).
Bring back val rpcs in versioned_rpc modules.

async_shell

Initial release.

async_smtp

Switched to PPX.
Follow Core & Async evolution.

async_ssl

Switched to ppx.

async_unix

In the periodic check for a Writer buffer have too old data, eliminated allocation and generally improved performance.

This eliminated a large source of allocation in a simple TCP pingpong benchmark bench/pingpong.
Removed allocation in the Async scheduler's code that detects the thread-pool being stuck. This involved switching it to use Time_ns rather than Time.

This eliminates a relatively large source of allocation in a simple TCP-pingpong benchmark bench/pingpong.
Switched to ppx.
Improved the Async scheduler's to allocate a handle_fired function once, rather than every time it calls advance_clock.
Added Fd_by_descr.find_exn, to avoid the option allocated by Fd_by_descr.find. Used it to reduce allocation in the Async scheduler.
Improved Reader.load_sexp* functions to behave better when loading from a non files, e.g. a pipe. Previously, it produced an empty error message because it mistakenly attempted to read the sexp a second time in order to determine the error position. Now it produces a good error message, but without the (impossible to obtain) error position.
In Async_unix.Syscall, added a zero-allocation syscall interface, removing sources of allocation as observed when running a simple TCP pingpong benchmark (found in bench/pingpong).
Added

val time_spent_waiting_for_io : unit -> Time_ns.Span.t

to Scheduler which returns the amount of time that the Async scheduler has spent in calls to epoll_wait (or select) since the start of the program.
Changed In_thread.Helper_thread.create from:

val create : ?priority:Priority.t -> ?name:string -> unit -> t Or_error.t

to:

val create : ?priority:Priority.t -> ?name:string -> unit -> t Deferred.t

Kept around the prior function, renamed as create_now.

Switching create to return a deferred allows it to, when there are no available threads, wait until one becomes available. This, in turn, avoids rare nondeterminstic failures in programs that make heavy use of the thread pool and create a helper thread, when the creation happens at just the wrong time, when no thread is available.

Split out Thread_safe_ivar from the internals of Thread_pool, so that it can be used in other tests, and in particular in a new unified test added by this feature.
Make Unix_syscalls.Stats bin-io-able.
Fixed a bug in Thread_safe.block_on_async*, in which the execution context wasn't properly restored before returning.
Add a function in Process that expects empty output, mirroring Shell.run.
Added Reader.read_one_iobuf_at_a_time, which is like read_one_chunk_at_a_time, except that the user-supplied handle_chunk function receives its data in an Iobuf.t, and uses the Iobuf position to communicate how much data was consumed.

This facilitates using reader in scenarios (such as with protogen) where Iobufs are expected (and presently allocated around the bigstring at each call) and the calculation of consumed bytes from the Iobuf is duplicated in few places.
Log.message used to always logs the message, even if its log level was too low. This has been fixed.
Add writer functions to schedule an iobuf to be written out.
Add Unix.Inet_addr.Stable.
Alter Async.Std.Socket.Address.Inet.Blocking_sexp to expose the polymorphic variant functions, so that you can include it in a separate polymorphic variant type.

Also, expose Async.Std.Socket.Address.Inet.__t_of_sexp__ to give a deprecation message, instead of a message about the function not existing.
Fixed a bug in Async's In_thread.Helper_thread, which wasn't finalizing helper threads, due to a bug in Thread_pool, which wasn't finalizing helper threads. The fix was to move the finalization out of Thread_pool, where we don't care about it, to In_thread.Helper_thread, where we do.

Added Scheduler.max_num_threads : unit -> int.
Make Epoll_file_descr_watcher trigger callbacks for error conditions such as closed pipes.

Testing

Three new unit tests, all validating appropriate behavior in the case that a Unix pipe is opened, then the read end is closed after reading only part of the data sent by the write end.
1. A test in writer.ml verifying that Writer.consumer_left is triggered. Before the fix to Epoll_file_descr_watcher, Writer.consumer_left would never become determined in this case.
2. A test in fd.ml verifying that Fd.ready_to is triggered for the writing fd. Before the fix to Epoll_file_descr_watcher, Fd.ready_to would never become determined in this case.
3. A test in linux_ext.ml verifying that Epoll.wait_timeout_after is triggered. This test shows that epoll reports the ERR flag for the file descriptor in this case, and therefore that Epoll_file_descr_watcher needs to pay attention to the ERR flag.
Added to Writer.write_sexp an optional ?terminate_with argument, that specifies how to terminate the string representation of the sexp. This also makes it clear that the default behavior, ~terminate_with:Space_if_needed, might append a space to the sexp you are outputting if its representation is not enclosed in either () or "" . Sexp.output_hum an Sexp.output_mach do not have this behavior, so porting non-async code to async could introduce unexpected differences in the output.
Add an Async wrapper for Core.Std.Unix.getifaddrs.

bignum

Switched to PPX.
The library used polymorphic compare, rather than Zarith.Q's compare, in a few locations. Fixed this.
Previously stable types in Bignum were defined with unstable types in the scope. Fixd this.
Update to zarith-1.4
Bignum.of_string needs to handle different formats for its input. The previous version of the code was trying to parse the common format (floats), and in case of failure, was attempting to use a different format (based on the error). This resulted in the string being parsed twice in some cases.

This version is a complete rewriting of of_string to do the parsing in one step. The new code for to_string encode an automaton and remembers the positions of the various elements of the string (depending on the format).

This feature uses a function which has been upstreamed in the new version of ZArith (1.4) which is a variant of the Zarith.of_string function to work with substrings. This variant alone is responsible for a big part of the performance improvement.

Summary of benchmarks

The new version of the code performs better than the original one in all cases. The performance improvement are variable depending on the micro benchmark. See below.

Follow ups

We also tried to implement the lexing engine using OCamllex. This makes for a much more concise description, but the performance are significantly lower. OCamllex produces code which allocates some table and some state, which is avoided in the hand written code. Also, it will allocate the sub strings matched.

Benchmark results

New version (patch for ZArith + of_substring, reimplementation of of_string)

┌─────────────────────────────────────────────────────────────────────┬──────────────┬───────────┬──────────┬──────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├─────────────────────────────────────────────────────────────────────┼──────────────┼───────────┼──────────┼──────────┼────────────┤ │ bigint\_bench.ml random │ 48_381.13ns │ 7_166.00w │ 1.24w │ 1.24w │ 45.12% │ │ bigint\_bench.ml:vs. Big\_int plus_self │ 293.96ns │ 72.00w │ │ │ 0.27% │ │ bigint\_bench.ml:vs. Big\_int plus_other │ 807.62ns │ 124.00w │ │ │ 0.75% │ │ bigint\_bench.ml:vs. Big\_int mult_self │ 353.98ns │ 91.00w │ │ │ 0.33% │ │ bigint\_bench.ml:vs. Big\_int mult_other │ 783.78ns │ 128.00w │ │ │ 0.73% │ │ bignum\_bench.ml:Bignum of\_string/to\_string of_string (decimal) │ 14_415.44ns │ 475.00w │ │ │ 13.44% │ │ bignum\_bench.ml:Bignum of\_string/to\_string of_string (scientific) │ 61_363.80ns │ 3_929.00w │ │ │ 57.23% │ │ bignum\_bench.ml:Bignum of\_string/to\_string of_string (fraction) │ 24_957.02ns │ 303.00w │ │ │ 23.28% │ │ bignum\_bench.ml:Bignum of\_string/to\_string to_string (decimal) │ 15_867.52ns │ 1_523.00w │ │ │ 14.80% │ │ bignum\_bench.ml:Bignum of\_string/to\_string to_string (scientific) │ 33_345.31ns │ 4_206.00w │ │ │ 31.10% │ │ bignum\_bench.ml:Bignum of\_string/to\_string to_string (fraction) │ 31_770.26ns │ 3_779.00w │ │ │ 29.63% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp of_sexp (decimal) │ 9_726.82ns │ 380.00w │ │ │ 9.07% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp of_sexp (scientific) │ 28_141.40ns │ 2_059.00w │ │ │ 26.25% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp of_sexp (fraction) │ 70_436.16ns │ 5_541.00w │ │ │ 65.69% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp to_sexp (decimal) │ 27_000.73ns │ 1_994.00w │ │ │ 25.18% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp to_sexp (scientific) │ 66_057.63ns │ 6_217.00w │ │ │ 61.61% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp to_sexp (fraction) │ 107_219.89ns │ 8_097.00w │ │ │ 100.00% │ │ bignum\_bench.ml:Bignum binprot roundtrip compact │ 5_997.81ns │ 581.00w │ │ │ 5.59% │ │ bignum\_bench.ml:Bignum binprot roundtrip classic │ 18_522.20ns │ 779.00w │ │ │ 17.27% │ │ bignum\_bench.ml:round round_decimal:0 │ 8_479.49ns │ 463.00w │ │ │ 7.91% │ │ bignum\_bench.ml:round round_decimal:3 │ 24_621.71ns │ 2_115.00w │ │ │ 22.96% │ │ bignum\_bench.ml:round round_decimal:6 │ 26_896.35ns │ 2_437.00w │ │ │ 25.09% │ │ bignum\_bench.ml:round round_decimal:9 │ 29_428.19ns │ 2_730.00w │ │ │ 27.45% │ │ bignum\_bench.ml:round round │ 8_452.31ns │ 459.00w │ │ │ 7.88% │ └─────────────────────────────────────────────────────────────────────┴──────────────┴───────────┴──────────┴──────────┴────────────┘

Original version

┌─────────────────────────────────────────────────────────────────────┬──────────────┬───────────┬──────────┬──────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├─────────────────────────────────────────────────────────────────────┼──────────────┼───────────┼──────────┼──────────┼────────────┤ │ bigint\_bench.ml random │ 51_218.04ns │ 7_166.00w │ 1.25w │ 1.25w │ 43.26% │ │ bigint\_bench.ml:vs. Big\_int plus_self │ 336.84ns │ 72.00w │ │ │ 0.28% │ │ bigint\_bench.ml:vs. Big\_int plus_other │ 837.73ns │ 124.00w │ │ │ 0.71% │ │ bigint\_bench.ml:vs. Big\_int mult_self │ 411.03ns │ 91.00w │ │ │ 0.35% │ │ bigint\_bench.ml:vs. Big\_int mult_other │ 808.03ns │ 128.00w │ │ │ 0.68% │ │ bignum\_bench.ml:Bignum of\_string/to\_string of_string (decimal) │ 29_650.60ns │ 2_415.00w │ │ │ 25.04% │ │ bignum\_bench.ml:Bignum of\_string/to\_string of_string (scientific) │ 92_495.93ns │ 6_465.00w │ │ │ 78.12% │ │ bignum\_bench.ml:Bignum of\_string/to\_string of_string (fraction) │ 39_482.77ns │ 2_060.00w │ │ │ 33.35% │ │ bignum\_bench.ml:Bignum of\_string/to\_string to_string (decimal) │ 16_195.93ns │ 1_523.00w │ │ │ 13.68% │ │ bignum\_bench.ml:Bignum of\_string/to\_string to_string (scientific) │ 34_227.78ns │ 4_059.00w │ │ │ 28.91% │ │ bignum\_bench.ml:Bignum of\_string/to\_string to_string (fraction) │ 32_856.17ns │ 3_779.00w │ │ │ 27.75% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp of_sexp (decimal) │ 19_745.71ns │ 2_149.00w │ │ │ 16.68% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp of_sexp (scientific) │ 51_024.99ns │ 3_853.00w │ │ │ 43.09% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp of_sexp (fraction) │ 88_884.15ns │ 7_819.00w │ │ │ 75.07% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp to_sexp (decimal) │ 32_812.27ns │ 2_498.00w │ │ │ 27.71% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp to_sexp (scientific) │ 77_518.77ns │ 6_369.00w │ │ │ 65.47% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp to_sexp (fraction) │ 118_402.78ns │ 8_907.00w │ │ │ 100.00% │ │ bignum\_bench.ml:Bignum binprot roundtrip compact │ 8_947.02ns │ 371.00w │ │ │ 7.56% │ │ bignum\_bench.ml:Bignum binprot roundtrip classic │ 22_799.74ns │ 1_039.00w │ │ │ 19.26% │ │ bignum\_bench.ml:round round_decimal:0 │ 8_176.74ns │ 463.00w │ │ │ 6.91% │ │ bignum\_bench.ml:round round_decimal:3 │ 25_798.77ns │ 2_115.00w │ │ │ 21.79% │ │ bignum\_bench.ml:round round_decimal:6 │ 28_561.23ns │ 2_437.00w │ │ │ 24.12% │ │ bignum\_bench.ml:round round_decimal:9 │ 30_861.38ns │ 2_730.00w │ │ │ 26.06% │ │ bignum\_bench.ml:round round │ 8_237.26ns │ 459.00w │ │ │ 6.96% │ └─────────────────────────────────────────────────────────────────────┴──────────────┴───────────┴──────────┴──────────┴────────────┘

Tentative version using OCamllex

┌─────────────────────────────────────────────────────────────────────┬──────────────┬────────────┬──────────┬──────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├─────────────────────────────────────────────────────────────────────┼──────────────┼────────────┼──────────┼──────────┼────────────┤ │ bigint\_bench.ml random │ 48_164.21ns │ 7_166.00w │ 1.25w │ 1.25w │ 39.99% │ │ bigint\_bench.ml:vs. Big\_int plus_self │ 285.84ns │ 72.00w │ │ │ 0.24% │ │ bigint\_bench.ml:vs. Big\_int plus_other │ 768.12ns │ 124.00w │ │ │ 0.64% │ │ bigint\_bench.ml:vs. Big\_int mult_self │ 343.14ns │ 91.00w │ │ │ 0.28% │ │ bigint\_bench.ml:vs. Big\_int mult_other │ 780.00ns │ 128.00w │ │ │ 0.65% │ │ bignum\_bench.ml:Bignum of\_string/to\_string of_string (decimal) │ 26_931.12ns │ 3_108.00w │ │ │ 22.36% │ │ bignum\_bench.ml:Bignum of\_string/to\_string of_string (scientific) │ 79_750.28ns │ 6_599.00w │ 0.11w │ 0.11w │ 66.21% │ │ bignum\_bench.ml:Bignum of\_string/to\_string of_string (fraction) │ 34_988.94ns │ 4_300.00w │ │ │ 29.05% │ │ bignum\_bench.ml:Bignum of\_string/to\_string to_string (decimal) │ 15_958.17ns │ 1_523.00w │ │ │ 13.25% │ │ bignum\_bench.ml:Bignum of\_string/to\_string to_string (scientific) │ 32_495.25ns │ 4_059.00w │ │ │ 26.98% │ │ bignum\_bench.ml:Bignum of\_string/to\_string to_string (fraction) │ 31_802.75ns │ 3_779.00w │ │ │ 26.40% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp of_sexp (decimal) │ 18_742.81ns │ 2_924.00w │ │ │ 15.56% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp of_sexp (scientific) │ 45_282.09ns │ 4_622.00w │ │ │ 37.60% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp of_sexp (fraction) │ 86_907.83ns │ 8_777.00w │ 0.15w │ 0.15w │ 72.16% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp to_sexp (decimal) │ 35_727.73ns │ 4_493.00w │ │ │ 29.66% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp to_sexp (scientific) │ 82_247.61ns │ 8_273.00w │ 0.13w │ 0.13w │ 68.29% │ │ bignum\_bench.ml:Bignum of\_sexp/to\_sexp to_sexp (fraction) │ 120_445.25ns │ 10_688.00w │ 0.12w │ 0.12w │ 100.00% │ │ bignum\_bench.ml:Bignum binprot roundtrip compact │ 6_734.49ns │ 371.00w │ │ │ 5.59% │ │ bignum\_bench.ml:Bignum binprot roundtrip classic │ 21_773.79ns │ 1_890.00w │ │ │ 18.08% │ │ bignum\_bench.ml:round round_decimal:0 │ 8_306.45ns │ 463.00w │ │ │ 6.90% │ │ bignum\_bench.ml:round round_decimal:3 │ 24_714.96ns │ 2_115.00w │ │ │ 20.52% │ │ bignum\_bench.ml:round round_decimal:6 │ 26_894.27ns │ 2_437.00w │ │ │ 22.33% │ │ bignum\_bench.ml:round round_decimal:9 │ 29_343.81ns │ 2_730.00w │ │ │ 24.36% │ │ bignum\_bench.ml:round round │ 8_296.05ns │ 459.00w │ │ │ 6.89% │ └─────────────────────────────────────────────────────────────────────┴──────────────┴────────────┴──────────┴──────────┴────────────┘

bin_prot

Bin_prot can be configured to use the primitives to read/write integers from bigarrays. This was never enabled due to missing tests that selecting this code path doesn't change the format.

This version add these tests and enable the use of the fast primitives.
Add benchmarks for all exposed bin_prot read/write functions. These are intended to check performance regressions.
Remove most use of cpp in bin_prot.

Replace the pre-processor conditionals by runtime one. This make the code a little less obfuscated.
Remove big literals so that the compiler does not complain in 32bit

core

N.B. Some interface change were made which are not listed here, as they are only cascading from core_kernel. Look at core_kernel's CHANGES.md file to get a complete history of the changes made for this release.

Add Core.Command.Arg_type.Export.time_zone
Fix Command.shape to run external programs only once to get their sexp.

Introduces a new variant of Command.t called Proxy. The Exec variant represents the top-level command of an external executable that has not yet been run; the Proxy variant represents an arbitrary subcommand that has been extracted by running an external executable. Command.exec constructs an Exec variant; Command.shape of an Exec variant runs the executable and generates a tree of Proxy variants representing all the information from the generated sexp, so the executable will not need to be re-run.
A version of recvmmsg that pre-allocates and reuses the iovec record. Profiling indicates this is a non-trivial amount of our I/O loop (under very heavy load).

Since nobody is using the heavyweight features of the existing recvmmsg, replace it with the lightweight one. This leads to minor but important changes to the interfaces of Iobuf.recvmmsg_assume_fd_nonblocking and Udp.recvmmsg_loop.
Switch to ppx.
Time.set_sexp_zone affects t_of_sexp

If we're willing to read sexps without zones, we should be willing to let you control what timezone they're read with.
Sped up Llimiter.
Make Interval.Int implement Container and Binary_searchable.
Add Identifiable.S to Unix.Cidr so that it supports hash tables. Update Unix.Cidr.t to normalize values, e.g. "192.168.1.101/24" ==> "192.168.1.0/24".
Added "noalloc" attribute to Linux_ext.unsafe_timerfd_settime.
In Iobuf, made some functions take:

(> write, _) Iobuf.t

rather than:

(read_write, _) Iobuf.t

if they only need to write to the iobuf and don't need to read it.
Time.of_string_abs didn't support ISO 8601 time zone strings without colons, or those specified as locations. This version adds support for time zones without colons
Unix.Passwd.getpwents takes the lock, partially applies Exn.protect, then releases the lock, then completes the application and actually runs stuff.
Command.file-completion

Files are now completed correctly when paths contain a directory.

Previously, completion when pointed at a directory would put a space at the end. This would cause the user to hit backspace every time a directory was in the path.
Add diff_weekdays and diff_weekend_days functions to date module.
Time.to_date_ofday_precise implements a complete inverse for of_date_ofday. This is needed to give the DWIM-est semantics to Schedule.t that we can think of.
Reduce allocation in Linux_ext.Epoll
Add Time_ns.Of_day.(add_exn, sub_exn, diff).
Adding head_padded_fixed_string to Iobuf.
Move Core_extended.Std.Sys.home to Core.Std.Sys.home_directory.
Add Iobuf.{read,write,input,output} akin to the bigstring versions.
Add expert iobuf functions for extracting bigstrings and iovecs that share the iobuf's underlying storage.
Add stable Int63.t conversions for types in Time_ns.
Rename DNS-based Inet_addr sexp conversions to expose their blocking nature.
Add Unix.Inet_addr.Stable.
Add [Core.Std.Schedule].
Fixed Iobuf_packet.iter, which behaved incorrectly if the packet didn't start at the lo_min of the iobuf. It used Iobuf.rewind when it should have used Iobuf.Lo_bound.restore.

Added @@deriving compare to Iobuf.Bound.
Add ability to Time.format for a specific time zone
Make more information available via Command.Shape.t

Expose machine-readable info on anonymous arguments.
Remove unnecessary rebinding of (^/) in core_filename.ml. It had one call site and wasn't exposed. The (^/) everyone uses comes from std.ml
Add getifaddrs to Core.Std.Unix.

Handles Packet (for interfaces that do not have an address on Linux systems only), IPv4 and IPv6 address families.
Implement Time_ns.Span.to_string_hum by analogy to Time.Span.to_string_hum. The code and tests are essentially copied from "lib/core/src/span.ml".
Remove our stubs for Unix.stat.

They were upstreamed in 4.02.2.

core_bench

Switched to ppx.

core_extended

N.B. Some interface changes occured in Core which are repercuted in this package, they are not all list in this file though.

Switched to PPX.
Upgrade Interval_map.t with monad operations.
Update the interval_map_intf.ml file to try to make the documentation clearer.

This mostly constitutes splitting out the core operations into a separate module type earlier in the file, so that their documentation occurs before the various more specific module types in reading order.

Various bits of documentation have been tweaked with examples or laws.
Add underscores to color print names Improve and uniformize the behavior of colorprintf functions at the cost of changing the type slightly
Fix core_extended stubs on openbsd

Closes #7 Closes #2
Move Core_extended.Std.Sys.home to Core.Std.Sys.home_directory.
Add a module whose type 'a t acts as a container of ordered items of type 'a (morally, a 'a list) but which supports efficient append operations.

Sometimes called a Rope, or Concatenable_list.
Expose the constructors of Ascii_table.Align.t so that we can write

Column.create ~align:Left ...

instead of

Column.create ~align:Align.left
Fix sexp diffing on records The wrong comparaison was leading to huge diff as soon as one field was missing on one side

core_kernel

Add Container.Make0 for monomorphic container types.
Improved the performance of the implementation of Bounded_int_table.find.
Switched to ppx
Remove references to Core_list from Sequence.
Added functions to Bigstring and Iobuf for reading unsigned 64-bit integers.
Move Comparable.bound to Maybe_bound.t. The purpose is to break up dependencies between the two.
Doubly_linked allocated during iteration. This became a large source of allocation for simple benchmarks like TCP pingpong (async/bench/pingpong). Some unnecessary allocations have been removed.
Added Timing_wheel.next_alarm_fires_at_exn, which is useful to avoid allocation when you know the timing wheel isn't empty.
Make versions of Binary_searchable.Make* that don't require a For_test argument. This allows Binary_searchable.Make to be used for types that don't easily convert from arrays.
Add Quickcheckable interface to Core and move generators/observers into type modules.

Renames core_list.ml to core_list0.ml, then adds a new core_list.ml with quickcheck generators and observers. This allows quickcheck.ml to use core_list0.ml without a dependency cycle.

The feature also moves the contents of quickcheck.mli into quickcheck_intf.ml.
Made Core.Unpack_buffer.Unpack_one.t be a unary type rather than a binary one, by hiding its partial_unpack type under an existential.

This makes it possible to make Unpack_one into a monad because we can combine two Unpack_one.t's with different partial_unpack types into a new Unpack_one.t with a different partial_unpack type.
https://github.com/janestreet/core\_kernel/pull/20 Core.Std module is not found when compiling lib_test/pool_caml_modify_check.ml.
Added an optional argument ?key_order for specifying the order of Map.to_alist output: either Increasing orDecreasing.

The default key order is no longer left unspecified: we're now committed to the `Increasing, which was the old behavior.
Add Sexpable.Of_sexpable2 functor, for symmetry with Binable.Of_binable2. Add sexpable.mli
Added a function for sequencing computations stored in a total map: Total_map.sequence.
Added Core.Bus, a publisher/subscriber system within the memory space of the program. This is a synchronous version of Async.Bus.
Added Core_map.fold2 (fold based on the contents of two maps side-by-side).
Core.Interfaces defines the Unit module type to be sig end. Increase uniformity with its other definitions by defining it to be Unit.S instead.
Adapt Core_random.int to accept larger values than 1 lsl 30.
Mark the Sexpable.Of_* and Binable.Of_* functors as stable.
In Core_char.int_is_ok, used by of_int and of_int_exn, use int compare instead of polymorphic compare.
Fix a few files where toplevel side effects might not be running when we don't pack libraries anymore and use -no-alias-deps.
In Char.For_quickcheck, memoize construction of the filtered chars generators, since if they are used once, they are likely to be used many times, and the construction is costly compared to generating a single char.
Extend Core_map to implement quickcheckable Extend Core_set to implement quickcheckable
In Avltree.add, replace ?(replace = true) with ~replace. This both makes the behavior more explicit, and saves some allocation occasionally.
Reimplement Avltree.iter directly, rather than using fold, which requires allocating a closure. This winds up being expensive during Hashtbl.iter.
Add a function in Blang to deal with boolean expressions E representing the membership of elements in a set, given a universe U and a function projecting each atoms of E to a subset of U.

Example:

{Blang.eval_set ~universe:js_tech resolve_named_set ("(or (and has-blue-eyes has-brown-hair) (and has-brown-eyes has-blue-hair))" |> Sexp.of_string |> t_of_sexp)}
Expose more functions in univ_map interface
Made Random.self_init by default raise if used in inline tests. One can opt out by passing ~allow_in_tests:true.
In core_hashtbl.ml, maybe_resize_table allocates the same closure in each iteration of a for loop. Allocate it just once.
Hashtbl.remove_one and Hashtbl.remove_multi are the same function, written twice. Remove remove_one and replace uses with remove_multi.
Bigstring.unsafe_{get,set}-{,u}int8 used the generic bigarray access function without a type annotation. As a result the compiler generated a call to the generic C function.

Fixed this by adding type annotations.
Add new functions to map that add common missing functionality and/or that makes the interface more uniform and consistent with other container modules.
Made Unpack_buffer.Unpack_one monadic so that users can easily compose file parsers

Added a couple simple parsers as examples and for testing.
Avoid use of polymorphic compare in Quickcheck. Make Quickcheck.Generator.bind_choice lazy: do not eagerly descend into all branches.

Reduces memory overhead by setting a threshold on the probability of choices that are remembered and discarded by Quickcheck.iter and friends.

Motivation: Currently, Quickcheck.iter and related functions guarantee never to repeat a choice from a generator. This winds up recording every choice ever made, which for a lot of generators is a prohibitive cost in space, and most of the recorded values are very unlikely to be repeated anyway.

Implementation: This feature sets a probability threshold below which choices will not be remembered. Choosing a fairly low, but still non-zero, threshold means values are still very unlikely to be repeated, but memory usage stays low.

As of this version, the benefits of "forgetting" unlikely-to-repeat values:

┌──────────────────────────────────────────┬──────────┬─────────┬──────────┬──────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├──────────────────────────────────────────┼──────────┼─────────┼──────────┼──────────┼────────────┤ │ quickcheck.ml:Quickcheck.iter remember │ 20.26ms │ 16.33Mw │ 100.85kw │ 100.85kw │ 100.00% │ │ quickcheck.ml:Quickcheck.iter forget │ 17.65ms │ 16.21Mw │ 34.83kw │ 34.83kw │ 87.10% │ └──────────────────────────────────────────┴──────────┴─────────┴──────────┴──────────┴────────────┘
Optimizations to:
- various Float.t validation functions
- various functions in Validate
- List.fold_right
Made the type of Option.compare compatible with @@deriving compare.
Fixed an example code fragment in a comment in applicative_intf.ml
In Core_hashtbl, the add_worker function used a bool ref both internally and to pass to Avltree to track whether a new key is added. This was allocated on every call to add or set, and set didn't even use its contents.

This version pre-allocates the bool ref inside each Core_hashtbl.t and reuses it. It still can't be a mutable field because it does need to be passed to Avltree.

After change:

┌───────────────────────────────────────────────────┬──────────────┬────────────┬────────────┬────────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├───────────────────────────────────────────────────┼──────────────┼────────────┼────────────┼────────────┼────────────┤ │ Hashtbl.set no collisions │ 84.73ns │ 3.00w │ 0.83w │ 0.83w │ 0.01% │ │ Hashtbl.set w/ collisions │ 112.46ns │ │ │ │ 0.02% │ │ Hashtbl.change no collisions │ 82.74ns │ 3.50w │ 0.53w │ 0.53w │ 0.01% │ │ Hashtbl.change w/ collisions │ 191.50ns │ 4.56w │ 1.15w │ 1.15w │ 0.03% │ │ Hashtbl.merge no collisions │ 292_976.43ns │ 26_669.00w │ 15_381.62w │ 12_305.62w │ 48.52% │ │ Hashtbl.merge w/ collisions │ 603_822.86ns │ 33_001.00w │ 20_037.22w │ 16_961.22w │ 100.00% │ │ Hashtbl.add_exn no resize, no collisions │ 80_992.57ns │ 3_088.00w │ 4_102.63w │ 3_077.63w │ 13.41% │ │ Hashtbl.add_exn no resize, w/ collisions │ 178_080.05ns │ 4_621.00w │ 5_668.61w │ 4_643.61w │ 29.49% │ │ Hashtbl.add_exn w/ resize, no collisions │ 176_442.98ns │ 16_403.00w │ 9_222.64w │ 6_148.64w │ 29.22% │ │ Hashtbl.add_exn w/ resize, w/ collisions │ 297_577.29ns │ 19_472.00w │ 12_292.13w │ 9_218.13w │ 49.28% │ └───────────────────────────────────────────────────┴──────────────┴────────────┴────────────┴────────────┴────────────┘

Before change:

┌───────────────────────────────────────────────────┬──────────────┬────────────┬────────────┬────────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├───────────────────────────────────────────────────┼──────────────┼────────────┼────────────┼────────────┼────────────┤ │ Hashtbl.set no collisions │ 104.88ns │ 5.00w │ 1.26w │ 1.26w │ 0.02% │ │ Hashtbl.set w/ collisions │ 114.33ns │ 2.00w │ │ │ 0.02% │ │ Hashtbl.change no collisions │ 85.79ns │ 4.50w │ 0.58w │ 0.58w │ 0.02% │ │ Hashtbl.change w/ collisions │ 198.75ns │ 5.56w │ 1.28w │ 1.28w │ 0.04% │ │ Hashtbl.merge no collisions │ 307_857.59ns │ 31_787.00w │ 15_380.91w │ 12_304.91w │ 58.19% │ │ Hashtbl.merge w/ collisions │ 529_054.02ns │ 38_119.00w │ 20_015.32w │ 16_939.32w │ 100.00% │ │ Hashtbl.add_exn no resize, no collisions │ 77_708.20ns │ 5_135.00w │ 4_101.83w │ 3_076.83w │ 14.69% │ │ Hashtbl.add_exn no resize, w/ collisions │ 180_950.23ns │ 6_668.00w │ 5_638.77w │ 4_613.77w │ 34.20% │ │ Hashtbl.add_exn w/ resize, no collisions │ 177_492.82ns │ 19_476.00w │ 9_237.07w │ 6_163.07w │ 33.55% │ │ Hashtbl.add_exn w/ resize, w/ collisions │ 285_298.72ns │ 22_545.00w │ 12_330.90w │ 9_256.90w │ 53.93% │ └───────────────────────────────────────────────────┴──────────────┴────────────┴────────────┴────────────┴────────────┘
In Core_hashtbl.add_worker, removed a match that avoided calling Avltree.add, but actually did hurt performance overall.

Perhaps at some point before cross-module inlining, this was a helpful optimization. Right now it bypasses the mutation inside Avltree, so replacing a value in a non-colliding bucket (a Leaf) causes unnecessary re-allocation of the leaf.

After changes:

┌───────────────────────────────────────────────────┬──────────────┬────────────┬────────────┬────────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├───────────────────────────────────────────────────┼──────────────┼────────────┼────────────┼────────────┼────────────┤ │ Hashtbl.set no collisions │ 52.19ns │ 2.00w │ │ │ │ │ Hashtbl.set w/ collisions │ 112.04ns │ 2.00w │ │ │ 0.02% │ │ Hashtbl.change no collisions │ 87.25ns │ 4.50w │ 0.58w │ 0.58w │ 0.02% │ │ Hashtbl.change w/ collisions │ 195.85ns │ 5.56w │ 1.29w │ 1.29w │ 0.04% │ │ Hashtbl.merge no collisions │ 308_164.10ns │ 31_787.00w │ 15_380.91w │ 12_304.91w │ 58.48% │ │ Hashtbl.merge w/ collisions │ 526_914.80ns │ 38_119.00w │ 20_013.81w │ 16_937.81w │ 100.00% │ │ Hashtbl.add_exn no resize, no collisions │ 76_983.60ns │ 5_135.00w │ 4_100.44w │ 3_075.44w │ 14.61% │ │ Hashtbl.add_exn no resize, w/ collisions │ 174_712.92ns │ 6_668.00w │ 5_667.47w │ 4_642.47w │ 33.16% │ │ Hashtbl.add_exn w/ resize, no collisions │ 176_681.57ns │ 19_476.00w │ 9_231.75w │ 6_157.75w │ 33.53% │ │ Hashtbl.add_exn w/ resize, w/ collisions │ 280_448.62ns │ 22_545.00w │ 12_293.32w │ 9_219.32w │ 53.22% │ └───────────────────────────────────────────────────┴──────────────┴────────────┴────────────┴────────────┴────────────┘

Before changes:

┌───────────────────────────────────────────────────┬──────────────┬────────────┬────────────┬────────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├───────────────────────────────────────────────────┼──────────────┼────────────┼────────────┼────────────┼────────────┤ │ Hashtbl.set no collisions │ 104.88ns │ 5.00w │ 1.26w │ 1.26w │ 0.02% │ │ Hashtbl.set w/ collisions │ 114.33ns │ 2.00w │ │ │ 0.02% │ │ Hashtbl.change no collisions │ 85.79ns │ 4.50w │ 0.58w │ 0.58w │ 0.02% │ │ Hashtbl.change w/ collisions │ 198.75ns │ 5.56w │ 1.28w │ 1.28w │ 0.04% │ │ Hashtbl.merge no collisions │ 307_857.59ns │ 31_787.00w │ 15_380.91w │ 12_304.91w │ 58.19% │ │ Hashtbl.merge w/ collisions │ 529_054.02ns │ 38_119.00w │ 20_015.32w │ 16_939.32w │ 100.00% │ │ Hashtbl.add_exn no resize, no collisions │ 77_708.20ns │ 5_135.00w │ 4_101.83w │ 3_076.83w │ 14.69% │ │ Hashtbl.add_exn no resize, w/ collisions │ 180_950.23ns │ 6_668.00w │ 5_638.77w │ 4_613.77w │ 34.20% │ │ Hashtbl.add_exn w/ resize, no collisions │ 177_492.82ns │ 19_476.00w │ 9_237.07w │ 6_163.07w │ 33.55% │ │ Hashtbl.add_exn w/ resize, w/ collisions │ 285_298.72ns │ 22_545.00w │ 12_330.90w │ 9_256.90w │ 53.93% │ └───────────────────────────────────────────────────┴──────────────┴────────────┴────────────┴────────────┴────────────┘
Add new functions to hashtbl that add common missing functionality and/or that makes the interface more uniform and consistent with other container modules.
Add a bunch of functions to list and array that add common missing functionality and/or that make their interfaces more uniform and consistent with other container modules.
Rewrite Hashtbl.merge to be simpler and faster.

After changes:

┌──────────────────────────────────────┬──────────┬─────────┬──────────┬──────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├──────────────────────────────────────┼──────────┼─────────┼──────────┼──────────┼────────────┤ │ Hashtbl.merge no collisions │ 172.57us │ 17.44kw │ 9.22kw │ 7.69kw │ 48.76% │ │ Hashtbl.merge w/ collisions │ 284.55us │ 20.61kw │ 11.53kw │ 9.99kw │ 80.41% │ │ Pooled_hashtbl.merge no collisions │ 260.57us │ 5.20kw │ 19.18kw │ 3.09kw │ 73.63% │ │ Pooled_hashtbl.merge w/ collisions │ 353.88us │ 5.20kw │ 19.18kw │ 3.09kw │ 100.00% │ └──────────────────────────────────────┴──────────┴─────────┴──────────┴──────────┴────────────┘

Before changes:

┌──────────────────────────────────────┬──────────┬─────────┬──────────┬──────────┬────────────┐ │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ├──────────────────────────────────────┼──────────┼─────────┼──────────┼──────────┼────────────┤ │ Hashtbl.merge no collisions │ 309.59us │ 31.79kw │ 15.38kw │ 12.30kw │ 48.91% │ │ Hashtbl.merge w/ collisions │ 526.67us │ 38.12kw │ 19.97kw │ 16.90kw │ 83.21% │ │ Pooled_hashtbl.merge no collisions │ 469.41us │ 7.32kw │ 35.29kw │ 3.12kw │ 74.16% │ │ Pooled_hashtbl.merge w/ collisions │ 632.96us │ 7.32kw │ 35.29kw │ 3.12kw │ 100.00% │ └──────────────────────────────────────┴──────────┴─────────┴──────────┴──────────┴────────────┘
Make Hashtbl functions raise an exception if a callback passed in as an argument mutates one of the hash tables being worked on.

Usually, though not always, this comes up for iteration functions. Once a hash table has been mutated, it is unsafe to continue operating on it, as its structure may have changed. Buckets and their contents may have been moved or resized; continuing may result in skipping key/value pairs, repeating key/value pairs, or executing unsafe operations.

This feature adds a mutation_allowed flag to hash tables. Each mutating operation first checks the flag, and raises if it is not set. Each operation with callbacks that must not mutate unsets the flag before calling the callbacks, and restores the flag's original value when it finishes.

We compared the timing of this implementation to an alternate implementation using a mutation counter, and the time and space used for this implementation was much better for iteration and within epsilon of the other for single-key operations like set.
Array function names related to zipping are all over the place. Make them match List, which has a nice uniform naming scheme.
- Rename combine -> zip_exn
- Rename split -> unzip
- (zip remains named as zip)
Add ~key and ~data labels to Hashtbl.filteri_inplace
Added Hash_set.to_hashtbl, by analogy to Set.to_map.
Since we are mutating avltrees in place, make sure the compiler sees the type parameters as invariant.

Tested that a segfaulting example doesn't compile anymore.
Add label f to Hashtbl.change, Map.change, & family.

Introduce the new function update in those modules, which enforces statically the presence of a resulting value

Example:

-|val Hashtbl.change : 'a t -> key -> ('a option -> 'a option) -> unit

+|val Hashtbl.change : 'a t -> key -> f:('a option -> 'a option) -> unit +|val Hashtbl.update : 'a t -> key -> f:('a option -> 'a) -> unit

The motivation for the introduction of update is that in an overwhelming majority of the places where Hashtbl.change is used in our codebase, it is statically known that a new value shall be computed and stored. The use of the dynamism offered by change, which can return an option, is error prone.

The addition of the label is considered acceptable in consideration to external libraries depending on core, because a missing label is just a warning, and we do not guarantee stability in the presence of -warn-error = true.
Changed Source_code_position.t from:

@@deriving bin_io, sexp

to:

@@deriving sexp_of

and made sexp_of use the human-readable format, "FILE:LINE:COL", rather than the unreadable format. Removed Source_code_position.t_hum, which is now obsolete.

If one wants a serialized source-code position, one can use Source_code_position.Stable.
Added Ref.set_temporarily, for temporarily setting a ref to a value for the duration of a thunk.

val set_temporarily : 'a t -> 'a -> f:(unit -> 'b) -> 'b
Add the function singleton : 'a -> 'a t in the stack containers. It cannot be added to Container.S directly because some container never have exactly 1 element.
Made Core.Array match Invariant.S1.
Change the interface of Make_iterable_binable* to give the control back to the user when deserializing Bin_protted data.

Improve the bin_prot deserialization of Maps and Sets. We construct a balanced tree directly instead of relying on Map.add / Set.add. This is possibile because the size of the map is known and elements are sorted.

The complexity goes down from n.log(n) to n.

In case the comparison function changes (and the invariant is not respected), there is a fallback to reconstruct the whole map from scratch.
Add a function to blit a Rope.t into a Buffer.t.
Hashtbl differs from some other core containers with idiosyncratic naming of iteration functions. Change to be consistent and to more closely match the conventions for List and Array.

Hashtbl:
- Copy iter -> iteri.
- Add a deprecation tag to iter.
Made Bag.invariant and Doubly_linked.invariant match Invariant.S1.
Map differs from some other core containers with idiosyncratic naming of iteration functions. The current Map name conventions are also internally inconsistent as well (ex: current Map.iter vs Map.map vs Map.mapi). Change to be consistent and to more closely match the conventions for List and Array.

Map:
- Copy filter -> filteri.
- Add a deprecation tag to filter.
Map differs from some other core containers with idiosyncratic naming of iteration functions. The current Map name conventions are also internally inconsistent as well (ex: current Map.iter vs Map.map vs Map.mapi). Change to be consistent and to more closely match the conventions for List and Array.

Map:
- Copy iter -> iteri.
- Add a deprecation tag to iter.
Made Core.Set_once match Invariant.S1.
Add Bigstring.concat.
For Core.Unique_id, exposed @@deriving typerep.
Expose Hashtbl.hashable, analogous to Map.comparator.
Adds a constant-time val mem_elt : 'a t -> 'a Elt.t -> bool to Doubly_linked and Bag
Add Ordering.to_int which can be useful when one is writing a comparison function. Instead of dealing with the int directly, one can return Ordering.t values and transform them later into ints.
Float.int_pow: Fast computation of x ** n when n is an integer.
Make Core_kernel.Std.Nothing.t enumerable. There's no particular reason not to.
Minor improvements to queue interface
Call Caml.Pervasives.do_at_exit before printing an exception and exiting

The default ocaml uncaught exception handler does this. It is especially useful for curses applications as the at_exit handler has a chance to put back the terminal in a good state before printing the exception and backtrace.

Do the same in Core and Async.
Removed big literals so that the compiler does not complain in 32bit
Add List.range', a generalization of List.range.
Add some functions to Map that are present in Hashtbl:
- remove_multi
- partition_tf
- partitioni_tf
- partition_map
- partition_mapi
Add a Map.nth_exn as a missing complementary function to nth
Renamed Validate.fail_sexp as fail_s, to follow our new *_s convention for Sexp.t-taking functions.
Sequence.split_n_eagerly returns a pair of sequences, but every element of the first sequence has already been evaluated by the time it returns. This feature just makes the first component of the tuple a list instead of a sequence, and renames split_n_eagerly to split_n.

Additionally, this feature adds a new chunks_exn function, which just applies split_n until the input sequence is empty.
Removed Timing_wheel's default alarm_precision, to force people to think about the precision they want when they create a timing wheel.
In Timing_wheel.Config.sexp_of_t, used @sexp_drop_default with level_bits.
Write a better-performing Array.filter_mapi function, and implement Array.filter_map, Array.filter_opt, Array.partitioni_tf, and Array.partition_tf in terms of it.

Slightly worse for zero-length input arrays, about unch'd if we're filtering out almost everything (eq_zero), better on most everything else.

┌──────────────────── │ Name │ ├──────────────────── │ core\_array.ml:filter old-filter-even:0 │ core\_array.ml:filter old-filter-even:1 │ core\_array.ml:filter old-filter-even:10 │ core\_array.ml:filter old-filter-even:100 │ core\_array.ml:filter old-filter-even:1000 │ core\_array.ml:filter old-filter-even:10000 │ core\_array.ml:filter old-filter-even:100000 │ core\_array.ml:filter new-filter-even:0 │ core\_array.ml:filter new-filter-even:1 │ core\_array.ml:filter new-filter-even:10 │ core\_array.ml:filter new-filter-even:100 │ core\_array.ml:filter new-filter-even:1000 │ core\_array.ml:filter new-filter-even:10000 │ core\_array.ml:filter new-filter-even:100000 │ core\_array.ml:filter old-filter-eq_zero:0 │ core\_array.ml:filter old-filter-eq_zero:1 │ core\_array.ml:filter old-filter-eq_zero:10 │ core\_array.ml:filter old-filter-eq_zero:100 │ core\_array.ml:filter old-filter-eq_zero:1000 │ core\_array.ml:filter old-filter-eq_zero:10000 │ core\_array.ml:filter old-filter-eq_zero:100000 │ core\_array.ml:filter new-filter-eq_zero:0 │ core\_array.ml:filter new-filter-eq_zero:1 │ core\_array.ml:filter new-filter-eq_zero:10 │ core\_array.ml:filter new-filter-eq_zero:100 │ core\_array.ml:filter new-filter-eq_zero:1000 │ core\_array.ml:filter new-filter-eq_zero:10000 │ core\_array.ml:filter new-filter-eq_zero:100000 │ core\_array.ml:filter old-filter-neq_zero:0 │ core\_array.ml:filter old-filter-neq_zero:1 │ core\_array.ml:filter old-filter-neq_zero:10 │ core\_array.ml:filter old-filter-neq_zero:100 │ core\_array.ml:filter old-filter-neq_zero:1000 │ │ core\_array.ml:filter old-filter-neq_zero:10000 │ │ core\_array.ml:filter old-filter-neq_zero:100000 │ core\_array.ml:filter new-filter-neq_zero:0 │ core\_array.ml:filter new-filter-neq_zero:1 │ core\_array.ml:filter new-filter-neq_zero:10 │ core\_array.ml:filter new-filter-neq_zero:100 │ core\_array.ml:filter new-filter-neq_zero:1000 │ │ core\_array.ml:filter new-filter-neq_zero:10000 │ │ core\_array.ml:filter new-filter-neq_zero:100000 │ core\_array.ml:filter old-filter_map-int:0 │ core\_array.ml:filter old-filter_map-int:1 │ core\_array.ml:filter old-filter_map-int:10 │ core\_array.ml:filter old-filter_map-int:100 │ core\_array.ml:filter old-filter_map-int:1000 │ core\_array.ml:filter old-filter_map-int:10000 │ core\_array.ml:filter old-filter_map-int:100000 │ core\_array.ml:filter new-filter_map-int:0 │ core\_array.ml:filter new-filter_map-int:1 │ core\_array.ml:filter new-filter_map-int:10 │ core\_array.ml:filter new-filter_map-int:100 │ core\_array.ml:filter new-filter_map-int:1000 │ core\_array.ml:filter new-filter_map-int:10000 │ core\_array.ml:filter new-filter_map-int:100000 │ core\_array.ml:filter old-filter_map-float:0 │ core\_array.ml:filter old-filter_map-float:1 │ core\_array.ml:filter old-filter_map-float:10 │ core\_array.ml:filter old-filter_map-float:100 │ │ core\_array.ml:filter old-filter_map-float:1000 │ │ core\_array.ml:filter old-filter_map-float:10000 │ │ core\_array.ml:filter old-filter_map-float:100000 │ │ core\_array.ml:filter new-filter_map-float:0 │ core\_array.ml:filter new-filter_map-float:1 │ core\_array.ml:filter new-filter_map-float:10 │ core\_array.ml:filter new-filter_map-float:100 │ │ core\_array.ml:filter new-filter_map-float:1000 │ │ core\_array.ml:filter new-filter_map-float:10000 │ │ core\_array.ml:filter new-filter_map-float:100000 │ │ core\_array.ml:filter old-filter_map-boxed:0 │ core\_array.ml:filter old-filter_map-boxed:1 │ core\_array.ml:filter old-filter_map-boxed:10 │ core\_array.ml:filter old-filter_map-boxed:100 │ │ core\_array.ml:filter old-filter_map-boxed:1000 │ │ core\_array.ml:filter old-filter_map-boxed:10000 │ │ core\_array.ml:filter old-filter_map-boxed:100000 │ │ core\_array.ml:filter new-filter_map-boxed:0 │ core\_array.ml:filter new-filter_map-boxed:1 │ core\_array.ml:filter new-filter_map-boxed:10 │ core\_array.ml:filter new-filter_map-boxed:100 │ │ core\_array.ml:filter new-filter_map-boxed:1000 │ │ core\_array.ml:filter new-filter_map-boxed:10000 │ │ core\_array.ml:filter new-filter_map-boxed:100000 │ │ core\_array.ml:filter old-partition_tf:0 │ core\_array.ml:filter old-partition_tf:1 │ core\_array.ml:filter old-partition_tf:10 │ core\_array.ml:filter old-partition_tf:100 │ core\_array.ml:filter old-partition_tf:1000 │ core\_array.ml:filter old-partition_tf:10000 │ core\_array.ml:filter old-partition_tf:100000 │ core\_array.ml:filter new-partition_tf:0 │ core\_array.ml:filter new-partition_tf:1 │ core\_array.ml:filter new-partition_tf:10 │ core\_array.ml:filter new-partition_tf:100 │ core\_array.ml:filter new-partition_tf:1000 │ core\_array.ml:filter new-partition_tf:10000 │ core\_array.ml:filter new-partition_tf:100000 └──────────────────── ────────────────────────────────┬─────────────────┬─────────────┬─────────────┬─────────────┬────────────┐ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ ────────────────────────────────┼─────────────────┼─────────────┼─────────────┼─────────────┼────────────┤ │ 12.37ns │ 9.00w │ │ │ │ │ 77.44ns │ 15.00w │ │ │ │ │ 207.10ns │ 36.00w │ │ │ │ │ 1_699.41ns │ 261.00w │ │ │ │ │ 56_320.50ns │ 1_009.00w │ 2_506.01w │ 1_004.01w │ 0.30% │ │ 469_134.89ns │ 10_009.00w │ 25_007.38w │ 10_005.38w │ 2.46% │ │ 4_421_742.22ns │ 100_009.00w │ 250_130.09w │ 100_128.09w │ 23.17% │ │ 13.87ns │ 14.00w │ │ │ │ │ 57.64ns │ 18.00w │ │ │ │ │ 196.28ns │ 35.00w │ │ │ │ │ 1_361.04ns │ 215.00w │ │ │ │ │ 21_473.76ns │ 1_014.00w │ 1_001.02w │ │ 0.11% │ │ 204_033.12ns │ 10_014.00w │ 10_001.14w │ 0.14w │ 1.07% │ │ 2_058_144.47ns │ 100_014.00w │ 100_002.00w │ 1.00w │ 10.78% │ │ 12.21ns │ 9.00w │ │ │ │ │ 71.23ns │ 15.00w │ │ │ │ │ 174.80ns │ 24.00w │ │ │ │ │ 1_212.70ns │ 114.00w │ │ │ │ │ 23_347.51ns │ 13.00w │ 1_007.00w │ 6.00w │ 0.12% │ │ 210_509.83ns │ 13.00w │ 10_007.00w │ 6.00w │ 1.10% │ │ 1_912_253.91ns │ 13.00w │ 100_007.01w │ 6.01w │ 10.02% │ │ 13.70ns │ 14.00w │ │ │ │ │ 56.56ns │ 18.00w │ │ │ │ │ 179.42ns │ 27.00w │ │ │ │ │ 1_254.49ns │ 117.00w │ │ │ │ │ 20_968.06ns │ 16.00w │ 1_001.02w │ │ 0.11% │ │ 204_299.82ns │ 16.00w │ 10_001.13w │ 0.13w │ 1.07% │ │ 2_019_283.81ns │ 16.00w │ 100_001.91w │ 0.91w │ 10.58% │ │ 12.14ns │ 9.00w │ │ │ │ │ 32.72ns │ 11.00w │ │ │ │ │ 219.18ns │ 48.00w │ │ │ │ │ 1_902.76ns │ 408.00w │ 0.12w │ 0.12w │ │ 82_032.44ns │ 2_007.00w │ 3_998.20w │ 1_997.20w │ 0.43% │ 850_234.44ns │ 20_007.00w │ 40_014.86w │ 20_013.86w │ 4.46% │ │ 7_345_941.05ns │ 200_007.00w │ 400_407.82w │ 200_406.82w │ 38.49% │ │ 13.66ns │ 14.00w │ │ │ │ │ 18.26ns │ 14.00w │ │ │ │ │ 201.04ns │ 43.00w │ │ │ │ │ 1_404.33ns │ 313.00w │ │ │ │ 22_829.70ns │ 2_012.00w │ 1_001.02w │ │ 0.12% │ 218_872.52ns │ 20_012.00w │ 10_001.21w │ 0.21w │ 1.15% │ │ 2_121_340.68ns │ 200_012.00w │ 100_002.77w │ 1.77w │ 11.12% │ │ 9.58ns │ 5.00w │ │ │ │ │ 68.46ns │ 11.00w │ │ │ │ │ 191.66ns │ 32.00w │ │ │ │ │ 1_492.60ns │ 257.00w │ │ │ │ │ 57_155.42ns │ 1_005.00w │ 2_507.01w │ 1_005.01w │ 0.30% │ │ 522_177.50ns │ 10_005.00w │ 25_008.54w │ 10_006.54w │ 2.74% │ │ 5_945_405.67ns │ 100_005.00w │ 250_170.69w │ 100_168.69w │ 31.15% │ │ 12.03ns │ 10.00w │ │ │ │ │ 53.63ns │ 14.00w │ │ │ │ │ 164.16ns │ 31.00w │ │ │ │ │ 1_263.42ns │ 211.00w │ │ │ │ │ 23_113.12ns │ 1_010.00w │ 1_001.02w │ │ 0.12% │ │ 218_152.23ns │ 10_010.00w │ 10_001.15w │ 0.15w │ 1.14% │ │ 2_217_307.86ns │ 100_010.00w │ 100_002.11w │ 1.11w │ 11.62% │ │ 9.32ns │ 5.00w │ │ │ │ │ 66.68ns │ 13.00w │ │ │ │ │ 182.86ns │ 42.00w │ │ │ │ 1_496.56ns │ 357.00w │ │ │ │ 76_479.74ns │ 2_005.00w │ 3_507.02w │ 2_005.02w │ 0.40% │ 694_999.59ns │ 20_005.00w │ 35_011.08w │ 20_009.08w │ 3.64% │ 8_694_669.26ns │ 200_005.00w │ 350_476.44w │ 200_474.44w │ 45.56% │ │ 12.29ns │ 10.00w │ │ │ │ │ 58.24ns │ 16.00w │ │ │ │ │ 142.67ns │ 41.00w │ │ │ │ 1_119.41ns │ 311.00w │ │ │ │ 14_262.66ns │ 2_010.00w │ 1_001.02w │ │ 0.07% │ 136_448.05ns │ 20_010.00w │ 10_001.23w │ 0.23w │ 0.71% │ 1_282_005.01ns │ 200_010.00w │ 100_003.14w │ 2.14w │ 6.72% │ │ 9.48ns │ 5.00w │ │ │ │ │ 71.16ns │ 13.00w │ │ │ │ │ 197.40ns │ 42.00w │ │ │ │ 1_762.40ns │ 357.00w │ │ │ │ 86_220.67ns │ 2_005.00w │ 3_507.02w │ 2_005.02w │ 0.45% │ 828_291.42ns │ 20_005.00w │ 35_011.84w │ 20_009.84w │ 4.34% │ 7_955_395.61ns │ 200_005.00w │ 350_441.44w │ 200_439.44w │ 41.68% │ │ 14.43ns │ 10.00w │ │ │ │ │ 59.24ns │ 16.00w │ │ │ │ │ 198.19ns │ 41.00w │ │ │ │ 1_580.21ns │ 311.00w │ │ │ │ 52_045.31ns │ 2_010.00w │ 2_011.01w │ 1_010.01w │ 0.27% │ 479_239.44ns │ 20_010.00w │ 20_012.42w │ 10_011.42w │ 2.51% │ 4_389_392.06ns │ 200_010.00w │ 200_135.09w │ 100_134.09w │ 23.00% │ │ 16.55ns │ 16.00w │ │ │ │ │ 128.08ns │ 29.00w │ │ │ │ │ 554.15ns │ 111.00w │ │ │ │ │ 4_853.58ns │ 921.00w │ 0.46w │ 0.46w │ 0.03% │ │ 201_289.06ns │ 5_016.00w │ 9_015.21w │ 5_010.21w │ 1.05% │ │ 1_796_749.87ns │ 50_016.00w │ 90_040.96w │ 50_035.96w │ 9.41% │ │ 19_084_871.85ns │ 500_016.00w │ 902_187.67w │ 502_182.67w │ 100.00% │ │ 28.29ns │ 23.00w │ │ │ │ │ 103.78ns │ 31.00w │ │ │ │ │ 504.10ns │ 96.00w │ │ │ │ │ 3_869.52ns │ 726.00w │ 0.23w │ 0.23w │ 0.02% │ │ 122_807.29ns │ 4_023.00w │ 5_013.04w │ 2_010.04w │ 0.64% │ │ 1_197_596.39ns │ 40_023.00w │ 50_020.05w │ 20_017.05w │ 6.28% │ │ 10_458_344.09ns │ 400_023.00w │ 500_590.94w │ 200_587.94w │ 54.80% │ ────────────────────────────────┴─────────────────┴─────────────┴─────────────┴─────────────┴────────────┘
Added Binable.Of_sexpable functor.
Install the sexp exception printer sooner so that we can get proper %test_result ... errors in things that come before core_kernel.
In Stable_unit_test.Make functors, include all test failures rather than just the first. This is useful for updating batches of expected bin_io results when stabilizing a module.
Remove an unnecessary cast in or_error.ml

core_profiler

Switched to ppx.
Minor adjustments to the command line of profiler_tool.exe:
- Make '-%' an alias for '-percentile'
- Make '-percentile' accept a comma-separated list of numbers
- Add '-median' argument that is equivalent to '-percentile 50'

email_message

Bugfixes and minor API improvements.

incremental

Add README.org to Incremental.
Added some type annotations based on comments by @def-lkb about lack of principality.
Switched to ppx.

jenga

Restructure the code in a way that allows to build binaries that statically link jenga with the rules. This is useful because some debugging/profiling tools don't work in the presence of dynamically loaded code very well.
Switch to PPX.
Change the gc info output by jenga so it shows heap size and top heap size, instead of live and heap size. The live part is not super useful given how random it is. I have seen cases where jenga was using 20GB during building and jenga reported a heap size of 13GB at the end so the top heap size avoids being tricked.
First half of the fixes no packing: sharing the structures of dependencies, so they take less space on disk (and in memory as well, when they are loaded from disk, but not really when building from scratch given the way we will use them).

The sexp format also has sharing, because it would also blow up in size otherwise (this is different from the interning of paths, where the interning saves a constant factor). And of course, it makes it possible to see the actual on-disk representation which is nice.

Also fix unhelpful error (contains no information) when the db can't be loaded.

Break the thing that avoids rerunning rules when the set of dependencies decreases. I think it was never useful anyway.
Better error on duplicate targets in the same rule.
To prevent running more than one jenga in a repository, use a local lock rather than an nfs one. We need a transition period though, so for now we use both kinds of locks. Building on nfs is slow, so I don't think there's any downside is not supporting nfs this way. And maybe inotify doesn't work. The upside is that we don't step into Lock.Nfs bugs where if a process is interrupted at the wrong time (when the two lock files are empty) the locks won't be cleaned up automatically, forcing someone to get rid of the lock files manually.

Also rename .jenga/.jenga.* to .jenga/*, because all these prefixes are annoying.
Added a couple of options to turn off some part of jenga, which I used to check how they impacted performance, and could still be handy later.
Optionally display additional information about much allocation was done, at the end of builds. Used it to try to improve memory usage of full tree builds without actually doing full tree builds.
Make stat'ing faster. Hash cons some tenacious that build mtimes map to avoid a huge increase of memory use.
Got rid of noise when stopping jenga.
Some changes to the implementation of Tenacious to improve memory efficiency.

Includes the following changes:
- Remove strong_refs field of Heart.fragile type and instead insert links from the clients Ring.t to its parent using Ring.keep_alive.
- Replace the Ring.t in the triggers field of Hearts.fragile with an Ivar.t since all uses of triggers were producing their own equivalent IVar.ts. Remove the functions broken by this because they were unused.
- Make the Tenacious.t type a concrete datatype, and optimize pure computations by partially evaluating this datatype directly in the pure case.
- Rather than building separate Heart.ts for cancellation and the result, split the cancellation heart into two hearts cancel and dep and then use dep as the result heart. This means that a tenacious is cancelled if either cancel or dep is broken, and it must return a heart representing the validity of its result combined with dep.
Cursory benchmarking indicates a 23% improvement in maximum resident set size and a 10% improvment in (user) execution time when building the lib directory from scratch.
Adding some sexp_of functions, since they're always missing and it's a pain when debugging.
Adding direct support for Dep.map. Even now that Tenacious is smarter, this still creates less binds. Doesn't seem to make much of a difference (perhaps 3-5% less allocation, on a null build of lib), but if nothing else, it's much less surprising to think that Dep.Map becomes Tenacious.Map.
Added Dep.List.concat.
Make it possible to turn off the behavior where jenga rejects commands that output on stderr.

It increases slightly the footprint of the in memory db, but the difference is tiny compared to the rest of the memory usage.
Added a few tests about Jenga_lib.Api.Reflect

ocaml_plugin

Switch to ppx.
Allow ppx-style code to be loaded by plugin-applications build using ocaml_plugin.
Follow Core & Async evolution.

patdiff

patdiff -location-style omake should print the line number of the first difference in each hunk, skipping context lines.
Switched to PPX.
Added binding in patdiff to use the newly minted colors of Ansi_terminal. This will be used notably by patdiff4 to produce better ddiff.

Also, have the module Color and Style implement and export Comparable.S. This is useful for example to dedup styles from a list of styles without relying on the polymorphic equality.
Make it so that if you pass -warn-if-no-trailing-newline-in-both false then you get the warning only when one file has a trailing newline and the other file does not.

If you pass -warn-if-no-trailing-newline-in-both true or omit this flag, then you get the current behavior of warning for each file independently.
Patdiff's unified-tests currently render colors codes in angle brackets. Change them to square brackets. Square brackets are word boundaries, so we'll get more legible diffs when tests fail.
Simple code change in patdiff to prepare more changes in patdiff4. This change is a pure refactoring and has zero runtime change. Just moving some functions around.
patdiff_core.ml is a very long module. start extracting module from it. start with format. in the process, expose in a private fashion the record Rule.t.
Continue on splitting the file patdiff_core.ml into smaller pieces. In this version, we extract each output mode into its own file.
Kill the generation of html diffs in patdiff. There are good third party tools that can convert efficiently ansi texts to html directly. We plan on simplifying a bit the patdiff source code to increase its maintainability, and dropping the requirement of producing html output seems a step in the right direction.

Some pointers:

http://www.pixelbeat.org/scripts/ansi2html.sh jane: app/ralloc/commander/ansi2html.ml

patience_diff

Switch to PPX.

ppx_assert

Update to follow evolution of Ppx_core.

ppx_bench

Update to follow Ppx_core evolution.
Mark attributes as handled inside explicitly dropped pieces of code.

So that a @@deriving inside a let%test dropped by ppx_inline_test_drop doesn't cause a failure.

ppx_bin_prot

Minor changes, nothing worth mentionning.

ppx_compare

Follow evolution of Ppx_core and Type_conv.

ppx_core

Kill the nonrec rewrite done by typerep. It is no longer needed since 4.02.2, we kept it only for compatibility with the camlp4 code.
Merlin uses @merlin.* ... attributes in different places. Which ppx_driver reports as unused.

Introduce the concept of reserved namespaces. When one declares the namespace "foo" as reserved then:
- foo.* will never get reported as unused
- it is impossible to Attribute.declare "foo.*"
Mark the "merlin" namespace as reserved by default.
Don't print:

Extension `foo' was not translated. Hint: Did you mean foo?
OCaml makes no distinctions between "foo" and {whatever|foo|whatever}. The delimiter choice is simply left to the user.

Do the same in our ppx rewriters: i.e. wherever we accept "foo", also accept {whatever|foo|whatever}.
Avoid stupid hints like this one:

Attribute default' was not used. Hint:default' is available for label declarations but is used here in the context of a label declaration. Did you put it at the wrong level?
Update the API for the common case of extension point expanders.

Make it simpler to define ppx rewriters that locally expand extension points, which is the majority of our non-type-conv rewriters.

Such expanders are run inside the same Ast_traverse.map in a top-down manner which:
- probably improve speed
- help with rewriters that capture a pretty-print of their payload
- help with rewriter that interpret some extension points in a special way inside their payload
Fix the order in which errors are reported by ppx rewriters. Make them be reported in the same order as they appear.
Mark attributes as handled inside explicitly dropped pieces of code.

So that a @@deriving inside a let%test dropped by ppx_inline_test_drop doesn't cause a failure.

ppx_csv_conv

ppx

An umbrella feature for development on ppx syntax extensions. All work except for rebasing should be done in subfeatures.
Affected files

ppx/ppx_csv_conv/src/ppx_csv_conv.ml
ppx/changes-for-public-release

Changes required for the first public release of our ppx rewriters

Improved the type_conv export to ppx_deriving
- improved how it works, to enable it we just need:
  
  Type_conv.Ppx_deriving_exporter.set (module Ppx_deriving)
- Type_conv.add_alias now takes as argument registered type_conv deriviers. This ensure that we can resolve aliases right from the registration time. This simplify the export to ppx_deriving
Split some ppx rewirters

Split some functions out of their main ppx_XXX library that does the registration with ppx_driver or ppx_type_conv.

For instance:
- ppx_sexp_conv defines sexp_of_quote that is used by ppx_assert
- ppx_here defines ast_of_pos that is used by ppx_fail and ppx_assert
Making ppx_assert depends on ppx_sexp_conv and ppx_here is not good as it enables @@deriving sexp and %sexp_of: ty even if the user only writes ppx_assert in the jbuild. This is especially problematic for the public release as it makes ppx_assert incompatible with ppx_deriving.

This feature moves these functions into libraries called ppx_XXX_expander (and ppx_inline_test_libname for ppx_inline_test).

To help reviewing the changes, you can run:

patdiff <(hg cat -r fe show -base ppx/ppx_compare/src/ppx_compare.ml) <(hg cat -r fe show -tip ppx/ppx_compare/expander/ppx_compare_expander.ml) patdiff <(hg cat -r fe show -base ppx/ppx_sexp_conv/src/ppx_sexp_conv.ml) <(hg cat -r fe show -tip ppx/ppx_sexp_conv/expander/ppx_sexp_conv_expander.ml)

Move Ppx_type_conv.Std.Type_conv_path to Ppx_core

It's not type-conv specific anymore and is needed by libraries that don't use type_conv otherwise.

Rename some old runtime libraries

pa_test_lib --> ppx_assert_lib pa_bench_lib --> ppx_bench_lib
Affected files

ppx/ppx_csv_conv/src/ppx_csv_conv.ml
ppx

An umbrella feature for development on ppx syntax extensions. All work except for rebasing should be done in subfeatures.
Affected files

ppx/ppx_csv_conv/src/ppx_csv_conv.ml
ppx/delete-make-at-the-end

Cleanup in type_conv: remove Type_conv.Generator_result.make_at_the_end, which was a hack to remove warnings. We can do it better now, and because this is only for signatures, the code generation issue what we had in simplify-type-conv-ignore-unused-warning doesn't apply.

For users

This feature moves a few values up in signatures. For instance in this interface sexp_of_t is now the first value instead of the second one as before this feature:
```
type t `@@deriving sexp_of`
val x : int
```
In some cases this caused the OCaml compiler to complain about items in the signature being re-ordered. In these cases the signature was adapted to match the expected ordering.
Affected files

ppx/ppx_csv_conv/src/ppx_csv_conv.ml
ppx

An umbrella feature for development on ppx syntax extensions. All work except for rebasing should be done in subfeatures.
Affected files

ppx/ppx_csv_conv/example/example.ml ppx/ppx_csv_conv/example/example.mli ppx/ppx_csv_conv/example/test.csv
ppx/ppx_csv_conv-example

Make ppx_csv_conv example do something.

Testing

Added inline test.
Affected files

ppx/ppx_csv_conv/example/example.ml ppx/ppx_csv_conv/example/example.mli ppx/ppx_csv_conv/example/test.csv

ppx_custom_printf

OCaml makes no distinctions between "foo" and {whatever|foo|whatever}. The delimiter choice is simply left to the user.

Do the same in our ppx rewriters: i.e. wherever we accept "foo", also accept {whatever|foo|whatever}.
Fix missing location in errors for broken custom printf example like:

printf !"%{sexp: int" 3;;
Update to follow Ppx_core evolution.

ppx_driver

Disable safety check when code transformations are used as standard "-ppx" rewriters.
Introduce reserved namespaces, see Ppx_core's changelog.

Pass errors as attribute with -dparsetree to avoid "Error while running external preprocessor".
Update to follow Ppx_core evolution.

ppx_enumerate

Update to follow type_conv evolution.

ppx_expect

Initial release.

ppx_fail

Added a README.md
Update to follow Ppx_core evolution.

ppx_fields_conv

The iter function generated by ppx_variants_conv and ppx_fields_conv allowed one to give function which returned values of arbitrary types as iter function. This release constraint these functions to return unit.

N.B. the signature generated by the use of @@deriving variants (resp. fields) in interface already constrained the type to unit.
Update to follow type_conv's evolution.
Add Fields.make_creator to ppx_fields_conv's readme, since it appears to not be all that deprecated.

ppx_here

Make ppx_here translate [%here] instead of _here_.
Update to follow Ppx_core evolution.

ppx_inline_test

Support literate-style .ml files that allow ocaml code interleaved with expected output annotations. Compiling with the ppx_expect_test generates a program that outputs the original source file, but with the actual output substituted for the expected-output annotations. Then we can pat-diff the original file against the output file.

Testing

Examples in the test/ and example/ folders.
Expect-tests can now be written inline in libraries by using let%expect_test.

The runtime library has been split into two components: the test runner, which collects the output of the test body, and registers enough information to construct the *.ml.corrected file from the input; and the test evaluator, which compares the test output against the expected output and generates the output files.
Update to follow Ppx_core evolution.
When an exception is raised inside a let%test_module, display the position and name of the TEST_MODULE, same as for the let%test.
Mark attributes as handled inside explicitly dropped pieces of code.

So that a @@deriving inside a let%test dropped by ppx_inline_test_drop doesn't cause a failure.

ppx_optcomp

Change the way optcomp resolve filenames in #import directives

Do the same as cpp, i.e. for relative filenames, consider they are relative to the directory of the file being parsed. This doesn't matter internally as build commands are always executed from the current directory, but it matters for the public release as everything is executed from the root.

ppx_sexp_conv

Trying to improve the tests in ppx_sexp_conv because they are a mess. At least all tests are automatic now. And more things are tested like the sexpification of exceptions.
Update to follow Type_conv and Ppx_core evolution.
Make ppx_sexp_conv correctly handle aliases to polymorphic variants:

type t = A @@deriving sexptype u = t@@deriving sexptype v = u | B @@deriving sexp

Before, v_of_sexp would never manage to read B. This problem is now fixed if you usesexp_polyonuinstead ofsexp, and if you don't, you get an "unbound value __u_of_sexp__". People should usesexp_polywhen they have a polymorphic variant type that is not syntactically a polymorphic variant, but in practice it's simpler to replacesexpbysexp_poly` when faced with the error above.

The need for sexp_poly should happen only in one new case: an implementation says type u = t@@deriving sexpbut the interface says `type u =A@@deriving sexp. (the old case where it was already needed is when you have an interface that says `type u = t `@@deriving sexp and in some other implementation you try to say type t = That_module.t | A @@deriving sexp`).

ppx_type_conv

Kill the nonrec rewrite done by typerep. It is no longer needed since 4.02.2, we kept it only for compatibility with the camlp4 code.
Cleanup in type_conv: remove Type_conv.Generator_result.make_at_the_end, which was a hack to remove warnings. We can do it better now, and because this is only for signatures, the code generation issue what we had in simplify-type-conv-ignore-unused-warning doesn't apply.
Update to follow Ppx_core evolution.

ppx_typerep_conv

Update following Ppx_core and Type_conv evolution.
Add a README.

ppx_variants_conv

The iter function generated by ppx_variants_conv and ppx_fields_conv allowed one to give function which returned values of arbitrary types as iter function. This feature constraint these functions to return unit.

N.B. the signature generated by the use of @@deriving variants (resp. fields) in interface already constrained the type to unit.
Update to follow Type_conv evolution.

re2

Switched to PPX.
Add Re2.Parser.any_string combinator.

There are no tests because any_string is constructed only from the tested API and there's almost no interesting properties of it that can be verified.

rpc_parallel

Switched to PPX.
Expose the connection_timeout argument in rpc_parallel. This argument exists in Rpc_parallel_core.Parallel, but it is not exposed in Rpc_parallel.Parallel.
Allow custom handling of missed async_rpc heartbeats.
Give a better error message when redirecting output on a remote box to a file path that does not exist.
remove unncessary chmod 700 call on the remote executable
Give a clear error message for the common mistake of not making the Parallel.Make_worker() functor application top-level
Make errors/exceptions in Rpc_parallel more observable
- Make stderr and stdout redirection mandatory in order to encourage logging stderr
- Clean up the use of monitors across Rpc_parallel
- Fix bug with exceptions that are sent directly to Monitor.main (e.g. Async_log does this)
Add the ability to explicitly initialize as a master and use some subcommand for the worker. This would allow writing programs with complex command structures that don't have to invoke a bunch of Rpc_parallel logic and start RPC servers for every command.
Add the ability to get log messages from a worker sent back to the master. In fact, any worker can register for the log messages of any other workers.

sexplib

Switch code in lib subdir to ppx-style.

textutils

Switched to PPX.
Fixed a bug where the computation of cell heights could cause division by zero in some cases.
Expose the constructors of Ascii_table.Align.t so that we can write

Column.create ~align:Left ...

instead of

Column.create ~align:Align.left

typerep

Add whether record fields are mutable.

typerep_extended

Switched to ppx.
Kill the nonrec rewrite done by typerep. It is no longer needed since 4.02.2, we kept it only for compatibility with the camlp4 code.

113.00.00

async

Added Async.Std.Printf module so that one doesn't unintentionally use blocking Core.Std.Printf functions in an Async program.

There was much pre-existing code that did this via:

: open Core.Std : open Async.Std

Async.Std.Printf defines blocking functions (e.g printf, eprintf) to cause a type error, but leaves Async-friendly functions (e.g. sprintf, ksprintf) untouched.

Replaced uses of Printf.*, with Core.Std.Printf.* where needed.

async_extended

Added a more raw interface to Delimited.Csv.

async_extra

Added Limiter module.

Implements an async aware throttling rate limiter on top of Core.Limiter.
Generalized Persistent_rpc_client to supports RPC connection types with additional information besides the Rpc.Connection.t itself.

For instance: Persistent_rpc_client.Versioned has Versioned_rpc.Connection_with_menu.t as its connection type.
Changed the Persistent_rpc_client.Make functor to not erase the type conn from its output module's signature.

This way, the output of Make can be fed to functors or functions that expect a module matching Persistent_rpc_client.S.
Moved Log from Async_extra to Async_unix, so that the scheduler can refer to it.
Fixed a bug where Persistent_rpc_client.close would hang waiting for a connection to close.

async_inotify

Added modify_event_selector optional parameter to Async_inotify.create.

async_kernel

Switched Lazy_deferred to use Or_error.t rather than ('a, exn) Result.t.

Note: There is difference in the run argument between Monitor.try_with and Monitor.try_with_or_error. In this module, the function is called already in a closure inside a bind, so that difference is acceptable.
Made Deferred match Invariant.S1.
Improved Async_kernel.Scheduler.run_cycles_until_no_jobs_remain to use Timing_wheel.fire_past_alarms rather than sleeping.
Added Quickcheck module.
Reworked the initialization of Monitor.try_with_ignored_exn_handling to log exceptions using Async.Log so that it doesn't rely on top-level effects, which may not happen without packed libraries.

async_rpc_kernel

Fixed race in Rpc that caused double connection cleanup.

Two errors, Connection_closed and a Writer error, (Uncaught_exn(monitor.ml.Error_((exn(\"writer error\"....)))))), occurring at the same time will cleanup the connection twice and call response_handler of open_queries twice with two different errors.

(((pid 31291) (thread_id 0)) ((human_readable 2015-05-25T10:47:18+0100) (int63_ns_since_epoch 1432547238929886105)) "unhandled exception in Async scheduler" ("unhandled exception" ((monitor.ml.Error_ ((exn ("Ivar.fill of full ivar" (Full _) lib/async_kernel/src/ivar0.ml:329:14)) (backtrace ("Raised at file \"error.ml\", line 7, characters 21-29" "Called from file \"rpc.ml\", line 101, characters 8-31" "Called from file \"connection.ml\", line 251, characters 8-172" "Called from file \"core_hashtbl.ml\", line 244, characters 36-48" "Called from file \"connection.ml\", line 248, characters 2-278" "Called from file \"async_stream.ml\", line 49, characters 53-56" "Called from file \"async_stream.ml\", line 21, characters 34-39" "Called from file \"job_queue.ml\", line 124, characters 4-7" "")) (monitor (((name main) (here ()) (id 1) (has_seen_error true) (is_detached false) (kill_index 0)))))) ((pid 31291) (thread_id 0)))))
Fixed bugs in Rpc in which a TCP connection's reader was closed before its writer.
In Versioned_rpc, eliminated an unnecessary Async cycle when placing RPC messages.
Added Rpc.Pipe_rpc.close_reason and Rpc.State_rpc.close_reason, which give the reason why a pipe returned by an RPC was closed.

These functions take the IDs that are returned along with the pipes by the dispatch functions, so the interface of dispatch did not need to change.
Made Rpc.Expert.dispatch expose that the connection was closed, just like One_way.Expert.dispatch.
Expose the name of the Versioned_rpc.Menu RPC.

async_smtp

Improve the async_smtp client interface so that it is suitable as a replacement for Core_extended.Std.Sendmail.

async_ssl

Added Ssl.Connection.close.

async_unix

Made Async dump core when it reports a "bug in async scheduler".

There is no change for toplevel unhandled user exceptions, for which Async does not dump core.
Added Dump_core_on_job_delay.dump_core function, which exposes the core-dumping functionality in the C stubs for Dump_core_on_job_delay.
Made Dump_core_on_job_delay.How_to_dump an ordinary variant and moved it into Async_kernel.Config.
Changed Thread_safe_pipe functions that write to the pipe to take an additional argument, an If_closed.t, that says how to behave if the pipe is closed.

The previous behavior is achieved with ~if_closed:Raise. One can also now use ~if_closed:Return to they return a variant reporting whether the pipe was closed, rather than raising.

Returning a variant allows callers to distinguish the pipe-closed case from other errors. This change also allows us to do a a single acquisition of the Async lock, with the pipe-closed check synchronously immediately preceding the operation, avoiding a race.
Added Fd.with_file_descr_deferred_exn.
Improved the performance of Clock.every, and in particular reduced its allocation.

It now allocates much less, especially with ~continue_on_error:false.

Handled Clock.every's ~stop argument directly using timing-wheel alarms, rather than using Deferred.choose.

Slightly changed the behavior of
```
Clock.every' f ~continue_on_error:false
```
in the corner case where f raises but its result also becomes determined. Prior to this feature, iteration would stop. After this feature, iteration will continue, because ~continue_on_error:false just looks at the deferred resulting from f. This doesn't affect:
```
Clock.every f ~continue_on_error:false
```
because if f raises, then there is no resulting deferred.

Benchmark:

+----------------------------------------------------+----------+------------+----------+----------+------------+ | Name | Time/Run | mWd/Run | mjWd/Run | Prom/Run | Percentage | +----------------------------------------------------+----------+------------+----------+----------+------------+ | [clock_ns.ml:Clock.every] ~continue-on-error:false | 54.21us | 91.03w | 0.36w | 0.36w | 22.06% | | [clock_ns.ml:Clock.every] ~continue_on_error:true | 245.80us | 93_208.27w | 7.31w | 7.31w | 100.00% | +----------------------------------------------------+----------+------------+----------+----------+------------+
Added to Clock.Event.t type parameters so that one can record a value in the event when it happens or is aborted, and read that value via Event.status.

type ('a, 'h) t val status : ('a, 'h) t -> [ Aborted of 'a |Happened of 'h | Scheduled_at of Time.t ] val run_at : Time.t -> ('z -> 'h) -> 'z -> (_, 'h) t val abort : ('a, 'h) t -> 'a -> [Ok | Previously_aborted of 'a |Previously_happened of 'h ]
Fixed a (never observed) race in the Async scheduler's closing of file descriptors.

Previously, when the number of active system calls on an Fd.t reached zero, the scheduler would call a closure that would immediately schedule the close() system call in a thread. It was possible (albeit very unlikely) that that close() would run before the scheduler got a chance to update the epoll set, violating the invariant that close is only ever called on fds not in the epoll set.

Now, the scheduler enqueues an ordinary Async job to do the close, and thus the close cannot happen until the next cycle, after the scheduler has updated the epoll set.
Changed Reader to treat read returning EPIPE as end-of-file rather than fail, to deal with OpenOnload.

This fixes an issue where reading from a TCP connection can return an EPIPE if the tcp connection is immediately closed. This happens when the application is running with onload and when the tcp connection is closed immediately after creation.
Reduced allocation of the Async scheduler's File_descr_watcher, by using callbacks to handle ready file descriptors.
Fixed In_thread.Helper_thread.create's error message if there are no available threads in the thread pool.

The error message is now constructed eagerly. It had been constructed lazily, so by the time it was rendered, the state might have changed, possibly making threads available. This leads to a nonsensical-looking error message that claims that there are no available threads, immediately followed by a list of available threads.
Moved Log from Async_extra to Async_unix, so that the scheduler can refer to it.
When Writer.with_file_atomic is unable to clean up its temp file, raise synchronously rather than asynchronously.

This eliminates complaints about an exception being thrown after a deferred has been computed.
Added Log.rotate to force log rotation.

val rotate : t -> unit Deferred.t.
Fixed Log rotation to correctly reset the size and number of lines.

bignum

Fixed a bug in the =Zarith= library's to_float function.

These fixes first introduce tests from the base distribution, and then backport a bugfix to the handling of to_float.

bin_prot

Switched build to use =config.h= rather than the command-line for preprocessor variables.
Switched from ARCH_SIXTYFOUR to JSC_ARCH_SIXTYFOUR.
Fixed to support 32-bit integers, which are used in js_of_ocaml.

Do not make too many assumptions on integer size. Integers are 32bit in Javascript.

Do not use the "get_float_offset" hack on 32bit as it cannot be implemented in javascript.

core_extended

Added to Interval_map a more complete set of operations.
Removed Core_extended.Sexp.filter_record, which has been superseded by Core.Std.Sexp.of_sexp_allow_extra_fields.
Added to Interval_map an Interval module, with the type of intervals used in an interval map.
In Color_print, added sprintf functions, and changed formatting to compose properly.

core_kernel

Added Float.int63_round_nearest_exn.

val int63_round_nearest_exn : t -> Core_int63.
Changed Hashtbl.sexp_of_t so that keys are sorted in increasing order.

This also applies to the sexp_of_t produced by Hashtbl.Make and Make_binable. Sorting by key is nice when looking at output, as well as in tests, so that the output is deterministic and so that diffs are minimized when output changes.
Added to Info, Error, and Or_error a Stable.V2 module, whose bin_io is the same as the unstable bin_io.
Replaced Map.prev_key and next_key with closest_key.

val closest_key : ('k, 'v, 'cmp) t -> [ Greater_or_equal_to |Greater_than | Less_or_equal_to |Less_than ] -> 'k -> ('k * 'v) option
Shared code between Monad.Make{,2} and Applicative.Make{,2}.
Added tests to make sure round_nearest and int63_round_nearest_exn don't allocate.
Added Lazy.T_unforcing module, with a custom sexp_of_t that doesn't force.

This serializer does not support round tripping, i.e. t_of_sexp. It is intended to be used in debug code or <:sexp_of< >> statements. E.g:
```
type t =
  { x : int Lazy.T_unforcing.t
  ; y : string
  }
with sexp_of
```
Extended Map.to_sequence and Set.to_sequence to take any combination of upper bound, lower bound, and direction.
Added Map.split.
Added Timing_wheel.fire_past_alarms, which fires alarms in the current time interval's bucket whose time is in the past.
Added a Total_map module, for maps where every value of the key type is present in the map.
Added Bigstring.compare and Bigstring.equal.
Split monad.ml into three files: monad.ml, monad.mli, and monad_intf.ml.
Removed the last remaining dependence of Core_kernel on Unix, moving Time_ns.pause functions to Core.
Added optional arguments to Hash_queue.create, ?growth_allowed and size, which then get passed to Hashtbl.create.
Added a ?strict:unit argument to functions that ordinarily create lazy sexps, like failwiths.
```
Info.create
Error.create
Error.failwiths
Error.failwithp
Or_error.error
```
This makes it easy to force a use to be strict, which is sometimes useful to accurately capture the state of a mutable data structure at the time the error happens, lest it change by the time the error is rendered.
Removed Interned_string module.
In Pooled_hashtbl, avoid trying to create arrays bigger than Sys.max_array_length.

The problem affected 32-bit platforms.
Added Quickcheck module.

Supports automated testing with randomly-generated inputs in the style of Haskell's Quickcheck library. Our adaptation supports flexible probability distributions for values of a given type and uniqueness guarantees for generated values.
Made Set.to_sequence and Set.split have the same interface as Map.to_sequence and Map.split, respectively.
Fixed Float and Timing_wheel to compile on 32-bit platforms.
Added Lazy.Stable.V1.
Added List.reduce_balanced, which is like reduce, but relies on associativity of f to make nesting of calls to f logarithmic rather than linear in the input list length.
Added String_id.Make_without_pretty_printer.
Restricted Time_ns.Span values to be less than 135 years, which ensures the corresponding float Time.Span values have microsecond precision.

Fixed a Time_ns test that recently started failing due to crossing the 135-year boundary.

Reducing the range of Time_ns.Span required adjusting the implementation of Core.Time_ns.Option.Stable.V1, which (accidentally, incorrectly) incorporated the (unstabilized) Core_kernel.Time_ns.Span.min_value as the representation of bid_none and .max_value as ask_none. The prior representation is preserved, but some previously allowed values are no longer allowed and now raise exceptions!
Added Rope module, the standard data structure for efficient string manipulation.
Added Sequence.unfold_with_and_finish, a variant of unfold_with that can continue the sequence after the inner sequence finishes.
Replaced Sequence.cycle with Sequence.cycle_list_exn, to work around a bug in Sequence.cycle raising on the empty sequence.

Sequence.cycle can cause an infinite loop if its input is empty. It is problematic to check whether the input sequence is empty.
```
* If we check it eagerly, we have to turn `cycle` into
  `cycle_eagerly_exn`, and it will evaluate the first element twice.

* If we check it lazily, we might raise an exception in a seemingly
  unrelated part of the code, and the usually-good habit of wrapping a
  function like `cycle_exn` in `try .. with ..`  would not catch it.
```
To get around these issues, [cycle] is changed to accept only lists as inputs, not sequences. It is now called [cycle_list_exn].
Fixed assumptions about the size of integers, to support compiling to Javascript, where integers are 32-bit.
Fixed build on Mac OSX.

Fix build when LINUX_EXT or TIMERFD are undefined.
Added Caml.Bytes.

Add an alias for Bytes in Caml. Fixes janestreet/core_kernel#46.
In Container, exposed polymorphic functions individually building container functions using fold or iter.

Exposed polymorphic functions in Core_kernel.Container for individually building each of the Container functions using fold or iter. E.g.:

type ('t, 'elt, 'accum) fold = 't -> init:'accum -> f:('accum -> 'elt -> 'accum) -> 'accum

type ('t, 'elt) iter = 't -> f:('elt -> unit) -> unit

val length : fold:('t, _, int ) fold -> 't -> int val exists : iter:('t, 'a) iter -> 't -> f:('a -> bool) -> bool
Added container.mli, which was sorely missing.
Added Doubly_linked.to_sequence.
Added Hash_queue.sexp_of_t.

core_profiler

Changed delta timers and probes so they record the total amount of time/value change between each start and pause.

custom_printf

Upgraded from OCaml 4.02.1 to 4.02.2

email_message

Extended and improved Email_message API.

fieldslib

Added Fields.Direct.set_all_mutable_fields, a function intended to guarantee when pooling records that one cannot forget to reinitialize some fields.

Obviously one could achieve this through something like Fields.Direct.iter, but we want a more efficient version that doesn't force the call side to create closures.

jenga

Treat output to stderr by an action as a failure.

ocaml_plugin

Made Ocaml_plugin.Plugin_cache.Config.t stable.

pa_bench

Add -pa-bench-drop-with-deadcode option.

pa_ounit

Add -pa-ounit-drop-with-deadcode option

pa_structural_sexp

Added support for records where some fields are optionally displayed.

re2

Improved Re2.find_submatches on big patterns with many submatches unmatched, e.g. =(ABC)|(DEF)|(GHI)|(KLM)|...=.

Without the fix:

+-------------------------------------------------------+--------------+------------+----------+----------+------------+ | Name | Time/Run | mWd/Run | mjWd/Run | Prom/Run | Percentage | +-------------------------------------------------------+--------------+------------+----------+----------+------------+ | [re2_internal.ml] find_submatches with many Nones:5 | 406.81ns | 30.00w | | | 0.08% | | [re2_internal.ml] find_submatches with many Nones:10 | 2_385.11ns | 207.00w | | | 0.47% | | [re2_internal.ml] find_submatches with many Nones:50 | 12_772.97ns | 2_072.00w | 0.33w | 0.33w | 2.53% | | [re2_internal.ml] find_submatches with many Nones:100 | 43_196.95ns | 7_191.00w | 2.03w | 2.03w | 8.56% | | [re2_internal.ml] find_submatches with many Nones:200 | 504_884.95ns | 29_316.00w | 16.05w | 16.05w | 100.00% | +-------------------------------------------------------+--------------+------------+----------+----------+------------+

With it:

+-------------------------------------------------------+--------------+-----------+----------+----------+------------+ | Name | Time/Run | mWd/Run | mjWd/Run | Prom/Run | Percentage | +-------------------------------------------------------+--------------+-----------+----------+----------+------------+ | [re2_internal.ml] find_submatches with many Nones:5 | 408.24ns | 30.00w | | | 0.12% | | [re2_internal.ml] find_submatches with many Nones:10 | 1_607.67ns | 163.00w | | | 0.48% | | [re2_internal.ml] find_submatches with many Nones:50 | 3_223.89ns | 563.00w | | | 0.96% | | [re2_internal.ml] find_submatches with many Nones:100 | 5_288.09ns | 1_063.00w | 0.20w | 0.20w | 1.58% | | [re2_internal.ml] find_submatches with many Nones:200 | 334_107.81ns | 2_063.00w | 0.79w | 0.79w | 100.00% | +-------------------------------------------------------+--------------+-----------+----------+----------+------------+
Fixed build on FreeBSD.

Excise direct mention of g++ from re2 Makefile, preferring the inbuilt CXX macro. This fixes the build on FreeBSD (yes, really).
Added an applicative interface to building/using regular expressions.
Made Re2 depend only on Core_kernel, not Core.

Fixes janestreet/re2#6

rpc_parallel

Fixed a file-descriptor leak

There was a file descriptor leak when killing workers. Their stdin, stdout, and stderr remain open. We now close them after the worker process exits.

112.35.00

async

Include some previously-omitted benchmarks

async_extended

Added Ltl module, an implementation of linear temporal logic, which can be used to run online queries on sequences of states.
Added Interactive.Job, for printing start/done messages for multiple simultaneous jobs.
Made Any_error be Applicative.
Added Command_rpc support for Versioned_rpc.Both_convert.Plain.

async_extra

Added to Log a better mechanism for catching and handling background errors, via set_on_error and an on_error argument to create.
Added Log.get_output : t -> Output.t list.
Changed Monitor.try_with so that errors after the initial return are written to the global error log, rather than ignored.
Added Monitor.try_with_or_error and try_with_join_or_error.

try_with_or_error is intended to someday be renamed as try_with. It also omits some of try_with's optional arguments: run and rest. Different from try_with, try_with_or_error uses ~run:`Now, which we now believe is a more sensible behavior.
Fixed a bug in Versioned_typed_tcp that causes spurious and repeated reconnects when user-level code disconnects.
Added Tcp.Server.create_sock, to create TCP servers that don't use Reader and Writer.
Changed Log.Level.arg to accept lowercase, uppercase, and capitalized words.

Replaced Unpack_sequence.unpack* functions with unpack_into_pipe and unpack_iter, for reduced allocation.

  module Unpack_from : sig
    type t =
    | Pipe   of string Pipe.Reader.t
    | Reader of Reader.t
  end

  val unpack_into_pipe
    :  from  : Unpack_from.t
    -> using : ('a, 'b) Unpack_buffer.t
    -> 'a Pipe.Reader.t * ('a, 'b) Unpack_result.t Deferred.t

  val unpack_iter
    :  from  : Unpack_from.t
    -> using : ('a, 'b) Unpack_buffer.t
    -> f     : ('a -> unit)
    -> ('a, 'b) Unpack_iter_result.t Deferred.t

Added to Log support for user-defined rotation schemes.
Added Log.is_closed.
Moved Async_extra.Rpc to its own library, Async_kernel_rpc, and abstracted its transport layer.

Async_kernel_rpc depends only on Async_kernel. This allows Async_rpc to be used in javascript or to try transports tuned for different use cases. Versioned_rpc was moved to Async_rpc_kernel as well.

Async_extra still provides an Rpc module with the Unix-dependent part:
- the Rpc.Transport module is augmented with Async_unix.{Reader,Writer} based transports
- the Rpc.Connection module is augmented with helpers for TCP based connections
In sexp-formatted Log messages, output the sexp on a single line rather than in multi-line "hum"an format.

This makes it possible to, among other things, easily grep such logs.
Fixed a (very small) space leak in Persistent_rpc_client.

The fix was to use Deferred.choose and Deferred.choice instead of Deferred.any and >>|. The old implementation added a callback to the t.close_started ivar every time the connection transitioned from connected to disconnected.
Added Persistent_rpc_client.create_generic, which is like create, but generic in the function used to connect.
Fixed a race condition in the Versioned_typed_tcp interface that caused a worker to miss a Connect message if the box is under high load.

Query_client.create is called from Worker_impl.create in a different async cycle than the following call to Query_client.listen (really, Tail.collect under the hood) which is made from Worker_impl.run.

When the load on the box is heavy (many workers starting and connecting at the same time), the OS might take away the CPU from the worker process between the two async cycles. The TCP socket gets connected while the process is still waiting for its turn, and eventually, when it's the worker's turn to grab the CPU, Async scheduler might process the TCP event earlier than Worker_impl.run.
Improved Udp.ready_iter to avoid intermediate exceptions by using Syscall_result.

UDP loops use that, so will benefit.

Adjust the implementation slightly as well: made the inner loop always exit on EAGAIN/EWOULDBLOCK to wait until ready, and give other Async jobs a chance to run after EAGAIN/EWOULDBLOCK in the outer loop.

async_kernel

Added Clock.Event.run_at and run_after.
Eliminated a space leak in Clock.with_timeout.

Previously Clock.with_timeout span d created a deferred that waited for span even if d (and hence with_timeout) became determined sooner. Now, with_timeout aborts the clock alarm if d becomes determined, which saves space in Async's timing wheel.
Added Clock.Event.fired, and removed the fired value that was returned by at and after.
```
  val fired : t -> [ `Happened | `Aborted ] Deferred.t
```
Added Clock.Event.reschedule_{at,after}.

Fixed the space leak that caused nested Monitor.try_with to use linear space rather than constant space.

Changed Monitor.try_with_ignored_exn_handling so that with Eprintf` or Run f, the error processing runs inMonitor.mainrather than in the monitor that calledMonitor.try_with`. This avoids space leaks due to chains of monitors, e.g.:

  open! Core.Std
  open! Async.Std

  let () =
    Monitor.try_with_ignored_exn_handling := `Run ignore;
    let num_monitors = 10_000_000 in
    let num_remaining = ref num_monitors in
    let rec loop n : unit =
      if n > 0
      then
        upon
          (Monitor.try_with
            (fun () ->
                loop (n - 1);
                return ()))
          (function
            | Error _ -> assert false
            | Ok () ->
              decr num_remaining;
              if !num_remaining = 0 then shutdown 0)
    in
    loop num_monitors;
    never_returns (Scheduler.go ());
  ;;

Added a unit test to detect if nested monitors leak space.

Removed Lazy_deferred.follow, which is not used in the tree anymore.

Removing this function allows Lazy_deferred.t to be implemented as:
```
type 'a t = 'a Deferred.t Lazy.t
```
although that's not done yet.
Added hooks to Async_kernel.Scheduler for js_of_ocaml.

This hook aims to be called every time one add a job to the scheduler (enqueue + timing_wheel).
Made Async_kernel not depend on thread.
Added Deferred.Memo, which wraps functions in Core.Memo to correctly handle asynchronous exceptions.
Renamed Pipe and Thread_safe_pipe functions that clear their input queue so that their name starts with transfer_in, to make it clearer that they side effect their input.

| old name | new name | | ------------------------- | ------------------------------ | | write' | transfer_in | | write_without_pushback' | transfer_in_without_pushback |

Added Pipe.init_reader, symmetric to Pipe.init.

val init        : ('a Writer.t -> unit Deferred.t) -> 'a Reader.t
val init_reader : ('a Reader.t -> unit Deferred.t) -> 'a Writer.t

Changed Async_kernel.Job_queue.run_jobs to call Exn.backtrace immediately after catching an unhandled exception

There should be no change in behavior. This change was to make it more clear that there is no intervening code that interferes with the global backtrace state.
Made Deferred functions that take an argument ?how:[Parallel | Sequential ] accept `Max_concurrent_jobs of int, which operates in a sequence in parallel, limited via a throttle.
Made Deferred.Or_error match Applicative.S.
Fixed Scheduler.run_cycles_until_no_jobs_remain so that it continues running if one has done Scheduler.yield.
Split the implementation of Deferred into a number of files, to solve some problems with circularities.

Split into:
- deferred.ml
- deferred_sequence.ml
- deferred_list.ml
- deferred_array.ml
- deferred_queue.ml
- deferred_map.ml
- deferred_result.ml
- deferred_option.ml
  
  For a sequence of multiple modules used to construct a module, switched from the Raw_* prefix convention to the numeric suffix convention. E.g. we now have Deferred0, Deferred1, Deferred.

async_parallel

Renamed Async_parallel as Async_parallel_deprecated; one should use Rpc_parallel instead.

async_rpc_kernel

Moved Async_extra.Rpc to its own library, Async_kernel_rpc, and abstracted its transport layer.

Async_kernel_rpc depends only on Async_kernel. This allows Async_rpc to be used in javascript or to try transports tuned for different use cases. Versioned_rpc was moved to Async_rpc_kernel as well.
Added Rpc.One_way module, for RPCs that do not expect any response or acknowledgment.
Sped up Rpc.Connection.

We have been able to send 6_000_000 (contentless) one-way messages per second under some idealized circumstances.
In Rpc, added an optional heartbeat_config argument to configure the heartbeat interval and timeout.
In Rpc, added some information to Connection_closed exceptions.
In Rpc.Implementations.create's on_unknown_rpc argument, added a connection_state argument to the `Call variant.

When using Rpc.Connection.serve, one can put client addresses in the connection_state, so this makes it possible to identify who sent the unknown RPC instead of just saying what the RPC's name and version are.
Added Rpc.One_way.Expert, which has an implement function that gives direct access to the internal buffer instead of using a bin-prot reader.
Made Rpc not raise an exception if a connection is closed due to heartbeating.
Reworked Async_rpc_kernel's Transport implementation to not require a transfer function similar to Async_unix.Writer.transfer.

Changed the RPC connection to flush the pipe when the writer is closed, which was the only behavior of Async_unix.Writer.transfer that had been relied on.

With this change, if someone closes the underlying writer by hand the pipes won't be flushed, which should be expected anyway.
Fixed an issue where "normal" Pipe_rpc errors caused the connection to shutdown.

Such errors include querying an unknown RPC or getting an exception raised by the RPC implementation shutdown. Now such errors behave like Rpc errors, i.e. they are completely ignored besides being put in the return value of Pipe_rpc.dispatch.

Errors that occur later in Pipe_rpc still cause the connection to close, since these should only occur if there is a bug somewhere.
In Rpc, connection-closing errors no longer raise top-level exceptions.

One can now call close_reason : t -> Info.t Deferred.t to get the reason the connection was closed.
Added One_way rpcs to Versioned_rpc.
Fixed Rpc to not write if the transport is closed.

The fix doesn't check that the writer is closed, but instead ensure that we don't try to write when we shouldn't. This means that it will still fail if the user close the transport by hand, which they shouldn't do.

This change requires transport implementations to fail after they have been closed. This is different from the semantics of Async_unix.Writer, but the latter is non-intuitive and makes it hard to check the correctness of the code. Moreover it is only required for Async_unix.Writer.with_flushed_at_close which we don't use.

async_ssl

Fix github issue #4 (some comments swapped).

async_unix

Made Unix.File_kind.t be Comparable, so it can be used in <:test_result< >>.
Reduced allocation in Async's scheduler in the common path.

The allocation was in Raw_scheduler.be_the_scheduler.compute_timeout, which was (statistically, based on perf) the largest single allocator in one of our applications. Now, it way down the list.

Note that the application is not a typical Async app in that it does not sit in epoll very much, due to the way we do low-latency I/O. This change will benefit everyone, but only a tiny bit.
Added Writer.write_bin_prot_no_size_header.

This is needed for Async RPC as it writes a different size on its own.
Fixed a bug in Writer.transfer, which didn't close the pipe when the consumer leaves.

Simplified the implementation of Writer.transfer:
- replaced the big loop by a simple iteration function on the pipe that just stop without filling its ivar when it sees the iteration should stop for other reason that `Eof on the pipe: writer closed, consumer left or stop requested by the user.
- replaced the various choose by a single one and deal with the closing reason only at this point.
Added Writer.close_started, symmetric to Writer.close_finished.

bignum

Upgraded from Zarith 1.2 to 1.3.
Removed dependence on Big_int.

bin_prot

Sped up bin_io of float array.

Bin_prot already had special fast handling for float array's but with bin_io did not use it except for the special type float_array. Now, there is fast handling for float array and its aliases, for example price array when type price = float.

Changed Size.bin_size_array, Write.bin_write_array and Read.bin_read_array short circuit to the fast path when it detects that float array is handled. Each of these functions receives a function for handling array elements and short circuits when the function for handling elements is equal to the function for handling floats, using physical equality of closures.
To cause short circuiting for aliases of float, changed bin_io so that aliased bin_io functions are equal the the bin_io functions of the original type. That is an optimization for itself regardless whether it's used for float. Before this change, every function generated for aliases were eta-expanded leading to different closures at runtime for each type.

Short circuiting needs to apply to the handling function rather than to the value at hand because:

the value is available only in size and write, and we need a way to make read work as well.
even when the value is a float at runtime, the handling of a specific float alias may have been overridden by a custom one.

Made a slight improvement to bin_read_float_array: since the array is going to be filled with read values, there is no need to fill it with 0. after allocation:

    let next = pos + size in
    check_next buf next;
-|  let arr = Array.create len 0. in
+|  let arr = Array.make_float len in
    unsafe_blit_buf_float_array buf arr ~src_pos:pos ~dst_pos:0 ~len;
    pos_ref := next;

The difference in speed when optimal and non optimal way of handling floats is used:

Name	Time/Run	mWd/Run	mjWd/Run
[bench.ml:float array] size non optimal	3_403.80ns	2_000.00w
[bench.ml:float array] size float_array	5.55ns
[bench.ml:float array] size Price.t array	6.18ns
[bench.ml:float array] write non optimal	7_839.89ns	2_000.00w
[bench.ml:float array] write float_array	292.42ns
[bench.ml:float array] write Price.t array	293.16ns
[bench.ml:float array] read non optimal	9_665.06ns	2_002.00w	1.00kw
[bench.ml:float array] read float_array	461.01ns	2.00w	1.00kw
[bench.ml:float array] read Price.t array	449.43ns	2.00w	1.00kw

There is no observed speed penalty for runtime check for short circuiting. The following benchmark shows the speed of handling int array without and with the check:

Name	Time/Run	mWd/Run	mjWd/Run
[bench.ml:float array] int array size	3_910.64ns
[bench.ml:float array] int array write	6_548.40ns
[bench.ml:float array] int array read	14_928.11ns	2.00w	1.00kw

Name	Time/Run	mWd/Run	mjWd/Run
[bench.ml:float array] int array size	3_906.86ns
[bench.ml:float array] int array write	5_874.63ns
[bench.ml:float array] int array read	14_225.06ns	2.00w	1.00kw

core

Tweaked Unix.stat's C code to reduce float rounding error, by using double-precision rather than single-precision floats.

Following Xavier's comment: http://caml.inria.fr/mantis/view.php?id=6285
Added Date.O module.
In Interval, exposed compare in stable types by having the appropriate modules match the Stable signature.
Added Iobuf.Fill.decimal and Poke.decimal, for efficiently writing integers in decimal format.
Removed Filename.O (and /^), since it's not used that much, and is inconsistent with the older operator for this that's exposed in Core.Std: ^/.
Improved Command autocompletion to work even if some arguments can't be parsed.

This is useful because the completion mode does not get fed precisely the same arguments that it would get if you hit RETURN.

As a simple example, if completion is set up for my-exe which takes a sexp as its first argument, then:
```
  my-exe '(a b)' <TAB>
```
will run my-exe in completion mode with:
```
  '(a b)'
```
as its first argument, rather than:
```
  (a b)
```
as would be passed if RETURN had been pressed instead. Since Sexp.of_string "'(a b)'" fails, in this example tab completion won't work.

Changed the internals a bit to make this possible. Most notably, the Parser module is now applicative rather than monadic, which required only a few simple changes to support.
Made Time_ns.to_time and Time_ns.Span.to_span round to the nearest microsecond in all cases.

Previously, Time_ns.Span.to_span sometimes rounded incorrectly for negative spans.
Added Time.Zone.prev_clock_shift, the analog of next_clock_shift.
```
val prev_clock_shift
  :  t
  -> before:Time_internal.T.t
  -> (Time_internal.T.t * Span.t) option
```
Implemented next_clock_shift and prev_clock_shift using Array.binary_search_segmented.
Added Lock_file.get_pid : string -> Pid.t option.
Added val random: unit -> int to Time_ns and Time_ns.Span.
Renamed Iobuf.sub as Iobuf.sub_shared.

This more closely matches Bigstring, and clarifies the semantics of Iobuf.sub vs sub in Blit_intf.S.
Added Iobuf blit modules: Blit, Blit_consume, Blit_fill, Blit_consume_and_fill.

Added Piecewise_linear.first_knot and last_knot.

val first_knot : t -> (key * value) option
val last_knot  : t -> (key * value) option

Made Unix.Cidr match Comparable.S_binable, and added Cidr.create and Cidr.netmask_of_bits.
Moved Unix.tm and Unix.strftime from Core_kernel to Core.
Made Crc.crc32 return Int63.t rather than int64, and added Crc.bigstring_crc32 and Iobuf.crc32.

Cleaned up old cruft in the C stubs for CRC checking.
Added Iobuf.narrow_lo and narrow_hi, which comprise Iobuf.narrow.
Changed Linux_ext.Timerfd, Epoll.wait, and Unix.select to use (int) Time_ns rather than (float) Time.

This avoids spurious float conversions and rounding problems.

Made all timeouts consistently treat negative timeouts as "timeout immediately".

This fixes an incorrect behavior of Linux_ext.Timerfd.set_after and set, which had been rounding to the nearest microsecond, which was particularly bad for time spans smaller than 500ns, which would be rounded to zero, and then would cause the timerfd to never fire. Now, the small span is directly fed to timerfd_settime. We also changed a span of zero to be treated as 1ns, to avoid the behavior of timerfd_settime 0, which causes the timerfd to be cleared and never fire.
Made Or_error match Applicative.S.
Added Command.Param, with the intention of one day replacing Command.Spec and providing an applicative interface for command-line parsing.

This change required lots of rearrangement of command.mli so that Command.Param and Command.Spec could share large portions of their interface. As a side effect, the interface is more sweeksy than before.
Added Command.shape, for exposing the shape of a command, including what subcommands its has.
Changed Syscall_result.to_result to return a preallocated object, and changed many uses to take advantage of this property.

Pattern matching on results is much clearer than if-analysis and avoids double-checking errors in many cases.

Syscall_result.to_result can only return preallocated results for a few Ok values from Syscall_result.Int, of course, and likewise for other large ok_value types. We have initially, and arbitrarily, limited preallocation to 64 errno's and 2048 ok_value's.
Added Sys.big_endian : bool, from Caml.Sys.
Disabled unit tests in Time_ns that started failing around 10:40pm NYC 2015-05-15.

The tests indicate an off-by-one-microsecond error in round tripping between Time.Span.t and Time_ns.Span.t.

core_bench

Exposed the equality of Core_bench.Std.Bench.Test.t with Core_bench.Test.t, so that one can get the name of a test.

This is useful for filtering based on test name.

core_extended

Removed the Stats_reporting module.
Renamed Quickcheck module to Quickcheck_deprecated. It's replaced by Janecheck, which for now is a separate library in the core_extended package, but will soon be merged into core.
Moved the Selector module to its own library. This is for internal reasons related for code review; it is included as a library within the core_extended package for now, but may move to another home in the future.
Added Extended_unix.terminal_width : int Lazy.t.
Added Interval_map module.
Added to Sendmail.send additional optional arguments: ?message_id:string, ?in_reply_to:string.

core_kernel

Added an Applicative interface to Core (a.k.a. idioms or applicative functors)
Generalized the signature of Hashtbl.merge_into to allow the types of src and dst to be different.
Made Day_of_week.of_string accept additional formats (integers 0-6, full day names).
Added Day_of_week.to_string_long, which produces the full day name.
Changed Hashtbl.add_exn to not create a new exception constructor when it raises due to a duplicate key.
Added Map.nth, which returns the nth element of a map, ordered by key rank.
Added Binable.Of_binable functors, similar to Sexpable.Of_sexpable

One should use Binable.Of_binable rather than the functionally equivalent Bin_prot.Utils.Make_binable.
Added Either module, with type ('a, 'b) t = First of 'a | Second of 'b.
Added to Univ_map a functor that creates a new Univ_map type in which the type of data is a function of the key's type, with the type function specified by the functor's argument.

Normally, a Univ_map.t stores ('a Key.t * 'a) pairs. This feature lets it store ('a Key.t * 'a Data.t) pairs for a given ('a Data.t).
Made Day_of_week.Stable be Comparable and Hashable.
Fixed a couple Exn unit tests that mistakenly relied on the global setting of Printexc.get_backtrace.

Now the tests locally set it to what they need.

This avoids unit-test failures when running with no OCAMLRUNPARAM set:
```
  File "exn.ml", line 130, characters 2-258: clear_backtrace threw "Assert_failure exn.ml:133:4".
      in TEST_MODULE at file "exn.ml", line 127, characters 0-1057
```
Renamed Monad.ignore as Monad.ignore_m, while preserving ignore = ignore_m in existing modules (e.g. Deferred) that used it.

We can later consider those modules on a case-by-case basis to see whether we want to remove ignore.
Added Set.symmetric_diff.
Added Timing_wheel.reschedule, which reschedules an existing alarm.
Added Applicative.S2, analogous to Monad.S2.
Added combinators to Either.
Added Hashtbl.add_or_error and create_with_key_or_error, which use Or_error and are more idiomatic ways of signalling duplicates.
Added Sexpable.Of_sexpable1 functor, for one-parameter type constructors.
Made Timing_wheel_ns keys be Int63.t rather than int, so that behavior is consistent on 32-bit and 64-bit machines.

Also, made Timing_wheel.Interval_num an abstract type.

Hid the bytes type in Core.Std, so that type errors refer to string rather than bytes.

Added Bytes module so that people can say Bytes.t if they need to.

Now we get reasonable error messages:

  String.length 13
  -->
  Error: This expression has type int but an expression was expected of type
          string

  "" + 13
  -->
  Error: This expression has type string but an expression was expected of type
          int

Modernized the coding style in Timing_wheel.
Replaced Unpack_buffer.unpack with unpack_into and unpack_iter, to avoid allocation.

Unpack_buffer.unpack created a (vector-backed) Core.Std.Queue for each call. When unpacking a buffer containing many values, resizing of the buffer can be costly and in some cases leads to promotions of short-lived data to the major heap.

The new functions avoid allocating the queue:
```
  val unpack_into : ('value, _) t -> 'value Queue.t     -> unit Or_error.t
  val unpack_iter : ('value, _) t -> f:('value -> unit) -> unit Or_error.t
```
Cleaned up the implementation of Gc.tune.
Change Unit implementation to use Identifiable.Make instead of applying functors separately.
Added val random: unit -> int to Int63.
Reworked Float.iround_*_exn functions to not allocate in the common case.
Added Fqueue.singleton and Fdeque.singleton.
Moved Unix.tm and Unix.strftime from Core_kernel to Core.

Added external time formatting:
```
float (* seconds *)-> string (* format *) -> string = "..."
```
Made String_id.Make call Pretty_printer.Register.
Changed String_id to allow the pipe character in identifiers.
Made List.compare have the usual type from with compare, val compare : ('a -> 'a -> int) -> 'a t -> 'a t -> int.

Previously, List.compare's type was:
```
  val compare : 'a t -> 'a t -> cmp:('a -> 'a -> int) -> int
```
Made stable Map's and Set's conform to the Stable1 interface.
Reworked Hashtbl.find_exn to not allocate.

Previously, Hashtbl.find_exn allocated because it called Hashtbl.find, which allocates an option (partially because Avltree allocates options in its find function).

jenga

Make .jenga.db be format version aware, and store .jenga files in a subdirectory.
Switch md5 computation to use a C binding, fixes #10.
Move jem.exe to jenga.exe monitor, jenga_offline.exe to jenga.exe offline, jenga.exe -cat-api to jenga.exe cat-api.
Remove Path.dotdot.
Adjust behaviour of Path.Repo.is_descendant to be more consistent.
Repo-root invariance: remove Path.Abs.the_root, Path.Rel.to_absolute_string, Path.Rel.create_from_absolute, and make Path.relative Path.the_root ".." fail.
Add a flag -sandbox-action to jenga build to run the action in an environment that (attempts to) detect missing dependencies or overly large sets of targets.
Don't interpret paths of shape ./foo.ext as aliases.
Always interpret Dep.path path as the file at path rather than the default alias for the directory at path
Make jenga look for the variable JENGA_OPTIONS, and use it to add a debug setting.
Fix handling of directories created during the build.
Basic support for symlink resolution.
Add an option to the monitor command to only display a single snapshot of the progress, rather than a continually updated pipe.
Fix a deadlock where File_access throttle and directory lock are obtained in the wrong order.

ocaml_plugin

In copy_source_files_to_working_dir, exclude files that start with a dot.

emacs creates temporary files that cannot be read with names like .#foo.ml, and attempting to copy those causes this function to fail.

pa_ounit

Made Pa_ounit_runtime not depend on OUnit anymore.

The dependency hasn't been needed in more than 2 years.

re2

Fixed a bug in Re2.find_all_exn, extant since 2014-01-23, in which it returns spurious extra matches.

Using pattern b and input aaaaaaaaaaaab is expected to return a single match at the end of the input but instead returned the match multiple times, approximately as many times as input length / min(match length, 1).

Added tests for this function and also get_matches which uses the same code.
Updated to new version of upstream library.

sexplib

Inline some calls that js_of_ocaml was unable to recognise as tail-recursive (cf. issue #14)

112.24.00

async

Keep up to date with interface changes in Async_kernel, Async_extra and Async_unix.

async_extended

Fixed misspelling in Command_rpc.Connection, renaming propogate_stderr as propagate_stderr.

async_extra

Changed Log to not eagerly run the rotation loop when an Output.Rotating_file is created.
Changed Log.Output.combine to write log outputs in sequence rather than parallel, to give the correct semantics when the same output is included multiple times in Log.create.

This fixed a test that was failing in lib_test/log_test.ml.
Remove Log.Rotation.t_of_sexp.
Made Command.async* functions flush stdout and stderr before calling shutdown, to avoid timeouts causing data to be dropped.

For now, we're making this change in Command rather than Writer. Writer already has at_shutdown handlers. We've observed that they don't behave well for command-line programs w.r.t. stderr. So, the thinking of this feature is to try out a different at_shutdown behavior, just for Command executables and just for stdout and stderr. If it works out, maybe we move it into Writer proper. Putting the change in Command for now reduces the scope of what is affected by the experiment, and hopefully correlates well with where the change is likely to help.
In Rpc, catch exceptions raised by blocking-RPC implementations.

Added functionality to Versioned_typed_tcp.Repeater.

Added to create an argument is_client_allowed : Client_name.t -> bool.

Added to start an argument:

on_connecting_error  : (client_name    : Client_name.t
                        -> server_name : Server_name.t
                        -> Error.t -> unit)

Fixed a race in Versioned_typed_tcp in which a message can be dropped between Server.create and Server.listen
Simplified the implementation of Rpc.

Rpc has an internal Response_handler module, which is just a record containing a response-handling function and an already_removed boolean field. It turns out that this is unnecessary: already_removed is set to true when the function returns remove`, but if it returnsremove` then it will also be removed from a hash table, and we only call the function immediately after looking it up in that hash table.

This wasn't always pointless: this function used to return deferred values and run inside a throttle. The simplification is only possible because we made it synchronous a while ago.
Added Tcp.Server.num_connections function.
Added creation functions for Versioned_rpc menus, for use in clients of an RPC proxy.

In Menu:
```
val create : Implementation.Description.t list -> t
```
In Connection_with_menu: val create_directly : Connection.t -> Menu.t -> t

These are for use in clients of an RPC proxy, which can't use the regular menu mechanism since they each need to have many menus (one for each potential target) but should only need to have one connection (to the proxy).
Added to Rpc expert submodules, Implementations.Expert and Rpc.Expert, with low-level access for implementing a proxy that can handle queries without knowing their names, types, etc. in advance.
Renamed Rpc.Implementation.Description as Rpc.Description.
Added Rpc.{Rpc,Pipe_rpc,State_rpc}.description accessor functions.
Added Rpc.Implementation.descriptions, which returns all RPCs in an Rpc.Implementations.t. This was needed for the rpc_discovery library: given an Implementations.t we want to advertise at
```
prefix/<rpc_name>/<rpc_version>/host_and_port = <host_and_port>
```
Added combinators to Rpc.Implementations: lift, add, add_exn.

async_kernel

Now depends on Core_kernel instead of Core.

Async_kernel.Clock uses Core_kernel.Time_ns and Core_kernel.Timing_wheel_ns rather than Core.Time and Core.Timing_wheel_float.
Added Async_kernel.Types module to deal with the mutual recrsion of Async_kernel's types.

This should help eliminate the complexity and make it easier to make changes without running into as many constraints due to module/type ordering.

Merged Types.Jobs into Types.Scheduler.
Improved the performance of Deferred.bind, eliminating an allocation in Ivar.connect.
Optimized Deferred.bind, removing a closure allocation by inlining Deferred.create.
Added Pipe.interleave_pipe, which is like interleave, but takes a pipe rather than a list.

async_ssl

By default OpenSSL ignores the result of certificate validation, so we need to tell it not to.
Expose session details such as checked certificates and negotiated version. Add session resumption.

async_unix

Made Process.env type equal Core.Std.Unix.env type, effectively adding the `Replace_raw variant.
Renamed Process.wait as collect_output_and_wait, and added a wait function that is a thin wrapper around waitpid.

Also renamed:
```
wait_stdout       --> collect_stdout_and_wait
wait_stdout_lines --> collect_stdout_lines_and_wait
```
Added Unix.getgrouplist, a wrapper around the eponymous function in core
Change the Async scheduler to run external actions immediately upon dequeueing them, rather than first enqueueing them in the normal job queue.

Also, made external actions be jobs rather than closures.
Changed Unix.Inet_addr.of_string_or_gethostbyname to not use a sequencer.

We had used a sequencer to workaround bugs in winbind, which we don't use anymore.

Reported on github: https://github.com/janestreet/async_unix/issues/4

bignum

Fixed exception raised by Bignum.sexp_of_t when the denominator is zero.

bin_prot

Minor commit: comments.

core

Renamed Dequeue as Deque.
Added Fdeque, a functional deque (a double-ended Fqueue, or a functional Deque).
Changed Fqueue's bin-io format, and added a stable type.

Deprecated deque-like functions in favor of Fdeque.
Added Fheap, a functional heap implementation based on pairing heaps.
Reverted the change to Or_error's bin-io format made in 112.17, going back to the format in 112.16 and before.
Added to Unix.env type a `Replace_raw variant, as used in exec and fork_exec.
Added Array.Permissioned module, which has a permissioned array type and permissioned versions of all the regular array functions.
Added Date.week_number : t -> int.

Added values to Day_of_week: of_int_exn, iso_8601_weekday_number, weekdays.

 val of_int_exn : int -> t
 val iso_8601_weekday_number : t -> int
 val weekdays : t list (- [ Mon; Tue; Wed; Thu; Fri ] *)

Switched Float IEEE functions to use Int63.t for the mantissa rather than int, so they work on 32-bit platforms.
Added a length field to the Map.t record, making Map.length O(1) rather than O(n).
Moved a fragment of Time_ns from Core to Core_kernel, enough so that Async_kernel can use Core_kernel.Time_ns and ultimately only depend on Core_kernel.
Fixed compilation of Time_ns 32-bit Linux.
Added Bounded_int_table.clear.
Fixed the module_name passed to Identifiable.Make for a number of modules.

The module name must be an absolute module path.

Reported here: https://github.com/janestreet/core/issues/52
Added Tuple.Binable functor, for making binable tuples.
Sped up a Time_stamp_counter unit test.

Time_stamp_counter unit test has an 18s unit test, which seems excessive. Take a couple of orders of magnitude off the number of iterations.
Added Time_ns.pause, whose implementation is the same as Time.pause.

This involved moving the nanosleep C code from Core to Core_kernel.

This was necessary so that Async_kernel can pause without introducing a dependence of Async on Core.
Made Core_kernel.Time_ns.Alternate_sexp use a similar format to Core.Time_ns.

This was needed so that Async_kernel can use a nice sexp format for time spans.
Changed Timing_wheel implementation to use Time_ns, and moved to Core_kernel.Timing_wheel_ns; made Core.Timing_wheel a wrapper around Timing_wheel_ns.

Generalized the timing-wheel interface to be parametric in Time, so that one interface applies to both Timing_wheel and Timing_wheel_ns.

Generalized the timing-wheel unit tests to a functor, Timing_wheel_unit_tests.Make, that is used to test both Timing_wheel_ns and Timing_wheel_float. Moved a few tests that depend on Time and Date from the functor into timing_wheel_float_unit_tests.ml.

Split out Timing_wheel.Debug into a separate functor, Timing_wheel_debug.Make.

This was done in so that Async_kernel can depend only on Core_kernel and not Core.
Added optional arguments to Command.group: ?body and ?preserve_subcommand_order.

preserve_subcommand_order : unit causes subcommands to be in the order they are specified, rather than sorted.

body : (path:string list -> unit) is called when no additional arguments are passed.
Added accessor function Command.summary : t -> string.
Fixed a bug in Time.Span robust comparison.
Changed Command's tab-completion bash code so that it is possible for programs to return completions containing spaces.

Actually knowing when and how to do so is difficult, because of course there's escaping to worry about. Adding helper functions to make that sort of thing manageable is left for future work.
In Command, made the -version and -build-info flags work at the top level when there are subcommands.
Added Sequence.interleaved_cartesian_product, which implements cartesian product of potentially infinite sequences.

core_extended

Added to Shell.set_defaults a ?preserve_euid:bool argument, which causes Shell to use bash -p.
Removed Array.Access_control, now that there is Core.Std.Array.Permissioned.
Removed Fast_int_div.

core_kernel

Added Time_ns module.

A fragment of Core.Std.Time_ns is now in Core_kernel.Std.Time_ns such that Async_kernel can use Time_ns and only depend on Core_kernel.
Renamed Dequeue as Deque. Dequeue remains for backward compatibility, but should not be used anymore. Use Deque instead.
Added Fdeque module, a functional version Deque. Deprecate deque-like functions in Fqueue.

custom_printf

Added syntax to use Sexp.to_string_mach rather than Sexp.to_string_hum.

Added syntax %{sexp#mach:<type>}, which is like %{sexp:<type>}, except it calls Sexplib.Sexp.to_string_mach instead of Sexplib.Sexp.to_string_hum.

In fact, you can put any identifier after the sexp# and it will call Sexplib.Sexp.to_string_<that identifier>; however, there are no other such functions right now.

jenga

Interns strings, significantly reducing memory use.

ocaml_plugin

Minor update: follow Async evolution.

pa_ounit

Added -verbose switch to time unit tests, so we can easily see which ones are taking a long time.

pa_test

Reduce code generated by pa_test in favor of more code in pa_test_lib.
- Less generated code means less time spent compiling it.
- Reducing code in pa_test.ml is good, because: a) Staging semantics make this code hard to understand. b) This code is written using the less familiar revised OCaml syntax. b) We get less code to migrate to syntax extensions.
Re-used location code in pa_here; side benefit that we get full path names in (Loc...) instead of just the file's basename.

patdiff

Minor update: doc.

patience_diff

Update references to Core.Std.Dequeue to refer to Core.Std.Deque

rpc_parallel

Added Parallel.State.get function, to check whether Rpc_parallel has been initialized correctly.
Added Map_reduce module, which is an easy-to-use parallel map/reduce library.

It can be used to map/fold over a list while utilizing multiple cores on multiple machines.

Also added support for workers to keep their own inner state.
Fixed bug in which zombie process was created per spawned worker.

Also fixed shutdown on remote workers
Made it possible for workers to spawn other workers, i.e. act as masters.
Made the connection timeout configurable and bumped the default to 10s.

sexplib

Minor update: documentation.

typerep

Remove unused "bin_proj" rewriter.

112.19.00

core_profiler

Initial release

112.17.00

async

Added tests and updated examples

async_extended

Added Interactive module for terminal interaction with users by command-line executables.

Interactive was previously in Iron_common.
In Process, added an ?env argument to some functions.
Allowed Command_rpc implementations to access the Rpc.Connection.t used to invoke them.

There is an option to invoke Command_rpc implementations via sexp communication instead of Rpc, so implementations are given a value of a variant type Sexp | Bin_io of Rpc.Connection.t.
Added Resource module, which abstracts the idea of acquiring and releasing a handle to a resource.

async_extra

Modernized code style in Async_extra.

This was mostly whitespace changes, plus deletions of unneeded module paths.
Added with sexp_of to Tcp.Where_to_listen for debugging.
In Versioned_typed_tcp, check that the writer on the other side is not closed in the Pass_on case.
Added a new way to implement an RPC, where the implementation doesn't return a deferred.

This "blocking" rpc implementation guarantees that the rpc will in fact be fully dispatched by the time the implementation returns.

This can be used to skip the deserialization of the query, and instead operate directly in the message contents as received.

Also, fixed a bug in which the query handler (and therefore the connection state) was being called before the internal async rpc handshake was finished.
Added an optional job_tag argument to Sequencer_table.enqueue, to display for debugging.
Added an optional argument to TCP-connection functions to control the local interface used to connect.

To implement this this, extended Tcp.connect to work on a bound socket.
Added with compare to Process.Output.t.
Added Process.Output.Stable module.
Exposed concrete rpc in Versioned_rpc.Both_convert.
Changed Cpu_usage to take its first sample after waiting, rather than immediately.

This fixes a problem where the first sample could be NAN or Inf.
Made Log buffer-age be unlimited, to avoid exceptions when log writes are blocked for long periods.
Improved Log.t_of_sexp's error message.
Changed Rpc.Connection.client and with_client to raise some errors which had been dropped during RPC dispatch.

Previously, errors dispatching Rpc.Rpc.t's were handled correctly and returned or raised by the relevant dispatch functions. However, errors that occurred in the middle of handling a Rpc.Pipe_rpc.t or Rpc.State_rpc.t were swallowed. This is because they happen after the dispatch functions have returned, and the dispatch interface doesn't allow for errors to occur in the middle of the pipe -- they must be raised to the monitor in effect when the Rpc.Connection.t is created. Errors could be raised to the effective monitor at the dispatch call, but the failure causes the entire connection to go into error, so the connection's monitor seems more appropriate. These errors weren't propagated to the caller because client and with_client both used Monitor.try_with without rest handling, causing /any/ errors caused while handling the connection (after the Connection.t has been returned to the user) to be dropped.
In Rpc, exposed some optional parameters from the Tcp module: ?max_pending_connections and ?buffer_age_limit.

async_kernel

Fixed a space leak in Clock.Event.abort, making it free the job that was created and stored in the timing wheel.
Moved Scheduler.yield from Async_unix.
Fixed a bug in Scheduler.yield_every, so that it doesn't initialize the scheduler until the staged function is called.
Added concat_map function to Monad_sequence interface.
Added Shutdown.shutdown_on_unhandled_exn.

Added some functions to Deferred.Or_error to parallel Core.Or_error: errorf, tag, tag_arg.

val errorf : ('a, unit, string, _ t) format4 -> 'a
val tag : 'a t -> string -> 'a t
val tag_arg : 'a t -> string -> 'b -> ('b -> Sexp.t) -> 'a t

Added Gc.Alarm, an Async-friendly wrapper around Core.Gc.Expert.Alarm.
Removed Gc.Expert, whose functions are superseded by Async-friendly functions in Gc proper.
Added Pipe.read_now_at_most.
Changed Pipe.merge to check whether its output is closed, and if so, stop rather than write to it (which raised).

Also, made Pipe.merge close its inputs whenever its output is closed.
Changed Clock.at to return Deferred.unit if it is supplied a time in the past.

Previously, it would create an empty ivar and a job to fill it that would run in the next cycle.
Changed Clock.Event.status to return Will_happen_at of Time.t``` rather thanWaiting```, if applicable.
Added Ivar.create_full.
Moved the use of Linux_ext to Async_unix.

This is one of the necessary steps in making Async_kernel depend on Core_kernel rather than Core.

async_parallel

Modernize the code

async_smtp

Moved from janestreet-alpha

async_ssl

moved ffi_bindings and ffi_stubgen in separate libraries

async_unix

Moved Scheduler.yield to Async_kernel.
Added Reader.load_annotated_sexp* functions.

These are like the existing Reader.load_sexp* functions, except they return annotated sexps rather than sexps. Having annotated sexps is useful so that one can report error positions to the user when processing values built by the t_of_sexp_ functions /after/ they return. I.e. when there aren't syntax errors in the sexps, but rather semantic errors detected later.
Removed noise and redundancy from Reader.load_sexp error messages.
Added Writer.save_sexps, analogous to Reader.load_sexps.
Made Writer errors raised by the background flush job include the entire Writer.t, rather than just the Fd.t.
Added to Writer.transfer an optional argument to limit the number of values read at once from the pipe.

The old behavior is to have no limit and remains the default.
Added to Writer some missing checks for functions that should ensure the input writer isn't closed.
Changed Scheduler.run_cycles_until_no_jobs_remain to pause so that alarms scheduled to fire in the past actually fire.

This is necessary because of the timing-wheel, which doesn't guarantee to fire an event until alarm-precision after it was scheduled.

Without this change, some tests unexpectedly fail, due to jobs not running that should have.

bignum

Added Bigint.random function, which produces a uniformly distributed value.

bin_prot

Added Bin_prot.Blob, formerly known as Core_extended.Wrapped, which has efficient handling of size-prefixed bin-io values in cases where serialization can be bypassed.

core

Deprecated the single-line files that simply include the corresponding Core_kernel module. Those are unnecessary, because people should use Core.Std.

We will keep these aliases around for a version before deleted them entirely.
Changed finalizers and signal handlers to, upon an unhandled exception, exit nonzero rather than asynchronously raise.
Removed Time.Zone.find_office.

Replaced uses with the still-blocking Time.Zone.find_exn
Made many changes to Time to make the time zone explicit instead of implicitly using the local timezone.

Added zone:Time.Zone.t parameters to many functions. In almost all cases, used ~zone:Time.Zone.local where previously it was implicit.

Removed of_local_ofday and to_local_ofday in favor of the explicit versions (with Time.Zone.local).

Removed Time.Zone.machine_zone () in favor of local.
Exported Core.Std.With_return.
Exposed Core.Std.with_return_option.
Fixed Time_ns.Ofday.of_span_since_start_of_day to check its input.
Changed Time_ns.to_span and of_span to round to microseconds, for round trippability.
Added Unix.Error module, for the Unix.error type.
Added Unix.Syscall_result, a new abstract type representing the result of a Unix system call as an int, to avoid allocation.

A lot of Unix system calls return an integer on success, so for ones that are called a lot, we can encode errors as -errno. This module abstracts this concept.
Changed Iobuf.recvmmsg functions to return the new Unix.Syscall_result.
Changed Unix.exec's ?env argument to support extending the environment in addition to replacing it.
Added with compare to Unix.Exit.t and Unix.Exit_or_signal.t.
Moved Backtrace to Core_kernel.

Deleted backtrace_stubs.c, now that we have Printexc.get_callstack.
Changed Bigstring.read_assume_fd_is_nonblocking and send_nonblocking_no_sigpipe to return Unix.Syscall_result.t, to reduce allocation.
Changed Iobuf.send_nonblocking_no_sigpipe to handle EINTR like EAGAIN, instead of raising.
Added Command.Spec.char.
Changed Process_env.parse_ssh_client to accept an SSH_CLIENT that is just IP address without ports.

core_bench

Updated code to follow some core changes

core_extended

Added functions to Low_level_debug to get a sexp or string representation of any type.

This could be handy when debugging polymorphic code.
Renamed String.is_substring to is_substring_deprecated. Use Core.String.is_substring instead.
Fixed a bug in Bin_io_utils.save.
Made Unix.Mac_address match Hashable.S.

core_kernel

Added List.is_prefix.

val List.is_prefix : 'a t -> prefix:'a t -> equal:('a -> 'a -> bool) -> bool

Made String_id.Make functor generative, which exposes that the result has type t = private string.

Previously the result of String_id.Make didn't expose type t = private string due to a type-checker bug:
- http://caml.inria.fr/mantis/view.php?id=6485
- http://caml.inria.fr/mantis/view.php?id=6011
Used generative functors, e.g. for Unique_id.

Used generative functors (new feature in 4.02) where previously we used dummy M : sig end arguments in the signature and (struct end) when applying the functor.

Just to note the difference between applicative and generative functors. Suppose we have:
```
module F (M : sig end) : sig type t end
```
and we apply it several times
```
module A = F (struct end)
module B = F (struct end)
module C = F (String)
module D = F (String)
```
Then we have that A.t <> B.t but C.t = D.t. This can lead to subtle bugs, e.g. Unique_id.Int (Unit). Note that it is perfectly valid to apply any module to F, even though that is certainly not what we want.

In 4.02, we can explicitly say that functor generates new types, i.e. it is generative. For this we use argument (). So F becomes
```
module F () : sig type t end
```
You can only apply F to () or (struct end) but each application yields a new type t.
```
module A = F ()
module B = F ()
module C = F (struct end)
module D = F (String) (* illegal *)
```
and now A.t, B.t and C.t are all different.

Note that F (struct end) is still allowed but was converted to to F () for consistency with signatures.

Propagated generativity where necessary. If inside a functor we use generative functor that creates new types, then we also need to make the enclosing functor generative.

For functors that don't create types (like Async.Log.Make_global), generative or applicative functors are the same, but the syntax of generative functors is lighter.
Exported Core_kernel.Std.With_return.
Exposed the record type of Source_code_position.t.
In Weak_hashtbl.create, exposed the ?growth_allowed and ?size arguments of the underlying Hashtbl.create.
Added with compare to Array.

Sped up Int.pow.

Benchmarks before:

Name	Time/Run	mWd/Run	Percentage
[int_math.ml:int_math_pow] random[ 5] x 10000	140_546.89ns		53.98%
[int_math.ml:int_math_pow] random[10] x 10000	173_853.08ns		66.77%
[int_math.ml:int_math_pow] random[30] x 10000	219_948.85ns		84.47%
[int_math.ml:int_math_pow] random[60] x 10000	260_387.26ns		100.00%
[int_math.ml:int_math_pow] 2 ^ 30	11.34ns
[int_math.ml:int_math_pow] 2L ^ 30L	21.69ns	3.00w
[int_math.ml:int_math_pow] 2L ^ 60L	22.95ns	3.00w

and after:

Name	Time/Run	mWd/Run	Percentage
[int_math.ml:int_math_pow] random[ 5] x 10000	105_200.94ns		80.78%
[int_math.ml:int_math_pow] random[10] x 10000	117_365.82ns		90.12%
[int_math.ml:int_math_pow] random[30] x 10000	130_234.51ns		100.00%
[int_math.ml:int_math_pow] random[60] x 10000	123_621.45ns		94.92%
[int_math.ml:int_math_pow] 2 ^ 30	8.55ns
[int_math.ml:int_math_pow] 2L ^ 30L	22.17ns	3.00w	0.02%
[int_math.ml:int_math_pow] 2L ^ 60L	22.49ns	3.00w	0.02%

Removed the old, deprecated permission phantom types (read_only, etc.) and replaced them with the new =Perms= types.

The old types had subtyping based on covariance and private types. The new types have subtyping based on contravariance and dropping capabilities.

Renamed read_only as read, since Perms doesn't distinguish between them.

The idiom for the type of a function that only needs read access changed from:
```
val f : _ t -> ...
```
to
```
val f : [> read ] t -> ...
```
This mostly hit Iobuf and its users.
Added String.is_substring.

Added With_return.prepend, and exposed With_return.t as contravariant.

(** [prepend a ~f] returns a value [x] such that each call to [x.return] first applies [f]
    before applying [a.return].  The call to [f] is "prepended" to the call to the
    original [a.return].  A possible use case is to hand [x] over to an other function
    which returns ['b] a subtype of ['a], or to capture a common transformation [f]
    applied to returned values at several call sites. *)
val prepend : 'a return -> f:('b -> 'a) -> 'b return

Moved the Gc module's alarm functionality into a new Gc.Expert.Alarm module.

The was done because the Gc alarms introduce threading semantics.
Exposed modules in Core_kernel.Std: Int_conversions, Ordered_collection_common
Removed Pooled_hashtbl from Hashable.S, to eliminate a dependency cycle between Int63 and Pool.

This was needed to use Int63 in Pool. Previously, Int63 <- Int <- Hashable <- Pool, which made it impossible to use Int63 in Pool.

So, we are removing the dependency Hashable <- Pool, simplifying Hashable to not include Pooled_hashtbl, and letting users call the Pooled_hashtbl functor directly when necessary.
Added to Pool.Pointer.Id conversions to and from Int63.
Made Pooled_hashtbl.resize allocate less.
Removed Pool.pointer_of_id_exn_is_supported, which was always true.
Added with compare to Info, Error, Or_error.
Moved Backtrace from Core
In C stubs, replaced intxx types by intxx_t.

Following this: http://caml.inria.fr/mantis/view.php?id=6517

Fixes #23
Removed Backtrace.get_opt, which is no longer necessary now that Backtrace.get is available on all platforms.
Added module types: Stable, Stable1, Stable2.
Exposed Core_kernel.Std.Avltree.
Removed from Binary_packing a duplicated exception, Pack_signed_32_argument_out_of_range.

Closes #26
Made Info, Error, and Or_error stable.

The new stable serialization format is distinct from the existing unstable serialization format in the respective modules, which wasn't changed.
Add Sequence.Step.sexp_of_t.

core_profiler

Initial release

custom_printf

Changed %{M.x} to call M.x instead of M.Format.x.

This is more natural and makes it easy to call arbitrary other functions.
Added syntax %{M#x}, which calls M.to_string_x.

This makes it easy to use existing modules, since calling functions to_string or to_string_x is already the convention.

email_message

Moved from janestreet-alpha

jenga

Fixed byte-compile targets to avoid stale artifact deletion of .for-byte-compile.cmt file when compilation fails.

This avoids polling jenga trigger loop.
Show Removed stale build artifact messages only when -act flag is given.
Extended Jenga API with val file_existence : Path.t -> unit t.

file-existence has same relationship to file-exists as glob_change has to glob_listing.
Fixed memory leak in tenacious hearts by using weak references and finalizers.
- Strip code for obsolete versions of hearts.
- Strip code for OLD_TENACIOUS=true.
- Ensure Ring support preemptive calls to detach. Add ring tests.
- Keep message showing Live(Kb-delta).
Fixed curly braces in globs.
Throttled calls to Writer.save, to fix the too many open files bug.

Throttled calls to Writer.save from save_description.run sharing the same throttle used for all FD access, with ~max_concurrent_jobs:500
Added to jenga -progress ~save=XXX, the number of calls to Writer.save in Progress.saves_run.

ocaml_plugin

Fixed spurious interface mismatch error when a plugin cache is shared by incompatible compilers.

When a plugin cache directory is used by several executables with incompatible cmis/compilers, and the cache config option try_old_cache_with_new_exec is set to true, this could lead to the following error:
```
Plugin failed: (ocaml_dynloader.ml.Dynlink_error "interface mismatch")
```
This feature fixes this.

Since it modifies some record, for later changes it seems easier and more conservative to allow field additions without breaking older version. Thus we allow extra fields in persisted records.
```
let t_of_sexp = Sexp.of_sexp_allow_extra_fields t_of_sexp
```
New executables can read both old and new caches, but old executables will either blow away new caches, or if the config says the cache is read-only, fail.

Take the chance to modernize part of the code.
Switched tests to unified tests.
Fixed bugs dealing with paths with spaces in them.
Check that plugins have the expected type before running them rather than after, which is what one would expect.

Also check that runtime and compile types match in check_ocaml_src_files and compile_ocaml_src_files_into_cmxs_file.

pa_ounit

only spend time to format test description when tests are run

patdiff

The call to Pcre.full_split in patdiff_core.ml rely on a bug of pcre-ocaml <= 7.1.2.

To get the same behavior with pcre-ocaml >= 7.1.3 we need to pass ~max:(-1).

See this bug for more details:

https://github.com/mmottl/pcre-ocaml/issues/1

rpc_parallel

Follow changes in Async RPC

sexplib

Added sexp_of_ support for GADTs, and remove the not-quite-working support for of_sexp.

textutils

Added a ~narrow argument to Text_graph.render

typerep

Split out typerep_extended which is now using core_kernel

typerep_extended

typerep_extended now use core_kernel

112.06.00

async_extended

Unwound a recent change to Mailbox where one invocation of receive would put aside items, preventing other invocations from noticing them.
Added Delimited.Row.nth_conv_exn, as a counterpart to get_conv_exn.
Fixed File_updates handling of identical mtimes.

async_extra

In Log, exposed the raw message.
Changed Rpc creators' connection_state to be a function that takes the connection and returns the state.

This makes it possible for the connection state to actually get a handle on the connection itself, which simplifies a number of idioms for using RPC. In particular, it makes it easier to respond with an RPC back to a client over client's own connection.
Fixed some nondeterministically failing tests.
In Log, made logs discard messages when their output list is empty.

Also, removed redundant tracking of current level.
Moved Udp.bind_to_interface_exn to Unix module in async_unix.
Added Versioned_typed_tcp.Repeater.

Repeater is used in the cases where we want to inspect and possibly alter the flow between a client and a server without having to change either the client or the server or the protocol between them.

async_kernel

Added Deferred.Sequence module, analogous to Deferred.List but for Core_kernel.Std.Sequence.
Modernized code style.

async_unix

Added Writer.behave_nicely_in_pipeline, which makes a program behave nicely when used in a shell pipeline where the consumer goes away.
Modernized code style.
Removed spurious overrides in Signal: set and signal.

These overrides are no longer necessary because the functions moved from Core.Signal to Core.Signal.Expert.
Moved async_extra's Udp.bind_to_interface_exn to Unix.

bignum

Added functions to round from Bignum.t to Bigint.t, and to convert Bigint.t into Bignum.t.

bin_prot

Sped up float and float array operations.
Removed a use of Obj.magic in code generated by pa_bin_prot for polymorphic variants that led to memory unsafety.

Previously, pa_bin_prot generated this kind of code for polymorphic variants:
```
  match Obj.magic (read_int buf pos) with
  | `A as x -> x
  | `B as x -> x
  | `C -> `C (read_float buf pos)
  | _ -> fail
```
and this caused the compiler to assume the result is an immediate value. To fix this we removed the as x -> x and used the computed integer hash.

core

Renamed Linux_ext.gettid as Unix.gettid, and added OpenBSD support.

SYS_gettid is not available on OpenBSD, but is used in Core_extended. See the mailing list discussion about this here:

https://groups.google.com/forum/#!topic/ocaml-core/51knlnuJ8MM

Seems like the OpenBSD alternative is:
```
pid_t        getthrid(void);
```
although it's not defined in any header file, which is a bit unfortunate.
Added Piecewise_linear.precache, which computes a lookup table that speeds up subsequent calls to Piecewise_linear.get.
Added Time_ns module, representing times as 63-bit integers of nanoseconds since the epoch.
Fixed build of unix_stubs.c on OpenBSD.
In Daemon, fixed an error message regarding WSTOPPED (fixes #47).
Added Time.Span.Stable.V2, with sexps that use new suffixes for microseconds (us) and nanoseconds (ns).

Time.Span.of_string supports the new format, but Time.Span.to_string doesn't yet produce it -- we plan to change that later, after the new of_string has made it out more widely.
Added Time.Span.to_string_hum, which gives more options for rendering time spans.
Merged the recvmmsg stubs in Bigstring and Iobuf.

Factored out a shared underlying recvmmsg call that both stubs use.

Restored -pedantic by avoiding a C99 feature (variable-length stack arrays).
Made Date.t abstract, and changed its representation from a 4-word record to an immediate int (packing year, month, day).
In Daemon, changed the permissions of the std{err,out} files generated during daemonization from 0o777 to 0o644.
Moved Thread_safe_queue from core to core_kernel.

This was done so that Async_kernel can use it, eliminating one of Async_kernel's dependencies on Core.

Thread_safe_queue_unit_tests remains Core, at least for now, because it has some dependencies on other stuff in Core.

core_bench

Solved a problem in which OCaml 4.02 was optimizing away benchmarks, making them meaningless.

core_extended

Sped up String.is_substring by replacing the OCaml implementation with a call to libc memmem.

memmem runs in 20% of the time, incurs minimal GC pressure, is portable among UNIXen that we target, AND it's clearer than the ML version.
Made Float_ref support bin_io and sexp.
Removed gettid, which is now available in Core.Unix.
Added Fast_int_div module, which speeds up integer division by a fixed divisor.
Moved Sexp.of_sexp_allow_extra_fields to core_kernel.

core_kernel

Made String_id have Stable_containers.Comparable.
Changed Gc.disable_compaction to require an allocation_policy.
Made Option match Invariant.S1.
Added Sequence.filter, compare, and sexp_of_t.

Added With_return.with_return_option, abstracting a common pattern of with_return.

  val with_return        : ('a return -> 'a  ) -> 'a
  val with_return_option : ('a return -> unit) -> 'a option

Install a handler for uncaught exceptions, using Printexc.set_uncaught_exception_handler, new in OCaml 4.02.
Changed Day_of_week representation to a normal variant.
Changed Exn.handle_uncaught so that if it is unable to print, it still does exit 1.
Added Sexp.of_sexp_allow_extra_fields, previously in Core_extended.Sexp.
Changed the implementation of Exn.raise_without_backtrace to use raise_notrace, new in OCaml 4.02.
Added Float functions for converting to and from IEEE sign/exponent/mantissa.
Added String.Caseless module, which compares and hashes strings ignoring case.
Reimplemented Type_equal.Id using extensible types (new in OCaml 4.02), removing a use of Obj.magic.

Changed Type_equal.Id.same_witness to return option rather than Or_error, which allows it to be implemented without allocation.
Removed a reference to the Unix module. Applications using core_kernel should be able to link without unix.cma again.
Made Char.is_whitespace accept \f and \v as whitespace, matching C.

core_profiler

Solved a problem in which OCaml 4.02 was optimizing away benchmarks, making them meaningless.

jenga

Support for user control of stale-artifact deletion, by allowing specification of an artifact-determination policy.
Expose jenga's internal (and better - only quotes when necessary) definition of Shell.escape in Api
Removed Action.shell from the API, superseded by Action.process.
Changed RPC interface as needed for build manager to switch from scraping error messages to RPCs.
Fixed jenga's per-rule memo table, which mistakenly kept stale values.
Show what target is being demanded, useful for debugging rules.
Run user action when persistent format changes.
When filtering buildable targets by globs, pay attention to the kinds allowed by the glob.

Specifically, if the kinds don't include `File (i.e. only include `Directory) then we should not see any buildable_targets in the filtered list.

ocaml_plugin

Stopped using the ~exclusive with Reader, because it doesn't work on read-only file systems.

It's not even needed because these files are written atomically.
Used a generative functor in the generated code, so the user code can apply generative functors at toplevel, or unpack first class modules that contain type components.
Fixed bug when mli file references something defined only in another ml.

Made it possible to compile a plugin in one process, and dynload the compiled cmxs file without starting async in another process.

This was done with two new APIs in Ocaml_dynloader.S:

  val compile_ocaml_src_files_into_cmxs_file
    : dynloader
    -> string list
    -> output_file:string
    -> unit Deferred.Or_error.t

  val blocking_load_cmxs_file : string -> t Or_error.t

Allowed plugins to optionally have a shebang line.
Made Ocaml_dynloader.find_dependencies also support files with shebang lines.

pa_bench

Made the code generated by pa_bench for BENCH not use ignore, because OCaml 4.02 will remove dead code in some cases, meaning the benchmarks are no longer measuring what they should. Instead the ignore is deep inside Core_bench, which is likely out of reach of the compiler.

The result of the user functions given to BENCH_FUN and BENCH_INDEXED are changed so they don't have to return unit and people are encouraged not to use ignore when these functions don't return unit (you will get the same warning though, i.e. a warning if the result of your function is a function too, thus preventing unintended partial applications).

For example, here are a few benchmarks and their output before the fix:

  let x = if Random.bool () then 100 else 1001
  let r = ref 0
  BENCH "ig-1" = 10 / x
  BENCH "ig-2" = ()
  BENCH "ig-3" = phys_equal (10 / x) (Obj.magic 0)
  BENCH "ig-4" = r := (10 / x)
  BENCH "ig-5" = r := x

  +----------------+----------+------------+
  | Name           | Time/Run | Percentage |
  +----------------+----------+------------+
  | [misc.ml] ig-1 |   3.92ns |     29.30% |
  | [misc.ml] ig-2 |   3.34ns |     24.95% |
  | [misc.ml] ig-3 |   3.91ns |     29.23% |
  | [misc.ml] ig-4 |  13.37ns |    100.00% |
  | [misc.ml] ig-5 |   3.24ns |     24.20% |
  +----------------+----------+------------+

Many of the the numbers above are much lower than they should be because of the implicit ignores inserted by the benchmark caused the division to to eliminated by the compiler. After the fix, the same benchmarks produced more meaningful numbers:

  +----------------+----------+------------+
  | Name           | Time/Run | Percentage |
  +----------------+----------+------------+
  | [misc.ml] ig-1 |  12.78ns |     94.55% |
  | [misc.ml] ig-2 |   3.23ns |     23.90% |
  | [misc.ml] ig-3 |  13.51ns |     99.94% |
  | [misc.ml] ig-4 |  13.52ns |    100.00% |
  | [misc.ml] ig-5 |   3.30ns |     24.40% |
  +----------------+----------+------------+

sexplib

Improved the implementation of Exn.sexp_of_t, using the unique id in exceptions in OCaml 4.02.

We use the identifier to map exception constructors to converters.

112.01.00

async

update tests

async_extended

Clarified an error in Rpc_proxy.

async_extra

Changed Persistent_rpc_client.connected to avoid returning a connection that is closed at the time it was called.
Optimized Rpc.implement so that if a server's implementation returns a determined deferred, then the output is immediately serialized and written out for the client.

This reduces memory consumption, improves throughput and latency. Measurements with the pipe_rpc_test program showed that a server went from processing 600_000 msg/sec, to 2_200_000 msg/sec before pegging the CPU.
Changed Log's output processor's batch size from 1_000 to 100.
Added Persistent_rpc_client.close and close_finished.
In Rpc.Connection.client and with_client, used the handshake_timeout as the timeout passed to Tcp.connect.

handshake_timeout was previously used only for the Rpc module's handshake timeout.
Changed Rpc.create's on_unknown_rpc argument, renaming \Ignoreas`Close_connection, and requiring`Callto return`Close_connectionor`Continue`.

\Ignore` was renamed because it was a poor name, since in fact it closed the connection.

Added a \Continue` option, whic allows one to keep the connection open.

Changed \Callto return`Continueor`Close_connection, where the oldunitreturn value meant`Close_connection`.
In Versioned_typed_tcp, enabled the use of "credentials" in the "Hello" message.

Propagate credentials to the user code when it arrives on the wire.

async_kernel

Optimized Monitor.try_with ~run:\Now fto return a determined deferred iff ()` returns a determined deferred.

Previously, Monitor.try_with ~run:\Now falways introduced aDeferred.map`, which made it impossible to do some optimizations that bypass the scheduler overhead.
Added an ASYNC_CONFIG field that causes the program to dump core if Async jobs are delayed too much.

The new field is dump_core_on_job_delay.
Switched Async_kernel from using Core.Sys to Pervasives.Sys eliminating one of the dependencies on Core.

async_unix

Changed Writer.transfer write pipe to close pipe when the writer, is closed.

Previously, Writer.transfer did not close the pipe when the underlying writer is closed. This was strange because:
1. Callers would have to consistently check for the writer being closed and close the Pipe.Readert= themselves
2. The analogous function Pipe.transfer closes the reader on similar circumstances.
The absence of the close was noticed as a bug in Rpc, which assumed that Writer.transfer did the close.
Fixed a bug in Scheduler.yield that caused it to pause for 50ms if there is no other pending work and no I/O.
Exposed type equivalence between Unix.Passwd.t and Core.Std.Unix.Passwd.t.
Changed Writer.write_bin_prot to use the new Bigstring.write_bin_prot.

bignum

Added Bignum.Bigint module, with arbitrary-precision integers based on Zarith, which is significantly faster than the Num.Big_int library.

bin_prot

In Write, improved some OCaml macros to name values and avoid calling C functions multiple times.

core

Removed vestigial code supporting OCaml 4.00.
Added Command support for flags that are passed one or more times.

Added Command.Spec.one_or_more and Command.Spec.non_empty_sequence to deal with the cases where you expect a flag or anonymous argument (respectively) to be passed one or (optionally) more times. This is common enough and distinct from the case where you want the argument passed zero or more times that it seems like we should canonize it in the library.
In Lock_file, made stale lock detection more robust.

Made Lock_file.create foo succeed if foo is absent and foo.nfs_lock file is present and stale. Previously, it would fail.
Removed Syslog.syslog's add_stderr argument; use the PERROR option instead.
Fixed unix_stubs.c compilation on NetBSD

Closes #45
Added Filename operators /^ and /@, and of_parts, like the same functions for Catalog paths.
Changed Iobuf functions that advance the iobuf to not also return a redundant number of bytes processed.

This avoids a small allocation (in the case of the int option functions) and normalizes the result (so the same information isn't returned two ways). Actually, it doesn't yet avoid the allocation in the implementation, as the corresponding Bigstring functions must still return the number of bytes processed, and currently do so as an option. We hope to eventually change that.

In the future I expect we will change unit to some error variant to also avoid the exception construction for EWOULDBLOCK/EAGAIN. We can even make Unix syscalls noalloc if we're careful.
In Unix module, added unit tests for Cidr.does_match.

core_bench

fixed legacy format string

core_extended

Added Float_ref module, which is like float ref but faster for sets due to bypassing the write barrier.

Benchmark results on Sandy Bridge:

| [float_ref.ml:] float ref set | 2_886.94ns | 8.00w | | | [float_ref.ml:] Float_ref.set | 355.76ns | 6.00w | | | [float_ref.ml:] float ref get | 415.52ns | 6.00w | | | [float_ref.ml:] Float_ref.get | 416.19ns | 6.00w | |
Added Bin_io_utils.Wrapped.t, which defines an 'a t with bin_io that supports size-prefixed serialization and deserialization.

Wrapped has two useful submodules, Opaque and Ignored, for efficient handling of size-prefixed bin-io values in cases where serialization can be bypassed. See the comments in the module for more details.

core_kernel

Removed vestigial code supporting OCaml 4.00.
Used {Hashable,Comparable}.S_binable in Day_of_week and Month.
Improved the performance of Set_once.set.
Added Type_equal.Lift3 functor.
Replaced occurrences of Obj.magic 0 with Obj.magic None.

With the former the compiler might think the destination type is always an integer and instruct the GC to ignore references to such values. The latter doesn't have this problem as options are not always integers.
Made String_id.of_string faster.
Added Bigstring functions for reading and writing the size-prefixed bin-io format.
- bin_prot_size_header_length
- write_bin_prot
- read_bin_prot
- read_bin_prot_verbose_errors
Added {Info,Error}.to_string_mach which produces a single-line sexp from an Error.t.
Added {Info,Error}.createf, for creation from a format string.
Added new Perms module with phantom types for managing access control.

This module supersedes the read_only, read_write, and immutable phantom types, which are now deprecated, and will be removed in the future. This module uses a different approach using sets of polymorphic variants as capabilities, and contravariant subtyping to express dropping capabilities.

This approach fixes a bug with the current phantom types used for Ref.Permissioned in which immutable types aren't guaranteed to be immutable:
```
let r = Ref.Permissioned.create 0
let r_immutable = (r :  (int, immutable) Ref.Permissioned.t)
let () = assert (Ref.Permissioned.get r_immutable = 0)
let () = Ref.Permissioned.set r 1
let () = assert (Ref.Permissioned.get r_immutable = 1)
```
The bug stems from the fact that the phantom-type parameter is covariant, which allows OCaml's relaxed value restriction to kick in, which allows one to create a polymorphic value, which can then be viewed as both immutable and read write. Here's a small standalone example to demonstrate:
```
module F (M : sig
            type +'z t
            val create : int -> _ t
            val get : _ t -> int
            val set : read_write t -> int -> unit
          end) : sig
  val t : _ M.t
end = struct
  let t = M.create 0
  let t_immutable = (t :  immutable M.t)
  let () =
    assert (M.get t_immutable = 0);
    M.set t 1;
    assert (M.get t_immutable = 1);
  ;;
end
```
The new approach fixes the problem by making the phantom-type parameter contravariant, and using polymorphic variants as capabilities to represent what operations are allowed. Contravariance allows one to drop capabilities, but not add them.
Added Int.Hex module, which has hexadecimal sexp/string conversions.
Added Gc.major_plus_minor_words, for performance reasons.

core_profiler

fixed legacy format string

custom_printf

Fixed uses of printf=-styleformat strings that have unspecified behavior in OCaml 4.02 and will become errors.
Support substitution in format string (lost with 4.02 compatibility)

jenga

Don't show noisy glob..changed messages except with -show-glob-changed flag.
Support shared build rules via ${jenga}/share.
Detect cycle in dep scheme instead of hanging.
Made standalone actions atomic, just like actions associated with target files.

Running actions and recording the result in the persistent .jenga.db should be performed atomically for standalone actions, as it is for actions which are associated with target files

ocaml_plugin

Changed to not use rm -r when it is expected to remove one file.

rpc_parallel

Initial import.

sexplib

Replaced occurrences of Obj.magic 0 with Obj.magic None.

With the former the compiler might think the destination type is always an integer and instruct the GC to ignore references to such values. The latter doesn't have this problem as options are not always integers.

textutils

Added Ascii_table.Table_char, to expose the special table border characters.

type_conv

Updated ast matching for 4.02

111.31.00

jenga

Switched API to composable generator schemes.
Support -api flag to show the embedded API.
New examples.

111.28.00

async_extra

Added to Versioned_rpc a non-functor interface.
Added Log.level, which returns the last level passed to set_level.
Enabled Async-RPC pushback in the Tcp_file protocol.

async_unix

Added Shutdown.set_default_force, which allows one to change the default force value used by shutdown.

This is useful for applications that call shutdown indirectly.
```
  val set_default_force : (unit -> unit Deferred.t) -> unit
```

core

Added Piecewise_linear.create_from_linear_combination.

val create_from_linear_combination : (t * float) list -> t Or_error.t

Added Time.is_{earlier,later} : Time.t -> than:Time.t -> bool, which are easier to read than Time.(<) and friends.
Added Command.exec, which allows one to include the Command hierarchy from one executable in another.

Command.exec takes the file path to an executable that uses the Command module and returns a Command.t that integrates the executable (by exec'ing it), including providing recursive help and autocompletion as if it were a standard Command.t.
Replaced most uses of Hashtbl.replace with Hashtbl.set.
Renamed Float.epsilon to robust_comparison_tolerance, to avoid confusion with epsilon_float.

core_extended

Implemented Int.gcd using binary GCD in C, for improved performance.
Added Bin_io_utils.Serialized, which stores a value in memory as its bin-io representation.

Writing such a value just blits the value.
Moved Text_block from Core_extended into Textutils.
Added modules Hashtbl2 and Hashtbl2_pair.

core_kernel

Added Pooled_hashtbl.resize function, to allow preallocating a table of the desired size, to avoid growth at an undesirable time.
Added Pooled_hashtbl.on_grow callback, to get information about hashtbl growth.
Changed Hashable.Make to not export a Hashable module.

The Hashable module previously exported was useless, and shadowed Core.Std.Hashable.
Moved Common.does_raise to Exn.does_raise, to make it easier to find.
Added Float.one, minus_one, and ~-. (fixes #12).
Removed Core.Std.unimplemented and renamed it as Or_error.unimplemented.

It is not used enough to live in the global namespace.

jenga

Fixed problem that caused rule failed to generate targets.

ocaml_plugin

Fixed a bug in tests that could leave the repository in a state where running the tests would fail.

The bug happened if the tests were interrupted after creating read-only directories but before cleaning then up.

pa_ounit

Added a flag to disable embedding of unit tests/inline benchmarks. (janestreet/core_kernel#13)

textutils

Moved Text_block from Core_extended into Textutils.

111.25.00

async

add a dns example

async_extra

Removed lazy from the core of Log.
Made Log.Message.t have a stable bin_io.

The Stable.V1 is the current serialization scheme, and Stable.V0 is the serialization scheme in 111.18.00 and before, which is needed to talk to older systems.
Changed Rpc to return Connection_closed if a connection ends before a response makes it to the caller.

Previously, the dispatch output was never determined.

Also, removed an unused field in one of the internal data structures of Async RPC.
In Versioned_rpc, added version:int argument to implement_multi functions.
In Versioned_rpc, the Pipe_rpc.Make functors now return an additional output functor.

Register' is like Register but has in its input module:
```
val response_of_model :
  Model.response Queue.t -> response Queue.t Deferred.t
```
rather than
```
val response_of_model : Model.response -> response
```
This is analogous to Pipe.map' and Pipe.map.
Added to Log a V2 stable format and better readers for time-varying formats.
In Log, added an optional ?time:Time.t argument to allow callers to pass in the logged time of an event rather than relying on Time.now ().

async_kernel

Fixed Clock.run_at_intervals to make the initial callback at an interval multiple.

Previously, if start was in the past, f would run immediately rather than waiting for an interval of the form start + i * span. Now it always waits for an interval, even the first time it runs.

async_parallel

improve error handling

async_unix

Added Unix.Addr_info and Name_info, which wrap getaddrinfo and getnameinfo.
Improved the scheduler's error message when the thread pool is stuck.

core

Added Gc.disable_compaction function.
Added Time.to_string_abs_trimmed, which prints a trimmed time and takes a zone argument.
Fixed unix_stubs.c to suppress a warning when building with some versions of gcc.
Changed Time.Zone to allow the zoneinfo location to be specified by an environment variable.

Closes #40
Fix compatibility with 4.02

core_extended

Moved Quickcheck from core.
Added [Int.gcd].

core_kernel

Fix build on FreeBSD

Closes #10

Added functions to Container interface: sum, min_elt, max_elt.

(** Returns the sum of [f i] for i in the container *)
val sum
  : (module Commutative_group.S with type t = 'sum)
  -> t -> f:(elt -> 'sum) -> 'sum

(** Returns a min (resp max) element from the collection using the provided [cmp]
    function. In case of a tie, the first element encountered while traversing the
    collection is returned. The implementation uses [fold] so it has the same
    complexity as [fold]. Returns [None] iff the collection is empty. *)
val min_elt : t -> cmp:(elt -> elt -> int) -> elt option
val max_elt : t -> cmp:(elt -> elt -> int) -> elt option

Made Core_hashtbl_intf more flexible. For instance supports modules that require typereps to be passed when creating a table.

Address the following issues:

The type ('a, 'b, 'z) create_options needs to be consistently used so that b corresponds with the type of data values in the returned hash table. The type argument was wrong in several cases.

Added the type ('a, 'z) map_options to Accessors so that map-like functions -- those that output hash tables of a different type than they input -- can allow additional arguments.
Fixed a bug in Dequeue's bin_prot implementation that caused it to raise when deserializing an empty dequeue.
Made Container.Make's interface match Monad.Make.
Deprecated infix or in favor of ||.
Simplified the interface of Arg (which was already deprecated in favor of Command).
Replaced Bag.fold_elt with Bag.filter.
Memo.general now raises on non-positive cache_size_bound.
Removed Option.apply.
Removed Result.call, Result.apply.
Moved Quichcheck to core_extended.

It should not be used in new code.

custom_printf

Fix 4.02 compatibility.

jenga

Switched to un-version-numbered API.
Renamed Tenacious_sample_lib.Tenacious to Tenacious_sample_lib.Tenacious_sample to avoid conflicts in the public release.
Write buildable_targets.list (on alias .info).

ocaml_plugin

ignore more warnings by default

patdiff

add a ?file_names argument to Compare_core.diff_strings

patience_diff

refactoring and more unit tests

sexplib

Fix compatibility with OCaml 4.02

111.21.00

async_extra

Added Sexp_hum Log.Output.format, which is useful for making logs more human readable.
Added with compare to Rpc.Implementation.Description.

async_ssl

Upgraded to use new ctypes and its new stub generation methods.

async_unix

Added Process.wait_stdout and wait_stdout_lines, which are like run and run_lines, but take a Process.t.

core

Fixed an issue where Time.Zone.init would not properly traverse the directory containing timezone information.
Added Time.With_utc_sexp, which has stable serialization of Time.t that is byte-for-byte equal across timezones.
Made Uuid stable.
Made Piecewise_linear stable.

core_kernel

Removed our custom C stub for closing channels, reverting to the one in the OCaml runtime.

A long time ago we found that the OCaml runtime did not release the lock before calling close on the fd underlying a channel. On some filesystems (e.g. smb, nfs) this could cause a runtime hang. We filed a bug with INRIA and wrote our own close function which In_channel calls to this day. The bug has long been fixed, and our function is probably buggy, so this reverts us to the runtime's close.

Added Float.{of,to}_int64_preserve_order, which implement the order-preserving zero-preserving bijection between non-NaN floats and 99.95% of Int64's.

Used the new function to improve one_ulp, which is now exposed:

  (** The next or previous representable float.  ULP stands for "unit of least precision",
      and is the spacing between floating point numbers.  Both [one_ulp `Up infinity] and
      [one_ulp `Down neg_infinity] return a nan. *)
  val one_ulp : [`Up | `Down] -> t -> t

Changed Map.symmetric_diff to return a Sequence.t instead of a list.
Added Sequence.filter_map.
Improved Stable_unit_test.Make_sexp_deserialization_test's error message so that it includes the expected sexp.

custom_printf

Fixed a bug in which custom-printf syntax was incompatible with labeled arguments.

For example, the preprocessor used to raise an exception on this code:
```
  let f ~labeled_arg:() fmt = ksprintf (fun _ -> ()) fmt in
  f !"hello" ~labeled_arg:()
```

jenga

Introduced jenga API v3, a small cleanup of v2 which has been planned for a while.

ocaml_plugin

Fixed a bug in ocaml_embed_compiler on 32-bit machines.

ocaml_embed_compiler tries to read the full contents of the file as a string, but the string might be too big on 32bits:

https://github.com/ocaml/opam-repository/pull/2062#issuecomment-43045491

patdiff

Added Patdiff_core.iter_ansi.

(** Iter along the lines of the diff and the breaks between hunks. Offers more flexibility
    regarding what the caller wants to do with the lines *)
val iter_ansi
  :  f_hunk_break:((int*int) -> (int*int) -> unit)
  -> f_line:(string -> unit)
  -> string Patience_diff.Hunk.t list
  -> unit

patience_diff

Added plain differ Plain_diff and use it in some cases for improved results.
Move modules under Patience_diff_lib.Std.

111.17.00

async_extra

Added module Persistent_rpc_client, an RPC client that attempts to reconnect when the connection is lost, until a new connection is established.
Significantly sped up the Rpc module by removing Bigstring serialization.

Performance of the two implementations was tested by building a simple client/server executable that would count major cycles. Sending 100 byte messages at a rate of 50k/second shows (on both sides of the RPC):

original:
- ~160 major cycles in 30s
- CPU usage around 60%
new:
- ~10 major cycles in 30s
- CPU usage <= 2%
Enabled a version of Pipe_rpc and State_rpc where the consumer can pushback on the producer if it can't consume the contents of the pipe fast enough.
Added Log.Level.arg : Log.Level.t Command.Spec.Arg_type.t for defining command lines that accept (and autocomplete) log levels.
Added Command.async_or_error and renamed Command.async_basic to Command.async, leaving async_basic a deprecated alias for the new name.

Command.async_or_error is similar to Command.basic and Command.async, but accepts a unit Or_error.t Deferred.t type.
Added Persistent_rpc_connection.current_connection, so that one can detect whether one is currently connected.
```
val current_connection : t -> Rpc.Connection.t option
```

async_inotify

Upgraded library to use inotify 2.0

async_kernel

Renamed Monitor.errors to Monitor.detach_and_get_error_stream and Monitor.error as Monitor.get_next_error.

The use of detach in the name is intended to make clear that errors do not propagate to the parent.

Added several other non-stream =Monitor= functions to capture common use cases of Monitor.detach_and_get_error_stream:
```
detach_and_get_next_error
detach_and_iter_errors
detach
```

bignum

Improved the performance of binprot deserialization by removing the allocation of an intermediate type.

core

Fixed a bug in Bigstring.really_recv if recv doesn't receive all the data it wants.

This bug has been around forever; it may not have caused trouble because Bigstring.really_recv (1) is barely used (the only use is in Bigstring.unmarshal_from_sock) and (2) passes recv the MSG_WAITALL flag, so it will read the full amount unless it gets interrupted by a signal.
Fixed Bigstring.read's handling of EINTR so that it retries rather than returning zero.

This fixes a bug introduced in 111.09 in the interaction between Bigstring.read and Async.Reader. Prior to 111.09, Bigstring.read would raise on EINTR, and Async.Reader would propagate the exception. From 111.09 to 111.16, Bigstring.read would return zero, which would confuse Async.Reader into thinking it reached EOF when it hadn't. From 111.17, Bigstring.read will retry and not return zero when not at EOF.

We believe the bug was rare, because otherwise we would have frequently seen EINTR exceptions prior to 111.09.

Added Command.Spec.apply and pair, which allow one to program more with Spec.param rather than Spec.t.

val apply : ('a -> 'b) param -> 'a param -> 'b param
val pair : 'a param -> 'b param -> ('a * 'b) param

Added Command.Spec.file, which builds an Arg_type value with the same autocompletion as Spec.file.

(** [file] defines an [Arg_type.t] that completes in the same way as
    [Command.Spec.file], but perhaps with a different type than [string] or with an
    autocompletion key. *)
val file
  :  ?key:'a Univ_map.Multi.Key.t
  -> (string -> 'a)
  -> 'a t

core_extended

Added some functions to Splay_tree:
- length
- keys
- data
- to_alist
- delete_{after,before}
- map
- map_range
- split.

core_kernel

In Bigstring, made many operations use compiler primitives new in OCaml 4.01.

Exposed Bigstring.get and set as compiler primitives in the interface.

Added Bigstring.unsafe_get_int64_{le,be}_trunc.
Made Error round trip exn, i.e. Error.to_exn (Error.of_exn exn) = exn.
Added to failwiths an optional ?here:Lexing.position argument.
Added with typerep to Flags.S.
Optimized List.dedup [] to return immediately.
Added data argument to polymorphic type Hashtbl_intf.Creators.create_options.

This allows implementations of Hashtbl_intf.Creators to have constructor arguments that depend on the type of both key and data values. For example:
```
module type Hashtbl_creators_with_typerep =
  Hashtbl_intf.Creators
  with type ('key, 'data, 'z) create_options
    =  typerep_of_key:'key Typerep.t
    -> typerep_of_data:'data Typerep.t
    -> 'z
```
Improved the interface for getting Monad.Make to define map in terms of bind.

Instead of passing a map function and requiring everyone who wants to define map using bind to call a special function, we use a variant type to allow the user to say what they want:
```
val map : [ `Define_using_bind
          | `Custom of ('a t -> f:('a -> 'b) -> 'b t)
          ]
```
Improved the performance of many Dequeue functions.

Previously, many Dequeue.dequeue-type functions worked by raising and then catching an exception when the dequeue is empty. This is much slower than just testing for emptiness, which is what the code now does.

This improves the performance of Async.Writer, which uses Dequeue.dequeue_front.

patdiff

Removed latex output.

patience_diff

Exposed Patience_diff.matches.

sexplib

Make the camlp4 dependency optional

111.13.00

async_extra

For Typed_tcp.create, added a Client_id.t argument to the auth callback.

async_unix

Added Scheduler.yield_every, which returns a function that calls Scheduler.yield every n-th call.

This is useful in circumstances where you don't have strict control over where a deferred is examined, as in a Deferred.List.iter.

bignum

Eliminated the dependence of Bignum on Re2, and reduced dependence from Core to Core_kernel.
Extended the rounding interface to bring it in line with int and float rounding.
Improved the performance of Bignum's binprot.

Bignum's binprot had been to just binprot the decimal string representation. This is both slow to do and unnecessarily big in the majority of cases. Did something better in the majority of cases and fell back to this representation in the exceptional case.
```
$ ./inline_benchmarks_runner
Estimated testing time 20s (2 benchmarks x 10s). Change using -quota SECS.
```
Name Time/Run mWd/Run Percentage

bignum0.ml:Stable:Bignum binprot roundtrip compact 7.87us 490.00w 32.88%

bignum0.ml:Stable:Bignum binprot roundtrip classic 23.94us 1_079.00w 100.00%

Name	Time/Run	mWd/Run	Percentage
bignum0.ml:Stable:Bignum binprot roundtrip compact	7.87us	490.00w	32.88%
bignum0.ml:Stable:Bignum binprot roundtrip classic	23.94us	1_079.00w	100.00%

core_extended

Moved Patience_diff out of Core_extended into its own library.

core_kernel

Added a Sequence module that implements polymorphic, on-demand sequences.

Also implemented conversion to Sequence.t from various containers.

Improved the explicitness and expressiveness of Binary_searchable.binary_search.

binary_search now takes an additional (polymorphic variant) argument describing the relationship of the returned position to the element being searched for.

  val binary_search
    :  ?pos:int
    -> ?len:int
    -> t
    -> compare:(elt -> elt -> int)
    -> [ `Last_strictly_less_than         (** {v | < elt X |                       v} *)
       | `Last_less_than_or_equal_to      (** {v |      <= elt       X |           v} *)
       | `Last_equal_to                   (** {v           |   = elt X |           v} *)
       | `First_equal_to                  (** {v           | X = elt   |           v} *)
       | `First_greater_than_or_equal_to  (** {v           | X       >= elt      | v} *)
       | `First_strictly_greater_than     (** {v                       | X > elt | v} *)
       ]
    -> elt
    -> int option

Added a new function, Binary_searchable.binary_search_segmented, that can search an array consisting of two segments, rather than ordered by compare.

(** [binary_search_segmented ?pos ?len t ~segment_of which] takes an [segment_of]
    function that divides [t] into two (possibly empty) segments:

    {v
      | segment_of elt = `Left | segment_of elt = `Right |
    v}

    [binary_search_segmented] returns the index of the element on the boundary of the
    segments as specified by [which]: [`Last_on_left] yields the index of the last
    element of the left segment, while [`First_on_right] yields the index of the first
    element of the right segment.  It returns [None] if the segment is empty.

    By default, [binary_search] searches the entire [t].  One can supply [?pos] or
    [?len] to search a slice of [t].

    [binary_search_segmented] does not check that [segment_of] segments [t] as in the
    diagram, and behavior is unspecified if [segment_of] doesn't segment [t].  Behavior
    is also unspecified if [segment_of] mutates [t]. *)
val binary_search_segmented
  :  ?pos:int
  -> ?len:int
  -> t
  -> segment_of:(elt -> [ `Left | `Right ])
  -> [ `Last_on_left | `First_on_right ]
  -> int option

Made Queue match Binary_searchable.S1.
Made Gc.Stat and Gc.Control match Comparable.
Fixed some unit tests in Type_immediacy that were fragile due to GC.

patience_diff

Moved Patience_diff out of Core_extended into its own library depending only on Core_kernel.

sexplib

In Sexplib.Std, renamed Macro as Sexp_macro.

type_conv

Removed some unused-value warnings when with is used in signatures.

Removed warnings in cases like:
```
include (module_expr : sig type t with bin_io end)
```

111.11.00

async

Updated the sound.ml example

async_extra

Made Log more fair with respect to other Async jobs, by working on fixed-length groups of incoming log messages.

Previously, Log had processed everything available. The change gives other Async jobs more of a chance to run.

async_kernel

Added Clock.run_at_intervals, which runs a job at regular intervals.

async_unix

Added val Scheduler.yield : unit -> unit Deferred.t, which becomes determined after the current cycle finishes.
Improved the behavior of the scheduler's thread pool when Thread.create raises.

With this improvement, when the thread pool is unable to create a thread, it presses on with the threads it has rather than raise. Subsequent attempts to add work to the thread pool will cause the pool to attempt to create the thread, as long as enough time has passed since the most recent thread-creation failure.

Before this change, the thread pool wouldn't handle a Thread.create exception, and the exception would get raised to whatever code happened to be calling the Thread_pool function that tried to create a thread, e.g. Thread_pool.add_work. This caused In_thread.run to unexpectedly raise, and in turn In_thread.syscall to unexpectedly raise, leading to:
```
"Fd.syscall_in_thread bug -- should be impossible"
```
Also, changed should be impossible text to please report, since there may be other lurking rare exceptions that In_thread.syscall can raise, and we'd like to hear about them.

We rely on thread-pool-stuck detection to report problems where the inability to create threads causes the inability of the thread pool to make progress. A tweak was needed to make that work -- now the thread-pool-stuck warning is based on whether the thread pool has unstarted work, rather than on whether the thread pool has an "available thread". The latter would no longer work, since it is now possible for the thread pool to have unstarted work and to appear to have an available thread, i.e. num_threads < max_num_threads.

core

Change some Bigstring functions to retry on EINTR rather than raise.

The following functions (and their unsafe versions) were affected:
- read
- really_read
- really_recv
- really_write
- really_send_no_sigpipe
Some other Bigstring functions, like input and output, already retried on EINTR, so this change has precedent.

All of the affected stubs raise Bigstring.IOError on failure, rather than Unix_error, which means the normal method for retrying on EINTR doesn't work. In particular Async.Reader didn't retry them, even though it was supposed to.

Additionally, the documentation for the following functions was corrected to say that they raise =Unix_error= rather than =IOError=:
- pread_assume_fd_is_nonblocking
- pwrite_assume_fd_is_nonblocking
Eliminated global binary-to-decimal tables computed at startup for converting Date and Of_day to string.

Used an efficient implementation of division by 10, rather than the sprintf tables in time_internal.ml. This result in much less allocation at startup and it is a bit faster:
- before:
Name Time/Run mWd/Run Percentage

Date.to_string 69.39ns 3.00w 100.00%
- after:
Name Time/Run mWd/Run Percentage

Date.to_string 53.38ns 3.00w 100.00%
Fixed Time.Zone tests so that they are deterministic.
Added Iobuf.to_string_hum, which produces a readable, multi-line representation of an iobuf, intended for debugging.
Fixed brittle unit tests of Command.

Name	Time/Run	mWd/Run	Percentage
Date.to_string	69.39ns	3.00w	100.00%

Name	Time/Run	mWd/Run	Percentage
Date.to_string	53.38ns	3.00w	100.00%

core_extended

For Flang, added ordering to fields, and added abs, min, and max to the language.
Removed Loggers module.

core_kernel

Added to String functions for substring search and replace, based on the KMP algorithm.

Here are some benchmarks, comparing Re2 for a fixed pattern, Mark's kmp from extended_string, and this implementation ("needle").

The pattern is the usual abacabadabacabae.... The text looks similar, with the pattern occurring at the very end.

For =Re2= and =Needle= search benchmarks, the pattern is preprocessed in advance, outside of the benchmark.

FWIW: I've also tried searches with pattern size = 32767, but =Re2= blows up, saying:

re2/dfa.cc:447: DFA out of memory: prog size 32771 mem 2664898

Name	Time/Run	mWd/Run	mjWd/Run	Prom/Run	Percentage
create_needle_15	102.56ns	21.00w
re2_compile_15	6_261.48ns		3.00w		0.01%
create_needle_1023	13_870.48ns	5.00w	1_024.01w		0.03%
re2_compile_1023	107_533.32ns		3.03w		0.24%
create_needle_8191	90_107.02ns	5.00w	8_192.01w		0.20%
re2_compile_8191	1_059_873.47ns		3.28w	0.28w	2.37%
create_needle_524287	6_430_623.96ns	5.00w	524_288.09w		14.35%
re2_compile_524287	44_799_605.83ns		3.77w	0.77w	100.00%
needle_search_15_95	349.65ns	4.00w
re2_search_15_95	483.11ns
mshinwell_search_15_95	1_151.38ns	781.01w
needle_search_15_815	2_838.85ns	4.00w
re2_search_15_815	3_293.06ns
mshinwell_search_15_815	8_360.57ns	5_821.07w	0.55w	0.55w	0.02%
needle_search_15_2415	8_395.84ns	4.00w			0.02%
re2_search_15_2415	9_594.14ns				0.02%
mshinwell_search_15_2415	24_602.09ns	17_021.16w	1.62w	1.62w	0.05%
needle_search_1023_6143	14_825.50ns	4.00w			0.03%
re2_search_1023_6143	40_926.59ns				0.09%
mshinwell_search_1023_6143	81_930.46ns	49_149.66w	1_025.65w	1.65w	0.18%
needle_search_1023_52223	126_465.96ns	4.00w			0.28%
re2_search_1023_52223	365_359.98ns				0.82%
mshinwell_search_1023_52223	527_323.73ns	371_715.39w	1_033.17w	9.17w	1.18%
needle_search_1023_154623	377_539.53ns	4.00w			0.84%
re2_search_1023_154623	1_001_251.93ns				2.23%
mshinwell_search_1023_154623	1_499_835.01ns	1_088_518.15w	1_033.19w	9.19w	3.35%
needle_search_8191_49151	115_223.31ns	4.00w			0.26%
re2_search_8191_49151	559_487.38ns				1.25%
mshinwell_search_8191_49151	653_981.19ns	393_219.50w	8_201.01w	9.01w	1.46%
needle_search_8191_417791	976_725.24ns	4.00w			2.18%
re2_search_8191_417791	4_713_965.69ns				10.52%
mshinwell_search_8191_417791	4_224_417.93ns	2_973_709.32w	8_202.37w	10.37w	9.43%
needle_search_8191_1236991	2_912_863.78ns	4.00w			6.50%
re2_search_8191_1236991	14_039_230.59ns				31.34%
mshinwell_search_8191_1236991	11_997_713.73ns	8_708_130.87w	8_202.47w	10.47w	26.78%

Added to Set functions for converting to and from a Map.t.
```
val to_map : ('key, 'cmp) t -> f:('key -> 'data) -> ('key, 'data, 'cmp) Map.t
val of_map_keys : ('key, _, 'cmp) Map.t -> ('key, 'cmp) t
```
This required adding some additional type trickery to Core_set_intf to indicate that the comparator for a given module may or may not be fixed.
Added an optional iter parameter to Container.Make.

A direct implementation of iter is often more efficient than defining iter in terms of fold, and in these cases, the results of Container.Make that are defined in terms of iter will be more efficient also.
Added Int.pow (and for other integer types), for bounds-checked integer exponentiation.

ocaml_plugin

Added a tag to exceptions coming from the toplevel execution of plugins so that we do not confuse them with exceptions coming from the library.

Also, added a function to check a plugin without executing it. And captured the common pattern of checking the compilation of a plugin in a Command.t offered in the library.

sexplib

Added error locations to Macro-expansion errors.

111.08.00

async_extra

Added Log.Message.add_tags, which extends a message with a list of key-value pairs.
```
  val add_tags : t -> (string * string) list -> t
```

async_kernel

Changed low-level error messages to use Sexp.to_string_hum rather than to_string_mach.

async_ssl

Improved the propagation of SSL errors to the caller.

async_unix

Added Sys.when_file_changes : string -> Time.t Pipe.Reader.t.
Added Time.now () to some error messages.

core

Improved Command to print a good error message if command-line parsing raises.

Command's Exn.handle_uncaught now protects the phase of parsing command-line arguments in addition to protecting the phase of running the main function as it did already.

core_kernel

Added Hashtbl.for_all and for_alli.
Added Float.to_padded_compact_string for converting a floating point number to a lossy, compact, human-readable representation.

E.g., 1_000_001.00 becomes "1m ".
Tweaked the form of the definition of Blang.Stable.V1.

Removed a type t_ that is not necessary now that we can use nonrec without triggering spurious warnings.

enumerate

Restricted the signature of the all value generated by with enumerate.

Now, in:
```
  type t = [ `Foo | ... ] with enumerate
```
the type of all will be t list, instead of [> t] list.

When taking unions of polymorphic variant types, inserted the appropriate coercions.

Fixes #1.

jenga

Fix a hang.

Jenga could reach a state with a non-zero todo-count, but have no jobs actually running, and then hang in this state forever. The hang would be evident from a progress line with not all targets built and with j=0+0 such as:
```
todo: 17 (100406 / 100423) j=0+0 con=149956 act=3303, finish at: 16:20
```

ocaml_plugin

Use ocamldep to generate the dependencies of an .ml file, if requested.

Added a function to find the dependencies of a module, but did not change the existing behavior and interface of the library if one does not choose to use this functionality.

pa_test

Made it possible to use <:test_eq< >> and friends in Core_kernel.

Removed the dependency of the code generated by pa_test on Core_kernel, so pa_test can be used there.

re2

Upgraded to upstream library version 20140304.

Added options Dot_nl and Never_capture.

111.06.00

async_extra

Added ?on_wouldblock:(unit -> unit) callback to Udp.recvmmsg_loop and recvmmsg_no_sources_loop.
For functions that create Rpc connections, added optional arguments: ?max_message_size:int and ?handshake_timeout:Time.Span.t.

These arguments were already available to Connection.create, but are now uniformly available to all functions that create connections.

async_kernel

Improved the performance of Pipe.filter_map by using batching.

async_ssl

Initial release

async_unix

In the Busy_pollers.t record, made the kernel_scheduler field be sexp_opaque.

Did this so that one doesn't get two copies of the kernel scheduler in sexps of the scheduler, which already has its own kernel_scheduler field.

core

Added inline benchmarks for =Iobuf= and =Time=.

Hera are some of the results from the new benchmarks, with some indexed tests dropped.

Name	Time/Run	mWd/Run	Percentage
[time.ml:Time] Time.to_string	848.74ns	249.98w	100.00%
[time.ml:Time] Time.to_ofday	59.66ns	38.00w	7.03%
[time.ml:Time] Time.now	39.78ns	2.00w	4.69%
[time.ml:Time] Time.Zone.find_office	83.64ns	4.00w	9.85%
[time.ml:Time] Time.Span.of_hr	3.71ns	2.00w	0.44%
[time.ml:Time] Time.Span.of_min	3.69ns	2.00w	0.44%
[time.ml:Time] Time.Span.of_sec	2.72ns		0.32%
[time.ml:Time] Time.Span.of_ms	6.02ns	2.00w	0.71%
[time.ml:Time] Time.Span.of_ns	5.98ns	2.00w	0.71%

Name	Time/Run	Percentage
[iobuf.ml:Blit tests] functor blit:5	15.53ns	7.66%
[iobuf.ml:Poke tests] char:0	4.11ns	2.03%
[iobuf.ml:Poke tests] uint8:0	5.35ns	2.64%
[iobuf.ml:Poke tests] int8:0	4.59ns	2.26%
[iobuf.ml:Poke tests] int16_be:0	5.19ns	2.56%
[iobuf.ml:Poke tests] int16_le:0	5.14ns	2.53%
[iobuf.ml:Poke tests] uint16_be:0	5.11ns	2.52%
[iobuf.ml:Poke tests] uint16_le:0	5.12ns	2.53%
[iobuf.ml:Poke tests] int32_be:0	5.17ns	2.55%
[iobuf.ml:Poke tests] int32_le:0	4.91ns	2.42%
[iobuf.ml:Poke tests] uint32_be:0	5.73ns	2.83%
[iobuf.ml:Poke tests] uint32_le:0	5.74ns	2.83%
[iobuf.ml:Poke tests] int64_be:0	5.33ns	2.63%
[iobuf.ml:Poke tests] int64_le:0	4.93ns	2.43%
[iobuf.ml:Peek tests] char:0	5.50ns	2.71%
[iobuf.ml:Peek tests] uint8:0	4.68ns	2.31%
[iobuf.ml:Peek tests] int8:0	4.91ns	2.42%
[iobuf.ml:Peek tests] int16_be:0	5.19ns	2.56%
[iobuf.ml:Peek tests] int16_le:0	4.90ns	2.42%
[iobuf.ml:Peek tests] uint16_be:0	5.17ns	2.55%
[iobuf.ml:Peek tests] uint16_le:0	5.10ns	2.51%
[iobuf.ml:Peek tests] int32_be:0	5.17ns	2.55%
[iobuf.ml:Peek tests] int32_le:0	4.92ns	2.42%
[iobuf.ml:Peek tests] uint32_be:0	5.45ns	2.69%
[iobuf.ml:Peek tests] uint32_le:0	5.46ns	2.69%
[iobuf.ml:Peek tests] int64_be:0	6.61ns	3.26%
[iobuf.ml:Peek tests] int64_le:0	6.31ns	3.11%

Re-implemented Thread_safe_queue to improve performance and reduce allocation.

The new implementation requires 3 words per element, down from the 7 words required by the old implementation.

The new implementation pools elements so that they can be reused, so there is no allocation in steady-state use.

The new implementation has dequeue_exn rather than dequeue, so that one can dequeue without allocating 2 words.

Eliminated create'. One should just use create and explicit calls to enqueue and dequeue_exn.

Eliminated dequeue_until_empty. One should use an explicit while loop guarded by length and using dequeue_exn.

Moved Thread_safe_queue from Core_kernel to Core, since it's thread related.

All in, there is now no allocation in a steady-state usage of enqueueing and dequeueing elements, as opposed to 9 words per enqueue+dequeue in the old implementation. This reduces the cost from enqueue+dequeue taking 166-216ns to enqueue+dequeue_exn taking 48-82ns (plus eliminating gc impacts). Here are some BENCH results, the first table being the old implementation, and the second table the new.

Name	Time/Run	mWd/Run	mjWd/Run
[thread_safe_queue.ml] enqueue + dequeue of immediate	183.89ns	9.00w	7.02w
[thread_safe_queue.ml] enqueue + dequeue of young object	216.69ns	11.00w	9.01w
[thread_safe_queue.ml] enqueue + dequeue_exn of old object	166.75ns	9.00w	7.02w

Name	Time/Run	mWd/Run
[thread_safe_queue.ml] enqueue + dequeue_exn of immediate	48.20ns
[thread_safe_queue.ml] enqueue + dequeue_exn of young object	81.96ns	2.00w
[thread_safe_queue.ml] enqueue + dequeue_exn of old object	48.30ns

Changed {Bigstring,Iobuf}.recvmmsg_assume_fd_is_nonblocking, when no message is available, to return a negative number rather than raise.

This was done for performance reasons, because raising an exception is expensive, due to the stashing of the backtrace and the string creation.
Added Iobuf.unsafe_resize.
Changed Bigstring.blit so that it doesn't release the OCaml lock on map_file bigstrings.

The old behavior of releasing the lock for blits of (small) bigstrings involving mmapped files was problematic and inconsistent. Its cost is high, and fundamentally any access to a mapped bigstring could cause some level of blocking.
Added time-related Arg_type.t values to Command.Spec.
Added module Type_immediacy, which has witnesses that express whether a type's values are always, sometimes, or never immediate.

This code used to be in the Typerep_immediate library in typerep.

core_kernel

Added inline benchmarks for Array

Here are some of the results from the new benchmarks, with some indexed tests dropped.

Name	Time/Run	mWd/Run	mjWd/Run
[core_array.ml:Alloc] create:0	13.65ns
[core_array.ml:Alloc] create:100	99.83ns	101.00w
[core_array.ml:Alloc] create:255	201.32ns	256.00w
[core_array.ml:Alloc] create:256	1_432.43ns		257.00w
[core_array.ml:Alloc] create:1000	5_605.58ns		1_001.01w
[core_array.ml:Blit.Poly] blit (tuple):10	87.10ns
[core_array.ml:Blit.Poly] blito (tuple):10	112.14ns	2.00w
[core_array.ml:Blit.Poly] blit (int):10	85.25ns
[core_array.ml:Blit.Poly] blito (int):10	107.23ns	2.00w
[core_array.ml:Blit.Poly] blit (float):10	84.71ns
[core_array.ml:Blit.Poly] blito (float):10	86.71ns	2.00w
[core_array.ml:Blit.Int] blit:10	19.77ns
[core_array.ml:Blit.Int] blito:10	23.54ns	2.00w
[core_array.ml:Blit.Float] blit:10	19.87ns
[core_array.ml:Blit.Float] blito:10	24.12ns	2.00w
[core_array.ml:Is empty] Polymorphic '='	18.21ns
[core_array.ml:Is empty] Array.equal	8.08ns	6.00w
[core_array.ml:Is empty] phys_equal	2.98ns
[core_array.ml:Is empty] Array.is_empty (empty)	2.98ns
[core_array.ml:Is empty] Array.is_empty (non-empty)	3.00ns

Moved Thread_safe_queue to core
Generalized the type of Exn.handle_uncaught_and_exit to (unit -> 'a) -> 'a.

In the case where handle_uncaught_and_exit succeeds, it can return the value of the supplied function.

It's type had been:
```
val handle_uncaught_and_exit : (unit -> never_returns) -> never_returns
```

Added Int.round* functions for rounding to a multiple of another int.

val round : ?dir:[ `Zero | `Nearest | `Up | `Down ] -> t -> to_multiple_of:t -> t

val round_towards_zero : t -> to_multiple_of:t -> t
val round_down         : t -> to_multiple_of:t -> t
val round_up           : t -> to_multiple_of:t -> t
val round_nearest      : t -> to_multiple_of:t -> t

These functions were added to Int_intf.S, implemented by Int, Nativeint, Int32, and Int64.

Various int modules were also lightly refactored to make it easier in the future to implement common operators available for all modules implementing the int interface via a functor to share the code.

jenga

Improved the error message when the same library is defined multiple times.
Fixed an issue where jenga sometimes would sometimes complain about a self cycle when foo.ml uses a module Foo.
With -no-notifiers, jenga doesn't use inotify to watch for file changes. This is useful for linting jengaroot.ml.
Allowed writing jenga rules which restrict dependencies from an initial conservative approximation to a more accurate set discovered after an action is run

re2

Added Re2.Std, so that one should now use Re2 via module Re2 = Re2.Std.Re2.

At some future date, we will rename the Regex module to Re2_internal to force the stragglers to update to the new convention.

typerep

Renamed Typerep libraries for more consistency with the rest of the framework.

  Typerep_kernel --> Typerep_lib
  Typerep_core   --> Typerep_extended
  Typereplib     --> Typerep_experimental

111.03.00

async_extra

Add ?max_connections:int argument to Rpc.Connection.serve.

max_connections is passed to Tcp.Server.create, and limits the number of connections that an Rpc server will accept.
Improved Log.Rotation:
- Made Log.Rotation.t abstract; use create rather than an explicit record.
- Added a `Dated naming_scheme.
- Add Log.Rotation.default, for getting a sensible default rotation scheme.
- Added an optional (but discouraged) option to symlink the latest log file.
- Every log rotation scheme has an associated Time.Zone.t.
- Changed the internal representation of Log.Rotation.t, but t_of_sexp is backwards compatible, so existing config files will continue to work.
Changed Udp.bind_any to use Socket.bind ~reuseaddr:false, to ensure a unique port.
Added Tcp.Server.listening_socket, which returns the socket the server is listening on.

Changed Tcp.Server so that if the listening socket is closed, the server is closed.
Added to Udp.Config.t a max_ready : int field to prevent UDP receive loops from starving other async jobs.
Improved File_tail to cut the number of fstat calls in half.

File_tail uses a stat loop to monitor a file and continue reading it as it grows. We had made two fstat invocations per loop iteration, using Async.Std.Unix.with_file which constructs an Fd.t and therefore does it own fstat. Switching to Core.Std.Unix.with_file with In_thread.run eliminated the extra fstat.

async_unix

Improved Socket.accept to abort and return `Socket_closed when the file descriptor underlying the socket is closed.
Added to Socket.bind a ?reuseaddr:bool argument, preserving the default as true.
Added Fd.close_started, which becomes determined when close is called on an Fd.t.

bin_prot

Fixed build on ARM.

core

Added Unix.Syslog module.
Changed Command.run to no longer ignore the first element of its ~argv parameter.
Made Time.Span.to_short_string show microsecond precision.

core_extended

Added Set_lang, a DSL for sets with constants, union, intersection, and difference.
In Process, use epoll rather than select when possible,

This prevents errors when selecting on file descriptors numbered greater than FD_SETSIZE (1024).
Removed Syslog module. There is now Unix.Syslog in core instead; the APIs are not compatible, but they are similar.

core_kernel

Added Error.to_string_hum_deprecated that is the same as Error.to_string_hum pre 109.61.
Changed Error.to_string_hum so that Error.to_string_hum (Error.of_string s) = s.

This fixed undesirable sexp escaping introduced in 109.61 and restores the pre-109.61 behavior for the special case of Error.of_string. A consequence of the removal of the custom to_string_hum converter in 109.61 was that:
```
Error.to_string_hum (Error.of_string s) =
    Sexp.to_string_hum (Sexp.Atom s)
```
That introduced sexp escaping of s.

Added to Doubly_linked functions for moving an element within a list.

val move_to_front : 'a t -> 'a Elt.t -> unit
val move_to_back  : 'a t -> 'a Elt.t -> unit
val move_after    : 'a t -> 'a Elt.t -> anchor:'a Elt.t -> unit
val move_before   : 'a t -> 'a Elt.t -> anchor:'a Elt.t -> unit

Improved Core_map_unit_tests.Unit_tests to allow arbitrary data in the map, not just ints.

This was done by eta expansion.

custom_printf

Simplified the code generated by pa_custom_printf to make it more readable.

re2

Fixed a bug with replace_exn and anchoring.

Fixed this bug:

$ R.replace_exn ~f:(fun _ -> "a") (R.create_exn "^") "XYZ";;
- : string = "aXaYaZa"

$ R.replace_exn ~f:(fun _ -> "a") (R.create_exn "^X") "XXXYXXZ";;
- : string = "aaaYXXZ"

textutils

Changed Textutils.Console to not reference Async.Log, so that building inline benchmark runners no longer requires building Async.

The change makes Textutils, and by extension Core_extended and Core_bench, not depend on Async.

110.01.00

async_extra

Added Cpu_usage.Sampler for directly sampling CPU usage.
Fixed Log.rotate to never raise.
Fixed two bugs in Log rotation.
- Log rotation had used the wrong date when checking whether it should rotate.
- Made Rotation.keep = \At_least` delete the oldest, rather than the newest, logs.

async_kernel

Added Deferred.Result.map_error.

core

Fixed Time unit tests that failed in London because of timezone dependence.
Added Iobuf.protect_window_and_bounds, which calls a user function and restores the iobuf's bounds afterwards.
Fixed compilation on OpenBSD, which doesn't support Unix.mcast_join's ?source : Inet_addr.t argument.

core_kernel

Changed Queue from a linked to an array-backed implementation.

Renamed the previous implementation to Linked_queue.

Renamed transfer, which was constant time, as blit_transfer, which is linear time.

Removed partial_iter. One can use with_return.

Added singleton, filter, get, set.
For Error and Info, changed to_string_hum to use sexp_of_t and Sexp.to_string_hum, rather than a custom string format.
Changed the output format of Validate.errors to be a sexp.
Added Hashtbl.of_alist_or_error and Map.of_alist_or_error.
Added String_id.Make functor, which includes a module name for better error messages.
Exposed Bucket.size.
Changed the default for Debug.should_print_backtrace to be false rather than true.

Usually the backtraces are noise.
Removed the tuning of gc parameters built in to Core, so that the default is now the stock OCaml settings.

Such tuning doesn't belong in Core, but rather done per application. Also, the Core settings had fallen way out of date, and not kept up with changes in the OCaml runtime settings. We have one example (lwt on async) where the Core settings significantly slowed down a program.
Added Exn.raise_without_backtrace, to raise without building a backtrace.

raise_without_backtrace never builds a backtrace, even when Backtrace.am_recording ().
Made with_return faster by using Exn.raise_without_backtrace.
Improved with_return to detect usage of a return after its creating with_return has returned.

ocaml_plugin

Added cmi's so that plugins can use lazy, recursive modules, and objects.

pa_test

For <:test_result< >>, renamed ~expected to ~expect

sexplib

Added with sexp support for mutually recursive types with common fields.

For instance:
```
  type a = { fl : float } and b = { fl : int } with sexp
```
Closes #3
Fixed the behavior of sexplib on private types.

sexplib used to ignore the private modifier, which means generated functions had the wrong type. Now, it generates a function with the right type for sexp_of and refuses to generate anything for of_sexp.
Added Macro.expand_local_macros, which macro expands sexps, failing on :include macros.
Fixed Macro's handling of nested :include's which was broken with respect to paths.

Prior to this fix, :include's were broken with respect to the path used to resolve the filename. Including a file outside the current directory which included another file relative to that one would break.

109.60.00

async_extra

Replaced Tcp_file.serve's ~port:int argument with Tcp.Where_to_listen.inet.

async_kernel

Changed the scheduler to clear a job from its queue when it runs the job, eliminating a performance regression from 109.57.

Clearing avoids spurious promotion of what would otherwise be dead data associated with already-executed jobs.

async_unix

Fixed a bug in detection of the thread pool being stuck that could overstate the amount of time the pool was stuck.

It had been incorrectly reporting the duration of the thread pool being stuck if the pool had no work in it and then got enough jobs to be stuck. In that situation, the duration included the time span before the pool got stuck. If the pool had been idle long enough, this could even spuriously abort the program.

comparelib

Fixed a type error in with compare of polymorphic variant inclusions.

core

Added Iobuf.unsafe_advance.

This can be used to benchmark inner loops that have redundant bounds checking, to see if the checks matter. For example, see the following two advance calls:

let rec process_buffer buf ~f =
  let len = Iobuf.length buf in
  if header_len <= len then
    let msg_len = header_len + Iobuf.Unsafe.Peek.uint16_be buf ~pos:0 in
    if msg_len <= len then begin
      let len = msg_len - header_len in
      Iobuf.advance buf header_len;
      f (Protocol.packed_of_iobuf buf);
      Iobuf.advance buf len;
      process_buffer buf ~f
    end

Added Weak_hashtbl.add_exn and sexp_of_t.
Fixed Lock_file.create to behave correctly if the target mountpoint is out of space.

Previously in this situation, Lock_file.create would create an empty lock and exit with exception trying to write pid/message there. Subsequent runs would not able to read pid out of empty pid file and blocking_create would block instead of removing defective lock.

core_kernel

Added Gc.keep_alive, which ensures its argument is live at the point of the call.
Added Sexp.With_text module, which keeps a value and the a sexp it was generated from, preserving the original formatting.

custom_printf

Compatibility with warning 7 (method override)

faillib

Compatibility with warning 7 (method override)

pipebang

Compatibility with warning 7 (method override)

sexplib

Moved unix-specific code to a new object section, sexplib_unix

type_conv

Compatibility with warning 7 (method override)

109.58.00

async_extra

Changed Cpu_usage to use Core.Percent instead of float where appropriate.
Made Bus.unsubscribe check that the subscriber is subscribed to the given bus.
Made Log.t support with sexp_of.
Fixed Tcp.on_port 0 to return the port actually being listened on, like Tcp.on_port_chosen_by_os.

Previously, a serverlistening on Tcp.on_port 0 would have its Tcp.Server.listening_on as 0, which of course is not the port the server is listening on.

async_kernel

Renamed the Async_core library as Async_kernel, to parallel Core_kernel.

Someday Async_core will depend only on Core_kernel, but not yet.
Added a thread-safe queue of "external actions" that is checked after each job.
Fixed a race condition in Clock.Event.abort.

Here is the race condition:
- Clock.Event.at adds an alarm, its value is a job (let's call it job1) with this run function:
```
let fire () =
  t := Happened;
  Ivar.fill ready `Happened;
```
- later a job (let's call it job2) aborting the clock event is queued in the async scheduler
- in the same cycle, the Timing_wheel.advance_clock fires the alarm and job1 scheduled
- at this point:
  - job1 and job2 are still pending
  - the alarm was removed so it is invalid
  - the clock event is still in the state Waiting
- job2 is executed before job1: the clock event is still in the Waiting state, so the abort tries to remove the alarm from the timing wheel: CRASH
  
  The bugfix is for Clock.Event.abort to check if the alarm has already been removed from the timing wheel and if so, don't remove it again.
Changed Monitor.try_with when run with ~rest:`Ignore, the default, so that the created monitor is detached from the monitor tree.

The detached monitor has no parent, rather than being a child of the current monitor. This will eliminate recently observed space leaks in Sequencer_table and Throttle, like:
```
let leak () =
  let seq = Throttle.Sequencer.create () in
  let rec loop n =
    Throttle.enqueue seq (fun () ->
      loop (n + 1);
      Deferred.unit
    )
    |> don't_wait_for
  in
  loop 0
```
Changed Async's scheduler to pool jobs rather than heap allocate them, decreasing the cost of a job by 30-40%.

Changed the main scheduler queue of jobs to be an Obj_array.t that is essentially a specialized Flat_queue (the specialization was necessary for speed).

Also, cleaned up the scheduler run-job loop.

With these changes, the cost of a simple job decreases significantly (30-40%), across a range of live data sizes. Here are the nanoseconds-per-job numbers for a microbenchmark with the old and new approaches.

| num live jobs | old ns/job | new ns/job | |---------------|------------|------------| | 1 | 74 | 53 | | 2 | 75 | 47 | | 4 | 76 | 41 | | 8 | 63 | 39 | | 16 | 62 | 38 | | 32 | 61 | 37 | | 64 | 61 | 37 | | 128 | 60 | 37 | | 256 | 60 | 38 | | 512 | 60 | 38 | | 1024 | 60 | 39 | | 2048 | 61 | 40 | | 4096 | 67 | 41 | | 8192 | 65 | 45 | | 16384 | 75 | 56 | | 32768 | 115 | 67 | | 65536 | 171 | 108 | | 131072 | 255 | 158 | | 262144 | 191 | 130 | | 524288 | 216 | 139 | | 1048576 | 238 | 152 |

See async/bench/nanos_per_job.ml for the benchmark.
Removed debug_space_leaks from Async's internals. It hadn't been used in years.

async_unix

Improved fairness of the async scheduler with respect to external threads, including I/O done in external threads.

The change is to add a thread-safe queue of "external actions" that is checked after each job.

Previously, when a job given to In_thread.run finished, In_thread.run would take the async lock, fill the result ivar and run a cycle. The problem is that in some situations, due to poor OS scheduling, the helper thread never had a chance to grab the lock. Now, In_thread.run tries to take the lock:
- if it can it does as before
- if it can't it enqueues a thunk in the external actions queue and wakes up the scheduler
With this change, the helper thread doing an In_thread.run will always quickly finish once the work is done, and the async scheduler will fill in the result ivar as soon as the current job finishes.
Fixed Epoll_file_descr_watcher.invariant to deal with the timerfd, which has the edge-triggered flag set.
Added Writer.write_gen, a generic functor for blitting directly to a writer's buffer.

core

Added Debug.should_print_backtrace : bool ref, to control whether Debug.am* functions print backtraces.
Added to Float inline benchmarks.
Moved all of the Gc module into Core_kernel.

Part of the Gc module used to be in Core because it used threads. But it doesn't use threads anymore, so can be all in Core_kernel.
Improved Iobuf support for copying to/from strings and bigstrings.

The new modules are Iobuf.{Consume,Peek}.To_{bigstring,string}. They match a Blit-like interface. We don't yet implement the Blit interface in all applicable contexts, but do use Blit.Make and expose some of the operations we want in the forms we expect them to take under a Blit interface.
Added Linux_ext.Timerfd.to_file_descr.
Added to Time.next_multiple an optional argument to control whether the inequality with after is strict.
Added Time.Zone.local, a lazily cached Time.Zone.machine_zone ().

This is the first stage in a plan to make explicit timezones more pervasive. First, they are made more convenient, by replacing the relatively wordy Time.Zone.machine_zone () with Time.Zone.local. This involves making the underlying timezone type lazy.

The next stage will be to remove machine_zone and use Time.Zone.local everywhere instead. Then (it is hoped) instead of of_local_whatever, we just say e.g. of_date_ofday Time.Zone.local and currently-implicitly-local functions will be able to switch over to explicit timezones without becoming too verbose.
Added Timing_wheel.Alarm.null.
Made Unix.File_descr.t have with sexp.

Closes janestreet/async_unix#3
Fixed OpenBSD compilation failures in C stubs.
Fixed Lock_file.is_locked to require read permission, not write permission, on the lock file.
Added to Unix.mcast_join an optional ?source:Inet_addr.t argument.

From pull-request on bitbucket: https://bitbucket.org/janestreet/core/pull-request/1/receive-source-specific-multicast/diff

core_bench

Added support for saving inline benchmark measurements to tabular files for easy loading into Octave.

core_extended

Cleaned up the Stats_reporting module

core_kernel

Moved all of the Gc module into Core_kernel.

Part of the Gc module used to be in Core because it used threads. But it doesn't use threads anymore, so can be all in Core_kernel.
Made Stable.Map and Set have with compare.
Added String.rev.

Closes janestreet/core#16

We will not add String.rev_inplace, as we do not want to encourage mutation of strings.
Made Univ_map.Key equivalent to Type_equal.Id.
Added Univ.view, which exposes Univ.t as an existential, type t = T : 'a Id.t * 'a -> t.

Exposing the existential makes it possible to, for example, use Univ_map.set to construct a Univ_map.tfrom a list of Univ.ts.

This representation is currently the same as the underlying representation, but to make changes to the underlying representation easier, it has been put in a module Univ.View.

core_profiler

Added support for saving inline benchmark measurements to tabular files for easy loading into Octave.

109.55.00

async_extra

Added Udp.recvmmsg_no_sources_loop, a specialization of recvmmsg_loop for improved performance.

This improvement was driven by profiling at high message rates.

async_unix

Fixed Scheduler.is_running to not initialize the scheduler.
Added Writer.make_write, which is a general function for blitting directly to a writer's buffer.
Added Writer.transfer', which is like Writer.transfer but allows async actions in the callback.

This was requested in pull request #1.
Added Writer.write_iobuf, which blits the contents of an iobuf directly to a writer's buffer.

core

Fixed building on FreeBSD and OpenBSD.
Added with typerep to many Core types.
Made open Core.Std support with typerep.
Added Iobuf.recvmmsg_assume_fd_is_nonblocking_no_options, a specialization of recvmmsg_assume_fd_is_nonblocking for improved performance.

This improvement was driven by profiling at high message rates.
Changed Unix.Rlimit.virtual_memory be an Or_error.t, for platforms where it is undefined.

core_bench

Improved bench.mli's generated docs and added some usage examples.

This also partly satisfies issue #3.

Added the ability to create groups of benchmarks with a common prefix.

For example, the prefix "Perf" below is created in created using create_group:

let command = Bench.make_command [
  Bench.Test.create ~name:"Time.now" (fun () ->
    ignore (Time.now ()));

  ...

  Bench.Test.create_group ~name:"Perf" [
    Bench.Test.create ~name:"TSC.now" ...

and the output shows:

Estimated testing time 7s (7 benchmarks x 1s). Change using -quota SECS.
┌───────────────────────────────────────────┬──────────┬─────────┬────────────┐
│ Name                                      │ Time/Run │ mWd/Run │ Percentage │
├───────────────────────────────────────────┼──────────┼─────────┼────────────┤
│ Time.now                                  │  41.38ns │   2.00w │     16.72% │
│ Calibrator.calibrate                      │ 247.42ns │  32.00w │    100.00% │
│ Perf/TSC.now                              │   7.84ns │         │      3.17% │
│ Perf/TSC.to_time                          │   9.35ns │   2.00w │      3.78% │
│ Perf/TSC.to_time (TSC.now ())             │  13.22ns │   2.00w │      5.34% │
│ Perf/TSC.to_nanos_since_epoch             │  10.83ns │         │      4.38% │
│ Perf/TSC.to_nanos_since_epoch(TSC.now ()) │  14.86ns │         │      6.00% │
└───────────────────────────────────────────┴──────────┴─────────┴────────────┘

core_extended

Added Service_command.acquire_lock_exn, for acquiring a service's lock.

core_kernel

Added with typerep to many Core types.
Changed Flat_queue to raise if the queue is mutated during iteration.
Improved Map.merge to run in linear time.

core_profiler

Improved bench.mli's generated docs and added some usage examples.

This also partly satisfies issue #3.

Added the ability to create groups of benchmarks with a common prefix.

For example, the prefix "Perf" below is created in created using create_group:

let command = Bench.make_command [
  Bench.Test.create ~name:"Time.now" (fun () ->
    ignore (Time.now ()));

  ...

  Bench.Test.create_group ~name:"Perf" [
    Bench.Test.create ~name:"TSC.now" ...

and the output shows:

Estimated testing time 7s (7 benchmarks x 1s). Change using -quota SECS.
┌───────────────────────────────────────────┬──────────┬─────────┬────────────┐
│ Name                                      │ Time/Run │ mWd/Run │ Percentage │
├───────────────────────────────────────────┼──────────┼─────────┼────────────┤
│ Time.now                                  │  41.38ns │   2.00w │     16.72% │
│ Calibrator.calibrate                      │ 247.42ns │  32.00w │    100.00% │
│ Perf/TSC.now                              │   7.84ns │         │      3.17% │
│ Perf/TSC.to_time                          │   9.35ns │   2.00w │      3.78% │
│ Perf/TSC.to_time (TSC.now ())             │  13.22ns │   2.00w │      5.34% │
│ Perf/TSC.to_nanos_since_epoch             │  10.83ns │         │      4.38% │
│ Perf/TSC.to_nanos_since_epoch(TSC.now ()) │  14.86ns │         │      6.00% │
└───────────────────────────────────────────┴──────────┴─────────┴────────────┘

pa_bench

Added support for inline benchmarks using the BENCH syntax, similar to TEST.

This feature allows users to specify inline benchmarks in any library.

One can specify a benchmark using the following syntax:
```
  BENCH "name" = expr
```
In the above, the value of expr is ignored. This creates a benchmark for expr, that is run using the inline_benchmarks_runner script from the command-line. This workflow is similar to that of inline unit tests.

One can specify benchmarks that require some initialization using BENCH_FUN. For example:
```
  BENCH_FUN "name" =
    let t = create () in
    (fun () -> test_something t)
```
The function returned on the RHS of BENCH_FUN should have type unit -> unit. One can specify benchmarks that have a variable parameter using BENCH_INDEXED. For example:
```
  BENCH_INDEXED "Array.create" len [1;10;100;1000] =
    (fun () -> Array.create ~len 0)
```
The above snippet measures the time taken to create arrays of different length. Indexed tests are useful in highlighting non-linearities in the execution time of functions.

We can group benchmarks together into modules and the output of inline_benchmarks_runner will reflect this grouping.
```
  BENCH_MODULE "Blit tests" = struct

    ..some benchmarks..

  end
```
For examples of all of the above see core_gc.ml and core_array.ml.

Only the generated inline_benchmarks_runner.exe depends on Core_bench and other libraries. The library that includes the the benchmarks itself does not have a dependency on Core_bench. Doing this is important so that we can add benchmarks to Core and still avoid cyclic dependencies. Finally, it is important to note that adding inline benchmarks should have little effect on the execution or module initialization time.

109.53.00

async

Bump version number

async_extra

Added module Bus, which is an intraprocess "broadcast" communication mechanism.
Added Tcp.to_inet_address and to_unix_address.
Added Tcp.to_socket which creates a Tcp.where_to_connect from a Socket.Address.Inet.t.
Module Weak_hashtbl is now implemented as a wrapper around Core.Weak_hashtbl.

No intended change in behavior.

async_unix

Changed the scheduler to calibrate Time_stamp_counter every second.
Improved error messages in the scheduler when epoll functions raise.
Made Scheduler.reset_in_forked_process close the epoll file descriptor.

bin_prot

Bump version number

core

Added Linux_ext.Epoll.close.
Added Weak_hashtbl module, moved from Async.

It had only been in Async to use Async's finalizers. The move to Core exposes a bit more with respect to finalization so that one can still implement Async.Weak_hashtbl, as well as do other things (e.g. use Weak_hashtbl in Incremental, which does not use Async).

Simplified the implementation of Weak_hashtbl to eliminate "entry ids". They were only used to avoid removing a table entry that was in use. But there is a more direct way to test for that -- just check if the weak is None or Some.
Added an autoload file for utop
Disabled warning 40 in corebuild

core_bench

Fixed a bug in Core_bench where the linear regression was sometimes supplied with spurious data.

This showed up when doing custom regressions that allow for a non-zero y-intercept.

core_extended

Fixed Flang and Olang to round-trip via sexps, i.e. (t_of_sexp (sexp_of_t t)) = t.

core_kernel

Added Float.to_string_round_trippable, which produces a string that loses no precision but (usually) uses as few digits as possible.

This can eliminate noise at the end (e.g. 3.14 not 3.1400000000000001243).

Benchmarks:

New sexp:

Name	Time/Run	mWd/Run	Percentage
new Float.sexp_of 3.14	463.28ns	6.00w	48.88%
new Float.sexp_of e	947.71ns	12.00w	100.00%

Old sexp:

Name	Time/Run	mWd/Run	Percentage
old Float.sexp_of 3.14	841.99ns	178.00w	98.03%
old Float.sexp_of e	858.94ns	178.00w	100.00%

Much of the speedup in the 3.14 case comes from the fact that format_float "%.15g" is much faster than sprintf "%.15g". And of course the above does not capture any of the benefits of dealing with shorter strings down the road.

Here are some detailed benchmarks of the various bits and pieces of what's going on here:

Name	Time/Run	mWd/Run	Percentage
format_float '%.15g' 3.14	335.96ns	2.00w	32.71%
format_float '%.17g' 3.14	394.18ns	4.00w	38.38%
format_float '%.20g' 3.14	459.79ns	4.00w	44.77%
format_float '%.40g' 3.14	638.06ns	7.00w	62.13%
sprintf '%.15g' 3.14	723.71ns	165.00w	70.47%
sprintf '%.17g' 3.14	803.44ns	173.00w	78.23%
sprintf '%.20g' 3.14	920.78ns	176.00w	89.66%
sprintf '%.40g' 3.14	990.09ns	187.00w	96.41%
format_float '%.15g' e	357.59ns	4.00w	34.82%
format_float '%.17g' e	372.16ns	4.00w	36.24%
format_float '%.20g' e	434.59ns	4.00w	42.32%
format_float '%.40g' e	592.78ns	7.00w	57.72%
sprintf '%.15g' e	742.12ns	173.00w	72.26%
sprintf '%.17g' e	747.92ns	173.00w	72.83%
sprintf '%.20g' e	836.30ns	176.00w	81.43%
sprintf '%.40g' e	1_026.96ns	187.00w	100.00%
valid_float_lexem 12345678901234567	76.29ns	9.00w	7.43%
valid_float_lexem 3.14	9.28ns	5.00w	0.90%
float_of_string 3.14	130.19ns	2.00w	12.68%
float_of_string 1234567890123456.7	184.33ns	2.00w	17.95%
to_string 3.14	316.47ns	7.00w	30.82%
to_string_round_trippable 3.14	466.02ns	9.00w	45.38%
to_string e	315.41ns	7.00w	30.71%
to_string_round_trippable e	949.12ns	15.00w	92.42%

Replaced Float.min_positive_value with min_positive_normal_value and min_positive_subnormal_value.
Added some functions to Float.O: abs, of_float, and Robustly_comparable.S.
Small improvements to the Heap module.

Implemented Heap.iter directly rather than in terms of fold.

In heap.ml, fixed the idiom for using Container.Make.
Added an Int.O and other Int*.O modules, with arithmetic operators, infix comparators, and a few useful arithmetic values.
Added Int.( ~- ), for unary negation.
Added Pool.unsafe_free.
Added Percent module.

core_profiler

Fixed a bug in Core_bench where the linear regression was sometimes supplied with spurious data.

This showed up when doing custom regressions that allow for a non-zero y-intercept.

ocaml_plugin

Bump version number

pa_ounit

Bump version number

pa_test

Bump version number

patdiff

Bump version number

re2

Bump version number

sexplib

Changed sexp_of_float to (usually) use as few digits as possible, without losing precision.
Split the part of Sexplib that depends on Num into a separate library, Sexplib_num.

This was done to eliminate the dependence of Core_kernel on Num, which is not usable on javascript.

textutils

Bump version number

type_conv

Bump version number

109.52.00

async_extra

Added module Cpu_usage, which publishes CPU-usage statistics for the running process.
Fixed Sequencer_table.enqueue so that there is no deferred between finding the state and calling the user function.

async_kernel

Changed Pipe.iter_without_pushback to never call f after Pipe.close_read has been called.

The new behavior is like Pipe.iter.
Changed the implementation of Pipe.fold_gen and Pipe.transfer_gen to be analogous to Pipe.iter_without_pushback, and in particular to process as many elements as possible before calling values_available.

async_unix

Fixed a bug in Unix.mkstemp, which had a race because it used Fd.create in a thread.

This bug affected Writer.with_file_atomic, save, save_lines, and save_sexp, and could cause corruption of the async scheduler data structures.
Changed async to never do set_nonblock on std{in,out,err}, which allows Core I/O to use the standard file descriptors simultaneously with async.

Before this change, the Core I/O libraries could (and sometimes did) fail due to Sys_blocked_io.
Changed Pipe.iter_without_pushback to never call f after Pipe.close_read has been called.

The new behavior is like Pipe.iter.

Changed the implementation of Pipe.fold_gen and Pipe.transfer_gen to be analogous to Pipe.iter_without_pushback, and in particular to process as many elements as possible before calling values_available.
Added ?expand_macros:bool argument to Reader.load_sexp* functions, to support the new Sexplib macros.
Added an optional argument to Process.run to accept nonzero exits as successful runs.

core

Added Unix.File_descr.equal.
Added Lock_file.Nfs.unlock, the Or_error version of unlock_exn.
Improved the detail of the error messages exposed by Lock_file.Nfs.create{,_exn}.
Added Unix.set_mcast_ifname, to control the interface used for UDP multicast traffic.

Added bindings for setsockopt IP_MULTICAST_IF.

See 6.3 in: http://www.tldp.org/HOWTO/Multicast-HOWTO-6.html
Changed Command argument processing to treat a single dash (-) as an anonymous argument rather than a flag.

This change follows the unix convention of passing - as an anonymous argument meaning stdin.
Added more bin-prot support to Iobuf: Consume.bin_prot, Fill.bin_prot, Peek.bin_prot, Poke.bin_prot.

Framing doesn't do much for Iobuf, so these are to be more standard, unframed accessors, as opposed to fill_bin_prot.
Added Core.Debug.am, amf, and ams, for outputting debugging messages showing the current source-code position.

Unfortunately, these aren't available in Core.Std.Debug, but only in Core.Debug. That will be fixed in 109.49.
Made Time_stamp_counter compile on non x86-64 platforms.
Made Core.Std.Debug be Core.Debug rather than Core_kernel.Debug.

This exposes the Debug.am* functions added in 109.48.

core_bench

Exposed an extensible form of make_command so that inline-benchmarking and the other tools can add more commandline flags.
A significant rewrite of Core_bench.

The rewrite provides largely the same functionality as the older version. The most visible external change is that the API makes it clear that Core_bench performs linear regressions to come up with its numbers. Further, it allows running user-specified multivariate regressions in addition to the built in ones.

The underlying code has been cleaned up in many ways, some of which are aimed at improving the implementation of inline benchmarking (the BENCH syntax, which has not yet been released).

core_extended

Removed Sexp.load_sexp_with_includes; one should use the new Sexplib.Macro functions.
Added Blang-like languages Flang and Olang.
- Flang -- terms over a field.
- Olang -- predicates over an ordered set.

core_kernel

Added to Binary_packing module functions for packing and unpacking signed 64-bit ints in little- and big-endian.
Changed the Comparator interfaces to no longer have with bin_io or with sexp.

The Comparator interfaces are now just about having a comparator.

Also, renamed type comparator as type comparator_witness. And, removed Comparator.S_binable, since one can use:
```
type t with bin_io
include Comparator.S with type t :` t
```
Changed Comparator.Make to return a module without a type t, like other *able functors,

This made it possible to remove the signature constraint when Comparator.Make is applied.
Made Comparable.S_binable be like Comparable.S and not have type t with sexp.

The following two functors now fail to type check:
```
module F1 (M : Comparable.S        ) : sig type t with sexp end ` M
module F2 (M : Comparable.S_binable) : sig type t with sexp end ` M
```
whereas previously F1 was rejected and F2 was accepted.
Changed the Monad.Make functor to require a val map argument.

This was done since we almost always want a specialized map, and we kept making the mistake of not overriding the generic one in the three places needed.

Added Monad.map_via_bind, which one can use to create a standard map function using bind and return.
Removed unnecessary signature constraints on the result of applying Monad.Make.

Some time ago, Monad.Make changed from returning:
```
S with type 'a t ` 'a M.t
```
to returning:
```
S with type 'a t :` 'a M.t
```
so we no longer need to constrain the result of Monad.Make at its uses to remove t.
Changed String.exists and String.for_all to iterate by increasing index rather than decreasing.
Added with compare to module Ref.
Made Flags be Comparable, with the order consistent with bitwise subset.
Cleaned up the implementation of Union_find.

Improvemed the code in union_find.ml:
- Removed an assert false.
- do not reallocate a parent node during compress. This should result in more stability for sets memory wise.
- Added implementation notes.
- Renamed internal variant constructors.
- Added unit tests.
Added Float.O, a sub-module intended to be used with local opens.

The idea is to be able to write expressions like:
```
Float.O.((3. + 4.) > 6. / 2.)
```
This idiom is expected to be extended to other modules as well.
Added a sexp_of_t converter to Type_equal.Id.
Replaced Univ.Constr with Type_equal.Id.
Added Debug.eprintf, analogous to eprint and eprints.

core_profiler

Exposed an extensible form of make_command so that inline-benchmarking and the other tools can add more commandline flags.
A significant rewrite of Core_bench.

The rewrite provides largely the same functionality as the older version. The most visible external change is that the API makes it clear that Core_bench performs linear regressions to come up with its numbers. Further, it allows running user-specified multivariate regressions in addition to the built in ones.

The underlying code has been cleaned up in many ways, some of which are aimed at improving the implementation of inline benchmarking (the BENCH syntax, which has not yet been released).

pa_ounit

Added -stop-on-error flag to inline_test_runner, to stop running tests at the first failure.

This is useful if the remaining tests are likely to fail too or just long to run.

pa_test

Added an optional ?message argument to <:test_eq<>>, <:test_result<>> and <:test_pred<>>.

re2

Fixed bugs in Re2.Regexp.find_all and Re2.Regexp.find_first.
- memory leaks on errors
- unlikely garbage in their return values, or even segfaults (especially unlikely for find_first)

sexplib

Added a Macro module, with load_sexp* functions that support file includes and templates.

The following new syntaxes are supported:

  (:include filename)
  (:let f (arg1 ... argn) sexp1 ... sexpn)
  (:use f (arg1 valn) ... (argn valn))
  (:concat a1 ... an)

Added support to with sexp for a subset of GADTs.

The new support is for types that use existentially quantified variables or plain variants written with GADT syntax.

Existentially quantified variables still have to be wrapped with sexp_opaque generate compiling code.
Fixed a type error in the code generated by with sexp in some cases of variant inclusions.

textutils

Added Console.Log module for writing colorized Async.Log messages.

type_conv

Removed comments from pretty-printed types in type_conv error messages.

109.47.00

async_extra

Added with sexp to Log.Output.machine_readable_format and format.

async_kernel

Fix a bug introduced in Monitor.error in 109.28, in which the error wasn't seen unless someone is listening to the monitor.

async_unix

Added Socket.Address.Inet.to_host_and_port.
Changed Fd_by_descr so that it actually calls Bounded_int_table.invariant.

bin_prot

Compilation fix for 32-bit systems

core

Added Time_stamp_counter module, which has fast (< 10 nanos) access to the hardware time-stamp counter.

This module provides the fast function Time_stamp_counter.now () which is our best effort high-performance cycle counter for a given platform. For x86 systems this retrieves the CPU's internal time stamp counter using the RDTSC instruction. For systems that do not have a RDTSC instruction, we fallback to using clock_gettime(CLOCK_MONOTONIC).

Here is a benchmark of execution time in nanos and allocations in words:
```
Name                            Time/Run   Minor
------------------------------- ---------- -------
Time.now                           39.02    2.00
TSC.now                             7.54
TSC.to_time                         4.88    2.00
TSC.to_time (TSC.now ())            8.54    2.00
TSC.to_time_nanos                   4.49
TSC.to_time_nanos(TSC.now ())       8.95
Calibrator.calibrate                 279   34.00
```
Historically, the rate of increment of the TSC (sometimes referred to as the TSC frequency) varied based of CPU overclocking, temperature, load etc. On modern Intel CPU's the TSC is expected to be stable. On Linux systems, the "constant_tsc" in /proc/cpuinfo indicates that the machine has a stable TSC rate. While this module assumes that the TSC is relatively stable, it can adapt to small variations in the TSC frequency.
Changed Daemon.daemonize and daemonize_wait to leave the umask alone by default.

Previously, these had alwasy set the umask to 0, which means that all app-harness programs and all binaries run from grass were creating world-writeable (0666) files by default.

core_kernel

Added Error.to_info and of_info.
Significantly sped up Float.iround_* functions.

For iround_down_exn, the new version appears to use about 25% of the CPU time of the old version on non-negative floats. For negative floats it uses around 60% of the CPU time.

Name Time (ns) % of max

old iround_down_exn pos 15.02 95.23

new iround_down_exn pos 3.75 23.75

old iround_down_exn neg 15.78 100.00

new iround_down_exn neg 9.80 62.10
Added Binary_searchable.Make functor to core, and used it in Array and Dequeue.
Fixed Bounded_int_table to match Invariant.S2.
Added to Pool support for 10-, 11-, and 12- tuples.
Added functions to the Gc module to get usage information without allocating.

Added these functions, all of type unit -> int:
```
minor_collections
major_collections
heap_words
heap_chunks
compactions
top_heap_words
```
They all satisfy:
```
Gc.f () = (Gc.quick_stat ()).Gc.Stat.f
```
They all avoid the allocation of the stat record, so one can monitor the garbage collector without perturbing it.

Name	Time (ns)	% of max
old iround_down_exn pos	15.02	95.23
new iround_down_exn pos	3.75	23.75
old iround_down_exn neg	15.78	100.00
new iround_down_exn neg	9.80	62.10

pa_test

Added new syntax <:test_result<>>, which is similar to <:test_eq<>>.

test_result is meant for the common case of comparing some hardcoded expected value against one returned by some code you're testing. In this case you can now write, for example:
```
<:test_result<int>> 3 ~expected:4
```
which behaves exactly the same way as:
```
<:test_eq<int>> 3 4
```
except that the error message is something like:
```
("got unexpected result"
  ((got 3) (expected 4)
    (Loc "File \"z.ml\", line 44, characters 25-25")))
```
rather than:
```
("comparison failed"
  (3 vs 4
    (Loc "File \"z.ml\", line 44, characters 25-25")))
```
The additional labels on the values help identify the problem earlier.

type_conv

Made type nonrec work when a type has both an equation and a representation.

For example:

type t = A of t
module T = struct
  type nonrec t = t = A of t
end

109.45.00

async_extra

Added ?abort:unit Deferred.t argument to Lock_file.waiting_create, Lock_file.Nfs.waiting_create and critical_section.

async_kernel

Removed internal Backpatched module.

Async used to use this module, but it doesn't anymore.

async_unix

Added Fd.every_ready_to Fd.interruptible_every_ready_to which register a callback to be called every time the fd becomes ready.

These can significantly reduce allocation.
Renamed Fd.ready_to_interruptible as Fd.interruptible_ready_to.
Changed Fd.ready_fold to use Fd.interruptible_ready_to, to improve its performance.

core

Added Core.Std.phys_same, which is like phys_equal, except has a more general type.
```
val phys_equal : 'a -> 'a -> bool
val phys_same  : _  -> _  -> bool
```
phys_same is useful when dealing with existential types, and one has a packed value and an unpacked value that one wants to check are physically equal. One can't use phys_equal in such a situation because the types are different.

Added Iobuf.set_bounds_and_buffer and set_bounds_and_buffer_sub, which make it easier to use with zero allocation.

(** [set_bounds_and_buffer ~src ~dst] copies bounds (ie limits + window) and shallowly
    copies the buffer from [src] to [dst].  [read_write] access is required on [src]
    because the caller might have [read_write] access to [dst], and would after the call
    then effectively have [read_write] access to [src]. *)
val set_bounds_and_buffer : src:(read_write, _) t -> dst:(_, seek) t -> unit

(** [set_bounds_and_buffer_sub ?pos ?len ~src ~dst ()] is a more efficient version of:
    [set_bounds_and_buffer ~src:(Iobuf.sub ?pos ?len src) ~dst].

    [set_bounds_and_buffer ~src ~dst] is not the same as
    [set_bounds_and_buffer_sub ~dst ~src ()], because the limits are narrowed in the
    latter case. *)
val set_bounds_and_buffer_sub
  :  ?pos:int
  -> ?len:int
  -> src:(read_write, _) t
  -> dst:(_, seek) t
  -> unit -> unit

Added ?timeout:Time.Span.t argument to Lock_file.blocking_create, Lock_file.Nfs.blocking_create and critical_section.

core_extended

Fixed Core_extended.Sys.groups to use Unix.Group.getbygid rather than Unix.Group.getbygid_exn.

This handles when a group is deleted and its gid remains in the cache, which causes Unix.Group.getbygid_exn to fail because the gid no longer resolves to a group.

core_kernel

Changed Blang.bind to short-circuit And, Or, and If expressions.

For example if bind t1 f false, thenbind (and_ t1 t2) false, and will not evaluate bind t2 f.
Renamed Dequeue.get as get_opt, and get_exn as get, to be consistent with other containers which don't use the _exn suffix for subscripting exceptions.
Removed Source_code_position.to_sexp_hum, in favor of sexp_of_t_hum, which works smoothly with with sexp.
Changed Flat_queue_unit_tests to run Flat_queue.invariant, which was mistakenly not being used.

ocaml_plugin

Made executables link without error even if no archive is embedded in them.

This is often the desired behavior (for inline tests of libraries using transitively ocaml-plugin for instance).

109.44.00

async_extra

Fixed a time-based race condition in Log rotation.

async_kernel

Documented that Throttle.enqueue t f never runs f immediately, and added unit tests.

bin_prot

Remove "unwrapped" pointers used by Bin_prot, with the bug from 109.41 fixed.

Unwrapped pointers cannot coexist with the remove-page-table optimization.

Removed all the C stubs for reading/writing and used instead either the new primitives of the next OCaml or standard OCaml code reading/writing integers byte by byte.

Since we don't have unsafe/safe functions anymore but only safe ones, removed all the bin_{read,write}_t_ functions.

Also renamed bin_read_t__ to __bin_read_t__ for the same reason as sexplib: to avoid confusion with the function generated for t_ and hide it in the toplevel.

core

Added val Day_of_week.num_days : from:t -> to_:t -> int.
Added Time.of_date_ofday_precise and Time.Zone.next_clock_shift, to deal with clocks going forward and backward.

Due to clocks going forward/backward, some local times occur twice, and some never occur at all. Time.of_date_ofday_precise identifies these cases and returns all of the relevant information.

Added accessors for Unix.Cidr: base_address and bits.

(** Accessors.
    - [base_address 192.168.0.0/24 ` 192.168.0.0]
    - [bits         192.168.0.0/24 ` 24]. *)
val base_address : t -> Inet_addr.t
val bits         : t -> int

core_kernel

Implemented Dequeue.iter directly, instead of as a specialization of fold.

Extended random tests to cover iter.

109.42.00

async_extra

Fixed Log.Blocking so that when async is running it writes the message in syslog before failing with an exception.

async_kernel

In ASYNC_CONFIG, replaced alarm_precision and timing_wheel_level_bits with timing_wheel_config.

This parallels the new Timing_wheel.Config module.

async_unix

Added Reader.drain.
```
val drain : t -> unit Deferred.t
```

Added Writer.with_close.

val with_close : t -> f:(unit -> 'a Deferred.t) -> 'a Deferred.t

bin_prot

Backed out the changes introduced in 109.41

core

Removed Zone from Core.Std; use Time.Zone instead.
Removed Time.Date; use Date instead.
Improved the performance of Piecewise_linear by using arrays with binary search on indices.

The previous implementation Piecewise_linear used (float * float) list (a list of (x,y) pairs) to represent piecewise linear functions, with a linear search through the knots to evaluate the function at a point. This type is now:
```
{ x : float array
; y : float array
}
```
and the implementation uses binary search to identify the correct array index.

Here are the costs of the get function under the old (list) and new (array) implementations for various numbers of knots:
```
knots |  old | new
------+------+-----
    1 |  11ns| 12ns
    2 |  18ns| 14ns
    5 |  27ns| 19ns
  100 | 182ns| 38ns
1000 |1974ns| 52ns
```
Added module Time.Ofday.Zoned, which is a pair of an Time.Ofday.t and a Time.Zone.t.
Added with compare to Time.Zone.Stable.t.
Added Timing_wheel functionality.
- Added Config module, which combines alarm_precision and timing_wheel_level_bits.
- Removed the need to supply a dummy value to create.
- Added mem and clear functions.
- Added functions for dealing with the interval number: interval_num, now_interval_num, interval_num_start, add_at_interval_num.
This makes it easier to use a timing wheel with int interval numbers, which are more efficient than than float times.

core_kernel

Added Array.is_sorted_strictly and List.is_sorted_strictly.

val is_sorted_strictly : 'a t -> cmp:('a -> 'a -> int) -> bool

Added Array.find_consecutive_duplicate and List.find_consecutive_duplicate.

val find_consecutive_duplicate : 'a t -> equal:('a -> 'a -> bool) -> ('a * 'a) option

Added Array.truncate, which changes (shortens) the length of an array.
```
val truncate : _ t -> len:int -> unit
```
Improved the debugging message in Bounded_int_table.remove to show the data structure's details.
Added Float.iround_lbound and iround_ubound, the bounds for rounding to int.
Added Hashtbl.similar, which is like equal, but allows the types of the values in the two tables to differ.
Added Pool.Pointer.phys_compare, which is analagous to phys_equal, and does not require an argument comparison function.
```
val phys_compare : 'a t -> 'a t -> int
```
Exposed that Pool.Debug's output types are the same as its input types.

109.41.00

async_parallel

Rename library from Parallel to Async_parallel

async_unix

Changed Reader to treat more errno responses to a read() system call as EOF rather than raise.

The following are now treated as EOF.
```
ENETDOWN
ENETRESET
ENETUNREACH
ETIMEDOUT
```
Improved the error message that async prints when the thread pool is stuck, including the time of the error.

bin_prot

Remove all uses of "unwrapped" pointers

Unwrapped pointers cannot coexist with the remove-page-table optimization.

Removed all the C stubs for reading/writing and used instead either the new primitives of the next OCaml or standard OCaml code reading/writing integers byte by byte.

Since we don't have unsafe/safe functions anymore but only safe ones, removed all the bin_{read,write}_t_ functions.

Also renamed bin_read_t__ to __bin_read_t__ for the same reason as sexplib: to avoid confusion with the function generated for t_ and hide it in the toplevel.

core

Added Command.Spec.map_anon and map_flag.

(** [map_flag flag ~f] transforms the parsed result of [flag] by applying [f] *)
val map_flag : 'a flag -> f:('a -> 'b) -> 'b flag

(** [map_anons anons ~f] transforms the parsed result of [anons] by applying [f] *)
val map_anons : 'a anons -> f:('a -> 'b) -> 'b anons

Fixed Unix.open_flag to compile with OCaml 4.01.

It needed the additional constructor O_CLOEXEC.

core_bench

Columns that have a + prefix are now always displayed, whereas columns that don't are displayed only if they have meaningful data.
Added the ability to reload saved metrics (benchmark test data) so that bench can re-analyze them.

core_kernel

Added Map.of_alist_reduce.

This function is a natural addition alongside of_alist_fold. Its advantage is that it does not require an init argument like of_alist_fold. Moreover, it does not involve option types, like List.reduce does in order to handle the empty list case.

core_profiler

Columns that have a + prefix are now always displayed, whereas columns that don't are displayed only if they have meaningful data.
Added the ability to reload saved metrics (benchmark test data) so that bench can re-analyze them.

ocaml_plugin

Added option -strict-sequence, which is set to true by default.

type_conv

Fixed the generated code of typerep and sexplib on sum types containing True or False.

Without this fix, typerep would wrong constructor names for Blang.t, for instance.

Variantslib has the same problem but applying the same fix there would still not make the generated code compile because the generated code would contain labels and variable named true or false.

Other syntax extensions should not be affected because they don't build strings from constructor names.

109.40.00

async_extra

Added to Udp.Config the ability to stop early, via stop : unit Deferred.t.

async_unix

Added value Socket.Type.unix_dgram, which represents a UNIX domain datagram socket.
Added UDP socket functionality: Socket.Opt.mcast_{loop,ttl} and Socket.mcast_{join,leave}.
Improved Fd.ready_fold to accept ?stop:(unit Deferred.t) rather than ?stop:('a -> bool).

core_extended

Added Stats_reporting.Delta, for recording deltas of values.

109.39.00

async_unix

Added "thread-local" storage, Scheduler.{find,with}_local, for LWT emulation.

(** [with_local key value ~f] runs [f] right now with the binding [key ` value].  All
    calls to [find_local key] in [f] and computations started from [f] will return
    [value]. *)
val with_local : 'a Univ_map.Key.t -> 'a option -> f:(unit -> 'b) -> 'b

(** [find_local key] returns the value associated to [key] in the current execution
    context. *)
val find_local : 'a Univ_map.Key.t -> 'a option

core_bench

Added support for additional predictors like minor/major GCs and compactions, using multi-variable linear regression.

Replaced linear regression with multi-variable linear regression. The original algorithm estimated the cost of a function f by using a linear regression of the time taken to run f vs the number of runs. The new version adds the ability to include additional predictors such as minor GCs, compactions etc.

This allows a more fine-grained split-up of the running costs of a function, distinguishing between the time spent actually running f and the time spent doing minor GCs, major GCs or compactions.
Added a forking option that allows benchmarks to be run in separate processes.

This avoids any influence (e.g. polluting the cache, size of live heap words) they might otherwise have on each other.

core_kernel

Implemented Heap.iter directly instead of in terms of fold.

core_profiler

Added support for additional predictors like minor/major GCs and compactions, using multi-variable linear regression.

Replaced linear regression with multi-variable linear regression. The original algorithm estimated the cost of a function f by using a linear regression of the time taken to run f vs the number of runs. The new version adds the ability to include additional predictors such as minor GCs, compactions etc.

This allows a more fine-grained split-up of the running costs of a function, distinguishing between the time spent actually running f and the time spent doing minor GCs, major GCs or compactions.
Added a forking option that allows benchmarks to be run in separate processes.

This avoids any influence (e.g. polluting the cache, size of live heap words) they might otherwise have on each other.

109.38.00

async_extra

In Rpc, exposed accessors for binary protocol values.

For example, this allows one to write a wrapper for Pipe_rpc that allows for the easy re cording and replaying of values the come over the pipe.

async_unix

Added Reader.of_pipe and Writer.of_pipe, for converting from string Pipe.t's.

These can be used to add arbitrary transformations (e.g. encryption, compression) to code that expects plain file- or socket-based readers and writers.

109.37.00

core

Command.run now calls Exn.handle_uncaught so you don't have to.
Fixes for building on FreeBSD.
Fixed Blang to build with OCaml 4.01.

In blang.mli:

Blang.t is a private variant type, Blang.Stable.V1.t is a private variant type, and client code knows Blang.t = Blang.Stable.V1.t. Previously, this was done in a strange way, using with type 'a t = private 'a t on the signature of Blang.Stable.V1. In addition to being strange, this line no longer builds in OCaml 4.01, which caused problems for building Real World Ocaml.

This patch changed the code to something much more straightforward, although not quite so straightforward as we expect to be able to achieve once a nonrec bug is fixed.

core_kernel

Added Core.Std.Poly as a short name for Core.Std.Polymorphic_compare.
Exposed module Core.Std.Decimal.

109.36.00

async_unix

Added Process.run_lines, which runs a process and returns stdout as a list of strings.

core_extended

In Sexp module, added ability to expand and compress bash-like brace wildcards.

core_kernel

Made Hashtbl.Poly.hash equal Caml.Hashtbl.hash, and changed changed String.hash and Float.hash to match OCaml's hash function.

Previously, Core.Poly.hash had been defined as:
```
let hash x = hash_param 10 100 x
```
This fell out of sync with OCaml's hash function, and was providing worse hash values.
Fixed Obj_array.singleton to never create a float array.

Also made it clearer that Obj_array.copy could never create a float array.
Changed Pool.create to allow zero-length pools.

Previously, Pool.create ~capacity:0 had raised, which made it easy to write code that blows up on edge cases for no apparent reason. For example, Heap.copy was written in a way that copying an empty heap would blow up (regardless of its capacity), and Heap.of_array would also blow up on an empty array.

Added String.split_lines.

(** [split_lines t] returns the list of lines that comprise [t].  The lines do
    not include the trailing ["\n"] or ["\r\n"]. *)
val split_lines : t -> t list

pa_ounit

Simplified so that it does not generate unnecessary top-level bindings.

It had been hiding quite a few unused import warnings.

textutils

In Ascii_table module, added support for displaying table using ASCII characters instead of Unicode.

This is motivated by the need to use Core_bench in contexts where the extended ASCII character set is not suitable for displaying tables.

The default style is the following:

┌─────────────────┬───────────┬──────────┐
│ Name            │ Time (ns) │ % of max │
├─────────────────┼───────────┼──────────┤
│ quick_stat      │     93.11 │   100.00 │
│ counters        │     33.24 │    35.70 │
│ allocated_bytes │     37.03 │    39.77 │
└─────────────────┴───────────┴──────────┘

The new style is as follows:

$ ./test_bench.exe gc -q 0.5 -ascii
Estimated testing time 1.5s (change using -quota SECS).

  Name              Time (ns)   % of max
----------------- ----------- ----------
  quick_stat            93.17     100.00
  counters              34.56      37.09
  allocated_bytes       37.06      39.78

109.35.00

async_extra

Added module Async.Udp, aimed at high-performance UDP applications.
Added module Lock_file.Nfs, which wraps the functions in Core.Std.Lock_file.Nfs.

async_kernel

Added new configuration options for Async, max_inter_cycle_timeout and max_num_jobs_per_priority_per_cycle.
```
  val max_inter_cycle_timeout : Time.Span.t
  val max_num_jobs_per_priority_per_cycle : int
```
These are configurable as usual via ASYNC_CONFIG.
Added an ASYNC_CONFIG option to debug the Shutdown module.
Added find and find_map to Deferred.Monad_sequence.

async_unix

Made some configuration possible via additional optional arguments to go_main.
```
?max_num_open_file_descrs:int
?max_num_threads:int
```

Made some aspects of the async scheduler configurable via functions in Scheduler.

val set_check_invariants                  : bool        -> unit
val set_detect_invalid_access_from_thread : bool        -> unit
val set_max_inter_cycle_timeout           : Time.Span.t -> unit
val set_record_backtraces                 : bool        -> unit

Added a dynamic check in Pipe that a consumer is used with the correct pipe.

Specifically, check that a consumer supplied to a read operation on a Pipe.Reader.t was previously created by add_consumer with that same reader.
Renamed Pipe.fold as fold_without_pushback and added Pipe.fold with an analogous type to Pipe.iter.
Fixed a bug in Pipe.merge, which did not always close the resulting pipe when the merge was finished.

This had prevented medusa regtests from working correctly.
In Writer, changed the default buffer_age_limit for files to `Unlimited.

THis was done for the same reason that we treat files specially in flush on close -- slowness will likely be resolved eventually with a file, unlike with a socket.

core

Added or_error functions in Unix.Exit_* types to unit Or_error.t.

This makes it easier to deal with combining with infix operators >>=? and >>|?

core_extended

Added stable versions of types contained in the Selector module.

core_kernel

Added with compare to List.Assoc.t.
Made Pooled_hashtbl.create handle non-positive and very large sizes in the same way as Core.Hashtbl.

Added is_error, is_ok, and does_raise to Core.Std.

let is_error ` Result.is_error
let is_ok    ` Result.is_ok
val does_raise : (unit -> _) -> bool

Reimplemented Heap and reworked the interface to be more standard.

The new implementation uses pairing heaps and Pool.
Added a module Pool.Unsafe, which is like Pool, except that create doesn't require an initial value.

This makes it unsafe to access pool pointers after they have been freed. But it is useful for situations when one isn't able to create an initial value, e.g. Core.Heap.
Removed Time.to_localized_string and Time.to_string_deprecated.

These did not include the time-zone offset. Instead, use Time.to_string and Time.to_string_abs, which do include the time-zone offset.
Exposed that Int63.t = private int on 64-bit machines.

This lets the OCaml compiler avoid caml_modify when dealing with it.

Added Gc stat functions that don't allocate: Gc.minor_words, Gc.major_words, Gc.promoted_words.

Added the following Gc functions:

Gc.minor_words : unit -> int
Gc.major_words : unit -> int
Gc.promoted_words : unit -> int

such that these functions cause no allocations by themselves. The assumption being that 63-bit ints should be large enough to express total allocations for most programs. On 32-bit machines the numbers may overflow and these functions are not as generally useful.

These functions were added because doing memory allocation debugging with Gc.quick_stat as the primary means of understanding allocations is difficult: tracking down allocations of the order of a few hundred words in a hot loop by putting in lots of quick_stat statements becomes too intrusive because of the memory allocations they cause.

Here are some benchmarks of existing Gc functions and the newly added functions:

$ ./test_bench.exe -q 2 -clear name time +alloc +time-err
Estimated testing time 12s (change using -quota SECS).

Name	Time (ns)	95% ci	Time R^2	Minor
quick_stat	92.16	+0.72 -0.64	1.00	23.00
counters	33.63	+0.26 -0.23	1.00	10.00
allocated_bytes	37.89	+0.34 -0.32	1.00	12.00
minor_words	4.63	+0.03 -0.02	1.00
major_words	4.36	+0.02 -0.02	1.00
promoted_words	4.10	+0.03 -0.02	1.00

faillib

New syntax extension to get improved location information on calls to failwiths.

Added failwithp, which is like failwiths but takes a _here_ argument:
```
  val failwithp : Lexing.position -> string -> 'a -> ('a -> Sexp.t) -> _
```
pa_fail automatically converts failwiths into failwithp _here_

If you don't want to compile with pa_fail, then you can manually write failwithp _here_. Otherwise, running the same source through pa_fail gets you location information for free.

ocaml_plugin

Changed the execution of plugin's toplevel to run in async instead of In_thread.run, unless a config parameter says otherwise.

textutils

Added new module Textutils.Text_graph for plotting text graphs on a terminal.

Here is an example density plot for minor_words from mcquote-stats.data.

  (354 (3%) values outside the range, bucket size is 38.58)
      74.00     3786 |----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----|
     112.58        0 |
     151.17        0 |
     189.75        0 |
     228.33        0 |
     266.92        0 |
     305.50        0 |
     344.08        0 |
     382.67        0 |
     421.25        0 |
     459.83     3653 |----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+-
     498.42        4 |
     537.00     1055 |----+----1----+----2----+--
     575.58      308 |----+---
     614.17       70 |-
     652.75      404 |----+----1
     691.33       71 |-
     729.92       32 |
     768.50       18 |
     807.08      151 |---
     845.67       17 |
     884.25        4 |
     922.83        8 |
     961.42       66 |-
    1000.00        0 |

Each - is approximately 37.860 units.

109.34.00

async_kernel

Added a function to Pipe that merges a list of sorted pipes

val Pipe.merge : 'a Reader.t list -> cmp:('a -> 'a -> int) -> 'a Reader.t
Improved the performance of Ivar.squash by removing the allocation of the list of ivars.

Instead, squash does one loop to find the end of the chain and a second loop to set all the indirections.
Changed Scheduler.run_cycles_until_no_jobs_remain to raise an exception if there is an unhandled exception when it finishes.

async_unix

Changed the scheduler to detect when the thread pool is stuck, i.e. when all threads are blocked and haven't made progress.

Added default handlers for when the thread pool is stuck, and the ability for the user to configure their own handlers.

Changed low-level I/O to not use nonblocking I/O for char devices.

This fixes a problem due to epoll not working with /dev/null. For example:

let () =
  Reader.read_line (Lazy.force Reader.stdin)
  >>> fun _ ->
  shutdown 0
;;
let () = never_returns (Scheduler.go ())

had failed when run like:

+ ./z.exe </dev/null
("bug in async scheduler"
((Unix.Unix_error "Operation not permitted" epoll_ctl "")
...

Made Unix.openfile use fstat to determine the kind of file for a file descriptor rather than assuming it's a regular file.
Improved the ASYNC_CONFIG option detect_invalid_access_from_thread by having it include a backtrace in the error message.

core

Added val Backtrace.get_opt : unit -> t option.

This is more convenient to use than Backtrace.get, which is an Or_error.t.
Moved functions for dealing with finalizers into the Gc.Expert module.

This was done to make people be very explicit and sure that they want finalizers, which are very hard to use because they essentially introduce multithreading semantics.

One should typically use async finalizers.
Eliminated the thread that had been used to sequentialize all finalizers.

core_extended

Improved Sexp.Diff.

Labeled arguments, put them in the right order (old before new), and rework the code to follow the same convention, and produce the output where deletions precede insertions.

core_kernel

Added a new module, Flat_queue, which is a queue of flat tuples.

This is essentially:
```
('a1 * .. * 'aN) Queue.t
```
However the queue is implemented as a Flat_array, so the tuples are layed out flat in the array and not allocated.
Improved Bounded_int_table.remove's error message when it detects an internal inconsistency.
Added new Debug module.
Changed Invariant.invariant to take _here_ rather than a string.
Made Float satisfy the Identifiable interface.

pa_test

Added new quotation expanders <:test_eq< >> and <:test_pred< >>.

These can be used to reduce boilerplate in testing code.
- <:test_eq< type >> expands to a function of type
```
  ?(f = (fun x y -> <:compare< type >> x y = 0))
-> ?here:Lexing.position list
-> type
-> type
-> unit
```
  that throws a nice exception if the values are not equal according to f.
- <:test_pred< type >> expands to a function of type
```
?here:Lexing.position list -> (type -> bool) -> type -> unit
```
  that throws a nice exception if the predicate fails on the value.

109.33.00

async_extra

Change Log.Global to by default send all output, including `Info, to stderr.

Replaced Log.Output.screen with Log.Output.stderr. There is now also and Log.Output.stdout.

109.32.00

async_extra

Added Dynamic_port_writer.

Dynamic_port_writer solves the problem of communicating a dynamically selected tcp port from a child process to its parent.

async_kernel

Improved the batching of Pipe.fold and other Pipe functions that handle batches.

Previously, such functions used a loop with Pipe.read. This didn't batch as well as it might. If values were put in the pipe after the read becomes determined but before the values are handled, then they wouldn't be handled until the next batch. Now, batching functions use values_available and then pull elements out of the pipe synchronously after waking up. This makes the batch as large as possible.

core

Normalized Command's help messages.

Made anonymous argument names uppercase and subcommand names lowercase.

In Iobuf, added duals to flip and snapshot to work on the high end of the window.

flip has been renamed to flip_lo. The dual of flip_lo is the newly added flip_hi, and shifts the window forward to continue reading, rather than back to switch from writing to reading, as flip_lo does.

flip_hi, in practice, needs snapshots of the upper bound of the window, we split Snapshot into Lo_bound and Hi_bound and introduced bounded versions of flip_lo, compact, and flip_hi to support buffers which are only partially filled, but have substructure, like packet buffers.

Here's the new API.

module type Bound = sig
  type ('d, 'w) iobuf

  (** Expose =t = private int= only if a =t= is stored in a mutable data structure
     somewhere and leads to a measurable =caml_modify= performance problem. *)
  type t with sexp_of

  val window : (_, _) iobuf -> t
  val limit  : (_, _) iobuf -> t
  val restore : t -> (_, seek) iobuf -> unit
end
module Lo_bound : Bound
module Hi_bound : Bound
val flip_lo         : (_, seek) t -> unit
val bounded_flip_lo : (_, seek) t -> Lo_bound.t -> unit
val flip_hi         : (_, seek) t -> unit
val bounded_flip_hi : (_, seek) t -> Hi_bound.t -> unit

core_bench

Changed -save to output compaction information.
Added indexed tests.

These are benchmarks of the form int -> unit -> unit, which can be profiled for a list of user specified ints.

core_kernel

Added val Option.merge: 'a t -> 'a t -> f:('a -> 'a -> 'a) -> 'a t.
Added val Validate.failf : ('a, unit, string, t) format4 -> 'a.
In Validated.Make_binable, made it possible to apply the validation function when un-bin-io-ing a value.
Added module Pooled_hashtbl to module type Hashable.

This is an alternative implementation to Core.Hashtbl. It uses a standard linked list to resolve hash collisions, and Pool to manage the linked-list nodes.

core_profiler

Changed -save to output compaction information.
Added indexed tests.

These are benchmarks of the form int -> unit -> unit, which can be profiled for a list of user specified ints.

ocaml_plugin

Fixed the slow and memory-consuming compilation of > 100MB .c files generated by ocaml_embed_compiler.

This was done by having them contain one big string instead of one big array.
Added more unused-value warnings in plugins.

If { Ui , M } are the modules that constitute a given plugin of expected module type S, then previously we generated a file like:
```
module Ui : sig
  ...
end = struct
  ...
end

module M : sig
  ...
end = struct
  ...
end

let () = ##register (M : S)
```
Doing that, we did not get unused variables:
1. for the toplevel of Ui if Ui does not have a mli.
2. for unused values of Ui and M if they have an mli exporting them.
OCaml plugin now allows one to get these warnings. Since (2) is rather annoying for utils kind of file, this comes only if a config flag is enabled.

re2

Fixed a bug in the C bindings that could cause a segfault.

Fixed a bug where mlre2__create_re in C can give OCaml a freed C string.

The bug was in:
```
if (!compiled->ok()) {
  compile_error = compiled->error().c_str();
  delete compiled;
  compiled = NULL;
  caml_raise_with_string(*caml_named_value("mlre2__Regex_compile_failed"),
      compile_error);
}
```
This is in mlre2__create_re if we fail to compile the regular expression. Notice how we delete the re object before we use the pts to its' error string. (Notice that in C++ c_str() returns a pointer to the internal data of the string object it does NOT create a copy and error() just returns a reference to the regular expression objects error string member *error_).

So if caml_raise_with_string has to allocate on the heap to create the exception or the copy of the string that might invalidate the ptr before we will copy it.

109.31.00

async_unix

Renamed Reader.read_one_chuck_at_a_time_until_eof as Reader.read_one_chunk_at_a_time, and added a new case to the handle_chunk callback.

The name change is to reflect that one can stop early, before EOF.

The new handle_chunk case allows one to specify the number of bytes that were read in the Stop case.

Also, changed read_one_chunk_at_a_time to use do_read instead of just locking the file without unlocking it. This allows subsequent read operations to read from the file.

core_kernel

Renamed some functions in module Lazy: dropped the lazy_ prefix from is_val, from_val, and from_fun.

ocaml_plugin

Fixed OCaml Plugin on CentOS 5 -- it had problems because the generated c files did not end with a newline.
Finished the transition from Command_deprecated to Command.

109.30.00

async_kernel

Added function Throttle.kill.

Throttle.kill aborts all jobs that have been enqueued but not started, and immediately aborts all jobs subsequently enqueued.

Split out Throttle debugging and unit-testing code into their own modules.
Changed the semantics of Throttle.enqueue on a dead throttle so that the exception is sent to the monitor rather than raised synchronously.

This gives more uniform treatment to the race between enqueueing a job and an already running job raising. Now the enqueued job is always aborted, whether enqueued before or after the raise.
Added an ASYNC_CONFIG option to print debug messages when Monitor.send_exn is called.

This is useful when one is debugging an application in which an exception is being unexpectedly swallowed.
Allow one to dynamically configure the behavior of =Monitor.try_with=.

This is to allow experimentation with different handling of asynchronous exceptions after Monitor.try_with has become determined.

async_unix

Changed aync's scheduler to use epoll rather than select by default.

This is based on a dynamic test to see whether timerfd_create works.
Added support for "busy polling".

This runs a thread that busy loops running user-supplied polling functions. The busy-loop thread is distinct from async's scheduler thread, but it acquires the async lock so that the user-supplied function can do ordinary async operations, e.g. fill an ivar.

Busy polling is useful for a situation like a shared-memory ringbuffer being used for IPC. One can poll the ringbuffer with a busy poller, and then when data is detected, fill some ivar that causes async code to handle the data.
Added Async.Fd.clear_nonblocking.

This clears the nonblocking bit on the file descriptor underlying the fd, and causes async to treat the fd as though it doesn't support nonblocking I/O.

This is useful for applications that want to share a file descriptor between async and non-async code and want to avoid Sys_blocked_io being seen by the non-async code.

core

Created submodule Core.Signal.Expert module.

This is for functions previously in Core.Signal that introduce multithreading semantics, and are hence hard to reason about and should only be used by experts.

core_bench

Report compaction stats

core_kernel

Added module, Core.Blit, which codifies the type, implementation, and unit-testing of blit functions.
Added remove_zero_flags option to Flags.Make, to support flags that are zero.

This fixes a problem with Flags.Make on CentOS 5 because O_CLOEXEC is 0 there.
Removed Pool.None, and folded Pool.Obj_array into Pool proper.

Pool.None had its day, but Pool.Obj_array dominates it, so we don't need it any more.

core_profiler

Report compaction stats

ocaml_plugin

Support for Mac OSX

Removed the dependency of ocaml-plugin on objcopy and /proc.

109.28.00

async_extra

Fixed an error message in Versioned_rpc that was swapping which versions were supported by the caller and the callee.

async_kernel

Eliminated a messy dependency cycle in async_core, so that Monitor.t no longer contains a Tail.t.

async_core was messy because of cycle between the following types:
```
Execution_context
--> Scheduler
--> Ivar
--> Deferred
--> Tail
--> Monitor
```
This messiness caused the need for the Raw signature, for the various Scheduler_dependent functors, for making various types polymorphic in execution_context, and then instantiating the polymorphism later.

The cause of the problem was that Monitor contained a Tail. We eliminated that, so that there is no longer a cycle, and defined things in order:
```
Monitor
Execution_context
Job
Scheduler
Ivar
Deferred
Tail
Stream
```
Replaced the errors tail from the monitor type:
```
errors : (exn, 'execution_context) Raw_tail.t;
```
with a list of handlers:
```
mutable error_handlers : (exn -> unit) list;
```
Cleaned up all the messiness caused by the cycle -- eliminated the Raw signature, the Scheduler_dependent functors, and the unnecessary polymorphism.

Cleaned up the long standing annoyance with Async.Stream, in which we couldn't expose the next sum type and people had to use annoying Stream.of_raw calls. Stream.of_raw is now gone.

core

Moved Timing_wheel from Zero.

core_extended

In Shell functions, made the amount of captured stderr/stdout configurable.

core_kernel

Moved all the contents of the Zero library into Core, mostly into Core_kernel.

We want to start using Zero stuff more in Core, which couldn't be done with Zero as a separate library.

Everything moved into Core_kernel, except for Timing_wheel, which moved into Core proper, due to its dependence on Time.
Renamed Flat_tuple_array as Flat_array.
Added Dequeue.{front,back}_index_exn

These are more efficient than using {front,back}_index and then Option.value_exn.
Exposed Core.String.unsafe_{get,set}.

pipebang

Fixed pa_pipebang in the toplevel.

pa_pipebang had registered an AST filter for implementations but not for the toplevel.

type_conv

Fixed an issue with type_conv in the toplevel.

Used AST filters for the _no_unused_value_warning_ machinery. type_conv modifies the parser but it didn't work well in the toplevel.

Removed the parsing_mli reference, an instead always add the special _no_unused_value_warning_ type and just strip it for signature items.

109.27.00

async_extra

Added function Versioned_typed_tcp.Client.shutdown.
Added new module Sequencer_table, which is a table of Throttle.Sequencer's indexed by keys.

async_kernel

Fixed Monitor.catch_stream to prevent missing a synchronous exception.

async_unix

Fixed a performance problem in the scheduler due to repeated calls of Timing_wheel.min_elt.

Timing_wheel.min_elt is an important part of async, since the scheduler calls it once per cycle to know when to timeout for epoll or select. This causes a problem if min_elt is slow and called repeatedly, which happens in an application where the next clock event is a second out, and yet there are lots of cycles per second.

Timing_wheel.min_elt now caches the minimum element, which eliminates the problem.
Fixed async's clock to work on 32-bit machines.

With the change to Timing_wheel in 109.22, async no longer worked on 32-bit machines, due to the clock overflowing. This is because it is initialized to Time.epoch, and can only handle 6 days.

The fix now in place is to start the clock at Time.now () rather than Time.epoch.
Added many functions to Async.Sys so that it looks more like Core.Sys.
Changed Reader.read_one_chunk_at_a_time_until_eof to not destroy the reader buffer.

Destroying the buffer failed if user code held on to the buffer.

comparelib

Changed how with compare treats option's so that None < Some, like Pervasives.compare.

Previously, with compare had treated Some < None.

core

Disabled use of recvmmssg, which isn't available on our CentOS 5 machines.
Defined Option.compare using with compare so that their comparisons are consistent.
Cleaned up the Dequeue module's interface and implementation.

The interface now matches the conventions used elsewhere in Core. The new implementation is also cleaner and more efficient.
Reimplemented the Stack module to improve performance, and renamed the old implementation as Linked_stack.

The new Stack is implemented with this type:
```
type 'a t `
  { dummy : 'a;
    mutable length : int;
    mutable elts : 'a array;
  }
```
Linked_stack is implemented with this type:
```
type 'a t `
  { mutable length : int;
    mutable elts : 'a list;
  }
```
Using an array rather than a linked list is a more efficient and traditional implementation. Pushing to the stack now does not require allocation, except in the rare case when the stack grows.

One downside is that Stack.clear is now O(n) rather than O(1).

This change also eliminates the Stack.Empty exception, so any code matching on that exception should fail to compile, and should be changed to depend on option-returning top and pop operations.
Improved Lock_file.Nfs.
- Allowed an arbitrary message to be stored and retreived.
- Fixed a case where create might throw an exception.
- Delete both files used for locking when we unlock.
Split Core into Core_kernel and Core.
Core_kernel is composed of all modules of Core that do not depend on unix or threads, and Core contains the rest and depends on Core_kernel.

The goal is to have a very portable standard library that can especially run on exotic environment such as Javascript.

So that code that directly refers to Core (rather than Core.Std) for modules that have moved to Core_kernel, we included "proxy" modules in Core that simply include the corresponding module from Core_kernel.
Fixed Core.Flags to build on 32-bit machines.

It had contained a unit test with an integer literal too large to be accepted by the compiler when building on a 32-bit machine.

core_bench

Added R^2 error estimation.

Adding this metric should give us a sense of how closely the given values fit a line. Even dots that are fairly scattered can give tight confidence intervals. We would like to have to number to have a sense of how much noise we have.

core_extended

In module Sexp, changed and renamed load_includes_in_sexp.

From:

val load_includes_in_sexp : ?max_depth:int -> Sexp.t -> Sexp.t

to:

val load_sexp_with_includes: ?max_depth:int -> ?buf:string -> string -> Sexp.t

Added function Sexp.Diff.to_string.
Previously the only option was to print to Out_channel.

core_profiler

Added R^2 error estimation.

Adding this metric should give us a sense of how closely the given values fit a line. Even dots that are fairly scattered can give tight confidence intervals. We would like to have to number to have a sense of how much noise we have.

custom_printf

Added missing registration of custom_printf's Camlp4 filter so that it works in the toplevel.

pa_ounit

Removed comments from test names displayed by pa_ounit.

Before:

File "iobuf.ml", line 141, characters 0-34: <<(** WHEN YOU CHANGE THIS, CHANGE iobuf_fields `...`>> threw ("Iobuf.create got nonpositive len" 0).

After:

File "iobuf.ml", line 141, characters 0-34: <<ignore (create ~len: 0)>> threw ("Iobuf.create got nonpositive len" 0).

109.24.00

async_extra

Made the Caller_converts interface in Versioned_rpc use the Connection_with_menu idea introduced in Both_convert.

async_kernel

Reworked the Throttle module.

Made both Throttle.t and Throttle.Sequencer.t instances of the same type, using a phantom type to distinguish them. Removed all Throttle.Sequencer functions -- one can now use the Throttle functions directly.

Added new functions:

(*** [max_concurrent_jobs t] returns the maximum number of jobs that [t] will run
     concurrently. *)
val max_concurrent_jobs : (_, _) t_ -> int

(*** [num_jobs_running t] returns the number of jobs that [t] is currently running.  It
     is guaranteed that if [num_jobs_running t < max_concurrent_jobs t] then
     [num_jobs_waiting_to_start t = 0].  That is, the throttle always uses its maximum
     concurrency if possible. *)
val num_jobs_running : (_, _) t_ -> int

(*** [num_jobs_waiting_to_start t] returns the number of jobs that have been [enqueue]d but
     have not yet started. *)
val num_jobs_waiting_to_start : (_ , _) t_ -> int

(*** [capacity_available t] becomes determined the next time that [t] has fewer than
     [max_concurrent_jobs t] running, and hence an [enqueue]d job would start
     immediately. *)
val capacity_available : (_, _) t_ -> unit Deferred.t

Replaced the Pipe used inside a Throttle with a Queue, and simplified the implementation. This fixed a bug in num_jobs_waiting_to_start, which could have missed a job that was not in the pipe but had not started.

async_unix

Changed Reader.close so that it frees the reader buffer using Bigstring.unsafe_destroy.

This is an improvement over the previous situation, in which the buffer wasn't freed until its finalizer fired.
Fixed a bug in Reader.read_bin_prot.

It was missing a try-with that could cause it to raise without cleaning up the reader.

core

Added module Core.Iobuf, a module aimed at zero-copy I/O.

An iobuf is a bigstring combined with a position and length, that defines a contiguous region of bytes in the bigstring. Operations on an iobuf operate relative to start of the region and cannot look outside the region.
Added module Core.Validated for creating an abstract type that ensures a validation function has been run.
Added function Bigstring.recvmmsg_assume_fd_is_nonblocking, which allows one to read a number of UDP packets with a single system call.
Fixed Unix.create_process on OSX.

109.23.00

core

Exposed Core.Std.Flags module.
Made the Heap module implement Container.S1.
Added module Ref.Permissioned, which is a ref with read_only / read_write access control.
Exposed the unique id in Type_equal.Id.

This allows, e.g. hash tables indexed by unique ids.
Removed the use of Obj.magic from the implementation of Type_equal.Id.same.

It is not needed because the Id.t contains a Uid.t and we can just use Uid.equal.

109.21.00

async_unix

Added Unix.remove.

core

Massively improved the signatures of Map and Set, both for readability and ocamldoc, as well as improved type error messages.

For instance the type of Int.Set.singleton was:
```
('a, 'comparator, 'a Core.Std.Int.Set.elt_ -> ('a, 'comparator) Core.Std.Int.Set.t_) Core.Core_set_intf.without_comparator
```
Now it is simply:
```
int -> Int.Set.t
```
Added an optional argument to Command.run that can be used to specify default flags from a user config file.

The optional argument can extend the command line based on the path to the command.
Rename module Weekday as Day_of_week.

The name Weekday conflicted with ordinary usage of "weekday" to mean Monday through Friday.
Changed sexp_of_t for {Month,Ofday,Time,Time.Span}.{Set,Map} to use the nice sexp format of the underlying atomic type.

Previously, the converter had used thes raw type (float, int, etc.). t_of_sexp still accepts both formats; we will remove the ability to accept the raw format in the distant future.

This output-format change was planned when we originally in 108.06b improved those t_of_sexp functions to accept both formats.
Added Unix.remove.
Removed some IFDEF's connected to OCaml <4 support.

109.20.00

async_kernel

Added the ability for a Throttle to have resources that are exclusively available to running jobs.

async_unix

Set close-on-exec for both ends of the pipe used to wake up the scheduler.

core

Wrapped Unix.wordexp in an Or_error.t since it is not available on all systems.
Added function Process_env.parse_ssh_client. This gets the address from which you're currently ssh'd in.
Added to Unix module the ability to get and set IP_MULTICAST_LOOP and IP_MULTICAST_TTL.
Exposed module Core.Std.Ref, which was previously only available via Core.Ref.
Remove Mutex.am_holding_mutex and changed the type of Mutex.try_lock.

With NPTL it is impossible to determine which thread is holding the lock. So, Mutex.am_holding_mutex is unimplementable. Also, Mutex.try_lock was incorrect because it claimed to raise if one was attempting to recursively lock. Since it's not possible to distinguish between recursive locking and the lock being held by another thread, we changed the type to make this clear:
```
val try_lock : t -> [ `Already_held_by_me_or_other | `Acquired ]
```
Removed our custom version of the OCaml runtime's core_sys_open function.

There used to be a bug in the OCaml runtime, PR#5069, in which open_{in,out}_gen could block while holding the OCaml lock, because they made a call to fcntl outside the blocking section. We had our own C code with the bug fix and re-exposed the fixed versions of the functions in Core.

The bug in OCaml has been fixed, so we have removed our patched function from Core.
In unix_stubs.c, switched from using FNM_FILE_NAME to FNM_PATHNAME.

The GNU project introduced FNM_FILE_NAME as a non-portable synonym for FNM_PATHNAME.

We were using pre-processor macros to define FNM_FILE_NAME as FNM_PATHNAME if unavailable, but it is simpler to just use the more portable FNM_PATHNAME everywhere.

ocaml_plugin

Removed a test that (rarely) failed nondeterministically.

sexplib

Renamed converter generated by with sexp for polymorphic variants so it is hidden from the toplevel.

of_sexp created a value named <type>_of_sexp__ to handle polymorphic variants. To hide it from the toplevel, we renamed it as __<type>_of_sexp__. We kept the __ suffix to avoid any confusion with a type named __<type>.

type_conv

Removed some warnings caused by generated signatures.
1. In signatures on local modules.
2. When there are duplicate signature items like in this example:
```
module Warnings : sig
  type t = private { foo : int } with fields (** used to say unused value foo *)
  val foo : string
end = struct
  type t = { foo : int } with fields
  let foo = "a"
end
```
3. In the signatures of all the parameters of functors that take multiple parameters; this used to work only for the last parameter.

109.19.00

async_extra

Added function Versioned_typed_tcp.Client.flushed : t -> [Flushed | Pending of Time.t Deferred.t ].

This exposes whether the underlying Writer.t has been flushed.

async_unix

Reworked a number of Reader functions to improve performance by avoiding deferreds.

This is a followup to the Reader improvements in 109.14, and eliminates some last vestiges of performance degradation that had been introduced in 109.04.
Added function Reader.lseek : t -> int64 -> mode:[<Set | End] -> int64 Deferred.t.

lseek t offset ~mode clears t's buffer and calls Unix.lseek on t's file descriptor.
Added function Writer.bytes_received : t -> int.
Added function Unix.mkfifo : ?perm:file_perm -> string -> unit Deferred.t, which was mistakenly missing.

This is a simple wrapper around Core.Unix.mkfifo.

core

Changed Time.to_string and Time.sexp_of_t to include the timezone.

This is an incompatible change with very old programs in which Time.of_string and Time.t_of_sexp did not support the timezone.

If you have programs that are:
- very old and do Time string/sexp handling
- rely on reading in time values without using Time.of_string and Time.t_of_sexp.
- rely on chains of writing/reading/writing times across machines and timezones where the time is always intended to be taken as the local time on the currently reading machine
you should recompile/review your code to make sure you won't have issues.
Added function List.remove_consecutive_duplicates : 'a t -> equal:('a -> 'a -> bool) -> 'a t.

This returns the input list with consecutive duplicates removed, and doesn't change the order of the remaining elements.
Added module User_and_group, which is a pair of a unix username and primary unix group.

The string/sexp converters follow the usual unix convention of <user>:<group>.
Added function Date.first_strictly_after : t -> on:Weekday.t -> t.

first_strictly_after t ~on:day_of_week returns the first occurrence of day_of_week strictly after t.
Added functor Type_equal.Lift.

It is always safe to conclude that if type a equals b, then type a X.t equals b X.t, for any type X.t. The OCaml type checker uses this fact when it can. However, sometimes, e.g. when using Type_equal.conv, one needs to explicitly use this fact to construct an appropriate Type_equal.t. The Type_equal.Lift* functors do this.
```
module Type_equal : sig
  type ('a, 'b) t
  ...
  module Lift (X : T1) : sig
    val lift : ('a, 'b) t -> ('a X.t, 'b X.t) t
  end
end
```

fieldslib

Made with fields generate the same functions in the Fields and Fields_of_* modules whether the type is called t or not.

109.18.00

async_unix

added Async.Unix.fcntl_{get,set}fl.

Made Reader and Writer detect if they are passed a file descriptor with incorrect permissions (O_WRONLY for Reader, O_RDONLY for Writer).

core

changed implementation of Array.sort to use introsort.

See http://en.wikipedia.org/wiki/Introsort.
tweaked a unit test in Core.Flags to not print a message to stderr.

pa_ounit

a number of improvements to inline_tests_runner, including a -verbose flag.
1. Made pa_ounit errors more readable.
2. Added -verbose flag.
3. Made the -only-test locations compatible with those displayed by the -verbose flag.
4. Renamed -display as -show-counts to avoid confusion with -verbose.
5. Improved errors when parsing the command line.
6. Updated the readme.
7. Added a -list-test-names which shows what tests would be run, if this option was not given.

109.17.00

async_extra

Added an option to Async.Log.Rotation to include the date in logfile names.

This is mostly for archiving purposes.
Made Versioned_rpc.Callee_converts.Pipe_rpc.implement_multi agree with Rpc.Pipe_rpc.implement on the type of pipe rpc implementations.
Improved the performance of Versioned_typed_tcp.

Avoided creating deferreds while reading the incoming messages.

core

Fixed Random.self_init, which was broken since 109.00.00 with the upgrade to OCaml 4.0

The fix changed the type signature expressed in core_random.ml of the standard OCaml caml_sys_random_seed C function from unit -> int from unit -> int array. That C function changed between OCaml 3.12 and 4.0.
Moved module Core_extended.Unix.Cidr into Core.Unix.
Wrapped Unix.wordexp into an Or_error.t to handle systems that does not implement it in the libc.
Fixed two other printer names
Added Array.int_blit and Array.float_blit, which are specialized fast blits for int array and float array.

For motivation underlying this change and other design alternatives please see Section 3 "Fast, Slow and Incorrect Array blits" of http://janestreet.github.com/ocaml-perf-notes.html
Added Unpack_buffer.Unpack_one.sexp for parsing sexps using the Unpack_buffer interface.

109.15.00

async_extra

In Rpc.client and Rpc.with_client, allowed the client to implement the rpcs.

Added a new optional argument: ?implementations:_ Client_implementations.t.
Added new module Versioned_rpc.Both_convert to allow the caller and callee to independently upgrade to a new rpc.

This is a new flavor of Versioned_rpc in which both sides do some type coercions.

async_unix

The epoll-based scheduler now supports sub-millisecond timeouts, using Linux_ext.Timerfd.

Async still uses the select-based scheduler by default. We plan to switch the default to epoll in a few weeks, once we have done more testing.
Eliminated module Work_group, which was for limiting the number of threads used by jobs.

This was a little-used module that significantly complicated the implementation of the Async thread pool.

One should consider using a Throttle instead.

Along the way, fixed a bug in Async helper threads in which the finalizer could fire too early, causing an unhandled exception. The fix involves relaxing the requirements on when Thread_pool.finished_with_helper_thread functions can be called, allowing it to be called while the helper thread still has work, but so long as no future work will be added.

core

Changed the tolerance of Time.Robustly_compare functions from 1E-7 to 1E-6.
Fixed the names of some toplevel pretty-printers, which referred to nonexistent modules.

Fix some of the pp's for Core which are used to install printers in the top-level. Some of the toplevel printers refer to non-existent modules like Core.Nativeint.pp; this feature changed to the correct name, like Core.Std.Nativeint.pp.

Added to module Unix functionality for getting and setting flags in the open-file-descriptor table.

module Open_flags : sig type t include Flags.S with type t :` t  ...  end
val fcntl_getfl : File_descr.t -> Open_flags.t
val fcntl_setfl : File_descr.t -> Open_flags.t -> unit

Added module Linux_ext.Timerfd.

This allows one to create a file descriptor that can be monitored by epoll or select and notify them at a certain time. It makes it possible to use epoll with sub-millisecond timeouts.
Added Version_util.application_specific_fields, which allows custom build-time information to be included in an executable.

109.14.00

async

Added function Monitor.kill, which kills a monitor and all its descendants.

This prevents any jobs from ever running in the monitor again.

async_unix

Fixed major performance degradation (since 109.04) in Reader.read* functions.

Added function Rpc.Implementation.map_inv.

val map_inv : 'a t -> f:('b -> 'a) -> 'b t

Add functions Reader.file_lines and Writer.save_lines.

These deal with files as lists of their lines.

val Reader.file_lines : string -> string list Deferred.t
val Writer.save_lines : string -> string list -> unit Deferred.t

Added a ?wakeup_scheduler:bool optional argument to functions in the Thread_safe module.

The default is true, which continues the behavior that has been in place since 109.09. However, once can use ~wakeup_scheduler:false to reduce CPU use, in return for increased latency (because the scheduler won't run a cycle immediately).

core

Fixed major performance problem with hashing in Int.Table.

Our Int.Table.replace was 3 times slower than polymorphic hash table and find was 8 times slower.

This was caused by using:
```
external seeded_hash_param : int -> int -> int -> 'a -> int = "caml_hash" "noalloc"
```
in Int.Table but:
```
external old_hash_param : int -> int -> 'a -> int = "caml_hash_univ_param" "noalloc"
```
everywhere else.

The seeded_hash_param was introduced in Caml 4.

We fixed this problem by changing Int.hash from:
```
let hash (x : t) = Hashtbl.hash x
```
to:
```
let hash (x : t) = if x >= 0 then x else ~-x
```
Added Bigstring.{pread,pwrite}, which allow reading and writing at a specific file offset.
Added module Nothing, which is a type with no values.

This is useful when an interface requires you to specify a type that you know will never be used in your implementation.
Changed Identifiable.Make so that it registers a pretty printer.

Identifiable.Make now uses Pretty_printer.Register. This requires all calls to Identifiable.Make to supply a val module_name : string.
Made Core.Zone match the Identifiable signature.
Made polymorphic equality always fail on Core.Map.t and Core.Set.t.

Before this change, polymorphic equality on a Core.Map or a Core.Set could either raise or return false. It returnd false if the data structures were unequal, and raised if the data structures were equal.

This is because their type definitions looked like:
```
type ('k, 'v, 'comparator) t =
  { tree : ('k, 'v) Tree0.t;
    comparator : ('k, 'comparator) Comparator.t;
  }
```
and polymorphic equality visits a block's fields in order. So, it will detect unequal trees and return false, but if the trees are equal, it will compare the comparators and raise because of the functional value.

This change reversed the order of the fields so polymorphic equality always fails.

custom_printf

initial import Added support for %{<Module>} in printf-style format strings.

If you put ! before a format string, it allows the use of a spec like %{<Module>} in the format string. For example, using %{Time} wraps Time.to_string around the appropriate argument.

It also allows different formats for a given type: %{<Module>.<identifier>} wraps <Module>.Format.<identifier> around the appropriate argument. For example, %{Float.pretty} would wrap Float.Format.pretty around the appropriate argument.

fieldslib

Made with fields expose first-class fields for private types while preserving privacy.

There is now an additional phantom type in a first-class field that prevents building or modifying elements of a private type.

One consequence of this change is that the Field.t type is now an abstract type -- it used to be exposed as a record type. So, one must, e.g., change field.Field.name to Field.name field.

109.13.00

async_kernel

Fixed Pipe.iter's handling of a closed pipe.

Fixed the handling by Pipe.iter and related foldy functions that handle one element at a time, which behaved surprisingly with a pipe whose read end has been closed. These functions had worked by reading a queue as a batch and then applying the user function to each queue element. But if the pipe's read end is closed during the processing of one queue element, no subsequent element should be processed. Prior to this fix, the iter didn't notice the pipe was closed for read until it went to read the next batch.
Renamed Pipe.read_one as Pipe.read_one', and added Pipe.read_one that reads a single element.

async_unix

Added Writer.write_line, which is Writer.write plus a newline at the end.
Added ?close_on_exec:bool argument to {Reader,Writer}.open_file and Async.Unix.open_file.

Made the default close_on_exec:true for Reader and Writer.
Added a compare function to Socket.Address.Inet.

core

Added Command.Spec.flags_of_args_exn, for compatibility with OCaml's standard library.

This function converts a Core.Std.Arg.t into a Command.Spec.t.
Made various modules Identifiable: Char, String, and the various Int modules.

In particular, Int being identifiable is useful, because one can now write:
```
module My_numeric_identifier : Identifiable ` Int
```
You might think that we could now delete String_id, and just write:
```
module My_string_identifier : Identifiable ` String
```
But this is not quite equivalent to using String_id, because String_id.of_string enforces that its argument is nonempty.
Removed module Space_safe_tuple, which became unnecessary in OCaml 4.00.0.

OCaml 4.00.0 included Fabrice's patch to fix the space leak that Space_safe_tuple was circumventing (PR#5288, commit SVN 11085).
Added Exn.to_string_mach, for single-line output.
Added Linux_ext.bind_to_interface, to improve security of UDP applications.
```
val bind_to_interface : (File_descr.t -> string -> unit) Or_error.t
```
This uses the linux-specifc socket option BINDTODEVICE to prevent packets being received from any interface other than one named.
Fixed Unix.mkdir_p on Mac OS X.

109.12.00

async_extra

Made explicit the equivalence between type Async.Command.t and type Core.Command.t.

async_unix

Fixed a bug in Fd.syscall_in_thread.

The bug could cause:
```
Fd.syscall_in_thread bug -- should be impossible
```
The bug was that syscall_in_thread raised rather than returning Error.
Changed Tcp.connect and Tcp.with_connect to also supply the connected socket.

Supplying the connected socket makes it easy to call Socket functions, e.g. to find out information about the connection with Socket.get{peer,sock}name. This also gives information about the IP address after DNS, which wouldn't otherwise be available.

One could reconstruct the socket by extracting the fd from the writer, and then calling Socket.of_fd with the correct Socket.Type. But that is both error prone and not discoverable.
Added Writer.schedule_bigsubstring, which parallels Writer.schedule_bigstring.

core

Add some functions to Byte_units.
- Added functions: to_string_hum, scale, Infix.//.
- Eliminated the notion of "preferred measure", so a Byte_units.t is just a float.
Improved the performance of Array.of_list_rev.

The new implementation puts the list elements directly in the right place in the resulting array, rather that putting them in order and then reversing the array in place.

Benchmarking shows that the new implementation runs in 2/3 the time of the old one.
Fixed Fqueue.t_of_sexp, which didn't work with sexp_of_t.

There was a custom sexp_of_t to abstract away the internal record structure and make the sexp look like a list, but there wasn't a custom t_of_sexp defined, so it didn't work.
Added Stable.V1 types for Host_and_port.
Removed Identifiable.Of_sexpable and Identifiable.Of_stringable, in favor of Identifiable.Make

Identifiable.Of_sexpable encouraged a terrible implementation of Identifiable.S. In particular, hash, compare, and bin_io were all built by converting the type to a sexp, and then to a string.

Identifiable.Of_stringable wasn't as obviously bad as Of_sexpable. But it still used the string as an intermediate, which is often the wrong choice -- especially for compare and bin_io, which can be generated by preprocessors.

Added Identifiable.Make as the replacement. It avoids using sexp conversion for any of the other operations.

Added List.intersperse and List.split_while.

These came from Core_extended.List.

val intersperse : 'a list -> sep:'a -> 'a list
val split_while : 'a list -> f:('a -> bool) -> 'a list ** 'a list

Added a functor, Pretty_printer.Register, for registering pretty printers. The codifies the idiom that was duplicated in lots of places:
```
let pp formatter t = Format.pp_print_string formatter (to_string t)
let () = Pretty_printer.register "Some_module.pp")
```

fieldslib

Added back Fields.fold to with fields for private types.

We had removed Fields.fold for private types, but this caused some pain. So we're putting it back. At some point, we'll patch with fields to prevent setting mutable fields on private types via the fields provided by fold.

sexplib

A tiny lexer improvement in lexer.mll. Used lexbuf.lex_{start|curr}_pos instead of lexbuf.lex_{start|curr}_p.pos_cnum for computing the length of a lexeme since the difference is the same.

109.11.00

async_extra

Exposed a version function in Pipe_rpc and State_rpc.

async_kernel

Extended Deferred.Or_error to parallel almost all of the Core.Or_error interface.
Improved the performance of Clock.at, and added a more efficient version, Clock.run_at.

Reworked the async heap of clock alarms to use async jobs as alarms.

Reworked Clock.at to use this and to not use abortable events, which is a performance improvement.

Added a more efficient version of Clock.at, for the common situation when one doesn't need a deferred.
```
(*** [run_at time ~f] is a more efficient version of [at time >>> f]. *)
val run_at : Time.t -> f:(unit -> unit) -> unit
```

async_unix

Added a check to fail if Scheduler.go is called more than once.

core

Added module Interned_string This has a functor for creating modules of interned strings. It uses the very simple mechanism of mapping distinct strings to consecutive ints.
Added value Hashtbl.find_and_remove.

fieldslib

with fields, for a type u that isn't named t, creates module Fields_of_u rather than module Fields. This allows one to us with fields on several types in the same structure.

109.10.00

async_extra

Fixed a race condition in Pipe_rpc and State_rpc. This race could cause an exception to be raised on connection closing.

async_unix

Added Shutdown.do_not_finish_shutdown_before. This allows one to add unit Deferred.t's that will delay the shutdown from finishing. The implementation is more efficient than using at_shutdown.

bin_prot

Improved error messages in presence of GADTs.

comparelib

Improved error messages in presence of GADTs.

core

Added |>, which means the same as |!, but is likely to replace it someday. This is mostly because |> is an accepted notation elsewhere, particularly in F#. In the future, we will consider eliminating |! in favor of |>, so as to avoid the duplication.
Made module Lazy into a monad.
Renamed List.stable_dedup_involving_an_application_of_the_set_functor as List.stable_dedup_staged. Made it use Staged.t to make explicit the expectation of partial application.
Added pretty printers for the toplevel to Error and Info.

fieldslib

Changed with fields on private types to not expose mutators or creators.

pa_ounit

Rewrote pa_ounit to simplify execution order and work better with functors.

Rewrote pa_ounit to solve its shortcomings with functors, namely that functors need to be applied with TEST_MODULE for their tests to be registered. The order of execution is also much simpler: tests are executed inline, at the toplevel (or functor application time). There is still a limitation: when a library doesn't have any occurrence of TEST, TEST_UNIT, or TEST_MODULE inside of it, the test runners are not set up, so tests inside of functors (from other libraries) will not be executed. Running inline_test_runner.exe is not going to run tests anymore; people should run the inline_test_runner script instead. Backtraces are now properly shown when exceptions are thrown.

sexplib

Improved error messages in presence of GADTs.
Made with sexp work with types containing as in signatures.

variantslib

Improved error messages in presence of GADTs.

109.09.00

async

Switched Async.Std's toplevel bindings for Deferred.Or_error's bind and map to use Deferred.Result. This allows them to be used with any 'error type, rather than just Error.t.

async_kernel

Fixed bug in Async.Throttle, in which jobs weren't started in order.

async_unix

Added module Thread_safe_pipe, for streaming data outside async into async. This a more efficient and feature-ful way to send a sequence of values from outside async into async than Thread_safe.pipe, which has been eliminated.
Changed functions in Thread_safe to always wake up the scheduler. Changed Thread_safe.run_in_async{,_exn} to not run a cycle, and instead rely on the scheduler to run the cycle.

core

In Core.Std, exposed Or_error.ok_exn and Or_error.error
Removed some values exported by Core.Std.

Removed some values from Core.Std that weren't widely used, or we didn't think should be exposed, including ascending, descending, and equal, which use polymorphic comparison, and we want to discourage.

Here's a guide to some of what was removed, and what one should now use instead.

| removed | replace with | |-----------------------------------+---------------------------------------| | Int_replace_polymorphic_compare | Int.Replace_polymorphic_compare | | ascending | Polymorphic_compare.ascending | | descending | Polymorphic_compare.descending | | equal | Polymorphic_compare.equal | | ifprintf | Printf.ifprintf | | sscanf | Scanf.sscanf | | Scan_failure | Scanf.Scan_failure | | string_of__of__sexp_of | Sexplib.Conv.string_of__of__sexp_of | | of_string__of__of_sexp | Sexplib.Conv.of_string__of__of_sexp | | type vec | type float64_vec |
Disallowed <:sexp_of< with two underscores; using a single underscore instead.
Added Command.Spec.Arg_type.of_alist_exn as an alternative for of_map. This captures the common pattern to create the map from an alist.
Improved the performance of Hashtbl. Constrained hashtbl size to a power of two and used a bitmask rather than mod operation for finding hash buckets.
Improved the performance of Univ, using the Type_equal GADT. The new implementation improves the run-time and space usage over the old one. In the old implementation, a Univ.t was represented as record with three fields: an exception, a string, and a closure. Creating a univ required allocating three heap blocks, the exception (3 words), the closure (3 words), and the three-field record (4 words). In the new implementation, a Univ.t is represented as a 2-field heap block containing the Constr.t and the value. Creating a univ allocates that single 3-word block, improving on the 10 words needed previously.

Matching on univs is also faster. In the old implementation, matching on a univ required making a function call, testing exception equality, and allocating a Some block. Now, it does just the test and allocation. Furthermore, it is possible to use does_match and match_exn to avoid the allocation.
Added Version_util.build_info_as_sexp.
Added _squelch_unused_module_warning_ to Comparable.S.Replace_polymorphic_compare.

sexplib

Fixed an unused rec warning in the code generated by pa_sexp in rare cases.

109.08.00

async_extra

Added module Async.Command This is Core.Command with additional async functions. In particular it contains a function async_basic that is exactly the same as Core.Command.basic, except that the function it wraps returns unit Deferred.t, instead of unit. async_basic will also start the async scheduler before the wrapped function is run, and will stop the scheduler when the wrapped function returns.

async_unix

Added module Async.Process This is a new module for creating and dealing with child processes.
For Writer.save, replaced the temp_prefix argument with temp_file.
Added Ivar.invariant function.
Added value Scheduler.fold_fields This lets one fold over the fields in the scheduler, eliminates an annoying place in catalog browser that reached into the internals of async to compute the sizes of the scheduler fields

core

Cleaned up and updated the README.
Changed executables to enable backtraces if OCAMLRUNPARAM is not set.
Changed Command so that executables show build info and version info This happens when an executatble is called as:

foo.exe version

Before this change, rather than display build info, executables would display the not-so-helpful:

(no option given - printing version)
Added back Float rounding functions with a hardcoded direction.
Exposed with bin_io and with compare for the =sexp_bool= type.
Added value Core.Never_returns.sexp_of_t.
Added values Or_error.tag{,_arg} These are analogous to Error functions of the same name.
Added functor Sexpable.Of_sexpable This is for serializing values of one type as though it were some other isomorphic type.
Added module Backtrace.Exn This exposes OCaml stdlib's Printexc functions for backtraces.
Added module Flags This implements Unix-style sets of flags that are represented as an int with various bits set, one bit for each flag, e.g., Linux_ext.Epoll.Flag.
Added module Uuid This module implements universally unique identifiers based on version 3 of the UUID specification. It used to be in Core_extended=
Added module Type_equal, which defines the "equality" GADT.

type_conv

Fixed type_conv to stop dropping parens in arguments such as:

type t = { a : int with default(1), sexp_drop_if(fun x -> (x + 1) * 2 = 4) } with sexp

109.07.00

async_unix

Changed the async scheduler so that if there are no upcoming events, it times out in 50ms rather than waiting forever.
Improved Reader.read_one_chunk_at_a_time_until_eof:
- the callback need not consume everything
- add \Eof_with_unconsumed_data` as a possible result
- grow internal buffer of the reader when needed
Added Shutdown.exit, removed Shutdown.shutdown_and_raise.
Added Scheduler.force_current_cycle_to_end.

core

Added a number of functions to =Bounded_int_table=: =equal=, =exists{,i}=, =for_all{,i}=, =filter_map{,i}=, =map{,i}=. Also added a functor, =Bounded_int_table.With_key=, that makes a bounded-int table binable and sexpable, and adds =of_alist= and =of_alist_exn=.
Added =Doubly_linked.iter_elt= and =Bag.iter_elt=.
Added =module Invariant=, which defines signatures that are to be included in other signatures to ensure a consistent interface to invariant-style functions.
Added =module Ordering=, which defines: =type t = Less | Equal | Greater=

109.06.00

core

Added [Map.symmetric_diff], for returning a list of differences between two maps. It has a fast-path implementation for maps that share a large amount of their internal structure.

109.05.00

async

Added val _squelch_unused_module_warning_ to Async.Std.

core

Updated [Core.Unix.stat] so that access, modify, and change times have nanosecond precision.
Fixed a bug in [Nano_mutex.invariant].
Simplified the implementation of [with_return] using a local explicit polymorphic type variable.

109.04.00

core

Fix [Backtrace.get], which was broken in 109.00, with the switch to OCaml 4.0.
Added [Heap.iter_el].

109.02.00

core

Add Char.of_string

2012-07-15

bin_prot

Rewrote README in Markdown and improved documentation.
Eliminated new warnings available in OCaml 4.00.

sexplib

Added support for S-expression default record fields.
Added syntax for S-expression comments and for nested block comments.
Fixed a few minor bugs and inconsistencies in the parsers and updated their whitespace handling to conform with the upcoming OCaml 4.00 compiler. The parser specification now also supports Menhir.
Rewrote README in Markdown and improved documentation.
Minor bug fix in the preprocessing module.
Eliminated new warnings available in OCaml 4.00.

type_conv

Added support for record field annotations and defaults.
Fixes for upcoming OCaml 4.00 release.

2012-02-28

bin_prot

Improved portability by better supporting the C99-standard and non-GNU compilers.

2011-11-10

bin_prot

Improved portability to older glibc distributions.

2011-09-18

sexplib

Improved documentation.

2011-09-15

bin_prot

Fixes to improve package dependency resolution.

sexplib

Fixes to improve package dependency resolution.

type_conv

Fixes to improve package dependency resolution.

2011-08-02

type_conv

Added missing module type case for "module type of".

2011-07-05

sexplib

Fixed a parser position bug. Parser positions passed by the user were not updated correctly.
Internal code beautification.

2011-07-04

bin_prot

Internal updates to sync with Jane Street.

sexplib

Internal updates to sync with Jane Street.

type_conv

Merged with Jane Street version. API changes:

Removed functions:
- Gen.ide
- Gen.idp
Removed location arguments from:
- type_is_recursive
- drop_variance_annotations

2011-06-29

bin_prot

Fixed bigstring layout bug, which should only affect value comparisons with OCaml 3.12.1 or later.
Made 64-bit detection more reliable on Mac OS X.

2011-01-30

sexplib

Fixed a code generation bug with toplevel entries.

Thanks to Yong Lu lyongu@gmail.com for the report!

2010-12-27

sexplib

Added support for MoreLabels.Hashtbl and improved reporting of error locations with preprocessor.

2010-12-26

sexplib

Worked around a compiler bug that is expected to be fixed in OCaml 3.12.1. This workaround temporarily removes the interface for module Conv (conv.mli), thus exposing the internals. This should not cause any problems for end users as long as they do not depend on the exported internal representations. The interface will become constrained again as soon as the fixed compiler is out.

2010-12-22

sexplib

Major release.

Merged with Jane Street version. This has caused an API-change that requires "open Sexplib.Conv" at the top of files that use the syntax extension.
Renamed functions:
- sexp_of_lazy -> sexp_of_lazy_t
- lazy_of_sexp -> lazy_t_of_sexp
Some standard library modules are now re-exported with predefined S-expression converters in module Conv.

type_conv

Merged with Jane Street version. No code changes.

2010-09-25

sexplib

Fixed inferred types of generated functions when dealing with arrow types.

type_conv

Added a missing type case to type_is_recursive. Thanks to Michael Wawrzoniak mhw@cs.princeton.edu for this patch!

2010-08-26

sexplib

Fixed a lexer bug when parsing comments.

2010-07-07

type_conv

Major changes for compatibility with OCaml 3.12.

2010-06-03

type_conv

Improved determination of type_conv paths. Thanks to Jacques Le Normand rathereasy@gmail.com for this patch!

2010-05-21

sexplib

Added support for sexp_bool record field annotations.

2010-05-18

sexplib

Improved performance of converting S-expressions to strings.

2010-04-12

sexplib

Changed API of Of_sexp_error exception.

2010-04-07

sexplib

Added of_(big)string_conv_exn functions.

2010-04-01

sexplib

Merged with Jane Street version.

Major new features (various functions):
- Type-annotated parsing for better error messages
- Greatly improved performance of exception converters

2010-03-20

bin_prot

Fixed linking of toplevels to require bigarrays.
Improved compilation on Mac OS X.

2010-03-17

bin_prot

Fixed small name capture bug.

2009-12-21

bin_prot

Updated contact information.

sexplib

Improved saving of files.

2009-10-12

sexplib

Added sexp_array record field extension.

2009-09-19

bin_prot

Added missing type cases for supporting variant types.
Fixed handling of variance annotations.

sexplib

Added missing variant type cases.
Fixed handling of variance annotations.

type_conv

Added missing type cases for supporting variant types.

2009-09-15

sexplib

Internal cleanups.

2009-07-28

sexplib

Added better support for conversion of exception types.

2009-07-27

bin_prot

Fixed build problem with gcc 4.4 due to stricter checking for empty macro arguments.

Thanks to Nobuyuki Tomiza nobuyuki.tomizawa@gmail.com for the patch!

2009-07-20

bin_prot

Merged tiny Jane Street improvements.

2009-07-03

bin_prot

Made byte swapping more portable.

2009-07-02

bin_prot

Added support for network byte order integers.

2009-06-23

sexplib

Fixed build problem.

Thanks to Sylvain Le Gall gildor@ocamlcore.org for the patch!

2009-05-08

sexplib

Fixed build problems on Windows and OCamlMakefile issues.

Thanks to Sylvain Le Gall gildor@ocamlcore.org for the patch!

2009-04-22

bin_prot

Added macro support for all kinds of vectors (vec, float32_vec, float64_vec) and matrices (mat, float32_mat, float64_mat), and for bigstrings (bigstring).

sexplib

Added macro support for all types of vectors, matrices, and for bigstrings.

2009-04-21

sexplib

Merged with Jane Street version, no user-relevant changes.

2009-04-16

bin_prot

Fixed a bug leading to an exception when writing extremely large values (>4 GB buffer size). Does not cause data corruption.

2009-03-09

sexplib

Merged with Jane Street version, no user-relevant changes.

2009-03-01

sexplib

Fixed build problem on Mac OS X by updating OCamlMakefile.

2009-01-20

sexplib

Automatically add S-expression pretty-printers to toplevels.

2009-01-14

type_conv

Added support for type converters that take arguments. Thanks to Jérémie Dimino jeremie@dimino.org for this patch!

Added support for deprecated OCaml syntax, since the compiler still supports it, too.

2008-10-22

type_conv

Fixed bug preprocessing labeled arguments in function types.

2008-10-18

type_conv

Fix for upcoming OCaml release 3.11.

2008-10-07

type_conv

Added a patch to improve handling of type conversion paths.

Thanks to David Rajchenbach-Teller David.Teller@ens-lyon.org for the patch!

2008-09-30

sexplib

Added a new feature: sexp_opaque. It prevents the need for / use of type converters for a given type in a particular type context.

Removed abstract types to unify them with this new concept.

2008-09-29

sexplib

Added a new feature: sexp_list. This is similar to the handling of sexp_option. By default an empty list is assumed for unspecified records using sexp_list as qualifier. Such record fields bound to empty lists will also not be printed anymore for better readability.

2008-09-23

sexplib

Added missing Not_found-exception to standard exception converters.

2008-08-20

sexplib

Removed dependency on threads. Fixed build problems.

type_conv

Added support for exception converters.

2008-08-08

sexplib

Nifty new feature: exceptions can now be converted to S-expressions, too! The "with sexp" syntax extension can be used with exceptions, thus registering a conversion function. A global exception conversion function can then be called to convert an arbitrary exception into an S-expression, which can then be printed out. This should greatly improve readability of uncaught exceptions while making life extremely easy for the developer.
Renamed the ParseError exception to Parse_error to be more compliant with Jane Street naming conventions.

2008-07-25

sexplib

Added utilities for conversion error handling. Minor fixes.

type_conv

Fixed bug concerning variance annotations in type definitions within structures.

2008-04-24

sexplib

Made Sexp-interface manifest.

2008-03-20

sexplib

Fixed META-file (missing num dependency).

2008-03-17

sexplib

Improved META-file.

type_conv

Improved META file and support of toplevel interpreter.

2008-03-13

sexplib

Fully allow function types in converters. Raise runtime exceptions on converting from S-expressions instead when function type encountered.

2008-02-11

sexplib

Fixed code generation problems with variance annotations in signatures, and empty types.

type_conv

Added support for handling variance annotations in signatures, and for empty types.

2007-12-17

sexplib

Added support for generating signature entries for S-expression converters. Thanks to Till Varoquaux till.varoquaux@gmail.com for the patch!

2007-11-29

sexplib

Added support for converting big_int, nat, num, and ratio.

2007-11-26

sexplib

Added support for parsing from bigstrings (char bigarrays).

2007-11-02

sexplib

Added syntax support for option types to use the ordinary sum type syntax. This should improve readability. The old syntax will be accepted, too, if Conv.read_old_option_format is set to true (this is currently the default). The old format will be used for writing if Conv.write_old_option_format is true (currently the default). The old syntax is deprecated and will probably not be supported by default in the near future. Reading new-style option values will always succeed.

2007-10-14

type_conv

Initial release.

2007-09-14

sexplib

Fixed bug in S-expression preprocessor concerning record field names.

2007-08-06

sexplib

Added support for converting functions to S-expressions.

2007-07-20

sexplib

Fixed position information and improved speed of S-expression parser. Fixed S-expression macro bug concerning contained polymorphic variants.

2007-06-28

sexplib

Improved Sexplib code generation.

2007-06-22

sexplib

Fixed escaping bug in S-expression parser.

2007-06-01

sexplib

Added correct handling of recursive types + test case.

2007-04-18

sexplib

Added missing conversion functions from S-expressions to pairs and triples.

2007-03-21

sexplib

Updated OCamlMakefile.

2007-03-02

sexplib

Improved error messages when parsing illegal type definitions.

2007-01-30

sexplib

Added triple conversions.

2006-11-22

sexplib

Updated OCamlMakefile.

2006-10-13

sexplib

Improved checking of records for extra or duplicate fields.

2006-09-06

sexplib

Added support for polymorphic record fields.

2006-09-05

sexplib

Added support for manifest types.

2006-08-16

sexplib

Improved error messages.

2006-07-28

sexplib

Added a new, hand-written S-expression parser that supports partial parsing and should be approx. 10x faster than the previous one.

2006-06-20

sexplib

Fixed a code generation problem leading to compilation errors concerning the use of type aliases within polymorphic variant type definitions.

This fix also solves potential erroneous appearances of backtracking exceptions in user code.

2006-03-21

sexplib

Added -for-pack option to Makefile and cleaned up distribution for a new public release.

2006-03-13

sexplib

Sexplib now accepts capitalized booleans.

2006-03-03

sexplib

Added customizable indentation levels.
Improved documentation.
Fixed API-problem concerning backward compatibility.

2006-03-01

sexplib

Added a missing flush for string conversions with a buffer.

2006-02-08

sexplib

Eliminated unused variable warnings in Sexplib-generated code.

2006-01-11

sexplib

Added functions for pretty-printing to buffers.
Improved performance of outputting S-expressions to channels.

2006-01-09

sexplib

Added functions load_sexp and load_sexps.

2006-01-04

sexplib

Changed float conversion from %E to %G (more readable).

2005-12-28

sexplib

Made machine representation for S-expressions more compact

2005-12-15

sexplib

Fixed a problem appearing with OCaml-release 3.08.4: CamlP4 obviously performs more strict checking on some constructs now and crashed with an exception when generating S-expression code for records containing only one field ("singleton tuple problem"). This problem is fixed now.

2005-11-25

sexplib

Fixed problem with type variables that could not be generalized.

2005-11-23

sexplib

Added a missing case in type definitions (path alias)

2005-11-17

sexplib

Major release: 2.0

Fixed a major design problem. The user now has to pass lex buffers instead of channels to input-functions. Reason: trailing characters in channels were lost due to ocamllex buffering them in the non-exposed lex buffer. This lex buffer is now exposed. The functions have been renamed ("input_X" -> "scan_X") to reflect this change.

2005-11-16

sexplib

Added label to conversion function "input_cnv_sexps".

2005-11-11

sexplib

Fixed a bug in the pretty-printer: strings in atoms were not escaped in the function "to_string_mach" (and therefore also "to_string").

2005-11-07

sexplib

Initial release.

Changelog

113.24.00

async

async_extended

async_extra

async_find

async_kernel

async_parallel

async_rpc_kernel

async_shell

async_smtp

async_ssl

async_unix

Testing

bignum

Summary of benchmarks

Follow ups

Benchmark results

bin_prot

core

core_bench

core_extended

core_kernel

Example:

core_profiler

email_message

incremental

jenga

ocaml_plugin

patdiff

patience_diff

ppx_assert

ppx_bench

ppx_bin_prot

ppx_compare

ppx_core

ppx_csv_conv

Improved the type_conv export to ppx_deriving

Split some ppx rewirters

Move Ppx_type_conv.Std.Type_conv_path to Ppx_core

Rename some old runtime libraries

For users

Testing

ppx_custom_printf

ppx_driver

ppx_enumerate

ppx_expect

ppx_fail

ppx_fields_conv

ppx_here

ppx_inline_test

Testing

ppx_optcomp

ppx_sexp_conv

ppx_type_conv

ppx_typerep_conv

ppx_variants_conv

re2

rpc_parallel

sexplib

textutils

typerep

typerep_extended

113.00.00

async

async_extended

async_extra

async_inotify

async_kernel

async_rpc_kernel

async_smtp

async_ssl

async_unix

bignum

bin_prot

core_extended

core_kernel

core_profiler

custom_printf

email_message