Why not […]?
There are already many serialization formats. Why invent a new one?
Why not JSON?
JSON is quite unsuitable for our purposes, really. It's slow to read and write (among other things because of string escaping), unless heroic efforts are deployed. It also doesn't specify anything about numbers and lacks the ability to represent byte strings and DAG references.
Why not CBOR?
CBOR is simpler, more efficient, more compact, and faster than JSON, and is one of the major sources of inspiration for Twine. However, it doesn't easily represent sharing within the same file.
Tags can be used to represent pointers (e.g. dasl uses tag 42 to mark external pointers to other content-addressed values), but CBOR libraries generally don't provide ways to read from an offset or to access the offset of a value being encoded for further reference.
Historical note about CBOR-pack
Before Twine, we were using our homegrown cbor-pack, a CBOR value of the shape {"h": [<cbor>, <cbor>, …, <cbor>], "k": <cbor>}. The key, k, would be the toplevel value (the entry point); the heap, h, would contain an array of CBOR values. Tag 6 would be used to refer to values in h by their index, so that 6(29) would actually represent the value h[29]. The downsides are that the whole array h would have to be parsed into a CBOR AST in memory during deserialization; and it's hard to directly write a stream of bytes during serialization (because h's array needs to be prefixed by a variable-width length).
Twine provides native sharing, and makes it easy to write bytes to some file or growing buffer during serialization, in one pass.
Why not Fleece?
Fleece is a binary encoding with a data model similar to JSON, but it also allows 0-copy reading and traversal of values, directly from a byte array (either in memory or memory-mapped). It is another big source of inspiration for Twine.
Compared to Fleece, Twine is somewhat simpler to implement, at the cost of some performance for 0-copy reading. Twine maps are not sorted by ascending keys; Twine arrays are not indexable in O(1).
However, Twine doesn't need to distinguish between wide and narrow collections because values don't have to fit in 2 or 4 bytes each. The complex bitpacking of Fleece is replaced by 4-bits tags à la CBOR.
Additionally, twine offers explicit references, which give users control over traversal of the DAG if they need it.