3 Commits

Author SHA1 Message Date
Steven Wu
be9c083cf7
[CAS] Add OnDiskGraphDB and OnDiskKeyValueDB (#114102)
Add OnDiskGraphDB and OnDiskKeyValueDB that can be used to implement
ObjectStore and ActionCache respectively. Those are on-disk persistent
storage that build upon OnDiskTrieHashMap and implements key functions
that are required by LLVMCAS interfaces.

This abstraction layer defines how the objects are hashed and stored on
disk. OnDiskKeyValueDB is a basic OnDiskTrieHashMap while OnDiskGraphDB
also defines:
* How objects of various size are store on disk and are referenced by
  the trie nodes.
* How to store the references from one stored object to another object
  that is referenced.

In addition to basic APIs for ObjectStore and ActionCache, other
advances database configuration features can be implemented in this
layer without exposing to the users of the LLVMCAS interface. For
example, OnDiskGraphDB has a faulty in function to fetch data from an
upstream OnDiskGraphDB if the data is missing.
2025-10-20 13:16:09 -07:00
Steven Wu
9e9edb5834
[CAS] Fix build for older macOS SDK (before 13.0) (#159133)
Fix a build failure and back-deployment problem for macOS before 13.0.
2025-09-16 17:50:09 +00:00
Steven Wu
f9cd2ee119
[CAS] Add MappedFileRegionArena (#114099)
Add MappedFileRegionArena which can be served as a file system backed
persistent memory allocator. The allocator works like a
BumpPtrAllocator,
and is designed to be thread safe and process safe.

The implementation relies on the POSIX compliance of file system and
doesn't work on all file systems. If the file system supports lazy tail
(doesn't allocate disk space if the tail of the large file is not used),
user has more flexibility to declare a larger capacity.

The allocator works by using a atomically updated bump ptr at a location
that can be customized by the user. The atomic pointer points to the
next available space to allocate, and the allocator will resize/truncate
to current usage once all clients closed the allocator.

Windows implementation contributed by: @hjyamauchi
2025-09-15 10:00:21 -07:00