5 Commits

Author SHA1 Message Date
Steven Wu
f7d32078e5
[CAS] Add OnDiskCASLogger to help logging/debugging CAS (#174462)
Adds a CAS logging interface to track mutations of the CAS and action
cache database files. This is designed to aid in the debugging of
data corruption and other issues with the low-level on-disk
representation.

To enable logging, set the environment variable LLVM_CAS_LOG to 1 or 2.
Level 2 includes logging of all allocations inside the index, action
cache, and datapool, which increases the overhead. The log file is
placed inside the top-level directory of the unified cache. For now it
is a human-readable textual format with one line per entry.
2026-01-16 09:48:41 -08:00
Steven Wu
be9c083cf7
[CAS] Add OnDiskGraphDB and OnDiskKeyValueDB (#114102)
Add OnDiskGraphDB and OnDiskKeyValueDB that can be used to implement
ObjectStore and ActionCache respectively. Those are on-disk persistent
storage that build upon OnDiskTrieHashMap and implements key functions
that are required by LLVMCAS interfaces.

This abstraction layer defines how the objects are hashed and stored on
disk. OnDiskKeyValueDB is a basic OnDiskTrieHashMap while OnDiskGraphDB
also defines:
* How objects of various size are store on disk and are referenced by
  the trie nodes.
* How to store the references from one stored object to another object
  that is referenced.

In addition to basic APIs for ObjectStore and ActionCache, other
advances database configuration features can be implemented in this
layer without exposing to the users of the LLVMCAS interface. For
example, OnDiskGraphDB has a faulty in function to fetch data from an
upstream OnDiskGraphDB if the data is missing.
2025-10-20 13:16:09 -07:00
Steven Wu
2aff3c6a6d
Re-land #161264: [CAS] Add OnDiskDataAllocator (#162112)
Fix the build configuration that has OnDiskCAS disabled.
2025-10-06 17:07:13 +00:00
Nico Weber
ebfb16a285 Revert "[CAS] Add OnDiskDataAllocator (#161264)"
This reverts commit 08e95405752e6d3475afdebe3bd5f2ff2ff17712.

Doesn't build on some bots,
see comments on https://github.com/llvm/llvm-project/pull/161264
2025-10-06 12:21:43 -04:00
Steven Wu
08e9540575
[CAS] Add OnDiskDataAllocator (#161264)
Add OnDiskDataAllocator, which is the data pool implementation inside a
OnDiskCAS that stores data in a single file. It is a based on
MappedFileRegionArena and wrapped inside a CAS database file.
2025-10-06 09:02:55 -07:00