Skip to content

Choose a cache location

Without a cache, every resolve re-downloads, and a resolve that verifies bytes it cannot store returns an error rather than discarding them silently. So a cache is effectively required; the decision is where.

The conventional location

dir, err := artifacts.UserCacheDir()
if err != nil {
    return err
}

client, err := artifacts.New(verifier, artifacts.WithCache(artifacts.NewDirCache(dir)))

UserCacheDir() returns <os.UserCacheDir()>/phpboyscout/artifacts: ~/.cache/phpboyscout/artifacts on Linux, ~/Library/Caches/phpboyscout/artifacts on macOS.

Note it is not named after your tool. Two tools belonging to the same user and resolving the same runtime share one copy. The alternative produces a directory named after whichever tool installed it first, which is how krites' resolver ended up populating a krites directory for a different tool entirely.

That sharing is within a user account, not across one. Cached files are written with mode 0600 (a consequence of the atomic temp-file-and-rename write), so a cache populated by one user is not readable by another. A genuinely multi-user shared cache needs each user to have their own, or a wrapper that relaxes the mode after Put returns.

NewDirCache takes an explicit path rather than calling UserCacheDir() itself. A library that decides where to write on someone's disk without being asked is a library that surprises somebody.

A pinned location

For a container image, a CI job, or anywhere the home directory is not meaningful:

artifacts.NewDirCache("/var/lib/mytool/artefacts")

The directory is created on first write, with mode 0755.

The layout

<root>/<artefact>/<version>/<file>

which mirrors the channel, so a cache is browsable and a stale entry is obvious to whoever is deleting it. Clearing one version is rm -rf <root>/onnxruntime/1.28.0.

What the cache checks

Bytes are verified on the way out as well as on the way in. A hit is re-hashed against a digest from a manifest verified in the same resolution, so an entry is never returned on the strength of having been verified once, at some point, by something.

An earlier version of this guide said the opposite: that a hit checked presence only, on the reasoning that re-hashing would make the cache slower than the download it avoids. That reasoning does not survive arithmetic: hashing tens of megabytes locally is cheaper than fetching them, and cheaper than the OpenPGP and WKD work already happening on the same path.

The consequence for choosing a location is smaller than it was, but not gone:

  • an entry someone has edited is detected and replaced, not served;
  • so a corrupt or tampered cache costs a re-download rather than a wrong artefact;
  • but the file is still opened by path after it is returned, so a local actor who can write to the directory between the check and your open is outside what this defends. That is the same caveat the trust model records.

A cache under /var/lib or /opt shared between users should still not be group- or world-writable, the failure is now noisy rather than silent, which is an improvement and not a licence.

If you want to audit a cache you did not populate:

err := cache.Verify(key, expectedDigest)

which re-reads the file and re-hashes it against a digest you supply, from client.Manifest, which is signed.

Concurrency

Put writes to a temporary file and renames, so a concurrent reader sees either no file or a complete one, never a half-written artefact that Has would then report as present. Two processes resolving the same version resolve to the same bytes, so whichever rename wins is correct.

Supplying your own

Cache is an interface. Implement it for an in-memory cache in tests, an object store, or a layout your deployment already has:

type Cache interface {
    Key(namespace string, ref Ref, file string) (Key, error)
    Path(key Key) string
    Has(key Key) bool
    Put(key Key, r io.Reader) (string, error)
    Verify(key Key, digest string) error
}

Put receives a reader that verifies as it streams, and returns an error before EOF if the bytes depart from the signed manifest. Return that error, as it is or wrapped with %w, so the caller can still match ErrUnverified. Resolve hands back the path Put returns without checking it again, so a Put that swallowed the error would serve unverified bytes. Install nothing at the final path until the copy has succeeded.

Verify runs on every hit. A failure there counts as a miss, and the artefact is downloaded again.

Key has no exported fields, so the only way to build one is DirCache.Key, which validates every component and refuses with ErrInvalidIdentifier. Your own Key method can delegate to it, for instance artifacts.NewDirCache("").Key(namespace, ref, file), and use Key.String() as the entry's path relative to wherever you store it.