Python API reference¶
The primary domain objects, generated from their docstrings. SnowDb is the
read/query surface; SnowDbManager wraps it with the admin/write surface. A
Dataset is one gridded dataset within a snowdb, described by a DatasetSpec.
This is a curated starting selection of the public surface — extend it as more of the library becomes a supported entry point.
SnowDb¶
snowtool.snowdb.db.SnowDb ¶
__init__ ¶
__init__(
config,
*,
zone_layer_providers=DEFAULT_ZONE_LAYER_PROVIDERS,
)
Build a snowdb from a root config.
The single constructor: it takes a :class:~snowtool.snowdb.config.RootConfig
-- loaded from a file (:meth:open) or built in code -- and resolves
everything the config defines (the root for relative links, the pourpoint
index/records locations, and each registered dataset). A dataset is either
embedded inline (its config carried in the link) or referenced by a path
link to a dataset.json; either way it is deserialized into a
:class:DatasetSpec and bound to its data directory (see
:meth:~snowtool.snowdb.config.DatasetConfig.resolve_data_dir).
The code follows the config rather than assuming paths.
bind_dataset_from_file ¶
bind_dataset_from_file(name, config_path)
Load a dataset config file and bind it into a :class:Dataset.
The single home for the "config file -> bound Dataset" tail: resolve the
path, parse+resolve it through the canonical
:func:~snowtool.snowdb.spec.load_dataset_spec, and bind with the
config file's own directory as the resolution base. Both the read
path-link branch (:meth:__init__) and the manager's staged-dataset
path go through here, so a not-yet-registered config binds exactly as a
later SnowDb.open will bind it. A malformed or unresolvable config
raises :class:~snowtool.exceptions.SnowDbConfigError from the loader
(not a raw pydantic/decode or bare ValueError).
open
classmethod
¶
open(
path,
*,
zone_layer_providers=DEFAULT_ZONE_LAYER_PROVIDERS,
)
Open a snowdb from its root config file -- the "from file" constructor.
path is the snowdb root directory (holding snowdb_conf.json) or the
config file itself. The config is required: a root without one is not a
snowdb this version understands, so this raises
:class:~snowtool.exceptions.SnowDbConfigError pointing at snowtool
init. The I/O half of construction: it reads + parses the root config,
then hands it to the constructor.
reopened ¶
reopened()
A fresh read view of this database re-read from disk.
The single fresh-state primitive: re-open\ s the root config (and
every dataset config/spec/grid it links) from self.root, carrying the
same injected zone_layer_providers, so a caller that must observe a
sibling write committed since this instance was built (an index update
that must fold a just-registered dataset's coverage rather than erase it)
reads the current on-disk truth instead of this open-time snapshot.
A database built in code with no root has nothing to re-open, so this
raises :class:~snowtool.exceptions.SnowDbConfigError.
pourpoint_paths ¶
pourpoint_paths()
The per-pourpoint record geojson under pourpoints/records/ (sorted).
pourpoints ¶
pourpoints(*, progress=NULL_PROGRESS)
Parse and return every stored pourpoint record (all basin-bearing).
Every stored record is basin-bearing (the import boundary,
_classify_sources, partitions point-only sources before they reach
records/), so each is constructed through
:meth:Pourpoint.from_basin_record -- the one guard enforcing that
invariant on read: a corrupt basin-less record raises the typed
:class:~snowtool.exceptions.IndexedPourpointMissingBasinError naming
its file, rather than the untyped ValueError a downstream
.geometry access would raise. progress reports the parse as one
tracked task, advancing once per record.
pourpoint_triplets ¶
pourpoint_triplets()
The station triplets of every stored pourpoint, read from filenames.
Record files are written named for the pourpoint's own triplet, so the filename is authoritative -- cheaper than parsing every record just for the triplet set (e.g. for set diffs or a coverage report).
pourpoint_record_path ¶
pourpoint_record_path(triplet)
The canonical records/<triplet>.geojson path (: -> _).
load_pourpoint ¶
load_pourpoint(triplet, *, index=None)
Parse the stored record for an indexed pourpoint triplet.
The index is the availability gate: only basin-bearing pourpoints are
indexed (PourpointIndex.build refuses point-only records), so a triplet
absent from the index -- anything dropped into records/ out of band
without a pourpoint reindex -- is not served and raises
:class:PourpointNotFoundError. Callers already holding the index
(e.g. a listing loop) pass it in to avoid re-reading it.
pourpoint_index ¶
pourpoint_index()
The persisted index.geojson manifest (empty if absent), mtime-cached.
Serves pourpoint list without parsing the (large) basin records; see
:mod:~snowtool.snowdb.pourpoint_index for the incremental-vs-reindex
maintenance contract. Cached and revalidated against the file's mtime: it
stats the index file and reloads only when the mtime differs from
the cached one (a missing file is cached as an empty index, mtime
None), so a single SnowDb -- e.g. an app-lifespan API instance --
stays correct after an out-of-band reindex at the cost of one stat per
access.
pourpoint_page ¶
pourpoint_page(
*, offset, limit, with_basins=False, contains=None
)
One page of the (triplet-sorted) index: entries paired with geometry.
The catalog read behind GET /pourpoints, kept in the domain so the
response model shrinks to pure feature/link shaping. contains, when
given, filters entries on their point (lon, lat) -- the caller
supplies the predicate (e.g. an OGC bbox containment test), keeping
this method free of any HTTP query type. The filtered count is the
total (second return value), computed before the offset/limit
slice so pagination reports the full match count. Each page entry is
paired with its basin polygon when with_basins -- a per-record
:meth:load_pourpoint (the expensive view; the index stores points only),
which enforces the indexed => basin-bearing invariant -- or None
otherwise (the caller uses the entry's point).
pourpoint_dataset_coverage ¶
pourpoint_dataset_coverage(triplet, dataset_name)
How fully dataset_name's grid covers pourpoint triplet's basin.
Read straight from the index's cached per-dataset coverage (computed at reindex/registration against each dataset's grid). A grid change is by definition a new dataset, so the cached value never goes stale -- and reading it avoids re-parsing the (large) basin record on every query.
A dataset registered after the index entry was written (a legacy
out-of-order registration, or before a pourpoint reindex) has no key
in the entry's coverage dict; that reads as
:attr:~snowtool.snowdb.coverage.Coverage.NONE (no coverage) rather than
an error, so a not-yet-recomputed dataset degrades to "off grid" instead
of a 500. Raises :class:~snowtool.exceptions.UnknownDatasetError if the
dataset is unknown, or :class:~snowtool.exceptions.PourpointNotFoundError
if the pourpoint is unindexed.
require_pourpoint_coverage ¶
require_pourpoint_coverage(
triplet, dataset_name, *, allow_partial=False
)
Query guard: raise unless dataset_name fully covers triplet.
The seam a stats/query call uses before reading rasters, closing the
silent-partial-stats gap. allow_partial permits a knowingly-clipped
query over a partially-covered pourpoint; a wholly off-grid one always
raises. Returns the computed :class:Coverage for callers that want to
log it.
dump_pourpoint ¶
dump_pourpoint(triplet, dest_dir)
Copy a stored pourpoint record out to dest_dir (round-trip / archive).
A pure read/export -- it copies a record out without touching the database,
so it lives on the read side even though the prune cascade
(:class:~snowtool.snowdb.manager.SnowDbManager) also uses it.
__getitem__ ¶
__getitem__(name)
Look up active dataset name.
Raises :class:~snowtool.exceptions.UnknownDatasetError (not
KeyError) for a name that is unregistered or registered but
inactive -- this surface serves only active datasets. A
registered-but-inactive name gets a pointed "activate it" hint instead
of a generic miss, since the fix differs.
registered_dataset ¶
registered_dataset(name, *, hint='')
Look up a registered dataset name (active or not).
The management/diagnostics counterpart to :meth:__getitem__: it
resolves anything registered (activation is irrelevant to ingest, zone
generation, and the report surfaces), where __getitem__ serves only
the active subset. A miss raises
:class:~snowtool.exceptions.UnknownDatasetError listing the registered
names, mirroring __getitem__'s wording. hint is an optional
trailing clause a caller with more context appends (e.g.
:meth:SnowDbManager.resolve_dataset pointing an unregistered name at
the path form) -- so both the CLI helper and the manager resolve through
this one lookup-with-error home instead of each rebuilding the message.
zone_layer_source ¶
zone_layer_source(name)
The generation source configured for zone-layer provider name.
The checked counterpart to indexing zone_layer_sources: a provider
with no configured source and no root to anchor its default against
(a database built in code) has no entry, so this raises the typed
:class:~snowtool.exceptions.ZoneLayerSourceNotConfiguredError naming
the fix (--source PROVIDER PATH or a sources config entry)
rather than the bare KeyError a direct index would surface deep in
generation. Only generation calls this; reads never touch sources.
SnowDbManager¶
snowtool.snowdb.manager.SnowDbManager ¶
Owns every write against a held :class:SnowDb (its read/query surface).
Built around an already-constructed :class:SnowDb (reachable as
:attr:db); :meth:open and :meth:initialize are the convenience
constructors that build the read database (or its layout) and wrap it.
Snapshot contract: self.db is a read snapshot fixed at construction --
no manager write refreshes it. A write that must observe its own (or a
sibling's) prior write reads the on-disk root config afresh
(:meth:_read_root_config) rather than consulting self.db.registered,
which reflects only the state at open time.
Concurrency: config and index writes are read-modify-write with no
cross-process locking, so they assume a single writer at a time -- two admin
commands mutating the root config or the pourpoint index concurrently can lose
an update (last save wins). Ingest is different: it only writes per-date
cogs/<date>/ directories, each committed by an atomic whole-directory swap,
so bulk ingest parallelizes freely across distinct dates. Just avoid
deliberately ingesting the same date from two processes at once.
open
classmethod
¶
open(
path,
*,
zone_layer_providers=DEFAULT_ZONE_LAYER_PROVIDERS,
)
Open the read :class:SnowDb at path and wrap it in a manager.
initialize
classmethod
¶
initialize(
path,
*,
zone_layer_providers=DEFAULT_ZONE_LAYER_PROVIDERS,
)
Create the base snowdb layout + an empty root config at path.
The one entry point that creates the root structure -- the
snowdb_conf.json root config (with no datasets registered; a dataset
exists only once :meth:register_dataset links it, and is served only
while its link is active), pourpoints/, and data/. Idempotent: an
existing config is loaded and left as is (its creation stamp and datasets
preserved). Returns a manager over the root -- its read database is empty
unless datasets were already registered.
register_dataset ¶
register_dataset(
name, dataset_config_path, *, active=True, coverage=None
)
Commit a dataset registration: the root-config write is the commit point.
Writes datasets[name] -> a link at dataset_config_path, stored
relative to the root when the config lives under the tree (a relocatable
tree) and absolute otherwise (a staged-elsewhere dataset). Re-registering a
name overwrites its link. active sets the link's visibility flag:
registration makes a dataset exist (manageable by name); only an active
one is served by readers (toggle later with :meth:set_dataset_active).
Returns the updated config.
coverage (a triplet -> :class:Coverage map, produced by
:meth:stage_dataset) is folded into every existing index entry under the
new dataset's key before the config is written. The two writes are
ordered index-first, config-second, and both are atomic (WS0), so every
crash window is safe: a crash after the index write leaves only a harmless
extra coverage key (readers still see the old dataset set from the config),
and a crash before the config write leaves readers seeing exactly the old
database. Without coverage (an out-of-band dataset register that skipped
staging) only the config is written; the missing coverage key reads as
Coverage.NONE until the next pourpoint reindex. Going live still
needs a service restart -- the SnowDb is built once at startup.
name must not read as a path token (see :func:_is_path_token);
registration is the single choke point that enforces this.
For a path link that already exists on disk, the config it points at
is parsed and resolved (:func:~snowtool.snowdb.spec.load_dataset_spec,
the same canonical loader :meth:SnowDb.open uses) before anything is
written, so a caller cannot commit a link to a config that exists but
fails to parse or resolve; a malformed or unresolvable config raises
:class:~snowtool.exceptions.SnowDbConfigError. A link to a missing
path is still committed as-is and only surfaces as the existing "dangling
link" error when a reader opens the database.
set_dataset_active ¶
set_dataset_active(name, active)
Toggle dataset name's active flag in the root config.
The activation half of the register/activate split: registration says a
dataset exists; this flips whether readers serve it. The config write is
the commit point (atomic, like registration), and a running API server
still needs a restart to see the change. Raises
:class:~snowtool.exceptions.UnknownDatasetError for a name the root
config does not register. Idempotent -- setting the current state
re-saves harmlessly.
resolve_dataset ¶
resolve_dataset(token)
Resolve a dataset NAME or a config path to a :class:Dataset.
The token is partitioned syntactically (see :func:_is_path_token), so
a name and a file can never shadow each other. A path token never
consults the catalog -- it must be an existing dataset config file (its
NAME taken from the parent directory), else
:class:~snowtool.exceptions.UnknownDatasetError. A name token never
touches the filesystem -- it resolves only against the root config's
registered datasets (active or not: management ops -- ingest, zone
generation, diagnostics -- never care about reader visibility); an
unregistered name raises the same error. To target an unregistered
(staged) config, pass its path.
stage_dataset ¶
stage_dataset(
name, dataset_config_path, *, progress=NULL_PROGRESS
)
Build everything a new dataset needs, all invisible to readers.
The staging half of the register split: it builds the dataset from its
config (:meth:SnowDb.bind_dataset_from_file, so it works before the
dataset is in self.db.datasets) and, entirely under data/<name>/
-- a directory a
reader ignores because datasets come only from the root config -- creates
the skeleton, rasterizes every indexed (basin-bearing) pourpoint's basin
onto the new grid, and computes each pourpoint's geometric coverage of
that grid. Zone layers are never generated here -- that is a separate
explicit operation (:meth:generate_zone_layers_for, which shares one
source read across datasets). Nothing here touches the root config or
the index, so a fresh SnowDb.open still does not see the dataset
until :meth:register_dataset commits it (passing back
:attr:StagedDataset.coverage).
progress reports each slow phase as a sequential tracked task: parsing
the pourpoint records, then the AOI rasterize pass. Coverage is not a
separate phase: every basin-bearing pourpoint is handed to
:meth:~snowtool.snowdb.pourpoint_manager.PourpointManager.rasterize_aois
regardless of grid fit -- an off-grid basin
(NONE coverage) has no window to burn, so its own Coverage.NONE
check skips it (reported in the returned rasterized.skipped, alongside
already-current rasters), rather than this method filtering it out first.
That same pass computes each basin's geometric coverage of the new grid
and surfaces it on :attr:AOIRasterizeResult.coverage, which this method
reads back into :attr:StagedDataset.coverage -- no second coverage pass.
Converge-by-default, like ingest: an existing skeleton is tolerated, and
rasterization rebuilds an AOI raster only when it is absent or its
provenance tag reads stale (a changed basin polygon or a format-version
bump). A byte-level forced rebuild is
:meth:~snowtool.snowdb.pourpoint_manager.PourpointManager.rasterize_aois
with rebuild=True (the pourpoint rasterize --rebuild command).
create_dataset ¶
create_dataset(
name,
config,
*,
nodata_mask_source=None,
progress=NULL_PROGRESS,
)
Stamp a brand-new dataset name from config: stage it, then
register it inactive -- the whole lifecycle the dataset create command
used to orchestrate step-by-step in the CLI.
Resolves the dataset's data directory the way a later SnowDb.open will
(:meth:~snowtool.snowdb.config.DatasetConfig.resolve_data_dir), writes
config beside its data as data/<name>/dataset.json so
:meth:stage_dataset can build from it and :meth:register_dataset can
link it, stages every artifact
(skeleton, AOI rasters, coverage -- but never zone layers; those are the
separate :meth:generate_zone_layers_for pass), and registers the staged
dataset. Converge-by-default like ingest and staging: the directory mkdir
and the config write are idempotent overwrites, and staging rebuilds an AOI
raster only when its provenance tag reads stale. When nodata_mask_source
is given (e.g. a template's packaged mask), it is copied into the dataset
directory and config is updated to reference it before either is used,
so the first staging pass burns AOI rasters with the mask already applied.
A config.data_dir must be absolute or omitted here (the convention
data/<name> under the root). A relative data_dir is refused with
:class:~snowtool.exceptions.SnowDbConfigError, because create is the one
call where it cannot mean what SnowDb.open will later take it to mean: a
stored relative data_dir resolves against the config file's own
directory, but create must decide where to put that config file from
data_dir -- there is no independent config home to anchor against, so the
two rules cannot agree (create would resolve against the root and open
against the config's dir, yielding a nested <dir>/<dir>). Omit it for the
convention, or give an absolute path; both round-trip identically through
SnowDb.open.
The one real invariant it enforces: an existing registration is never
clobbered. Registration happens only when name is not already in the
root config -- so a re-create of a live dataset never deactivates it or
relinks its config out from under readers (its active state and link
survive verbatim). A fresh registration is committed inactive
(active=False) with the staged coverage folded into the index, so the
dataset exists (manageable by name) but stays invisible to readers until an
explicit :meth:set_dataset_active. Returns a :class:CreatedDataset
carrying the staging result and whether this call registered the dataset.
generate_zone_layers_for ¶
generate_zone_layers_for(
datasets,
provider_names=None,
*,
source_overrides=None,
force=False,
options=None,
progress_factory=None,
)
Generate zone layers across datasets with one shared read per provider.
For each selected provider it resolves the source once (an override from
source_overrides, else the configured default) and reads it a single time
over the combined extent of every dataset that enables that provider -- so
standing up N datasets that share a provider pays that provider's expensive
source read once, not N times (terrain's aspect, for instance, must be
computed at the source resolution, so sharing the read is the whole point).
datasets are passed as objects (registered or merely staged), so
activation is irrelevant here: zone layers live under data/<name>/
regardless of whether the root config links the dataset; only the datasets
that enable a provider are targeted (the rest have no such zone layer).
provider_names limits the providers (default: the union of every
dataset's enabled providers); an unknown name -- selected or overridden --
raises :class:~snowtool.exceptions.UnknownZoneLayerProviderError.
options carries engine knobs (e.g. terrain's workers/
block_size). progress_factory builds a per-provider reporter
(default: silent). Returns {provider_name: {dataset_name: hash}}, with
provider keys that targeted no dataset omitted.
Dataset¶
snowtool.snowdb.dataset.Dataset ¶
A :class:DatasetSpec bound to its data/<name>/ directory.
Owns the per-dataset filesystem layout (aoi-rasters/, the per-provider
zone-layer subdirs, cogs/) and the operations on it; grid/variables are
reached through self.spec.
coverage_domain
property
¶
coverage_domain
The static region this dataset can serve (for AOI coverage).
grid_geometry
property
¶
grid_geometry
This grid's authoritative COG-write geometry, as one value.
The single home for the transform/CRS/tile_size/shape an ingester's
grid-aligned rasters write with (see :class:GridGeometry): derived once
here from the grid + spec, so no ingester re-derives or re-threads them.
nodata_mask_pair
cached
property
¶
nodata_mask_pair
The nodata mask paired with its sha256 digest, or None with no mask.
The single source for both the mask path and its provenance hash: both
halves come from the same config field, so the pair is never
half-specified. It couples them into the one value write_aoi_raster
wants (path + digest) instead of two positionally-tied arguments; the
digest alone is reachable via :attr:nodata_mask_hash. A configured mask
whose file is missing raises :class:NodataMaskError.
Cached per instance so a convergence loop over hundreds of pourpoints hashes the file once, not once per AOI. Management ops build short-lived Datasets, so a swapped mask file is picked up by the next run.
nodata_mask_hash
property
¶
nodata_mask_hash
sha256 of the configured nodata-mask file; None with no mask.
The digest half of :attr:nodata_mask_pair (the cache lives there), as
aoi_provenance wants just the hash.
ensure_skeleton ¶
ensure_skeleton()
Create any missing part of this dataset's directory skeleton.
Converge-by-default and idempotent: builds the dataset dir plus its
aoi-rasters/ and cogs/ subdirs with exist_ok=True and never
clobbers, so a fresh call and a re-run over a fully- or partially-built
skeleton both succeed. Returns whether the skeleton was incomplete
before this call (i.e. this call made it) -- the caller uses it to
report new-vs-existing.
Zone layers (terrain, land cover, ...) are not built here: each needs a
source and is generated separately by
:meth:SnowDbManager.generate_zone_layers_for (so generation can share one
source read across every dataset).
zone_target ¶
zone_target(provider)
This dataset's grid as a target for provider's generation engine.
rasterize_aoi ¶
rasterize_aoi(pourpoint, *, rebuild=False)
Burn pourpoint's basin onto this dataset's grid as an AOI raster.
Converge-by-default: build when the raster is missing or stale (see
:meth:aoi_raster_is_current), skip when it is already current --
rebuild=True forces a rebuild regardless of current state. Returns
True when a raster was written, False when the existing one was
already current and nothing was written; a caller that wants the raster
itself opens it with :meth:load_aoi_raster.
The tile window is clamped to the grid (see
:func:~snowtool.snowdb.grid.bounding_tiles), so a basin straddling a
grid edge burns only its in-grid portion; a basin entirely outside the
grid raises :class:~snowtool.exceptions.GeometryOutsideGridError (the
batch paths pre-filter those by coverage instead of calling this).
aoi_raster_hash ¶
aoi_raster_hash(station_triplet)
The AOI-geometry hash an existing AOI raster was burned from.
Reads only the COG's tags (no array decode); returns None if the
raster does not exist or predates the AOI_HASH_TAG tagging.
aoi_raster_is_current ¶
aoi_raster_is_current(pourpoint)
Whether a burned AOI raster exists AND has no actionable issue.
Delegates to :func:~snowtool.snowdb.aoi_raster.aoi_raster_issues -- the
same health check doctor reports with -- so "current" means: the file
exists, is readable, is on the current grid, has its tile-bbox tag, and
its stored SNOWTOOL_AOI_HASH matches pourpoint's geometry (plus
this dataset's nodata-mask hash) and the current format version. An
empty-but-otherwise-current raster
(:class:~snowtool.snowdb.issues.EmptyArtifact) still counts as current --
rebuilding it would produce the same empty raster, so
:meth:rasterize_aoi correctly skips it; a corrupt/unreadable raster, a
grid move, or a changed basin do not, and force a rebuild.
remove_aoi_raster ¶
remove_aoi_raster(station_triplet)
Delete this dataset's burned AOI raster for triplet; True if present.
ingest ¶
ingest(source, *, force=False, progress=NULL_PROGRESS)
Ingest a source artifact into per-date COGs, via this dataset's ingester.
Drives spec.ingester's per-date plan through the generic
:func:~snowtool.snowdb.ingest.run_ingest (which computes the versioned
source hash and commits each date); raises if the dataset has no
configured ingester. Returns an
:class:~snowtool.snowdb.ingest.IngestResult splitting the dates written
from those skipped as already current. progress reports each date's
per-variable COG writes (see :meth:write_date_cogs).
write_date_cogs ¶
write_date_cogs(
date,
out_names,
build_rasters,
*,
source_hash,
force=False,
progress=NULL_PROGRESS,
)
Write a date's already-on-grid rasters into cogs/<YYYYMMDD>/ atomically.
The dataset-agnostic write side of ingest: it owns the date directory.
out_names are the COG filenames this date will land (the ingester derives
them cheaply from source metadata); build_rasters is a deferred callable
producing the rasters (which know how to write themselves as COGs) -- it is
invoked only if the date is not skipped, so an already-current date never
pays to read the source. source_hash is the versioned hash of the source
artifact this date came from (see
:data:~snowtool.snowdb.ingest.INGEST_FORMAT_VERSION); it is both
stamped on every COG (via the ingester's SOURCE_HASH tag) and used by the
skip check below. Returns True if the date dir was (re)built, False if
it was skipped as already current.
The whole per-date directory is the unit of commit. Writes stage into a
temp dir beside the target (:func:~snowtool.snowdb.atomic.staged_dir) and
are swapped in wholesale, so (a) a crash mid-ingest never leaves a partial
date on disk -- a reader sees the wholly-old dir or the wholly-new one --
and (b) stale COGs from a prior, differently-named source vanish by
construction rather than lingering beside the new ones and making a variable
unresolvable.
Completeness is enforced at date granularity. Before any filesystem work the
declared out_names must cover every spec variable, so a source short a
required input variable raises :class:IncompleteDatasetDataError up front;
after writing, every spec variable must resolve to exactly one COG in the
staged dir or the swap is abandoned and the existing date dir left untouched.
Idempotent-skip granularity is likewise per-date, not per-file: without
force a date is skipped only when its dir already holds exactly the
COGs this call would write (complete and free of stale members) and their
stored SOURCE_HASH equals source_hash. The filename set alone is not
enough: source filenames embed provenance, so a renamed re-release is
caught by a name mismatch, but a re-release under the same filename with
different bytes would keep the names identical -- the hash equality catches
that, forcing a rebuild. A missing tag (a date dir written before hashing)
also reads as stale. Any divergence rebuilds the whole date dir; force
always rebuilds.
load_aoi_raster ¶
load_aoi_raster(station_triplet)
Open the burned AOI raster for triplet (the stats read input).
Raises :class:FileNotFoundError (pointing at pourpoint rasterize) when the
raster has not been built for this dataset, so a stats query surfaces a
clean missing-prerequisite error rather than a bare open failure.
available_dates ¶
available_dates()
Every date with an ingested cogs/<YYYYMMDD>/ directory, ascending.
Returns the full set; a caller wanting a date window filters the result
through a :class:~snowtool.snowdb.query.DateRangeQuery (the one
inclusive-optional-bounds predicate), so the bounds logic lives in one
place.
variable_path ¶
variable_path(d, variable)
The single COG for variable on date d, or None if absent.
The one-variable form of :meth:resolve_variables: same directory
listing, same :meth:_glob_matches engine, same duplicate-COG raise -- so
a date this reports resolved is exactly one the completeness check counts
complete.
resolve_variables ¶
resolve_variables(d, variables)
Resolve each of variables to its single COG for date d from one
directory listing.
The canonical variable resolver (:meth:variable_path is the one-variable
form): the date dir is listed once and every requested variable is matched
against that single listing, so a multi-variable query pays one directory
read per date, not one per (date, variable) -- which is what lets a
wide-archive query's up-front completeness check fail fast. A variable whose
glob matches no file is omitted (:meth:RasterCollection.validate turns the
resulting partial date into the typed integrity error); a variable matching
more than one file raises :class:IncompleteDatasetDataError.
unresolved_variables ¶
unresolved_variables(d)
Spec variable keys unresolved (missing or duplicated) for date d.
Tolerant by design: a duplicated COG makes a key unresolved (not a
raise), so the completeness report can surface a corrupt date as a
finding instead of crashing on it. An absent date directory yields every
key (nothing is present). variable_path keeps its strict raise for
the query path.
aoi_raster_triplets ¶
aoi_raster_triplets()
Station triplets that have a burned aoi-rasters/<triplet>.tif.
remove_date ¶
remove_date(d, *, dry_run=False)
Delete a date's cogs/<YYYYMMDD>/ directory; True if it existed.
dry_run reports whether the date dir exists without deleting it.
DatasetSpec¶
snowtool.snowdb.spec.DatasetSpec ¶
crs
cached
property
¶
crs
The grid's CRS (pyproj), the single source for every CRS-derived
value -- is_geographic, cell_area, and the dataset's rasterio
write CRS (:attr:Dataset.grid_crs).
coverage_domain
cached
property
¶
coverage_domain
The static region this dataset can serve.
Used by AOI coverage classification: the dataset's footprint when it
declares one (e.g. a MODIS block minus a never-ingested tile), else the
full grid-extent rectangle -- so a basin over a permanently-empty hole is
not reported as fully covered.
is_geographic
cached
property
¶
is_geographic
Whether cell area varies across the grid (geographic CRS) or is
constant (projected/linear CRS). Drives whether an AOI raster burns
per-row geodesic area or the constant cell_area.
cell_area
cached
property
¶
cell_area
The constant per-cell area, in square metres. Only meaningful on a projected grid; raises on a geographic grid, where area varies by latitude and the AOI raster burns per-row geodesic area instead.
griffine reports a projected grid's planar cell area in the CRS's own
linear units squared, so it is converted to m^2 here -- every area we
emit (the area_m2 field, the CSV column) is metres regardless of the
grid's units.
zone_params ¶
zone_params(provider_name, layer_key)
The configured default query params for one zone layer.
None when the provider/layer is not configured, or is configured
with no params -- either way the scheme's own defaults apply, so
callers pass the result to :meth:ZoneScheme.configured uniformly.
from_config
classmethod
¶
from_config(config, name)
Deserialize a :class:~snowtool.snowdb.config.DatasetConfig into a spec.
The config's grid, variables, zones and footprint are already the
domain types, so they carry straight over; only the ingester name is
resolved to the concrete ingester from the registry (None for a
read-only/derived dataset). name is supplied separately because the
config does not carry one -- it comes from where the config is registered.
Raises a bare :class:ValueError for an unknown ingester name; prefer
:func:load_dataset_spec whenever a config path is in hand.