tsdive ingest¶

Build an archive from a CSV or parquet export.

The usage walkthrough has a section on it.

Usage¶

usage: tsdive ingest [-h] [--out OUT] [--meta META]
                     [--timestamp-col TIMESTAMP_COL]
                     [--timestamp-format FORMAT] [--dayfirst] [--sep CHAR]
                     [--decimal CHAR] [--encoding NAME]
                     [--value-col VALUE_COL] [--quality-col QUALITY_COL]
                     [--init-meta FILE|DIR] [--wide] [--tag-col COL]
                     [--meta-dir DIR] [--tags A,B,...] [--quality-suffix S]
                     [--source-id ID] [--tz TZ]
                     [--assume-quality GOOD|UNCERTAIN|BAD] [--overwrite]
                     [--json] [--no-color]
                     source

Options¶

Argument	Value	Default	Description
`source`		required	CSV or parquet export: one tag, one column per tag with --wide, or one row per tag and timestamp with --tag-col
`--out`	`OUT`		archive to create; with --wide or --tag-col, directory for one archive per tag
`--meta`	`META`		JSON file of tag metadata (the tsdive.meta object); a key it does not define raises SchemaError
`--timestamp-col`	`TIMESTAMP_COL`	`timestamp`	timestamp column of the export
`--timestamp-format`	`FORMAT`		strptime format every timestamp must match, like '%d/%m/%Y %H:%M:%S'
`--dayfirst`			read numeric dates day first, so 01/02/2026 is 1 February; without it or --timestamp-format a date that reads both ways raises SchemaError
`--sep`	`CHAR`		column separator of a CSV export, like ';' (default ,)
`--decimal`	`CHAR`		decimal mark of a CSV export, like ',' (default .)
`--encoding`	`NAME`		text encoding of a CSV export, like cp1252 (default utf-8)
`--value-col`	`VALUE_COL`		value column of a single-tag or --tag-col export (default value)
`--quality-col`	`QUALITY_COL`		quality column of a single-tag or --tag-col export (default quality)
`--init-meta`	`FILE\|DIR`		write a metadata template to FILE and stop, then fill it in and ingest with --meta FILE; with --wide or --tag-col, write a .json template per tag into DIR, then ingest with --meta-dir DIR
`--wide`			read one column per tag and write one archive per tag into --out, named by point_id
`--tag-col`	`COL`		read the tag of each row from COL and write one archive per tag into --out, named by point_id
`--meta-dir`	`DIR`		directory of .json metadata files, one per tag
`--tags`	`A,B,...`		tags to ingest; omitted, every column that is not the timestamp or a quality column (--wide), or every value of --tag-col
`--quality-suffix`	`S`		with --wide, each tag's quality column is
`--source-id`	`ID`		identity.source_id written into every template (--init-meta)
`--tz`	`TZ`		IANA zone the source's naive timestamps are in; they are localised to it and converted to UTC. Timestamps that already carry an offset ignore this
`--assume-quality`	`GOOD\|UNCERTAIN\|BAD`		declare a quality for a source that has no quality column; recorded on the archive and reported by every profile of it
`--overwrite`			replace an existing archive at --out, or template under --init-meta
`--json`			print one JSON object and nothing else
`--no-color`			plain text, no ANSI colour (NO_COLOR does the same)

Example¶

The example exports the FIC-101 demo archive to fic101.csv and its metadata to fic101.json first, then ingests them again.

$ tsdive ingest fic101.csv --out archive/plant1/FIC101.PV.parquet --meta fic101.json
wrote     archive/plant1/FIC101.PV.parquet
tag       demo:FIC101.PV   562 rows   quality from column quality