Skip to content

API reference / Reading and ingest

tsdive.init_long_meta

init_long_meta(
    source: str | Path,
    *,
    out_dir: str | Path,
    source_id: str,
    tag_col: str,
    timestamp_col: str = "timestamp",
    value_col: str = "value",
    quality_col: str = "quality",
    tags: Sequence[str] | None = None,
    overwrite: bool = False,
    sep: str | None = None,
    decimal: str | None = None,
    encoding: str | None = None,
) -> list[Path]

Write one metadata template per tag of a long export, one row per tag and timestamp.

The tags are tags when given, else every value of tag_col in order of first appearance. Each template at out_dir / f"{safe_filename(tag)}.json" carries identity (source_id and the tag as point_id), name (the tag) and every optional key of tsdive.meta set to null. When quality_col exists, quality_codes lists every raw code of the tag's rows, filled as init_meta fills it. Each string among a tag's numeric values is added to it as null. The file is read as ingest reads it, sep, decimal and encoding included. ingest_long reads the templates from out_dir.

Raises:

Type Description
SchemaError

unreadable input, a missing timestamp, value or tag column, a row without a tag, or a tag in tags that the column does not hold.

ValueError

two tags that share one file name.

FileExistsError

a template exists and overwrite is False.

Examples:

>>> import pandas as pd
>>> import tsdive
>>> pd.DataFrame({"tag": ["FIC101.PV", "TIC101.PV", "FIC101.PV", "TIC101.PV"],
...               "ts": ["2024-03-01T00:00:00Z"] * 2 + ["2024-03-01T00:01:00Z"] * 2,
...               "v": [61.0, 180.2, 62.0, 180.4],
...               "q": ["Good", "Good", "Questionable", "Good"]}
...              ).to_csv("long.csv", index=False)
>>> paths = tsdive.init_long_meta("long.csv", out_dir="meta", source_id="plant1",
...                               tag_col="tag", timestamp_col="ts", value_col="v",
...                               quality_col="q")
>>> [path.as_posix() for path in paths]
['meta/FIC101.PV.json', 'meta/TIC101.PV.json']
>>> import json
>>> json.loads(paths[0].read_text(encoding="utf-8"))["quality_codes"]
{'Good': 'GOOD', 'Questionable': None}