Choose a clean baseline window¶

screen, spc, mspc and compare judge a window against a baseline. A bad baseline gives confident wrong answers, so this guide picks one step by step on the demo flow, to judge the two hours from 04:00 to 06:00.

1. Profile the history¶

$ tsdive profile data/demo/fic101_demo.parquet
demo:FIC101.PV  FIC-101 flow
coverage 0.933   GOOD 561/562   censored yes   gaps 1

window    2024-03-30 20:00:00Z -> 2024-03-31 06:00:00Z  (10 h)
contract  TIME_WEIGHTED  RECORDED  NONE  stepped no  digest d59d433d9c62
units     m3/h -> cubic meters per hour

Coverage
  coverage 0.933   valid 0.998   gaps 1   data-loss gaps 1   longest 40 min
  2024-03-30 23:00:00Z -> 23:40:00Z   40 min   unknown (no rule matched)

Quality
  GOOD 561   UNCERTAIN 1   BAD 0
  unmapped codes, treated UNCERTAIN: SENSOR DRIFT

Range
[11 more lines not shown]

Two stretches are off limits: the 40-minute outage from 23:00, and the half hour pinned at 100 m3/h from 02:00. A baseline over the clipped stretch is refused, and one over the outage is refused by mspc.

2. Segment it¶

tsdive segment cuts the archive into stretches of steady level, with each stretch's median and MAD:

$ tsdive segment data/demo/fic101_demo.parquet
demo:FIC101.PV  6 segments   5 breakpoints   censored yes

window    2024-03-30 20:00:00Z -> 2024-03-31 06:00:00Z   usable 561
method    PELT L2 on MAD-scaled values   penalty 18.99 (3*log n)   min-size 10

  #  start                 end                     n   median     mad
  1  2024-03-30 20:00:00Z  2024-03-30 21:55:00Z  116    62.54  0.1911
  2  2024-03-30 21:56:00Z  2024-03-31 00:09:00Z   95    61.66  0.2427
  3  2024-03-31 00:10:00Z  2024-03-31 02:00:00Z  111    62.56  0.1826
  4  2024-03-31 02:01:00Z  2024-03-31 02:29:00Z   29    100.0       0
  5  2024-03-31 02:30:00Z  2024-03-31 03:56:00Z   86    61.37  0.1696
  6  2024-03-31 03:57:00Z  2024-03-31 06:00:00Z  124    62.52  0.2387

Segment 4 is the clipped half hour: median 100.0, MAD 0. The window from 04:00 lies in segment 6, median 62.52 m3/h.

3. Pick a segment at the window's operating point¶

The nearest clean segment is 5, right before the window. Its median is 61.37 m3/h, more than one m3/h below the window's level:

$ tsdive screen data/demo/fic101_demo.parquet \
    --baseline 2024-03-31T02:30:00Z/2024-03-31T03:56:00Z \
    --window 2024-03-31T04:00:00Z/2024-03-31T06:00:00Z
demo:FIC101.PV  flagged 102 of 121 (84.3%)

baseline  2024-03-31 02:30:00Z -> 03:56:00Z   GOOD 86   censored no
window    2024-03-31 04:00:00Z -> 06:00:00Z
method    MAD   center 61.37   scale 0.2514   k 3.0   limits [60.62, 62.13]
caveat    provisional: one baseline for every regime in the window
[8 more lines not shown]

84% of the window is flagged. Nothing is wrong with the flow: segment 5 ran at a lower level, and its sigma of 0.2514 m3/h makes the step up to 62.5 look like a fault. Segment 1 ran at 62.54 m3/h, the window's level:

$ tsdive screen data/demo/fic101_demo.parquet \
    --baseline 2024-03-30T20:00:00Z/2024-03-30T21:55:00Z \
    --window 2024-03-31T04:00:00Z/2024-03-31T06:00:00Z
demo:FIC101.PV  flagged 0 of 121 (0.0%)

baseline  2024-03-30 20:00:00Z -> 21:55:00Z   GOOD 116   censored no
window    2024-03-31 04:00:00Z -> 06:00:00Z
method    MAD   center 62.54   scale 0.2833   k 3.0   limits [61.69, 63.39]
caveat    provisional: one baseline for every regime in the window

0 of 121 flagged. The flow from 04:00 behaves like the flow at 20:00.

Rules of thumb¶

Same operating point: throughput, grade, recipe and controller mode as the window. The segment medians show the levels.
Clean: no clipped sample, which censored no confirms, and no outage when you run mspc or compare's joint tables.
Long enough: at least 30 GOOD samples; a few hundred give steadier limits. Segment 1 holds 116.
Normal: a period the operators call normal, not the week of a known upset.

When the window visits several operating points, a single baseline cannot fit it. Write the segments as a MODE archive with segment --mode-out and screen per regime with --mode, as baselines and the MAD screen shows.

If the baseline is refused¶

window is censored: the baseline holds a clipped sample. Move it off the stretch segment shows at the range end.
only 20 GOOD history samples; 30 required: widen it, and check it lies inside the archive's time range.
aligned coverage 0.867 below required 0.95 from mspc: move it off the outage.