H15 parameters

Test parameters to use for HI15 QA

[1]:

import pandas as pd
import matplotlib.pyplot as plt

# Jupyter magic to make plots display interactive
# must install ipympl (Ipython-matplotlib) and nodejs
from ipywidgets.embed import embed_minimal_html
%matplotlib widget

import sys
sys.path.append("../")
from post_gce_qc import qaqc, data_transfer, cross_probe_qc, main

[3]:

prov = data_transfer.LoadProvisionalData(file_n='../config_new.yaml', strtyr=2019, endyr=2025, fname_base='MS00413_PPT_L1_5min_')
prov.load_ppt_data()
df = prov.pivot_on_probe(prov.df, 'H15', '02')

[13]:

df.ACC.plot(grid=True)

[13]:

<Axes: xlabel='Date'>

[5]:

qa = qaqc.QaRules(df, {'precision':0.2})

Double precip

[6]:

qa.flag_double_precip(min_ppt_nprecision=2, flat_tank_nprecision=1, flat_ppt_nprecision=1)

[8]:

qa.qa_events.duplicate[qa.qa_events.duplicate==True]

[8]:

Series([], Name: duplicate, dtype: bool[pyarrow])

Let’s check for empty tanks, that’s when this problem usually happens.

[14]:

plt.figure()
df.INST.plot(grid=True)

[14]:

<Axes: xlabel='Date'>

[26]:

strt, end = pd.to_datetime('2/4/24 1800'), pd.to_datetime('2/4/24 2000')

df.loc[strt:end]

[26]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2024-02-04 18:00:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:05:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:10:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:15:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:20:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:25:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:30:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:35:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:40:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:45:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:50:00 0.0 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:55:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:00:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:05:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:10:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:15:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:20:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:25:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:30:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:35:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:40:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:45:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:50:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:55:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 20:00:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA> 
[28]:

strt, end = pd.to_datetime('2/28/24 1500'), pd.to_datetime('2/28/24 1700')

df.loc[strt:end]

[28]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2024-02-28 15:00:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:05:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:10:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:15:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:20:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:25:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:30:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:35:00 0.42 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:40:00 0.42 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:45:00 0.42 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:50:00 0.62 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 15:55:00 0.62 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-28 16:00:00 0.83 <NA> 0.62 W 1284.030029 W
2024-02-28 16:05:00 1.04 <NA> 0.21 <NA> 1284.23999 <NA>
2024-02-28 16:10:00 1.04 <NA> 0.0 <NA> 1284.23999 <NA>
2024-02-28 16:15:00 1.04 <NA> 0.0 <NA> 1284.23999 <NA>
2024-02-28 16:20:00 1.04 <NA> 0.0 <NA> 1284.23999 <NA>
2024-02-28 16:25:00 1.25 <NA> 0.21 <NA> 1284.449951 <NA>
2024-02-28 16:30:00 1.46 <NA> 0.21 <NA> 1284.660034 <NA>
2024-02-28 16:35:00 1.66 <NA> 0.21 <NA> 1284.869995 <NA>
2024-02-28 16:40:00 1.87 <NA> 0.21 <NA> 1285.079956 <NA>
2024-02-28 16:45:00 2.08 <NA> 0.21 <NA> 1285.280029 <NA>
2024-02-28 16:50:00 2.08 <NA> 0.0 <NA> 1285.280029 <NA>
2024-02-28 16:55:00 2.08 <NA> 0.0 <NA> 1285.280029 <NA>
2024-02-28 17:00:00 2.29 <NA> 0.21 <NA> 1285.48999 <NA> 
[15]:

strt, end = pd.to_datetime('6/6/19 1530'), pd.to_datetime('6/6/19 1630')

df.loc[strt:end]

[15]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2019-06-06 15:30:00 112.099998 <NA> 0.0 <NA> 1707.530029 <NA>
2019-06-06 15:35:00 112.599998 <NA> 0.0 <NA> 1707.530029 <NA>
2019-06-06 15:40:00 112.800003 <NA> 0.0 <NA> 1707.530029 <NA>
2019-06-06 15:45:00 4.99 <NA> 0.0 R 1707.530029 R
2019-06-06 15:50:00 2.7 <NA> 0.0 <NA> 1707.530029 <NA>
2019-06-06 15:55:00 7.91 <NA> 2.91 F 1710.439941 F
2019-06-06 16:00:00 7.91 <NA> 0.0 <NA> 1710.439941 <NA>
2019-06-06 16:05:00 7.91 <NA> 0.0 <NA> 1710.439941 <NA>
2019-06-06 16:10:00 8.11 <NA> 0.2 <NA> 1710.650024 <NA>
2019-06-06 16:15:00 8.11 <NA> 0.0 <NA> 1710.650024 <NA>
2019-06-06 16:20:00 8.11 <NA> 0.0 <NA> 1710.650024 <NA>
2019-06-06 16:25:00 8.11 <NA> 0.0 <NA> 1710.650024 <NA>
2019-06-06 16:30:00 8.11 <NA> 0.0 <NA> 1710.650024 <NA> 
[16]:

strt, end = pd.to_datetime('3/8/23 1115'), pd.to_datetime('3/8/23 1200')

df.loc[strt:end]

[16]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2023-03-08 11:15:00 138.199997 <NA> 0.2 <NA> 986.070007 <NA>
2023-03-08 11:20:00 138.199997 <NA> 0.0 <NA> 986.070007 <NA>
2023-03-08 11:25:00 1.66 <NA> 0.0 R 986.070007 R
2023-03-08 11:30:00 0.21 <NA> 0.0 <NA> 986.070007 <NA>
2023-03-08 11:35:00 9.78 <NA> 8.11 J 994.179993 J
2023-03-08 11:40:00 10.2 <NA> 0.42 <NA> 994.599976 <NA>
2023-03-08 11:45:00 10.2 <NA> 0.0 <NA> 994.599976 <NA>
2023-03-08 11:50:00 10.2 <NA> 0.0 <NA> 994.599976 <NA>
2023-03-08 11:55:00 10.2 <NA> 0.0 <NA> 994.599976 <NA>
2023-03-08 12:00:00 10.2 <NA> 0.0 <NA> 994.599976 <NA>

None of those are doubled, but they should all be caught by empty tank.

Empty Tank

[20]:

qa.flag_empty_tank(pause_nsteps=2)

[30]:

qa.df_orig[qa.qa_events.tank_empty==True]

[30]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2024-02-04 18:50:00 0.0 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 18:55:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>
2024-02-04 19:00:00 0.21 <NA> 0.0 <NA> 1283.410034 <NA>

Uhoh. Only one of them actually hit 0… I don’t see any sort of E flags from GCE…hmmm. It turns out that a lot of those low periods had antifreeze added, but were just over-drains, so they did not go below zero. This should be caught by drain and recharge.

Drain Recharge

A few moments have been identified where the tank was overdrained and recharged with antifreeze. These infrequent events were added to manual flags, but should be able to be caught by the drain_recharge_flagging_wrapper.

[31]:

# use defaults
qa.drain_recharge_flagging_wrap()

[34]:

pd.options.display.min_rows = 30
qa.df_orig[qa.qa_events.drain_event==True]

[34]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2018-10-08 16:00:00 26.42 <NA> 0.0 R 38.700001 R
2018-10-08 16:05:00 26.42 <NA> 0.0 <NA> 38.700001 <NA>
2018-10-08 16:10:00 26.42 <NA> 0.0 <NA> 38.700001 <NA>
2018-11-19 16:15:00 67.620003 <NA> 0.0 R 176.679993 R
2018-11-19 16:20:00 67.620003 <NA> 0.0 <NA> 176.679993 <NA>
2018-11-19 16:25:00 67.620003 <NA> 0.0 <NA> 176.679993 <NA>
2018-12-19 14:35:00 94.199997 <NA> 0.0 R 531.159973 R
2018-12-19 14:40:00 94.199997 <NA> 0.0 <NA> 531.159973 <NA>
2018-12-19 14:45:00 94.199997 <NA> 0.0 <NA> 531.159973 <NA>
2018-12-19 19:10:00 94.199997 <NA> 0.0 <NA> 531.460022 <NA>
2018-12-19 20:30:00 94.199997 <NA> 0.0 <NA> 531.460022 <NA>
2018-12-19 21:10:00 94.199997 <NA> 0.0 <NA> 531.460022 <NA>
2018-12-19 21:50:00 94.199997 <NA> 0.0 <NA> 531.460022 <NA>
2018-12-19 22:30:00 94.199997 <NA> 0.0 <NA> 531.460022 <NA>
2018-12-19 23:35:00 94.199997 <NA> 0.0 <NA> 531.460022 <NA>
... ... ... ... ... ... ...
2024-06-05 17:45:00 27.67 <NA> 0.0 <NA> 1849.459961 <NA>
2024-06-05 17:50:00 27.66 <NA> 0.0 <NA> 1849.459961 <NA>
2024-06-13 13:55:00 28.290001 <NA> 0.0 <NA> 1850.709961 <NA>
2024-06-14 10:20:00 26.01 <NA> 0.0 <NA> 1850.719971 <NA>
2024-06-14 13:15:00 28.09 <NA> 0.0 <NA> 1850.719971 <NA>
2024-06-14 13:20:00 28.09 <NA> 0.0 <NA> 1850.719971 <NA>
2024-06-14 14:10:00 28.299999 <NA> 0.0 <NA> 1850.719971 <NA>
2024-06-14 14:15:00 28.299999 <NA> 0.0 <NA> 1850.719971 <NA>
2024-06-14 14:40:00 28.299999 <NA> 0.0 <NA> 1850.719971 <NA>
2024-06-14 14:45:00 28.299999 <NA> 0.0 <NA> 1850.719971 <NA>
2024-06-14 14:50:00 28.09 <NA> 0.0 <NA> 1850.719971 <NA>
2024-09-02 01:35:00 83.800003 <NA> 0.0 <NA> 1905.689941 <NA>
2024-09-21 13:55:00 18.52 <NA> 0.0 R 1933.48999 R
2024-09-21 14:00:00 18.52 <NA> 0.0 <NA> 1933.48999 <NA>
2024-09-21 14:05:00 18.52 <NA> 0.0 M 1933.48999 M

384 rows × 6 columns

[35]:

qa.df_orig[(qa.qa_events.drain_event==True)&(qa.df_orig.TOT > 0)]

[35]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2019-06-06 15:55:00 7.91 <NA> 2.91 F 1710.439941 F
2019-11-18 15:55:00 31.629999 <NA> 11.65 <NA> 126.129997 <NA>
2021-12-19 12:00:00 39.560001 <NA> 0.21 <NA> 645.369995 <NA>
2022-05-18 15:25:00 28.719999 <NA> 0.21 <NA> 1513.099976 <NA>
2023-03-08 11:35:00 9.78 <NA> 8.11 J 994.179993 J
2024-03-20 13:20:00 36.619999 <NA> 0.21 <NA> 1445.910034 <NA>

With the NOAH IV’s, which are recharged after every drain, the 75th percentile of recharge values is used, but this is such a small sample, it may not be very practical. All of the large values here are from known recharge events, but the 0.21 values could be real rain. We need to check.

[36]:

qa.df_orig.loc[(qa.qa_events.drain_event==True)&(qa.df_orig.TOT > 0), 'TOT'].describe(percentiles=[0.75])

[36]:

count         6.0
mean     3.883333
std       4.89108
min          0.21
50%          1.56
75%          6.81
max         11.65
Name: TOT, dtype: double[pyarrow]

Yeah, that seems really high. We’ll keep it at 2 mm.

[38]:

qa.qa_flags[(qa.qa_events.drain_event==True)&(qa.df_orig.TOT > 0)]

[38]:
Q U C SetNA Set0 E
Date
2019-06-06 15:55:00 False False False True False True
2019-11-18 15:55:00 False False False True False True
2021-12-19 12:00:00 False False False False False False
2022-05-18 15:25:00 False False False False False False
2023-03-08 11:35:00 False False False True False True
2024-03-20 13:20:00 True False False False False False

Well it’s good that only one of the 0.21 values is being flagged, and it’s just a Q. Let’s check it out all the same.

[39]:

strt, end = pd.to_datetime('12/19/21 1100'), pd.to_datetime('12/19/21 1300')
qa.df_orig.loc[strt:end]

[39]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2021-12-19 11:00:00 257.799988 <NA> 0.2 <NA> 642.059998 <NA>
2021-12-19 11:05:00 258.200012 <NA> 0.4 <NA> 642.460022 <NA>
2021-12-19 11:10:00 258.600006 <NA> 0.4 <NA> 642.859985 <NA>
2021-12-19 11:15:00 258.799988 <NA> 0.2 <NA> 643.059998 <NA>
2021-12-19 11:20:00 259.200012 <NA> 0.4 <NA> 643.460022 <NA>
2021-12-19 11:25:00 259.700012 <NA> 0.5 <NA> 643.960022 <NA>
2021-12-19 11:30:00 259.899994 <NA> 0.2 <NA> 644.159973 <NA>
2021-12-19 11:35:00 260.5 <NA> 0.6 <NA> 644.76001 <NA>
2021-12-19 11:40:00 260.899994 <NA> 0.4 <NA> 645.159973 <NA>
2021-12-19 11:45:00 260.899994 <NA> 0.0 <NA> 645.159973 <NA>
2021-12-19 11:50:00 39.349998 <NA> 0.0 R 645.159973 R
2021-12-19 11:55:00 39.349998 <NA> 0.0 <NA> 645.159973 <NA>
2021-12-19 12:00:00 39.560001 <NA> 0.21 <NA> 645.369995 <NA>
2021-12-19 12:05:00 39.970001 <NA> 0.41 <NA> 645.780029 <NA>
2021-12-19 12:10:00 39.970001 <NA> 0.0 <NA> 645.780029 <NA>
2021-12-19 12:15:00 39.970001 <NA> 0.0 <NA> 645.780029 <NA>
2021-12-19 12:20:00 40.18 <NA> 0.21 <NA> 645.98999 <NA>
2021-12-19 12:25:00 40.389999 <NA> 0.21 <NA> 646.200012 <NA>
2021-12-19 12:30:00 40.389999 <NA> 0.0 <NA> 646.200012 <NA>
2021-12-19 12:35:00 40.599998 <NA> 0.21 <NA> 646.409973 <NA>
2021-12-19 12:40:00 40.810001 <NA> 0.21 <NA> 646.619995 <NA>
2021-12-19 12:45:00 41.009998 <NA> 0.2 <NA> 646.820007 <NA>
2021-12-19 12:50:00 41.220001 <NA> 0.21 <NA> 647.030029 <NA>
2021-12-19 12:55:00 41.43 <NA> 0.21 <NA> 647.23999 <NA>
2021-12-19 13:00:00 41.849998 <NA> 0.42 <NA> 647.659973 <NA>

And that didn’t get flagged, so that looks good.

[40]:

strt, end = pd.to_datetime('5/18/22 1425'), pd.to_datetime('5/18/22 1625')
qa.df_orig.loc[strt:end]

[40]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2022-05-18 14:25:00 298.0 <NA> 0.0 <NA> 1511.790039 <NA>
2022-05-18 14:30:00 298.0 <NA> 0.0 <NA> 1511.790039 <NA>
2022-05-18 14:35:00 298.0 <NA> 0.0 <NA> 1511.790039 <NA>
2022-05-18 14:40:00 298.0 <NA> 0.0 <NA> 1511.790039 <NA>
2022-05-18 14:45:00 298.0 <NA> 0.0 <NA> 1511.790039 <NA>
2022-05-18 14:50:00 298.200012 <NA> 0.2 <NA> 1511.98999 <NA>
2022-05-18 14:55:00 299.100006 <NA> 0.9 <NA> 1512.890015 <NA>
2022-05-18 15:00:00 299.100006 <NA> 0.0 <NA> 1512.890015 <NA>
2022-05-18 15:05:00 299.100006 <NA> 0.0 <NA> 1512.890015 <NA>
2022-05-18 15:10:00 99.099998 <NA> 0.0 R 1512.890015 R
2022-05-18 15:15:00 28.719999 <NA> 0.0 R 1512.890015 R
2022-05-18 15:20:00 28.51 <NA> 0.0 R 1512.890015 R
2022-05-18 15:25:00 28.719999 <NA> 0.21 <NA> 1513.099976 <NA>
2022-05-18 15:30:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 15:35:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 15:40:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 15:45:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 15:50:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 15:55:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 16:00:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 16:05:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 16:10:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 16:15:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 16:20:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>
2022-05-18 16:25:00 28.719999 <NA> 0.0 <NA> 1513.099976 <NA>

That also seems ok, and did not get flagged. A little suspicious how little rain there is, but the results seem good.

[41]:

strt, end = pd.to_datetime('3/20/24 1220'), pd.to_datetime('3/20/24 1420')
qa.df_orig.loc[strt:end]

[41]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2024-03-20 12:20:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:25:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:30:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:35:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:40:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:45:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:50:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:55:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 13:00:00 162.300003 <NA> 0.2 <NA> 1445.5 <NA>
2024-03-20 13:05:00 162.5 <NA> 0.2 <NA> 1445.699951 <NA>
2024-03-20 13:10:00 151.899994 <NA> 0.0 R 1445.699951 R
2024-03-20 13:15:00 36.41 <NA> 0.0 R 1445.699951 R
2024-03-20 13:20:00 36.619999 <NA> 0.21 <NA> 1445.910034 <NA>
2024-03-20 13:25:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:30:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:35:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:40:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:45:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:50:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:55:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 14:00:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 14:05:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 14:10:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 14:15:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 14:20:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA> 
[44]:

qa.qa_flags[(qa.qa_flags.Q==True)]

[44]:
Q U C SetNA Set0 E
Date
2024-03-20 13:20:00 True False False False False False 
[45]:

qa.qa_flags[(qa.qa_flags.E==True)]

[45]:
Q U C SetNA Set0 E
Date
2019-06-06 15:55:00 False False False True False True
2019-11-18 15:55:00 False False False True False True
2023-03-08 11:35:00 False False False True False True

That all looks pretty reasonable, but the running average was meant for a window of 4 x15min time steps, so let’s rerun with 12 for the 5 min timestep adn see if it behaves at least as well or a little better.

[48]:

# reset
qa.qa_flags[['Q', 'E', 'SetNA']] = False
qa.qa_events[['drain_event', 'neg_delta_tank']] = False

[49]:

# change run_avg_window to 1 hour
qa.drain_recharge_flagging_wrap(drain_threshold=-25, event_window=3, max_recharge=2.67, runavg_nstd=2, runavg_wind=12)

[50]:

qa.qa_flags[(qa.qa_flags.Q==True)]

[50]:
Q U C SetNA Set0 E
Date
2022-05-18 15:25:00 True False False False False False
2024-03-20 13:20:00 True False False False False False 
[51]:

qa.qa_flags[(qa.qa_flags.E==True)]

[51]:
Q U C SetNA Set0 E
Date
2019-06-06 15:55:00 False False False True False True
2019-11-18 15:55:00 False False False True False True
2023-03-08 11:35:00 False False False True False True 
[52]:

qa.df_orig[(qa.qa_events.drain_event==True)&(qa.df_orig.TOT > 0)]

[52]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2019-06-06 15:55:00 7.91 <NA> 2.91 F 1710.439941 F
2019-11-18 15:55:00 31.629999 <NA> 11.65 <NA> 126.129997 <NA>
2021-12-19 12:00:00 39.560001 <NA> 0.21 <NA> 645.369995 <NA>
2022-05-18 15:25:00 28.719999 <NA> 0.21 <NA> 1513.099976 <NA>
2023-03-08 11:35:00 9.78 <NA> 8.11 J 994.179993 J
2024-03-20 13:20:00 36.619999 <NA> 0.21 <NA> 1445.910034 <NA>

Yeah, the 5/18 seemed like the logic should have flagged it, even though it may be fine. It would be a random bit of rain during a site visit.

Repeating Value Precip

Sometimes the simple_pre.m program seems to get stuck and repeats the same accumulation every 5 min for hours or days. These need to be filtered out.

[53]:

# try defaults
qa.flag_repeating_val_precip()

[54]:

qa.df_orig[qa.qa_events.duplicate==True]

[54]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date 
[55]:

qa.precision

[55]:

0.2

That could mean that this problem doesn’t exist here. But it could be parameterized wrong. So I’ll try a few different permutations.

[56]:

# up the tank oscillations to 2x precision
qa.flag_repeating_val_precip(min_ppt_nprecision=0, flat_ppt_nprecision=0, flat_tank_nprecision=2, min_number_repeating=5)

[57]:

qa.df_orig[qa.qa_events.duplicate==True]

[57]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2018-11-23 08:10:00 108.0 <NA> 0.2 <NA> 217.059998 <NA>
2018-11-23 08:15:00 108.199997 <NA> 0.2 <NA> 217.259995 <NA>
2018-11-23 08:20:00 108.400002 <NA> 0.2 <NA> 217.460007 <NA>
2018-11-23 08:25:00 108.599998 <NA> 0.2 <NA> 217.660004 <NA>
2018-11-23 08:30:00 108.800003 <NA> 0.2 <NA> 217.860001 <NA>
2018-11-23 08:35:00 109.0 <NA> 0.2 <NA> 218.059998 <NA>
2018-11-23 08:40:00 109.199997 <NA> 0.2 <NA> 218.259995 <NA>
2018-11-30 20:30:00 216.199997 <NA> 0.2 <NA> 325.26001 <NA>
2018-11-30 20:35:00 216.399994 <NA> 0.2 <NA> 325.459991 <NA>
2018-11-30 20:40:00 216.600006 <NA> 0.2 <NA> 325.660004 <NA>
2018-11-30 20:45:00 216.800003 <NA> 0.2 <NA> 325.859985 <NA>
2018-11-30 20:50:00 217.0 <NA> 0.2 <NA> 326.059998 <NA>
2018-11-30 20:55:00 217.199997 <NA> 0.2 <NA> 326.26001 <NA>
2018-11-30 21:00:00 217.399994 <NA> 0.2 <NA> 326.459991 <NA>
2018-11-30 21:05:00 217.600006 <NA> 0.2 <NA> 326.660004 <NA>
... ... ... ... ... ... ...
2024-04-29 12:40:00 142.5 <NA> 0.2 <NA> 1641.079956 <NA>
2024-05-04 14:35:00 89.300003 <NA> 0.2 <NA> 1699.859985 <NA>
2024-05-04 14:40:00 89.5 <NA> 0.2 <NA> 1700.060059 <NA>
2024-05-04 14:45:00 89.699997 <NA> 0.2 <NA> 1700.26001 <NA>
2024-05-04 14:50:00 89.900002 <NA> 0.2 <NA> 1700.459961 <NA>
2024-05-04 14:55:00 90.099998 <NA> 0.2 <NA> 1700.660034 <NA>
2024-05-04 15:00:00 90.300003 <NA> 0.2 <NA> 1700.859985 <NA>
2024-05-04 15:05:00 90.5 <NA> 0.2 <NA> 1701.060059 <NA>
2024-06-02 15:10:00 190.0 <NA> 0.2 <NA> 1800.560059 <NA>
2024-06-02 15:15:00 190.199997 <NA> 0.2 <NA> 1800.76001 <NA>
2024-06-02 15:20:00 190.399994 <NA> 0.2 <NA> 1800.959961 <NA>
2024-06-02 15:25:00 190.600006 <NA> 0.2 <NA> 1801.160034 <NA>
2024-06-02 15:30:00 190.800003 <NA> 0.2 <NA> 1801.359985 <NA>
2024-06-02 15:35:00 191.0 <NA> 0.2 <NA> 1801.560059 <NA>
2024-06-02 15:40:00 191.199997 <NA> 0.2 <NA> 1801.76001 <NA>

1187 rows × 6 columns

Well that looks like pretty valid precip. The original test of default parameters didn’t have any false positives, so I’m pleasantly surprised that this data set doesn’t seem to have this problem.

Tank Fluctuations

Don’t accumulate precip during normal diurnal fluctuations of the tank.

[59]:

qa.flag_precip_during_tank_flux(tank_col='INST', ppt_col='TOT', fluxprecision=5, accprecision=5,
                                     zero_precision=2, fluxwindow=289, accwindow='1D', extend_ahead=3)

[60]:

qa.df_orig[qa.qa_events.diurnal_flux==True]

[60]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date

The PAT used at HI15 does seem to have a lot less signal noise, but this still seems a little unlikely to me. Let’s try adjusting a few of these parameters and see if we don’t find any fluctuations.

[62]:

qa.flag_precip_during_tank_flux(tank_col='INST', ppt_col='TOT', fluxprecision=5, accprecision=3,
                                     zero_precision=2, fluxwindow=289, accwindow='1D', extend_ahead=3)

[63]:

qa.df_orig[qa.qa_events.diurnal_flux==True]

[63]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date 
[64]:

qa.flag_precip_during_tank_flux(tank_col='INST', ppt_col='TOT', fluxprecision=5, accprecision=2,
                                     zero_precision=2, fluxwindow=289, accwindow='1D', extend_ahead=3)

[65]:

qa.df_orig[qa.qa_events.diurnal_flux==True]

[65]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date 
[66]:

qa.flag_precip_during_tank_flux(tank_col='INST', ppt_col='TOT', fluxprecision=3, accprecision=2,
                                     zero_precision=2, fluxwindow=289, accwindow='1D', extend_ahead=3)

[67]:

qa.df_orig[qa.qa_events.diurnal_flux==True]

[67]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2024-06-13 13:20:00 28.92 <NA> 0.83 <NA> 1850.709961 <NA> 
[70]:

plt.close(3)

[72]:

f = qaqc.ApplyFlags(qa.df_orig.index, precision=0.2)
f.import_provisional_data(qa.df_orig)
f.apply_QaRules_flags(qa.qa_events, qa.qa_flags)

day = pd.to_datetime('6/12/24')
f.plot_flagged_day(day, 'H15', tdelta='3D', auto_qa_event=qa.qa_events)

[72]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2024-06-12 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

ReAssess drain

First, the drain event looks wrong. Let’s redo that with a longer running window.

Currently set to default of tank < runavg + precision with a running window of 3 timesteps. In the original testing of NOAHIV’s that was 45 minutes, which would be a window of 9 for all the other loggers.

[73]:

# reset
qa.qa_flags[['Q', 'E', 'SetNA']] = False
qa.qa_events[['drain_event', 'neg_delta_tank']] = False

[82]:

# change event_window (for finding drains) to 1 hour45 min
qa.drain_recharge_flagging_wrap(drain_threshold=-25, event_window=9, max_recharge=2.67, runavg_nstd=2, runavg_wind=12)

[75]:

qa.qa_flags[(qa.qa_flags.Q==True)]

[75]:
Q U C SetNA Set0 E
Date
2018-10-08 16:40:00 True False False False False False
2019-06-06 16:10:00 True False False False False False
2020-01-28 15:45:00 True False False False False False
2020-01-28 15:55:00 True False False False False False
2020-11-18 18:30:00 True False False False False False
2023-03-08 11:40:00 True False False False False False
2023-04-19 13:05:00 True False False False False False
2023-04-19 13:10:00 True False False False False False
2023-04-19 13:15:00 True False False False False False
2024-01-20 17:10:00 True False False False False False
2024-03-20 13:20:00 True False False False False False 
[76]:

qa.qa_flags[(qa.qa_flags.E==True)]

[76]:
Q U C SetNA Set0 E
Date
2019-06-06 15:55:00 False False False True False True
2019-11-18 15:55:00 False False False True False True
2023-03-08 11:35:00 False False False True False True 
[79]:

qa.df_orig[(qa.qa_flags.Q==True)&(qa.df_orig.TOT > 0)]

[79]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2018-10-08 16:40:00 26.629999 <NA> 0.21 <NA> 38.91 <NA>
2019-06-06 16:10:00 8.11 <NA> 0.2 <NA> 1710.650024 <NA>
2020-01-28 15:45:00 23.93 <NA> 0.42 <NA> 865.789978 <NA>
2020-01-28 15:55:00 24.549999 <NA> 0.42 <NA> 866.409973 <NA>
2020-11-18 18:30:00 45.389999 <NA> 0.84 W 298.839996 W
2023-03-08 11:40:00 10.2 <NA> 0.42 <NA> 994.599976 <NA>
2023-04-19 13:05:00 25.389999 <NA> 0.21 <NA> 1364.699951 <NA>
2023-04-19 13:10:00 25.6 <NA> 0.21 <NA> 1364.910034 <NA>
2023-04-19 13:15:00 26.02 <NA> 0.42 <NA> 1365.339966 <NA>
2024-01-20 17:10:00 27.68 <NA> 0.42 <NA> 1123.589966 <NA>
2024-03-20 13:20:00 36.619999 <NA> 0.21 <NA> 1445.910034 <NA>

That added a whole lot of Q’s, let’s try just a 30 min window.

[83]:

# reset
qa.qa_flags[['Q', 'E', 'SetNA']] = False
qa.qa_events[['drain_event', 'neg_delta_tank']] = False

[84]:

# change event window to 30 min for finding drains
qa.drain_recharge_flagging_wrap(drain_threshold=-25, event_window=6, max_recharge=2.67, runavg_nstd=2, runavg_wind=12)

[87]:

qa.df_orig[(qa.qa_flags.Q==True)]

[87]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2019-06-06 16:10:00 8.11 <NA> 0.2 <NA> 1710.650024 <NA>
2020-01-28 15:45:00 23.93 <NA> 0.42 <NA> 865.789978 <NA>
2023-03-08 11:40:00 10.2 <NA> 0.42 <NA> 994.599976 <NA>
2024-03-20 13:20:00 36.619999 <NA> 0.21 <NA> 1445.910034 <NA> 
[88]:

qa.df_orig[(qa.qa_flags.E==True)]

[88]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2019-06-06 15:55:00 7.91 <NA> 2.91 F 1710.439941 F
2019-11-18 15:55:00 31.629999 <NA> 11.65 <NA> 126.129997 <NA>
2023-03-08 11:35:00 9.78 <NA> 8.11 J 994.179993 J 
[93]:

end

[93]:

Timestamp('2024-03-20 18:20:00')

[94]:

strt = pd.to_datetime('6/6/19 1530')
end = strt+ pd.to_timedelta('1h')

qa.df_orig.loc[strt:end]

[94]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2019-06-06 15:30:00 112.099998 <NA> 0.0 <NA> 1707.530029 <NA>
2019-06-06 15:35:00 112.599998 <NA> 0.0 <NA> 1707.530029 <NA>
2019-06-06 15:40:00 112.800003 <NA> 0.0 <NA> 1707.530029 <NA>
2019-06-06 15:45:00 4.99 <NA> 0.0 R 1707.530029 R
2019-06-06 15:50:00 2.7 <NA> 0.0 <NA> 1707.530029 <NA>
2019-06-06 15:55:00 7.91 <NA> 2.91 F 1710.439941 F
2019-06-06 16:00:00 7.91 <NA> 0.0 <NA> 1710.439941 <NA>
2019-06-06 16:05:00 7.91 <NA> 0.0 <NA> 1710.439941 <NA>
2019-06-06 16:10:00 8.11 <NA> 0.2 <NA> 1710.650024 <NA>
2019-06-06 16:15:00 8.11 <NA> 0.0 <NA> 1710.650024 <NA>
2019-06-06 16:20:00 8.11 <NA> 0.0 <NA> 1710.650024 <NA>
2019-06-06 16:25:00 8.11 <NA> 0.0 <NA> 1710.650024 <NA>
2019-06-06 16:30:00 8.11 <NA> 0.0 <NA> 1710.650024 <NA> 
[95]:

# reset
qa.qa_flags[['Q', 'E', 'SetNA']] = False
qa.qa_events[['drain_event', 'neg_delta_tank']] = False

[96]:

# change event window to 30 min for finding drains
qa.drain_recharge_flagging_wrap(drain_threshold=-25, event_window=5, max_recharge=2.67, runavg_nstd=2, runavg_wind=12)

[97]:

qa.df_orig[(qa.qa_flags.Q==True)]

[97]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2023-03-08 11:40:00 10.2 <NA> 0.42 <NA> 994.599976 <NA>
2024-03-20 13:20:00 36.619999 <NA> 0.21 <NA> 1445.910034 <NA> 
[98]:

qa.df_orig[(qa.qa_flags.E==True)]

[98]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2019-06-06 15:55:00 7.91 <NA> 2.91 F 1710.439941 F
2019-11-18 15:55:00 31.629999 <NA> 11.65 <NA> 126.129997 <NA>
2023-03-08 11:35:00 9.78 <NA> 8.11 J 994.179993 J 
[99]:

strt = pd.to_datetime('3/8/23 1100')
end = strt+ pd.to_timedelta('1h')

qa.df_orig.loc[strt:end]

[99]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2023-03-08 11:00:00 137.800003 <NA> 0.0 <NA> 985.869995 <NA>
2023-03-08 11:05:00 137.800003 <NA> 0.0 <NA> 985.869995 <NA>
2023-03-08 11:10:00 138.0 <NA> 0.0 <NA> 985.869995 <NA>
2023-03-08 11:15:00 138.199997 <NA> 0.2 <NA> 986.070007 <NA>
2023-03-08 11:20:00 138.199997 <NA> 0.0 <NA> 986.070007 <NA>
2023-03-08 11:25:00 1.66 <NA> 0.0 R 986.070007 R
2023-03-08 11:30:00 0.21 <NA> 0.0 <NA> 986.070007 <NA>
2023-03-08 11:35:00 9.78 <NA> 8.11 J 994.179993 J
2023-03-08 11:40:00 10.2 <NA> 0.42 <NA> 994.599976 <NA>
2023-03-08 11:45:00 10.2 <NA> 0.0 <NA> 994.599976 <NA>
2023-03-08 11:50:00 10.2 <NA> 0.0 <NA> 994.599976 <NA>
2023-03-08 11:55:00 10.2 <NA> 0.0 <NA> 994.599976 <NA>
2023-03-08 12:00:00 10.2 <NA> 0.0 <NA> 994.599976 <NA>

The Q flag on 0.42 seems very reasonable. That very well could have been the float slowly turning the PAT after recharge, or the recharge being poured in partially before the 5 min mark and partially after. Let’s check that other Q.

[105]:

strt = pd.to_datetime('3/20/24 1215')
end = strt+ pd.to_timedelta('2h')

qa.df_orig.loc[strt:end]

[105]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2024-03-20 12:15:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:20:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:25:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:30:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:35:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:40:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:45:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:50:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 12:55:00 161.899994 <NA> 0.0 <NA> 1445.300049 <NA>
2024-03-20 13:00:00 162.300003 <NA> 0.2 <NA> 1445.5 <NA>
2024-03-20 13:05:00 162.5 <NA> 0.2 <NA> 1445.699951 <NA>
2024-03-20 13:10:00 151.899994 <NA> 0.0 R 1445.699951 R
2024-03-20 13:15:00 36.41 <NA> 0.0 R 1445.699951 R
2024-03-20 13:20:00 36.619999 <NA> 0.21 <NA> 1445.910034 <NA>
2024-03-20 13:25:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:30:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:35:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:40:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:45:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:50:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 13:55:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 14:00:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 14:05:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 14:10:00 36.41 <NA> 0.0 <NA> 1445.910034 <NA>
2024-03-20 14:15:00 36.619999 <NA> 0.0 <NA> 1445.910034 <NA>

That one’s hard to say, it’s right after the drain. Now let’s see if the drain events got fixed in our fluctuation testing.

Tune flux to Find More Bounce

In the graph above, there is some false precip during signal bounce. Let’s see if we can improve performance a little bit.

[107]:

qa.flag_precip_during_tank_flux(tank_col='INST', ppt_col='TOT', fluxprecision=3, accprecision=2,
                                     zero_precision=3, fluxwindow=289, accwindow='1D', extend_ahead=3)

[108]:

qa.df_orig[qa.qa_events.diurnal_flux==True]

[108]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2024-06-13 13:20:00 28.92 <NA> 0.83 <NA> 1850.709961 <NA> 
[109]:

qa.flag_precip_during_tank_flux(tank_col='INST', ppt_col='TOT', fluxprecision=2, accprecision=2,
                                     zero_precision=3, fluxwindow=289, accwindow='1D', extend_ahead=3)

[110]:

qa.df_orig[qa.qa_events.diurnal_flux==True]

[110]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2022-02-23 18:40:00 61.830002 <NA> 0.18 <NA> 858.169983 <NA>
2024-06-13 13:20:00 28.92 <NA> 0.83 <NA> 1850.709961 <NA> 
[111]:

del f

[112]:

f = qaqc.ApplyFlags(qa.df_orig.index, precision=0.2)
f.import_provisional_data(qa.df_orig)
f.apply_QaRules_flags(qa.qa_events, qa.qa_flags)

day = pd.to_datetime('6/12/24')
f.plot_flagged_day(day, 'H15', tdelta='3D', auto_qa_event=qa.qa_events)

[112]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2024-06-12 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

[116]:

plt.close(6)

[117]:

day = pd.to_datetime('2/23/22')
f.plot_flagged_day(day, 'H15', tdelta='2D', auto_qa_event=qa.qa_events)

[117]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2022-02-23 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

[118]:

day = pd.to_datetime('2/22/22')
f.plot_flagged_day(day, 'H15', tdelta='2D', auto_qa_event=qa.qa_events)

[118]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2022-02-22 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

If it weren’t for the drop at the end, this seems like some pretty legitimate, slow accumulating precip. I think this is false, and over tuned. Let’s fine the lightest tuning that will still catch the other date.

[129]:

qa.qa_events.diurnal_flux = False

[130]:

qa.flag_precip_during_tank_flux(tank_col='INST', ppt_col='TOT', fluxprecision=4, accprecision=2,
                                     zero_precision=3, fluxwindow=289, accwindow='1D', extend_ahead=3)

[131]:

qa.df_orig[qa.qa_events.diurnal_flux==True]

[131]:
INST INST_Flag TOT TOT_Flag ACC ACC_Flag
Date
2024-06-13 13:20:00 28.92 <NA> 0.83 <NA> 1850.709961 <NA>

Search for Bounce

I’m going to quickly zoom around the tank to see if I can find any instances of flux that we’re missing.

[132]:

plt.figure()
qa.df_orig.INST.plot(grid=True)

[132]:

<Axes: xlabel='Date'>

[134]:

plt.close(9)

[135]:

day = pd.to_datetime('7/3/19')
f.plot_flagged_day(day, 'H15', tdelta='9D', auto_qa_event=qa.qa_events)

[135]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2019-07-03 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

[136]:

day = pd.to_datetime('7/9/19')
f.plot_flagged_day(day, 'H15', tdelta='1D', auto_qa_event=qa.qa_events)

[136]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2019-07-09 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

[137]:

plt.figure()
qa.df_orig.INST.plot(grid=True)

[137]:

<Axes: xlabel='Date'>

[138]:

day = pd.to_datetime('8/12/19')
f.plot_flagged_day(day, 'H15', tdelta='5D', auto_qa_event=qa.qa_events)

[138]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2019-08-12 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

[139]:

plt.figure()
qa.df_orig.INST.plot(grid=True)

[139]:

<Axes: xlabel='Date'>

[140]:

day = pd.to_datetime('8/8/20')
f.plot_flagged_day(day, 'H15', tdelta='10D', auto_qa_event=qa.qa_events)

[140]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2020-08-08 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

[141]:

plt.figure()
qa.df_orig.INST.plot(grid=True)

[141]:

<Axes: xlabel='Date'>

[143]:

plt.close(16)

[144]:

day = pd.to_datetime('6/25/21')
f.plot_flagged_day(day, 'H15', tdelta='10D', auto_qa_event=qa.qa_events)

[144]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2021-06-25 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

[145]:

plt.figure()
qa.df_orig.INST.plot(grid=True)

[145]:

<Axes: xlabel='Date'>

[146]:

day = pd.to_datetime('7/9/23')
f.plot_flagged_day(day, 'H15', tdelta='10D', auto_qa_event=qa.qa_events)

[146]:

(<Axes: xlabel='Date', ylabel='Precip (mm)'>,
 <Axes: title={'center': 'H15 - 2023-07-09 00:00:00'}, xlabel='Date', ylabel='Tank Height (mm)'>)

The accumulation algorithm in simple_pre.m is behaving itself exceptionally well here when the signal noise picks up. It is a much more stable signal than the tank level gauges. I don’ think we’re missing any problematic fluctuation-accumulation here.

[ ]: