From d0f7ba321b1b38f6108b614cb08b69f8c1ccb447 Mon Sep 17 00:00:00 2001 From: Julian Pawlowski Date: Mon, 6 Apr 2026 12:58:02 +0000 Subject: [PATCH] docs: document statistics optimization and fix entity ID examples Add coverage for the state_class/statistics table optimization across both user and developer documentation. docs/user/docs/configuration.md: - Add 'Price Sensor Statistics' section explaining that only 3 sensors write to the HA statistics database (current_interval_price, current_interval_price_base, average_price_today) - Fix incorrect entity ID examples: remove non-existent _override suffix from recorder exclude globs, Developer Tools example, and seasonal automation example (actual IDs: number.*_best_price_flexibility etc.) docs/developer/docs/recorder-optimization.md: - Add 'Long-Term Statistics Optimization (state_class)' section covering the statistics/statistics_short_term table dimension, which is distinct from _unrecorded_attributes (state_attributes table) - Documents the MONETARY device_class constraint (MEASUREMENT blocked by hassfest, only TOTAL or None valid), the 3 sensors keeping TOTAL with rationale, the 23 sensors set to None, and ~88% write reduction - Includes comparison table: _unrecorded_attributes vs state_class Impact: Users now understand the built-in statistics optimization and have correct recorder exclude examples. Developers understand both optimization layers and their interaction. --- docs/developer/docs/recorder-optimization.md | 67 ++++++++++++++++++++ docs/user/docs/configuration.md | 24 +++++-- 2 files changed, 85 insertions(+), 6 deletions(-) diff --git a/docs/developer/docs/recorder-optimization.md b/docs/developer/docs/recorder-optimization.md index 3579674..50cc494 100644 --- a/docs/developer/docs/recorder-optimization.md +++ b/docs/developer/docs/recorder-optimization.md @@ -283,6 +283,73 @@ LIMIT 5; - ✅ Attributes still available via `entity.attributes` in templates/automations - ✅ Only prevents **storage** in Recorder, not runtime availability +## Long-Term Statistics Optimization (`state_class`) + +**This is a second, independent mechanism** that controls writes to the HA `statistics` and `statistics_short_term` tables — distinct from `_unrecorded_attributes`, which only affects the `state_attributes` table. + +### Why This Matters + +- The `state_attributes` table (controlled by `_unrecorded_attributes`) is **auto-purged** after ~10 days +- The `statistics`/`statistics_short_term` tables (controlled by `state_class`) are **never auto-purged** — they grow unbounded + +This makes `state_class=TOTAL` on many sensors the primary cause of long-term database bloat for users with long-running installations. + +### HA Constraint: MONETARY Device Class + +For sensors with `device_class=SensorDeviceClass.MONETARY`, only two `state_class` values are valid: + +| `state_class` | Statistics written | Frontend effect | +|---|---|---| +| `TOTAL` | ✅ Yes — unbounded growth | Statistics line-chart on entity detail page | +| `None` | ❌ No | States timeline only (History panel, "Show More") | +| `MEASUREMENT` | ❌ Blocked by hassfest | — | + +`MEASUREMENT` causes a hassfest validation error for MONETARY sensors, leaving only `TOTAL` or `None`. + +### Implementation Decision + +Only 3 of 26 MONETARY sensors keep `state_class=TOTAL` — those where long-term history is genuinely useful: + +| Sensor | Reason | +|---|---| +| `current_interval_price` | Long-term price trend (weeks/months) | +| `current_interval_price_base` | Required for Energy Dashboard | +| `average_price_today` | Seasonal daily average tracking | + +All other 23 MONETARY sensors use `state_class=None`: +- Forecast/future sensors (`next_avg_*h`) +- Daily snapshots (`lowest/highest_price_today/tomorrow`) +- Rolling windows (`trailing/leading_24h_*`) +- Next/previous interval sensors + +**Effect of `state_class=None`:** +- ✅ Short-term state history (States timeline, ~10 days) still works normally +- ✅ Templates, automations, and attributes are unaffected +- ❌ Statistics line-chart removed from entity detail page for these sensors +- ❌ No writes to `statistics`/`statistics_short_term` tables + +### Expected Impact + +Going from 26 → 3 sensors writing to the statistics tables: +- **~88% reduction** in statistics table writes +- Prevents the primary cause of long-term database bloat +- Existing statistics data is retained (only new writes stop) + +### Relationship to `_unrecorded_attributes` + +These are two independent mechanisms targeting different tables: + +| Mechanism | Table affected | Purged? | Controls | +|---|---|---|---| +| `_unrecorded_attributes` | `state_attributes` | ✅ ~10 days | Which attributes are stored per state write | +| `state_class=None` | `statistics`, `statistics_short_term` | ❌ Never | Whether long-term statistics are written at all | + +Both optimizations work together. `_unrecorded_attributes` reduces the size of each state write; `state_class=None` eliminates an entire category of unbounded writes. + +### Implementation File + +- **`custom_components/tibber_prices/sensor/definitions.py`** — `state_class` set per-sensor in `SensorEntityDescription` + ## References - [HA Developer Docs - Excluding State Attributes](https://developers.home-assistant.io/docs/core/entity/#excluding-state-attributes-from-recorder-history) diff --git a/docs/user/docs/configuration.md b/docs/user/docs/configuration.md index 07923d3..07379cd 100644 --- a/docs/user/docs/configuration.md +++ b/docs/user/docs/configuration.md @@ -129,7 +129,7 @@ Each configuration entity includes a detailed description attribute explaining w **Note:** For **Number entities**, Home Assistant displays a history graph by default, which hides the attributes panel. To view the `description` attribute: 1. Go to **Developer Tools → States** -2. Search for the entity (e.g., `number._best_price_flexibility_override`) +2. Search for the entity (e.g., `number._best_price_flexibility`) 3. Expand the attributes section to see the full description **Switch entities** display their attributes normally in the entity details view. @@ -148,7 +148,7 @@ automation: action: - service: number.set_value target: - entity_id: number._best_price_flexibility_override + entity_id: number._best_price_flexibility data: value: 10 # Stricter than default 15% ``` @@ -169,13 +169,25 @@ recorder: exclude: entity_globs: # Exclude all Tibber Prices configuration entities - - number.*_best_price_*_override - - number.*_peak_price_*_override - - switch.*_best_price_*_override - - switch.*_peak_price_*_override + - number.*_best_price_* + - number.*_peak_price_* + - switch.*_best_price_* + - switch.*_peak_price_* ``` This is especially useful if: - You rarely change these settings - You want the smallest possible database footprint - You don't need to see the history graph for these entities + +#### Price Sensor Statistics + +The integration also minimizes long-term statistics growth for price sensors. Only 3 sensors write to the HA statistics database (which is never auto-purged): + +- **Current Electricity Price** — Long-term price trend over weeks/months +- **Current Electricity Price (Energy Dashboard)** — Required for Energy Dashboard integration +- **Today's Average Price** — Seasonal price comparison + +All other price sensors (forecasts, rolling averages, daily min/max, future averages) have long-term statistics disabled. Their **state history** (the step chart in the History panel) still works normally for ~10 days — only the smooth statistics line-chart on the entity detail page is absent for these sensors. + +No configuration changes are needed — this optimization is built into the integration.