jpawlowski/hass.tibber_prices
1
0
Fork 0
mirror of https://github.com/jpawlowski/hass.tibber_prices.git synced 2026-07-28 01:26:49 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

chore(style): normalize Markdown list indentation across all docs

Browse source 
Convert four-space-indented list items (`-   item`) to standard two-space
(`- item`) in AGENTS.md, CONTRIBUTING.md, README.md, and all Docusaurus
documentation pages (developer and user, including versioned snapshots).
No content changes.

Release-Notes: skip
This commit is contained in:
Julian Pawlowski 2026-04-12 14:15:31 +00:00
parent e163a47d57
commit aa9a1200b8
339 changed files with 16987 additions and 12955 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

83
AGENTS.md
View file

@ -49,7 +49,6 @@ When working with the codebase, Copilot MUST actively maintain consistency betwe
**When to discuss in chat vs. direct file changes:** **When to discuss in chat vs. direct file changes:**
- **Make direct changes when:** - **Make direct changes when:**
- Clear, straightforward task (fix bug, add function, update config) - Clear, straightforward task (fix bug, add function, update config)
- Single approach is obvious - Single approach is obvious
- User request is specific ("add X", "change Y to Z") - User request is specific ("add X", "change Y to Z")
@ -205,29 +204,24 @@ Skip planning for:
**Planning Document Lifecycle:** **Planning Document Lifecycle:**
1. **Planning Phase** (WIP in `/planning/`) 1. **Planning Phase** (WIP in `/planning/`)
- Create `planning/<feature>-refactoring-plan.md` - Create `planning/<feature>-refactoring-plan.md`
- Iterate freely (git-ignored, no commit pressure) - Iterate freely (git-ignored, no commit pressure)
- AI can help refine without polluting git history - AI can help refine without polluting git history
- Multiple revisions until plan is solid - Multiple revisions until plan is solid
2. **Implementation Phase** (Active work) 2. **Implementation Phase** (Active work)
- Use plan as reference during coding - Use plan as reference during coding
- Update plan if issues discovered - Update plan if issues discovered
- Track progress through phases - Track progress through phases
- Test after each phase - Test after each phase
3. **Completion Phase** (After implementation) 3. **Completion Phase** (After implementation)
- **Option A**: Move to `docs/development/` if lasting value - **Option A**: Move to `docs/development/` if lasting value
- Example: `planning/module-splitting-plan.md` → `docs/development/module-splitting-plan.md` - Example: `planning/module-splitting-plan.md` → `docs/development/module-splitting-plan.md`
- Update status to "✅ COMPLETED" - Update status to "✅ COMPLETED"
- Commit as historical reference - Commit as historical reference
- **Option B**: Delete if superseded - **Option B**: Delete if superseded
- Plan served its purpose - Plan served its purpose
- Code and AGENTS.md are source of truth - Code and AGENTS.md are source of truth
@ -310,12 +304,14 @@ After successful refactoring:
**Root Directory (`custom_components/tibber_prices/`):** **Root Directory (`custom_components/tibber_prices/`):**
**✅ ALLOWED in root:** **✅ ALLOWED in root:**
- Platform modules: `__init__.py`, `sensor.py` (deprecated, now `sensor/`), `binary_sensor.py` (deprecated, now `binary_sensor/`), future platforms - Platform modules: `__init__.py`, `sensor.py` (deprecated, now `sensor/`), `binary_sensor.py` (deprecated, now `binary_sensor/`), future platforms
- Core integration files: `const.py`, `manifest.json`, `services.yaml`, `diagnostics.py`, `data.py`, `migrations.py` - Core integration files: `const.py`, `manifest.json`, `services.yaml`, `diagnostics.py`, `data.py`, `migrations.py`
- Translation directories: `translations/`, `custom_translations/` - Translation directories: `translations/`, `custom_translations/`
- Brand images: `brand/` (icon.png, dark_icon.png, logo.png, dark_logo.png + `@2x` variants) — served via HA brands proxy API (HA ≥ 2026.4), silently ignored on older versions - Brand images: `brand/` (icon.png, dark_icon.png, logo.png, dark_logo.png + `@2x` variants) — served via HA brands proxy API (HA ≥ 2026.4), silently ignored on older versions
**❌ PROHIBITED in root:** **❌ PROHIBITED in root:**
- Utility modules (use `/utils/` package instead) - Utility modules (use `/utils/` package instead)
- Helper functions (use `/utils/` or appropriate package) - Helper functions (use `/utils/` or appropriate package)
- Data transformation logic (use `/utils/` or `/coordinator/`) - Data transformation logic (use `/utils/` or `/coordinator/`)
@ -368,6 +364,7 @@ After successful refactoring:
**When Adding New Files:** **When Adding New Files:**
**Before creating a new file in root, ask:** **Before creating a new file in root, ask:**
1. Is this a new HA platform? → OK in root (e.g., `switch.py`, `number.py`) 1. Is this a new HA platform? → OK in root (e.g., `switch.py`, `number.py`)
2. Is this a utility/helper? → Goes in `/utils/` or `/entity_utils/` 2. Is this a utility/helper? → Goes in `/utils/` or `/entity_utils/`
3. Is this coordinator-related? → Goes in `/coordinator/` 3. Is this coordinator-related? → Goes in `/coordinator/`
@ -389,7 +386,6 @@ After successful refactoring:
**Key Patterns:** **Key Patterns:**
- **Dual translation system**: Standard HA translations in `/translations/` (config flow, UI strings per HA schema), supplemental in `/custom_translations/` (entity descriptions not supported by HA schema). Both must stay in sync. Use `async_load_translations()` and `async_load_standard_translations()` from `const.py`. When to use which: `/translations/` is bound to official HA schema requirements; anything else goes in `/custom_translations/` (requires manual translation loading). **Schema reference**: `/schemas/json/translation_schema.json` provides the structure for `/translations/*.json` files based on [HA's translation documentation](https://developers.home-assistant.io/docs/internationalization/core). - **Dual translation system**: Standard HA translations in `/translations/` (config flow, UI strings per HA schema), supplemental in `/custom_translations/` (entity descriptions not supported by HA schema). Both must stay in sync. Use `async_load_translations()` and `async_load_standard_translations()` from `const.py`. When to use which: `/translations/` is bound to official HA schema requirements; anything else goes in `/custom_translations/` (requires manual translation loading). **Schema reference**: `/schemas/json/translation_schema.json` provides the structure for `/translations/*.json` files based on [HA's translation documentation](https://developers.home-assistant.io/docs/internationalization/core).
- **Select selector translations**: Use `selector.{translation_key}.options.{value}` structure (NOT `selector.select.{translation_key}`). Translation keys map to JSON in `/translations/*.json` following the HA schema structure. - **Select selector translations**: Use `selector.{translation_key}.options.{value}` structure (NOT `selector.select.{translation_key}`). Translation keys map to JSON in `/translations/*.json` following the HA schema structure.
**CRITICAL Rules:** **CRITICAL Rules:**
@ -460,6 +456,7 @@ The integration uses **4 distinct caching layers** with automatic invalidation:
- **Why**: Avoid expensive calculation (~100-500ms) when data unchanged (70% CPU saving) - **Why**: Avoid expensive calculation (~100-500ms) when data unchanged (70% CPU saving)
**Cache Invalidation Coordination**: **Cache Invalidation Coordination**:
- Options change → Explicit `invalidate_config_cache()` on both DataTransformer and PeriodCalculator - Options change → Explicit `invalidate_config_cache()` on both DataTransformer and PeriodCalculator
- Midnight turnover → Clear persistent + transformation cache, period cache auto-invalidates via hash - Midnight turnover → Clear persistent + transformation cache, period cache auto-invalidates via hash
- Tomorrow data arrival → Hash mismatch triggers period recalculation only - Tomorrow data arrival → Hash mismatch triggers period recalculation only
@ -542,6 +539,7 @@ custom_components/tibber_prices/
### Dependency Flow (Calculator Pattern) ### Dependency Flow (Calculator Pattern)
**Clean Separation:** **Clean Separation:**
``` ```
sensor/calculators/ → sensor/attributes/ (Volatility only - Hybrid Pattern) sensor/calculators/ → sensor/attributes/ (Volatility only - Hybrid Pattern)
sensor/calculators/ → sensor/helpers/ (DailyStat, RollingHour - Pure functions) sensor/calculators/ → sensor/helpers/ (DailyStat, RollingHour - Pure functions)
@ -553,6 +551,7 @@ sensor/helpers/ ✗ (NO imports from calculators/)
``` ```
**Why this works:** **Why this works:**
- **One-way dependencies**: Calculators can import from attributes/helpers, but NOT vice versa - **One-way dependencies**: Calculators can import from attributes/helpers, but NOT vice versa
- **No circular imports**: Reverse direction is empty (verified Jan 2025) - **No circular imports**: Reverse direction is empty (verified Jan 2025)
- **Clean testing**: Each layer can be tested independently - **Clean testing**: Each layer can be tested independently
@ -562,6 +561,7 @@ sensor/helpers/ ✗ (NO imports from calculators/)
**Background:** During Nov 2025 refactoring, Trend and Volatility calculators retained attribute-building logic to avoid duplicating complex calculations. This creates a **backwards dependency** (calculator → attributes) but is INTENTIONAL. **Background:** During Nov 2025 refactoring, Trend and Volatility calculators retained attribute-building logic to avoid duplicating complex calculations. This creates a **backwards dependency** (calculator → attributes) but is INTENTIONAL.
**Pattern:** **Pattern:**
1. **Calculator** computes value AND builds attribute dict 1. **Calculator** computes value AND builds attribute dict
2. **Core** stores attributes in `cached_data` dict 2. **Core** stores attributes in `cached_data` dict
3. **Attributes package** retrieves cached attributes via: 3. **Attributes package** retrieves cached attributes via:
@ -569,6 +569,7 @@ sensor/helpers/ ✗ (NO imports from calculators/)
- `_add_timing_or_volatility_attributes()` for volatility sensors - `_add_timing_or_volatility_attributes()` for volatility sensors
**Example (Volatility):** **Example (Volatility):**
```python ```python
# sensor/calculators/volatility.py # sensor/calculators/volatility.py
from custom_components.tibber_prices.sensor.attributes import ( from custom_components.tibber_prices.sensor.attributes import (
@ -591,6 +592,7 @@ def get_volatility_attributes(self) -> dict | None:
``` ```
**Trade-offs:** **Trade-offs:**
- ✅ **Pro**: Complex logic stays in ONE place (no duplication) - ✅ **Pro**: Complex logic stays in ONE place (no duplication)
- ✅ **Pro**: Calculator has full context for attribute decisions - ✅ **Pro**: Calculator has full context for attribute decisions
- ❌ **Con**: Violates strict separation (calculator builds attributes) - ❌ **Con**: Violates strict separation (calculator builds attributes)
@ -603,6 +605,7 @@ def get_volatility_attributes(self) -> dict | None:
All calculator modules use `TYPE_CHECKING` correctly: All calculator modules use `TYPE_CHECKING` correctly:
**Pattern:** **Pattern:**
```python ```python
# Runtime imports (used in function bodies) # Runtime imports (used in function bodies)
from custom_components.tibber_prices.const import CONF_PRICE_RATING_THRESHOLD_HIGH from custom_components.tibber_prices.const import CONF_PRICE_RATING_THRESHOLD_HIGH
@ -617,23 +620,27 @@ if TYPE_CHECKING:
``` ```
**Rules:** **Rules:**
- ✅ **Runtime imports**: Functions, classes, constants used in code → OUTSIDE TYPE_CHECKING - ✅ **Runtime imports**: Functions, classes, constants used in code → OUTSIDE TYPE_CHECKING
- ✅ **Type-only imports**: Only used in type hints → INSIDE TYPE_CHECKING - ✅ **Type-only imports**: Only used in type hints → INSIDE TYPE_CHECKING
- ✅ **Coordinator import**: Always in base.py, inherited by all calculators - ✅ **Coordinator import**: Always in base.py, inherited by all calculators
**Verified Status (Jan 2025):** **Verified Status (Jan 2025):**
- All 8 calculators (base, interval, rolling_hour, daily_stat, window_24h, volatility, trend, timing, metadata) use TYPE_CHECKING correctly - All 8 calculators (base, interval, rolling_hour, daily_stat, window_24h, volatility, trend, timing, metadata) use TYPE_CHECKING correctly
- No optimization needed - imports are already categorized optimally - No optimization needed - imports are already categorized optimally
### Import Anti-Patterns to Avoid ### Import Anti-Patterns to Avoid
❌ **DON'T:** ❌ **DON'T:**
- Import from higher layers (attributes/helpers importing from calculators) - Import from higher layers (attributes/helpers importing from calculators)
- Use runtime imports for type-only dependencies - Use runtime imports for type-only dependencies
- Create circular dependencies between packages - Create circular dependencies between packages
- Import entire modules when only needing one function - Import entire modules when only needing one function
✅ **DO:** ✅ **DO:**
- Follow one-way dependency flow (calculators → attributes/helpers) - Follow one-way dependency flow (calculators → attributes/helpers)
- Use TYPE_CHECKING for type-only imports - Use TYPE_CHECKING for type-only imports
- Import specific items: `from .helpers import aggregate_price_data` - Import specific items: `from .helpers import aggregate_price_data`
@ -646,6 +653,7 @@ if TYPE_CHECKING:
**Core Challenge:** **Core Challenge:**
The period calculation applies **three independent filters** that ALL must pass: The period calculation applies **three independent filters** that ALL must pass:
1. **Flex filter**: `price ≤ daily_min × (1 + flex)` 1. **Flex filter**: `price ≤ daily_min × (1 + flex)`
2. **Min_Distance filter**: `price ≤ daily_avg × (1 - min_distance/100)` 2. **Min_Distance filter**: `price ≤ daily_avg × (1 - min_distance/100)`
3. **Level filter**: `rating_level IN [allowed_levels]` 3. **Level filter**: `rating_level IN [allowed_levels]`
@ -655,6 +663,7 @@ The period calculation applies **three independent filters** that ALL must pass:
When `daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)`, the flex filter permits intervals that the min_distance filter blocks, causing zero periods despite high flexibility. When `daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)`, the flex filter permits intervals that the min_distance filter blocks, causing zero periods despite high flexibility.
Example: daily_min=10 ct, daily_avg=20 ct, flex=50%, min_distance=5% Example: daily_min=10 ct, daily_avg=20 ct, flex=50%, min_distance=5%
- Flex allows: ≤15 ct - Flex allows: ≤15 ct
- Distance allows: ≤19 ct - Distance allows: ≤19 ct
- But combined: Only intervals ≤15 ct AND ≤19 ct AND matching level → Distance becomes dominant constraint - But combined: Only intervals ≤15 ct AND ≤19 ct AND matching level → Distance becomes dominant constraint
@ -685,11 +694,13 @@ Example: daily_min=10 ct, daily_avg=20 ct, flex=50%, min_distance=5%
**Configuration Guidance:** **Configuration Guidance:**
**Recommended Flex Ranges:** **Recommended Flex Ranges:**
- **With relaxation enabled**: 10-20% base flex (relaxation will escalate as needed) - **With relaxation enabled**: 10-20% base flex (relaxation will escalate as needed)
- **Without relaxation**: 20-35% direct flex (no automatic escalation) - **Without relaxation**: 20-35% direct flex (no automatic escalation)
- **Anti-pattern**: Base flex >30% with relaxation enabled → causes rapid escalation and filter conflicts - **Anti-pattern**: Base flex >30% with relaxation enabled → causes rapid escalation and filter conflicts
**Key Constants** (defined in `coordinator/period_handlers/core.py`): **Key Constants** (defined in `coordinator/period_handlers/core.py`):
```python ```python
MAX_SAFE_FLEX = 0.50 # 50% absolute maximum MAX_SAFE_FLEX = 0.50 # 50% absolute maximum
MAX_OUTLIER_FLEX = 0.25 # 25% for stable outlier detection MAX_OUTLIER_FLEX = 0.25 # 25% for stable outlier detection
@ -698,6 +709,7 @@ FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # WARNING at 30% base flex
``` ```
**Relaxation Strategy** (`coordinator/period_handlers/relaxation.py`): **Relaxation Strategy** (`coordinator/period_handlers/relaxation.py`):
- Per-day independent loops (each day escalates separately based on its needs) - Per-day independent loops (each day escalates separately based on its needs)
- Hard cap: 3% absolute maximum increment per step (prevents explosion from high base flex) - Hard cap: 3% absolute maximum increment per step (prevents explosion from high base flex)
- Default configuration: 11 flex levels (15% base → 18% → 21% → ... → 48% max) - Default configuration: 11 flex levels (15% base → 18% → 21% → ... → 48% max)
@ -705,6 +717,7 @@ FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # WARNING at 30% base flex
- Each flex level tries all filter combinations before increasing flex further - Each flex level tries all filter combinations before increasing flex further
**Period Boundary Behavior** (`coordinator/period_handlers/period_building.py`): **Period Boundary Behavior** (`coordinator/period_handlers/period_building.py`):
- Periods can **cross midnight** (day boundaries) naturally - Periods can **cross midnight** (day boundaries) naturally
- Reference price locked to **period start day** for consistency across the entire period - Reference price locked to **period start day** for consistency across the entire period
- Pattern: "Uses reference price from start day of the period for consistency" (same as period statistics) - Pattern: "Uses reference price from start day of the period for consistency" (same as period statistics)
@ -712,6 +725,7 @@ FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # WARNING at 30% base flex
- This prevents artificial splits at midnight when prices remain favorable across the boundary - This prevents artificial splits at midnight when prices remain favorable across the boundary
**Default Configuration Values** (`const.py`): **Default Configuration Values** (`const.py`):
```python ```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection) DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -722,6 +736,7 @@ DEFAULT_RELAXATION_ATTEMPTS_PEAK = 11 # 11 steps: 20% → 50% (3% increment
The relaxation increment is **hard-coded at 3% per step** in `relaxation.py` for reliability and predictability. This prevents configuration issues with high base flex values while still allowing sufficient escalation to the 50% hard maximum. The relaxation increment is **hard-coded at 3% per step** in `relaxation.py` for reliability and predictability. This prevents configuration issues with high base flex values while still allowing sufficient escalation to the 50% hard maximum.
**Dynamic Scaling Table** (min_distance adjustment): **Dynamic Scaling Table** (min_distance adjustment):
``` ```
Flex Scale Example (min_distance=5%) Flex Scale Example (min_distance=5%)
------------------------------------------- -------------------------------------------
@ -737,12 +752,14 @@ Flex Scale Example (min_distance=5%)
**Testing Scenarios:** **Testing Scenarios:**
When debugging period calculation issues: When debugging period calculation issues:
1. Check flex value: Is base flex >30%? Reduce to 15-20% if using relaxation 1. Check flex value: Is base flex >30%? Reduce to 15-20% if using relaxation
2. Check logs for "scaled min_distance": Is it reducing too much? May need lower base flex 2. Check logs for "scaled min_distance": Is it reducing too much? May need lower base flex
3. Check filter statistics: Which filter blocks most intervals? (flex, distance, or level) 3. Check filter statistics: Which filter blocks most intervals? (flex, distance, or level)
4. Check relaxation warnings: INFO at 25%, WARNING at 30% indicate suboptimal config 4. Check relaxation warnings: INFO at 25%, WARNING at 30% indicate suboptimal config
**See:** **See:**
- **Theory documentation**: `docs/developer/docs/period-calculation-theory.md` (comprehensive mathematical analysis, conflict conditions, configuration pitfalls) - **Theory documentation**: `docs/developer/docs/period-calculation-theory.md` (comprehensive mathematical analysis, conflict conditions, configuration pitfalls)
- **Implementation**: `coordinator/period_handlers/` package (core.py, relaxation.py, level_filtering.py, period_building.py) - **Implementation**: `coordinator/period_handlers/` package (core.py, relaxation.py, level_filtering.py, period_building.py)
- **User guide**: `docs/user/docs/period-calculation.md` (simplified user-facing explanations) - **User guide**: `docs/user/docs/period-calculation.md` (simplified user-facing explanations)
@ -866,6 +883,7 @@ _Check-only flow (CI/CD-oriented):_
``` ```
_Agent behavior rules:_ _Agent behavior rules:_
- If asked to "fix", "format", "auto-heal", or "make it pass" → start with fix/format scripts. - If asked to "fix", "format", "auto-heal", or "make it pass" → start with fix/format scripts.
- If asked only to "verify", "validate", or "CI parity" → use check scripts. - If asked only to "verify", "validate", or "CI parity" → use check scripts.
- After applying fixes, run the relevant check script once to confirm a clean state. - After applying fixes, run the relevant check script once to confirm a clean state.
@ -913,7 +931,6 @@ When changes are complete and ready for testing:
1. **Ask user to test**, don't execute `./scripts/develop` yourself 1. **Ask user to test**, don't execute `./scripts/develop` yourself
2. **Provide specific test guidance** based on what changed in this session: 2. **Provide specific test guidance** based on what changed in this session:
- Which UI screens to check (e.g., "Open config flow, step 3") - Which UI screens to check (e.g., "Open config flow, step 3")
- What behavior to verify (e.g., "Dropdown should show translated values") - What behavior to verify (e.g., "Dropdown should show translated values")
- What errors to watch for (e.g., "Check logs for JSON parsing errors") - What errors to watch for (e.g., "Check logs for JSON parsing errors")
@ -944,9 +961,11 @@ When changes are complete and ready for testing:
**Purpose:** Keep commit guidance centralized and avoid duplicated/contradictory rules. **Purpose:** Keep commit guidance centralized and avoid duplicated/contradictory rules.
**Authoritative commit-message instructions:** **Authoritative commit-message instructions:**
- Use `.github/instructions/commit-messages.instructions.md` for commit type/scope, Impact footer style, and release-notes skip trailers. - Use `.github/instructions/commit-messages.instructions.md` for commit type/scope, Impact footer style, and release-notes skip trailers.
**Critical behavior rules (still enforced here):** **Critical behavior rules (still enforced here):**
1. **Commit execution**: Only run `git commit` when the user explicitly asks to commit. A one-time request to commit does not authorize future commits without asking again. 1. **Commit execution**: Only run `git commit` when the user explicitly asks to commit. A one-time request to commit does not authorize future commits without asking again.
2. **Commit message generation**: When the user asks only for a commit message, generate the message and stop — do not run `git commit`. The user will commit themselves. 2. **Commit message generation**: When the user asks only for a commit message, generate the message and stop — do not run `git commit`. The user will commit themselves.
3. **git push**: Never suggest or execute `git push`. The user always handles pushing themselves. 3. **git push**: Never suggest or execute `git push`. The user always handles pushing themselves.
@ -955,6 +974,7 @@ When changes are complete and ready for testing:
6. When suggesting commits, include exact files to stage. 6. When suggesting commits, include exact files to stage.
**Internal/unreleased fixes:** **Internal/unreleased fixes:**
- If a fix never affected released users, mark commit body with one trailer so release notes can exclude it: - If a fix never affected released users, mark commit body with one trailer so release notes can exclude it:
- `Release-Notes: skip` - `Release-Notes: skip`
- `User-Impact: none` - `User-Impact: none`
@ -966,21 +986,18 @@ When changes are complete and ready for testing:
**Multiple Options Available:** **Multiple Options Available:**
1. **Helper Script** (recommended, foolproof) 1. **Helper Script** (recommended, foolproof)
- Script: `./scripts/release/prepare VERSION` - Script: `./scripts/release/prepare VERSION`
- Bumps manifest.json version → commits → creates tag locally - Bumps manifest.json version → commits → creates tag locally
- You review and push when ready - You review and push when ready
- Example: `./scripts/release/prepare 0.3.0` - Example: `./scripts/release/prepare 0.3.0`
2. **Auto-Tag Workflow** (safety net) 2. **Auto-Tag Workflow** (safety net)
- Workflow: `.github/workflows/auto-tag.yml` - Workflow: `.github/workflows/auto-tag.yml`
- Triggers on manifest.json changes - Triggers on manifest.json changes
- Automatically creates tag if it doesn't exist - Automatically creates tag if it doesn't exist
- Prevents "forgot to tag" mistakes - Prevents "forgot to tag" mistakes
3. **Local Script** (testing, preview, and updating releases) 3. **Local Script** (testing, preview, and updating releases)
- Script: `./scripts/release/generate-notes [FROM_TAG] [TO_TAG]` - Script: `./scripts/release/generate-notes [FROM_TAG] [TO_TAG]`
- Parses Conventional Commits between tags - Parses Conventional Commits between tags
- Supports multiple backends (auto-detected): - Supports multiple backends (auto-detected):
@ -1006,7 +1023,6 @@ When changes are complete and ready for testing:
``` ```
4. **GitHub UI Button** (manual, PR-based) 4. **GitHub UI Button** (manual, PR-based)
- Uses `.github/release.yml` configuration - Uses `.github/release.yml` configuration
- Click "Generate release notes" when creating release - Click "Generate release notes" when creating release
- Works best with PRs that have labels - Works best with PRs that have labels
@ -1119,7 +1135,6 @@ USE_AI=false ./scripts/release/generate-notes
**Backend Comparison:** **Backend Comparison:**
- **GitHub Copilot CLI** (`copilot`): - **GitHub Copilot CLI** (`copilot`):
- ✅ AI-powered semantic understanding - ✅ AI-powered semantic understanding
- ✅ Smart grouping of related commits into single release notes - ✅ Smart grouping of related commits into single release notes
- ✅ Interprets "Impact:" sections for user-friendly descriptions - ✅ Interprets "Impact:" sections for user-friendly descriptions
@ -1128,7 +1143,6 @@ USE_AI=false ./scripts/release/generate-notes
- ⚠️ Output may vary between runs - ⚠️ Output may vary between runs
- **git-cliff** (template-based): - **git-cliff** (template-based):
- ✅ Fast and consistent - ✅ Fast and consistent
- ✅ 1:1 commit to release note line mapping - ✅ 1:1 commit to release note line mapping
- ✅ Highly configurable via `cliff.toml` - ✅ Highly configurable via `cliff.toml`
@ -1223,6 +1237,7 @@ python -m json.tool custom_components/tibber_prices/translations/de.json > /dev/
This project uses **two complementary tools** with different responsibilities: This project uses **two complementary tools** with different responsibilities:
**Pyright (Type Checker)** - Catches type safety issues: **Pyright (Type Checker)** - Catches type safety issues:
- ✅ Type mismatches (`str` passed where `int` expected) - ✅ Type mismatches (`str` passed where `int` expected)
- ✅ None-safety violations (`Optional[T]` used as `T`) - ✅ None-safety violations (`Optional[T]` used as `T`)
- ✅ Missing/wrong type annotations - ✅ Missing/wrong type annotations
@ -1232,6 +1247,7 @@ This project uses **two complementary tools** with different responsibilities:
- 🔍 **Always run first** - catches design issues early - 🔍 **Always run first** - catches design issues early
**Ruff (Linter + Formatter)** - Enforces code style and patterns: **Ruff (Linter + Formatter)** - Enforces code style and patterns:
- ✅ Code formatting (line length, indentation, quotes) - ✅ Code formatting (line length, indentation, quotes)
- ✅ Import ordering (stdlib → third-party → local) - ✅ Import ordering (stdlib → third-party → local)
- ✅ Unused imports/variables - ✅ Unused imports/variables
@ -1358,11 +1374,13 @@ def get_timestamp() -> str:
### Pyright Configuration ### Pyright Configuration
Project uses `typeCheckingMode = "basic"` in `pyproject.toml`: Project uses `typeCheckingMode = "basic"` in `pyproject.toml`:
- Balanced between strictness and pragmatism - Balanced between strictness and pragmatism
- Catches real bugs without excessive noise - Catches real bugs without excessive noise
- Compatible with Home Assistant's typing style - Compatible with Home Assistant's typing style
**Key settings:** **Key settings:**
```toml ```toml
[tool.pyright] [tool.pyright]
include = ["custom_components/tibber_prices"] include = ["custom_components/tibber_prices"]
@ -1374,6 +1392,7 @@ typeCheckingMode = "basic"
**CRITICAL: When generating code, always aim for Pyright `basic` mode compliance:** **CRITICAL: When generating code, always aim for Pyright `basic` mode compliance:**
✅ **DO:** ✅ **DO:**
- Add type hints to all function signatures (parameters + return types) - Add type hints to all function signatures (parameters + return types)
- Use proper type annotations: `dict[str, Any]`, `list[dict]`, `str | None` - Use proper type annotations: `dict[str, Any]`, `list[dict]`, `str | None`
- Handle Optional types explicitly (None-checks before use) - Handle Optional types explicitly (None-checks before use)
@ -1381,6 +1400,7 @@ typeCheckingMode = "basic"
- Prefer explicit returns over implicit `None` - Prefer explicit returns over implicit `None`
❌ **DON'T:** ❌ **DON'T:**
- Leave functions without return type hints - Leave functions without return type hints
- Ignore potential `None` values in Optional types - Ignore potential `None` values in Optional types
- Use `Any` as escape hatch (only when truly needed) - Use `Any` as escape hatch (only when truly needed)
@ -1399,6 +1419,7 @@ typeCheckingMode = "basic"
3. **Home Assistant API has incomplete typing** 3. **Home Assistant API has incomplete typing**
**ALWAYS include explanation:** **ALWAYS include explanation:**
```python ```python
# ✅ GOOD - Explains why ignore is needed # ✅ GOOD - Explains why ignore is needed
result = tz.localize(dt) # type: ignore[attr-defined] # pytz-specific method result = tz.localize(dt) # type: ignore[attr-defined] # pytz-specific method
@ -1410,6 +1431,7 @@ result = tz.localize(dt) # type: ignore
### Integration with VS Code ### Integration with VS Code
Pylance (VS Code's Python language server) uses the same Pyright engine: Pylance (VS Code's Python language server) uses the same Pyright engine:
- **Red squiggles** = Type errors (must fix) - **Red squiggles** = Type errors (must fix)
- **Yellow squiggles** = Warnings (should fix) - **Yellow squiggles** = Warnings (should fix)
- Hover for details, Cmd/Ctrl+Click for definitions - Hover for details, Cmd/Ctrl+Click for definitions
@ -1539,6 +1561,7 @@ When renaming entity keys or changing sensor value units/semantics across releas
This is a Home Assistant standard to avoid naming conflicts between integrations and ensure clear ownership of classes. This is a Home Assistant standard to avoid naming conflicts between integrations and ensure clear ownership of classes.
**Naming Pattern:** **Naming Pattern:**
```python ```python
# ✅ CORRECT - Integration prefix + semantic purpose # ✅ CORRECT - Integration prefix + semantic purpose
class TibberPricesApiClient: # Integration + semantic role class TibberPricesApiClient: # Integration + semantic role
@ -1558,6 +1581,7 @@ class TibberPricesSensorCalculatorTrend: # Too verbose, import path shows loca
``` ```
**IMPORTANT:** Do NOT include package hierarchy in class names. Python's import system provides the namespace: **IMPORTANT:** Do NOT include package hierarchy in class names. Python's import system provides the namespace:
```python ```python
# The import path IS the full namespace: # The import path IS the full namespace:
from custom_components.tibber_prices.coordinator.price_data_manager import TibberPricesPriceDataManager from custom_components.tibber_prices.coordinator.price_data_manager import TibberPricesPriceDataManager
@ -1569,6 +1593,7 @@ from custom_components.tibber_prices.sensor.calculators.trend import TibberPrice
``` ```
**Home Assistant Core follows this pattern:** **Home Assistant Core follows this pattern:**
- `TibberDataCoordinator` (not `TibberCoordinatorDataCoordinator`) - `TibberDataCoordinator` (not `TibberCoordinatorDataCoordinator`)
- `MetWeatherData` (not `MetCoordinatorWeatherData`) - `MetWeatherData` (not `MetCoordinatorWeatherData`)
- `MetDataUpdateCoordinator` (not `MetCoordinatorDataUpdateCoordinator`) - `MetDataUpdateCoordinator` (not `MetCoordinatorDataUpdateCoordinator`)
@ -1576,6 +1601,7 @@ from custom_components.tibber_prices.sensor.calculators.trend import TibberPrice
Use semantic prefixes that describe the PURPOSE, not the package location. Use semantic prefixes that describe the PURPOSE, not the package location.
**When prefix is required:** **When prefix is required:**
- ✅ All public classes (used across multiple modules) - ✅ All public classes (used across multiple modules)
- ✅ All exception classes - ✅ All exception classes
- ✅ All coordinator classes - ✅ All coordinator classes
@ -1584,6 +1610,7 @@ Use semantic prefixes that describe the PURPOSE, not the package location.
- ✅ All data classes (dataclasses, NamedTuples) used as public APIs - ✅ All data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:** **When prefix can be omitted:**
- 🟡 Private helper classes used only within a single module (prefix class name with `_` underscore) - 🟡 Private helper classes used only within a single module (prefix class name with `_` underscore)
- 🟡 Type aliases and callbacks (e.g., `TimeServiceCallback` is acceptable) - 🟡 Type aliases and callbacks (e.g., `TimeServiceCallback` is acceptable)
- 🟡 Small NamedTuples used only for internal function returns (e.g., within calculators) - 🟡 Small NamedTuples used only for internal function returns (e.g., within calculators)
@ -1593,6 +1620,7 @@ Use semantic prefixes that describe the PURPOSE, not the package location.
**Private Classes (Module-Internal):** **Private Classes (Module-Internal):**
If you create a helper class that is ONLY used within a single module file: If you create a helper class that is ONLY used within a single module file:
```python ```python
# ✅ CORRECT - Private class with underscore prefix # ✅ CORRECT - Private class with underscore prefix
class _InternalHelper: class _InternalHelper:
@ -1604,11 +1632,13 @@ result = _InternalHelper().process()
``` ```
**When to use private classes:** **When to use private classes:**
- ❌ **DON'T** use for code organization alone - if it deserves a class, it's usually public - ❌ **DON'T** use for code organization alone - if it deserves a class, it's usually public
- ✅ **DO** use for internal implementation details (e.g., state machines, internal builders) - ✅ **DO** use for internal implementation details (e.g., state machines, internal builders)
- ✅ **DO** use for temporary refactoring helpers (mark as `# TODO: Make public` if it grows) - ✅ **DO** use for temporary refactoring helpers (mark as `# TODO: Make public` if it grows)
**Example of genuine private class use case:** **Example of genuine private class use case:**
```python ```python
# In coordinator/price_data_manager.py # In coordinator/price_data_manager.py
class _ApiRetryStateMachine: class _ApiRetryStateMachine:
@ -1680,7 +1710,6 @@ We use **Pyright** for static type checking:
**Log Level Strategy:** **Log Level Strategy:**
- **INFO Level** - User-facing results and high-level progress: - **INFO Level** - User-facing results and high-level progress:
- Compact 1-line summaries (no multi-line blocks) - Compact 1-line summaries (no multi-line blocks)
- Important results only (success/failure outcomes) - Important results only (success/failure outcomes)
- No indentation (scannability) - No indentation (scannability)
@ -1688,7 +1717,6 @@ We use **Pyright** for static type checking:
- Example: `"Day 2025-11-11: Success after 1 relaxation phase (2 periods)"` - Example: `"Day 2025-11-11: Success after 1 relaxation phase (2 periods)"`
- **DEBUG Level** - Detailed execution trace: - **DEBUG Level** - Detailed execution trace:
- Full context headers with all relevant configuration - Full context headers with all relevant configuration
- Step-by-step progression through logic - Step-by-step progression through logic
- Hierarchical indentation to show call depth/logic structure - Hierarchical indentation to show call depth/logic structure
@ -1855,7 +1883,6 @@ When writing or updating user-facing documentation (`docs/user/docs/` or `docs/d
Understanding **how** good documentation emerges is as important as knowing what makes it good: Understanding **how** good documentation emerges is as important as knowing what makes it good:
- **Live Understanding vs. Code Analysis** - **Live Understanding vs. Code Analysis**
- ✅ **DO:** Write docs during/after active development - ✅ **DO:** Write docs during/after active development
- When implementing complex logic, document it while the "why" is fresh - When implementing complex logic, document it while the "why" is fresh
- Use real examples from debugging sessions (actual logs, real data) - Use real examples from debugging sessions (actual logs, real data)
@ -1866,7 +1893,6 @@ Understanding **how** good documentation emerges is as important as knowing what
- No user perspective: What's actually confusing? - No user perspective: What's actually confusing?
- **User Feedback Loop** - **User Feedback Loop**
- Key insight: Documentation improves when users question it - Key insight: Documentation improves when users question it
- Pattern: - Pattern:
1. User asks: "Does this still match the code?" 1. User asks: "Does this still match the code?"
@ -1876,19 +1902,16 @@ Understanding **how** good documentation emerges is as important as knowing what
- Why it works: User questions force critical thinking, real confusion points get addressed - Why it works: User questions force critical thinking, real confusion points get addressed
- **Log-Driven Documentation** - **Log-Driven Documentation**
- Observation: When logs explain logic clearly, documentation becomes easier - Observation: When logs explain logic clearly, documentation becomes easier
- Why: Logs show state transitions ("Baseline insufficient → Starting relaxation"), decisions ("Replaced period X with larger Y"), and are already written for humans - Why: Logs show state transitions ("Baseline insufficient → Starting relaxation"), decisions ("Replaced period X with larger Y"), and are already written for humans
- Pattern: If you spent hours making logs clear → use that clarity in documentation too - Pattern: If you spent hours making logs clear → use that clarity in documentation too
- **Concrete Examples > Abstract Descriptions** - **Concrete Examples > Abstract Descriptions**
- ✅ **Good:** "Day 2025-11-11 found 2 periods at flex=12.0% +volatility_any (stopped early, no need to try higher flex)" - ✅ **Good:** "Day 2025-11-11 found 2 periods at flex=12.0% +volatility_any (stopped early, no need to try higher flex)"
- ❌ **Bad:** "The relaxation algorithm uses a configurable threshold multiplier with filter combination strategies" - ❌ **Bad:** "The relaxation algorithm uses a configurable threshold multiplier with filter combination strategies"
- Use real data from debug sessions, show actual attribute values, demonstrate with timeline diagrams - Use real data from debug sessions, show actual attribute values, demonstrate with timeline diagrams
- **Context Accumulation in Long Sessions** - **Context Accumulation in Long Sessions**
- Advantage: AI builds mental model incrementally, sees evolution of logic (not just final state), understands trade-offs - Advantage: AI builds mental model incrementally, sees evolution of logic (not just final state), understands trade-offs
- Disadvantage of short sessions: Cold start every time, missing "why" context, documentation becomes spec-writing - Disadvantage of short sessions: Cold start every time, missing "why" context, documentation becomes spec-writing
- Lesson: Complex documentation benefits from focused, uninterrupted work with accumulated context - Lesson: Complex documentation benefits from focused, uninterrupted work with accumulated context
@ -2214,6 +2237,7 @@ def _get_sensor_attributes(self) -> dict | None:
``` ```
**Why direct method over Callable pattern?** **Why direct method over Callable pattern?**
- **Simpler**: No lambda/Callable indirection, clearer stack traces - **Simpler**: No lambda/Callable indirection, clearer stack traces
- **More HA-standard**: Most Core integrations use direct methods - **More HA-standard**: Most Core integrations use direct methods
- **Better performance**: ~2x faster (~0.1-0.5μs vs 0.2-0.8μs per call) - **Better performance**: ~2x faster (~0.1-0.5μs vs 0.2-0.8μs per call)
@ -2225,6 +2249,7 @@ def _get_sensor_attributes(self) -> dict | None:
Both platforms now use **identical signatures and patterns** (unified Nov 2025): Both platforms now use **identical signatures and patterns** (unified Nov 2025):
**Sensor Platform (`sensor/attributes.py`):** **Sensor Platform (`sensor/attributes.py`):**
```python ```python
def build_extra_state_attributes( def build_extra_state_attributes(
entity_key: str, entity_key: str,
@ -2243,6 +2268,7 @@ def build_extra_state_attributes(
``` ```
**Binary Sensor Platform (`binary_sensor/attributes.py`):** **Binary Sensor Platform (`binary_sensor/attributes.py`):**
```python ```python
async def build_async_extra_state_attributes( async def build_async_extra_state_attributes(
entity_key: str, entity_key: str,
@ -2262,6 +2288,7 @@ def build_sync_extra_state_attributes(...) -> dict | None:
``` ```
**Key Points:** **Key Points:**
- **Architectural consistency**: Both platforms use direct method pattern (not Callable) - **Architectural consistency**: Both platforms use direct method pattern (not Callable)
- **Naming consistency**: Both use `_get_sensor_attributes()` method name - **Naming consistency**: Both use `_get_sensor_attributes()` method name
- **Parameter consistency**: Both builders accept `sensor_attrs` parameter - **Parameter consistency**: Both builders accept `sensor_attrs` parameter
@ -2395,7 +2422,6 @@ If the answer to any is "no", make the name more explicit.
After the sensor.py refactoring (completed Nov 2025), sensors are organized by **calculation method** rather than feature type. Follow these steps: After the sensor.py refactoring (completed Nov 2025), sensors are organized by **calculation method** rather than feature type. Follow these steps:
1. **Determine calculation pattern** - Choose which group your sensor belongs to: 1. **Determine calculation pattern** - Choose which group your sensor belongs to:
- **Interval-based**: Uses time offset from current interval (e.g., current/next/previous) - **Interval-based**: Uses time offset from current interval (e.g., current/next/previous)
- **Rolling hour**: Aggregates 5-interval window (2 before + center + 2 after) - **Rolling hour**: Aggregates 5-interval window (2 before + center + 2 after)
- **Daily statistics**: Min/max/avg within calendar day boundaries - **Daily statistics**: Min/max/avg within calendar day boundaries
@ -2407,7 +2433,6 @@ After the sensor.py refactoring (completed Nov 2025), sensors are organized by *
**IMPORTANT — After adding/renaming entities**: Run `./scripts/docs/generate-sensor-reference` to regenerate the multi-language sensor reference table. The `scripts/check` and CI will fail if the reference is stale. **IMPORTANT — After adding/renaming entities**: Run `./scripts/docs/generate-sensor-reference` to regenerate the multi-language sensor reference table. The `scripts/check` and CI will fail if the reference is stale.
2. **Add entity description** to appropriate sensor group in `sensor/definitions.py`: 2. **Add entity description** to appropriate sensor group in `sensor/definitions.py`:
- `INTERVAL_PRICE_SENSORS`, `INTERVAL_LEVEL_SENSORS`, or `INTERVAL_RATING_SENSORS` - `INTERVAL_PRICE_SENSORS`, `INTERVAL_LEVEL_SENSORS`, or `INTERVAL_RATING_SENSORS`
- `ROLLING_HOUR_PRICE_SENSORS`, `ROLLING_HOUR_LEVEL_SENSORS`, or `ROLLING_HOUR_RATING_SENSORS` - `ROLLING_HOUR_PRICE_SENSORS`, `ROLLING_HOUR_LEVEL_SENSORS`, or `ROLLING_HOUR_RATING_SENSORS`
- `DAILY_STAT_SENSORS` - `DAILY_STAT_SENSORS`
@ -2417,7 +2442,6 @@ After the sensor.py refactoring (completed Nov 2025), sensors are organized by *
- `DIAGNOSTIC_SENSORS` - `DIAGNOSTIC_SENSORS`
3. **Add handler mapping** in `sensor/core.py` → `_get_value_getter()` method: 3. **Add handler mapping** in `sensor/core.py` → `_get_value_getter()` method:
- For interval-based: Use `_get_interval_value(interval_offset, value_type)` - For interval-based: Use `_get_interval_value(interval_offset, value_type)`
- For rolling hour: Use `_get_rolling_hour_value(hour_offset, value_type)` - For rolling hour: Use `_get_rolling_hour_value(hour_offset, value_type)`
- For daily stats: Use `_get_daily_stat_value(day, stat_func)` - For daily stats: Use `_get_daily_stat_value(day, stat_func)`
@ -2437,19 +2461,16 @@ After the sensor.py refactoring (completed Nov 2025), sensors are organized by *
The refactoring consolidated duplicate logic into unified methods in `sensor/core.py`: The refactoring consolidated duplicate logic into unified methods in `sensor/core.py`:
- **`_get_interval_value(interval_offset, value_type, in_euro=False)`** - **`_get_interval_value(interval_offset, value_type, in_euro=False)`**
- Replaces: `_get_interval_price_value()`, `_get_interval_level_value()`, `_get_interval_rating_value()` - Replaces: `_get_interval_price_value()`, `_get_interval_level_value()`, `_get_interval_rating_value()`
- Handles: All interval-based sensors (current/next/previous) - Handles: All interval-based sensors (current/next/previous)
- Returns: Price (float), level (str), or rating (str) based on value_type - Returns: Price (float), level (str), or rating (str) based on value_type
- **`_get_rolling_hour_value(hour_offset, value_type)`** - **`_get_rolling_hour_value(hour_offset, value_type)`**
- Replaces: `_get_rolling_hour_average_value()`, `_get_rolling_hour_level_value()`, `_get_rolling_hour_rating_value()` - Replaces: `_get_rolling_hour_average_value()`, `_get_rolling_hour_level_value()`, `_get_rolling_hour_rating_value()`
- Handles: All 5-interval rolling hour windows - Handles: All 5-interval rolling hour windows
- Returns: Aggregated value (average price, aggregated level/rating) - Returns: Aggregated value (average price, aggregated level/rating)
- **`_get_daily_stat_value(day, stat_func)`** - **`_get_daily_stat_value(day, stat_func)`**
- Replaces: `_get_statistics_value()` (calendar day portion) - Replaces: `_get_statistics_value()` (calendar day portion)
- Handles: Min/max/avg for calendar days (today/tomorrow) - Handles: Min/max/avg for calendar days (today/tomorrow)
- Returns: Price in subunit currency units (cents/øre) - Returns: Price in subunit currency units (cents/øre)
@ -2481,13 +2502,11 @@ Edit `utils/price.py` or `utils/average.py`. These are stateless pure functions
The config flow is split into three separate flow handlers: The config flow is split into three separate flow handlers:
1. **User Flow** (`config_flow/user_flow.py`) - Initial setup and reauth 1. **User Flow** (`config_flow/user_flow.py`) - Initial setup and reauth
- `async_step_user()` - API token input - `async_step_user()` - API token input
- `async_step_select_home()` - Home selection - `async_step_select_home()` - Home selection
- `async_step_reauth()` / `async_step_reauth_confirm()` - Reauth flow - `async_step_reauth()` / `async_step_reauth_confirm()` - Reauth flow
2. **Subentry Flow** (`config_flow/subentry_flow.py`) - Add additional homes 2. **Subentry Flow** (`config_flow/subentry_flow.py`) - Add additional homes
- `async_step_user()` - Select from available homes - `async_step_user()` - Select from available homes
- `async_step_init()` - Subentry options - `async_step_init()` - Subentry options
@ -2569,6 +2588,7 @@ Only after consulting the official HA docs did we discover the correct pattern:
- ❌ **Using standard library datetime**: Use `dt_util.now()` instead of `datetime.now()`. - ❌ **Using standard library datetime**: Use `dt_util.now()` instead of `datetime.now()`.
**See code for correct patterns:** **See code for correct patterns:**
- Async operations: `api/client.py` - Async operations: `api/client.py`
- Exception handling: `coordinator/core.py` - Exception handling: `coordinator/core.py`
- Translations: `sensor/definitions.py` (translation_key usage) - Translations: `sensor/definitions.py` (translation_key usage)
@ -2580,10 +2600,12 @@ Only after consulting the official HA docs did we discover the correct pattern:
**CRITICAL: Always exclude non-essential attributes from Recorder to prevent database bloat.** **CRITICAL: Always exclude non-essential attributes from Recorder to prevent database bloat.**
**Implementation:** **Implementation:**
- Use `_unrecorded_attributes = frozenset({...})` as **class attribute** in entity classes - Use `_unrecorded_attributes = frozenset({...})` as **class attribute** in entity classes
- See `sensor/core.py` and `binary_sensor/core.py` for current implementation - See `sensor/core.py` and `binary_sensor/core.py` for current implementation
**What to exclude:** **What to exclude:**
1. **Descriptions/help text** - `description`, `usage_tips` (static, large) 1. **Descriptions/help text** - `description`, `usage_tips` (static, large)
2. **Large nested structures** - `periods`, `data`, `*_attributes` dicts (>1KB) 2. **Large nested structures** - `periods`, `data`, `*_attributes` dicts (>1KB)
3. **Frequently changing diagnostics** - `icon_color`, `cache_age`, status strings 3. **Frequently changing diagnostics** - `icon_color`, `cache_age`, status strings
@ -2592,14 +2614,15 @@ Only after consulting the official HA docs did we discover the correct pattern:
6. **Redundant/derived data** - `price_spread`, `diff_%` (calculable from other attrs) 6. **Redundant/derived data** - `price_spread`, `diff_%` (calculable from other attrs)
**What to keep:** **What to keep:**
- `timestamp` (always), all price values, `cache_age_minutes`, `updates_today` - `timestamp` (always), all price values, `cache_age_minutes`, `updates_today`
- Period timing (`start`, `end`, `duration_minutes`), price statistics - Period timing (`start`, `end`, `duration_minutes`), price statistics
- Boolean status flags, `relaxation_active` - Boolean status flags, `relaxation_active`
**When adding new attributes:** **When adding new attributes:**
- Will this be useful in history 1 week from now? No → Exclude - Will this be useful in history 1 week from now? No → Exclude
- Can this be calculated from other attributes? Yes → Exclude - Can this be calculated from other attributes? Yes → Exclude
- Is this >100 bytes and not essential? Yes → Exclude - Is this >100 bytes and not essential? Yes → Exclude
**See:** `docs/developer/docs/recorder-optimization.md` for detailed categories and impact analysis **See:** `docs/developer/docs/recorder-optimization.md` for detailed categories and impact analysis

6
CONTRIBUTING.md
View file

@ -73,14 +73,17 @@ Impact: <user-visible effects>
**Types:** `feat`, `fix`, `docs`, `refactor`, `chore`, `test` **Types:** `feat`, `fix`, `docs`, `refactor`, `chore`, `test`
For full commit-message rules (including release-note skip trailers for internal/unreleased fixes), see: For full commit-message rules (including release-note skip trailers for internal/unreleased fixes), see:
- `.github/instructions/commit-messages.instructions.md` - `.github/instructions/commit-messages.instructions.md`
Important trailers for commits that should NOT appear in release notes: Important trailers for commits that should NOT appear in release notes:
- `Release-Notes: skip` - `Release-Notes: skip`
- `User-Impact: none` - `User-Impact: none`
- `Released-Bug: no` - `Released-Bug: no`
**Example:** **Example:**
```bash ```bash
git commit -m "feat(sensors): add daily average price sensor git commit -m "feat(sensors): add daily average price sensor
@ -119,6 +122,7 @@ See `.github/instructions/commit-messages.instructions.md` for detailed commit-m
- **Python version**: 3.13+ - **Python version**: 3.13+
Always run before committing: Always run before committing:
```bash ```bash
./scripts/lint ./scripts/lint
``` ```
@ -144,6 +148,7 @@ Documentation is organized in two Docusaurus sites:
- Navigation via `docs/developer/sidebars.ts` - Navigation via `docs/developer/sidebars.ts`
**When adding new documentation:** **When adding new documentation:**
1. Place file in appropriate `docs/*/docs/` directory 1. Place file in appropriate `docs/*/docs/` directory
2. Add to corresponding `sidebars.ts` for navigation 2. Add to corresponding `sidebars.ts` for navigation
3. Update translations when changing `translations/en.json` (update ALL language files) 3. Update translations when changing `translations/en.json` (update ALL language files)
@ -153,6 +158,7 @@ Documentation is organized in two Docusaurus sites:
Report bugs via [GitHub Issues](../../issues/new/choose). Report bugs via [GitHub Issues](../../issues/new/choose).
**Great bug reports include:** **Great bug reports include:**
- Quick summary and background - Quick summary and background
- Steps to reproduce (be specific!) - Steps to reproduce (be specific!)
- Expected vs. actual behavior - Expected vs. actual behavior

2
README.md
View file

@ -104,7 +104,7 @@ The integration provides **100+ entities** across sensors, binary sensors, switc
<img src="https://raw.githubusercontent.com/jpawlowski/hass.tibber_prices/main/docs/user/static/img/entities-overview.jpg" width="400" alt="Entity list showing dynamic icons for different price states"> <img src="https://raw.githubusercontent.com/jpawlowski/hass.tibber_prices/main/docs/user/static/img/entities-overview.jpg" width="400" alt="Entity list showing dynamic icons for different price states">
| Category | Highlights | Count | | Category | Highlights | Count |
|----------|-----------|-------| | ----------------------- | ----------------------------------------------------------------------------- | ----- |
| **💰 Prices** | Current, next & previous interval price + rolling hour averages | 6+ | | **💰 Prices** | Current, next & previous interval price + rolling hour averages | 6+ |
| **📊 Statistics** | Daily min/max/avg for today & tomorrow, 24h trailing & leading windows | 12+ | | **📊 Statistics** | Daily min/max/avg for today & tomorrow, 24h trailing & leading windows | 12+ |
| **🔮 Forecasts** | Next 1h–12h average prices, price outlook & trajectory sensors | 20+ | | **🔮 Forecasts** | Next 1h–12h average prices, price outlook & trajectory sensors | 20+ |

31
docs/developer/docs/api-reference.md
View file

@ -56,7 +56,7 @@ query {
Fetches quarter-hourly prices: Fetches quarter-hourly prices:
```graphql ```graphql
query($homeId: ID!) { query ($homeId: ID!) {
viewer { viewer {
home(id: $homeId) { home(id: $homeId) {
currentSubscription { currentSubscription {
@ -76,6 +76,7 @@ query($homeId: ID!) {
``` ```
**Parameters:** **Parameters:**
- `homeId`: Tibber home identifier - `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY` - `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data) - `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits ## Rate Limits
Tibber API rate limits (as of 2024): Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token - **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute - **Burst limit:** 100 requests per minute
Integration stays well below these limits: Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day - Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day - User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home - **Total:** ~100 requests/day per home
@ -106,6 +109,7 @@ Integration stays well below these limits:
``` ```
**Fields:** **Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR) - `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone - `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE) - `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -119,6 +123,7 @@ Integration stays well below these limits:
``` ```
Supported currencies: Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh - `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh - `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh - `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses ### Common Error Responses
**Invalid Token:** **Invalid Token:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Unauthorized", "message": "Unauthorized",
"extensions": { "extensions": {
"code": "UNAUTHENTICATED" "code": "UNAUTHENTICATED"
} }
}] }
]
} }
``` ```
**Rate Limit Exceeded:** **Rate Limit Exceeded:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Too Many Requests", "message": "Too Many Requests",
"extensions": { "extensions": {
"code": "RATE_LIMIT_EXCEEDED" "code": "RATE_LIMIT_EXCEEDED"
} }
}] }
]
} }
``` ```
**Home Not Found:** **Home Not Found:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Home not found", "message": "Home not found",
"extensions": { "extensions": {
"code": "NOT_FOUND" "code": "NOT_FOUND"
} }
}] }
]
} }
``` ```
Integration handles these with: Integration handles these with:
- Exponential backoff retry (3 attempts) - Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors - ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures - ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation ## Data Transformation
Raw API data is enriched with: Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals - **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals - **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average - **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
--- ---
💡 **External Resources:** 💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview) - [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer) - [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token) - [Get API Token](https://developer.tibber.com/settings/access-token)

17
docs/developer/docs/architecture.md
View file

@ -147,7 +147,7 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance: The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory | | Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------| | ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB | | **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB | | **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB | | **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
@ -196,7 +196,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components ### Core Components
| Component | File | Responsibility | | Component | File | Responsibility |
|-----------|------|----------------| | --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling | | **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance | | **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) | | **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
@ -210,7 +210,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025): The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility | | Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------| | ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators | | **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) | | **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) | | **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
@ -219,6 +219,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities | | **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`): **Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access - `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous) - `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows - `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata - `metadata.py` - Home/metering metadata
**Benefits:** **Benefits:**
- 58% reduction in core.py (2,170 → 909 lines) - 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation) - Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator - Independent testability for each calculator
@ -238,7 +240,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities ### Helper Utilities
| Utility | File | Purpose | | Utility | File | Purpose |
|---------|------|---------| | ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation | | **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations | | **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic | | **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025): Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`): **Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals - `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows - `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg - `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics - `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`): **Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods - Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.) - Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`): **Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access - Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc. - Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching) - Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`): **Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation - Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes - Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()` - Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:** **Benefits:**
- Minimal code duplication across 80+ sensors - Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation) - Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing - Easy to extend: Add sensor → choose pattern → add to routing
@ -334,7 +341,7 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization ### CPU Optimization
| Optimization | Location | Savings | | Optimization | Location | Savings |
|--------------|----------|---------| | ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks | | Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation | | Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations | | Lazy logging | Throughout | ~15% on log-heavy operations |

42
docs/developer/docs/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts. **Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:** **What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total) - **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query - **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation - **Timestamps**: Last update times for validation
**Lifetime:** **Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time) - **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily) - **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage - **Survives**: HA restarts via persistent Storage
@ -36,6 +38,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:** **Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator): 1. **Midnight turnover** (Timer #2 in coordinator):
```python ```python
# coordinator/day_transitions.py # coordinator/day_transitions.py
def _handle_midnight_turnover() -> None: def _handle_midnight_turnover() -> None:
@ -45,6 +48,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
``` ```
2. **Cache validation on load**: 2. **Cache validation on load**:
```python ```python
# coordinator/cache.py # coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool: def is_cache_valid(cache_data: CacheData) -> bool:
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc. **Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:** **What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names - **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions - **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:** **Lifetime:**
- **Forever** (until HA restart) - **Forever** (until HA restart)
- No invalidation during runtime - No invalidation during runtime
**When populated:** **When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py` - At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once - Lazy loading: If translation missing, attempts file load once
**Access pattern:** **Access pattern:**
```python ```python
# Non-blocking synchronous access from cached data # Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en") description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:** **What is cached:**
### DataTransformer Config Cache ### DataTransformer Config Cache
```python ```python
{ {
"thresholds": {"low": 15, "high": 35}, "thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
### PeriodCalculator Config Cache ### PeriodCalculator Config Cache
```python ```python
{ {
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60}, "best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,10 +128,12 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Lifetime:** **Lifetime:**
- Until `invalidate_config_cache()` is called - Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle - Built once on first use per coordinator update cycle
**Invalidation trigger:** **Invalidation trigger:**
- **Options change** (user reconfigures integration): - **Options change** (user reconfigures integration):
```python ```python
# coordinator/core.py # coordinator/core.py
@ -132,6 +144,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Performance impact:** **Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs - **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs - **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update) - **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed. **Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:** **What is cached:**
```python ```python
{ {
"best_price": { "best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Cache key:** Hash of relevant inputs **Cache key:** Hash of relevant inputs
```python ```python
hash_data = ( hash_data = (
today_signature, # (startsAt, rating_level) for each interval today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
``` ```
**Lifetime:** **Lifetime:**
- Until price data changes (today's intervals modified) - Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters) - Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data) - Recalculated at midnight (new today data)
@ -179,6 +195,7 @@ hash_data = (
**Invalidation triggers:** **Invalidation triggers:**
1. **Config change** (explicit): 1. **Config change** (explicit):
```python ```python
def invalidate_config_cache() -> None: def invalidate_config_cache() -> None:
self._cached_periods = None self._cached_periods = None
@ -193,10 +210,12 @@ hash_data = (
``` ```
**Cache hit rate:** **Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged) - **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00) - **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:** **Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts) - **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup) - **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache) - **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy **Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:** **What is cached:**
```python ```python
{ {
"timestamp": ..., "timestamp": ...,
@ -224,6 +244,7 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks. **Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:** **Current behavior:**
- Caches **only enriched price data** (price + statistics) - Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache) - **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when: - Invalidated when:
@ -232,6 +253,7 @@ hash_data = (
- New update cycle begins - New update cycle begins
**Architecture:** **Architecture:**
- DataTransformer: Handles price enrichment only - DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache) - PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches - Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow ## Cache Invalidation Flow
### User Changes Options (Config Flow) ### User Changes Options (Config Flow)
``` ```
User saves options User saves options
↓ ↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
``` ```
### Midnight Turnover (Day Transition) ### Midnight Turnover (Day Transition)
``` ```
Timer #2 fires at 00:00 Timer #2 fires at 00:00
↓ ↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
``` ```
### Tomorrow Data Arrives (~13:00) ### Tomorrow Data Arrives (~13:00)
``` ```
Coordinator update cycle Coordinator update cycle
↓ ↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
``` ```
**No cache invalidation cascades:** **No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update) - Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch) - Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change) - Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load) - Translation cache is **never invalidated** (read-only after load)
**Thread safety:** **Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop) - All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model) - No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics ## Performance Characteristics
### Typical Operation (No Changes) ### Typical Operation (No Changes)
``` ```
Coordinator Update (every 15 min) Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid) ├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
``` ```
### After Midnight Turnover ### After Midnight Turnover
``` ```
Coordinator Update (00:00) Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day) ├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
``` ```
### After Config Change ### After Config Change
``` ```
Options Update Options Update
├─> Cache invalidation: `<`1ms ├─> Cache invalidation: `<`1ms
@ -382,7 +412,7 @@ Options Update
## Summary Table ## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose | | Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------| | ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls | | **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O | | **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups | | **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
@ -392,12 +422,14 @@ Options Update
**Total memory overhead:** ~116KB per coordinator instance (main + subentries) **Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:** **Benefits:**
- 97% reduction in API calls (from every 15min to once per day) - 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation) - 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check) - 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup) - Zero file I/O during runtime (translations cached at startup)
**Trade-offs:** **Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems) - Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented) - Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues - Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues ## Debugging Cache Issues
### Symptom: Stale data after config change ### Symptom: Stale data after config change
**Check:** **Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log) 1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed? 2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger? 3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init. **Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating ### Symptom: Period calculation not updating
**Check:** **Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()` 1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash` 2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs 3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs. **Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today ### Symptom: Yesterday's prices shown as today
**Check:** **Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py` 1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2) 2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs 3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration. **Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations ### Symptom: Missing translations
**Check:** **Check:**
1. `async_load_translations()` called at startup? 1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`? 2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys 3. Cache population: `_TRANSLATIONS_CACHE` keys

3
docs/developer/docs/coding-guidelines.md
View file

@ -41,12 +41,14 @@ class TimeService:
``` ```
**When prefix is required:** **When prefix is required:**
- Public classes used across multiple modules - Public classes used across multiple modules
- All exception classes - All exception classes
- All coordinator and entity classes - All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs - Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:** **When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore) - Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`) - Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns - Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:** **Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring: Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md` 1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames 2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module 3. Test thoroughly after each module

17
docs/developer/docs/contributing.md
View file

@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
``` ```
**Branch naming:** **Branch naming:**
- `feature/` - New features - `feature/` - New features
- `fix/` - Bug fixes - `fix/` - Bug fixes
- `docs/` - Documentation only - `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md). Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:** **Run checks frequently:**
```bash ```bash
./scripts/type-check # Pyright type checking ./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix) ./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
``` ```
Run your test: Run your test:
```bash ```bash
./scripts/test tests/test_your_feature.py -v ./scripts/test tests/test_your_feature.py -v
``` ```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
``` ```
**Commit types:** **Commit types:**
- `feat:` - New feature - `feat:` - New feature
- `fix:` - Bug fix - `fix:` - Bug fix
- `docs:` - Documentation - `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance - `chore:` - Maintenance
**Add scope when relevant:** **Add scope when relevant:**
- `feat(sensors):` - Sensor platform - `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator - `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation - `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max) Title: Short, descriptive (50 chars max)
Description should include: Description should include:
```markdown ```markdown
## What ## What
Brief description of changes Brief description of changes
## Why ## Why
Problem being solved or feature rationale Problem being solved or feature rationale
## How ## How
Implementation approach Implementation approach
## Testing ## Testing
- [ ] Manual testing in Home Assistant - [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated - [ ] Unit tests added/updated
- [ ] Type checking passes - [ ] Type checking passes
- [ ] Linting passes - [ ] Linting passes
## Breaking Changes ## Breaking Changes
(If any - describe migration path) (If any - describe migration path)
## Related Issues ## Related Issues
Closes #123 Closes #123
``` ```
### PR Checklist ### PR Checklist
Before submitting: Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md) - [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`) - [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`) - [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For ### What Reviewers Look For
✅ **Good:** ✅ **Good:**
- Clear, self-explanatory code - Clear, self-explanatory code
- Appropriate comments for complex logic - Appropriate comments for complex logic
- Tests covering edge cases - Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns - Follows existing patterns
❌ **Avoid:** ❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones - Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes - Mixing unrelated changes
- Missing tests for new features - Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On ## Finding Issues to Work On
Good first issues are labeled: Good first issues are labeled:
- `good first issue` - Beginner-friendly - `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions - `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements - `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
--- ---
💡 **Related:** 💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup - [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide - [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests - [Testing](testing.md) - Writing tests

33
docs/developer/docs/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical? ## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to: Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable - **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases - **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks - **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅ #### 1.1 Listener Cleanup ✅
**What is tested:** **What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`) - Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`) - Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`) - Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners - Binary sensor cleanup removes ALL registered listeners
**Why critical:** **Why critical:**
- Each registered listener holds references to Entity + Coordinator - Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak - Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed - With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()` - `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()` - `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()` - `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅ #### 1.2 Timer Cleanup ✅
**What is tested:** **What is tested:**
- Quarter-hour timer is cancelled and reference cleared - Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared - Minute timer is cancelled and reference cleared
- Both timers are cancelled together - Both timers are cancelled together
- Cleanup works even when timers are `None` - Cleanup works even when timers are `None`
**Why critical:** **Why critical:**
- Uncancelled timers continue running after integration unload - Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks - HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates - Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()` - `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅ #### 1.3 Config Entry Cleanup ✅
**What is tested:** **What is tested:**
- Options update listener is registered via `async_on_unload()` - Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()` - Cleanup function is correctly passed to `async_on_unload()`
**Why critical:** **Why critical:**
- `entry.add_update_listener()` registers permanent callback - `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates - Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))` - Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:** **Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration) - `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()` - `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation #### 2.1 Config Cache Invalidation
**What is tested:** **What is tested:**
- DataTransformer config cache is invalidated on options change - DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated - PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update - Trend calculator cache is cleared on coordinator update
**Why critical:** **Why critical:**
- Stale config → Sensors use old user settings - Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods - Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis - Stale trend cache → Outdated trend analysis
**Code Locations:** **Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()` - `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()` - `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()` - `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal #### 3.1 Persistent Storage Removal
**What is tested:** **What is tested:**
- Storage file is deleted on config entry removal - Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss) - Cache is saved on shutdown (no data loss)
**Why critical:** **Why critical:**
- Without storage removal: Old files remain after uninstallation - Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart - Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}` - Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:** **Code Locations:**
- `__init__.py` → `async_remove_entry()` - `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py` **File:** `tests/test_timer_scheduling.py`
**What is tested:** **What is tested:**
- Quarter-hour timer is registered with correct parameters - Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters - Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer) - Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly - Midnight turnover detection works correctly
**Why critical:** **Why critical:**
- Wrong timer parameters → Entities update at wrong times - Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem - Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py` **File:** `tests/test_sensor_timer_assignment.py`
**What is tested:** **What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys - All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys - All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap) - Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked - Sensor and binary sensor platforms are checked
**Why critical:** **Why critical:**
- Wrong timer assignment → Sensors update at wrong times - Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem - Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management ### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks **Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds) - `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms) - `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:** **Why no tests needed:**
- No long-running tasks (all < 2 seconds) - No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically - HA's event loop handles short tasks automatically
- Task exceptions are already logged - Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup ### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented **Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session) - `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created - No new sessions are created
- HA manages session lifecycle automatically - HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory ### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache **Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total - Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading - Module-level cache without re-loading
- Practically no memory issue - Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity ### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop` **Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days) - Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults - Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors - 80+ sensors access `coordinator.data` without errors
**Structure:** **Structure:**
```python ```python
coordinator.data = { coordinator.data = {
"user_data": {...}, "user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory ### 10. Service Response Memory
**Current Status:** HA's response lifecycle **Current Status:** HA's response lifecycle
- HA automatically frees service responses after return - HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call - ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code - No response accumulation in integration code
@ -208,7 +235,7 @@ coordinator.data = {
### ✅ Implemented Tests (41 total) ### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage | | Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------| | ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% | | Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% | | Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% | | Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
@ -222,7 +249,7 @@ coordinator.data = {
### 📋 Analyzed but Not Implemented (Nice-to-Have) ### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale | | Category | Status | Rationale |
|----------|--------|-----------| | ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) | | Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) | | API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) | | Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
@ -230,6 +257,7 @@ coordinator.data = {
| Service Response Memory | 📋 | HA automatically frees service responses | | Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:** **Legend:**
- ✅ = Fully tested or pattern verified correct - ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues) - 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested ### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests: All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks) - ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks) - ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners) - ✅ Config entry cleanup works (no dangling listeners)

18
docs/developer/docs/debugging.md
View file

@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages ### Key Log Messages
**Coordinator Updates:** **Coordinator Updates:**
``` ```
[custom_components.tibber_prices.coordinator] Successfully fetched price data [custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data [custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
``` ```
**Period Calculation:** **Period Calculation:**
``` ```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0% [custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods [custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
``` ```
**API Errors:** **API Errors:**
``` ```
[custom_components.tibber_prices.api] API request failed: Unauthorized [custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s [custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -67,6 +70,7 @@ Restart Home Assistant to apply.
### Set Breakpoints ### Set Breakpoints
**Coordinator update:** **Coordinator update:**
```python ```python
# coordinator/core.py # coordinator/core.py
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
``` ```
**Period calculation:** **Period calculation:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]: def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
``` ```
**Flags:** **Flags:**
- `-v` - Verbose output - `-v` - Verbose output
- `-s` - Show print statements - `-s` - Show print statements
- `-k pattern` - Run tests matching pattern - `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns ### Useful Test Patterns
**Print coordinator data:** **Print coordinator data:**
```python ```python
def test_something(coordinator): def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}") print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
``` ```
**Inspect period attributes:** **Inspect period attributes:**
```python ```python
def test_periods(hass, coordinator): def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', []) periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading ### Integration Not Loading
**Check:** **Check:**
```bash ```bash
grep "tibber_prices" config/home-assistant.log grep "tibber_prices" config/home-assistant.log
``` ```
**Common causes:** **Common causes:**
- Syntax error in Python code → Check logs for traceback - Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync` - Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*` - Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating ### Sensors Not Updating
**Check coordinator state:** **Check coordinator state:**
```python ```python
# In Developer Tools > Template # In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }} {{ states.sensor.tibber_home_current_interval_price.last_updated }}
``` ```
**Debug in code:** **Debug in code:**
```python ```python
# Add logging in sensor/core.py # Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s", _LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong ### Period Calculation Wrong
**Enable detailed period logs:** **Enable detailed period logs:**
```python ```python
# coordinator/period_handlers/period_building.py # coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s", _LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
``` ```
**Check filter statistics:** **Check filter statistics:**
``` ```
[period_building] Flex filter blocked: 45 intervals [period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals [period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy ### Remote Debugging with debugpy
Add to coordinator code: Add to coordinator code:
```python ```python
import debugpy import debugpy
debugpy.listen(5678) debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL ### IPython REPL
Install in container: Install in container:
```bash ```bash
uv pip install ipython uv pip install ipython
``` ```
Add breakpoint: Add breakpoint:
```python ```python
from IPython import embed from IPython import embed
embed() # Drops into interactive shell embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
--- ---
💡 **Related:** 💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests - [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment - [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure - [Architecture](architecture.md) - Code structure

1
docs/developer/docs/intro.md
View file

@ -168,6 +168,7 @@ Documentation is organized in two Docusaurus sites:
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory) - **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:** **Best practices:**
- Use clear examples and code snippets - Use clear examples and code snippets
- Keep docs up-to-date with code changes - Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation - Add new pages to appropriate `sidebars.ts` for navigation

17
docs/developer/docs/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals ## Performance Goals
Target metrics: Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms) - **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor - **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms) - **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching ### Caching
**1. Persistent Cache** (API data): **1. Persistent Cache** (API data):
```python ```python
# Already implemented in coordinator/cache.py # Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY) store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
``` ```
**2. Translation Cache** (in-memory): **2. Translation Cache** (in-memory):
```python ```python
# Already implemented in const.py # Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {} _TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
``` ```
**3. Config Cache** (invalidated on options change): **3. Config Cache** (invalidated on options change):
```python ```python
class DataTransformer: class DataTransformer:
def __init__(self): def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading ### Lazy Loading
**Load data only when needed:** **Load data only when needed:**
```python ```python
@property @property
def extra_state_attributes(self) -> dict | None: def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations ### Bulk Operations
**Process multiple items at once:** **Process multiple items at once:**
```python ```python
# ❌ Slow - loop with individual operations # ❌ Slow - loop with individual operations
for interval in intervals: for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices ### Async Best Practices
**1. Concurrent API calls:** **1. Concurrent API calls:**
```python ```python
# ❌ Sequential (slow) # ❌ Sequential (slow)
user_data = await fetch_user_data() user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
``` ```
**2. Don't block event loop:** **2. Don't block event loop:**
```python ```python
# ❌ Blocking # ❌ Blocking
result = heavy_computation() # Blocks for seconds result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks ### Avoid Memory Leaks
**1. Clear references:** **1. Clear references:**
```python ```python
class Coordinator: class Coordinator:
async def async_shutdown(self): async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
``` ```
**2. Use weak references for callbacks:** **2. Use weak references for callbacks:**
```python ```python
import weakref import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures ### Efficient Data Structures
**Use appropriate types:** **Use appropriate types:**
```python ```python
# ❌ List for lookups (O(n)) # ❌ List for lookups (O(n))
if timestamp in timestamp_list: if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls ### Minimize API Calls
**Already implemented:** **Already implemented:**
- Cache valid until midnight - Cache valid until midnight
- User data cached for 24h - User data cached for 24h
- Only poll when tomorrow data expected - Only poll when tomorrow data expected
**Monitor API usage:** **Monitor API usage:**
```python ```python
_LOGGER.debug("API call: %s (cache_age=%s)", _LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age) endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates ### Smart Updates
**Only update when needed:** **Only update when needed:**
```python ```python
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
"""Fetch data from API.""" """Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection ### State Class Selection
**Affects long-term statistics storage:** **Affects long-term statistics storage:**
```python ```python
# ❌ MEASUREMENT for prices (stores every change) # ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size ### Attribute Size
**Keep attributes minimal:** **Keep attributes minimal:**
```python ```python
# ❌ Large nested structures (KB per update) # ❌ Large nested structures (KB per update)
attributes = { attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
--- ---
💡 **Related:** 💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers - [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design - [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools - [Debugging](debugging.md) - Profiling tools

117
docs/developer/docs/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic. **Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:** **Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point - `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering - `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy - `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max. **Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be within flex% ABOVE daily minimum # Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex) in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Flex: 15% - Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15) - Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better. **Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be at least min_distance% BELOW daily average # Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100)) meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Min Distance: 5% - Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95) - Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone - Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:** **Implementation:**
```python ```python
# Sensor classification uses user config # Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10) user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,21 +113,25 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5% #### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:** **Given:**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh - Daily Max: 20 ct/kWh
**Flex Filter (50%):** **Flex Filter (50%):**
``` ```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
``` ```
**Min Distance Filter (5%):** **Min Distance Filter (5%):**
``` ```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
``` ```
**Conflict:** **Conflict:**
- Interval at 14.8 ct/kWh: - Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS) - ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL) - ❌ Distance: 14.8 > 14.25 (FAIL)
@ -132,11 +142,13 @@ Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
### Mathematical Analysis ### Mathematical Analysis
**Conflict condition for Best Price:** **Conflict condition for Best Price:**
``` ```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100) daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
``` ```
**Typical values:** **Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5% - Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25` - Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%) - Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases. **Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:** **Formula:**
```python ```python
if flex > 0.20: # 20% threshold if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20 flex_excess = flex - 0.20
@ -159,7 +172,7 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):** **Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale | | Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------| | ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant | | ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction | | 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction | | 30% | 0.75 | 3.75% | Moderate reduction |
@ -167,6 +180,7 @@ if flex > 0.20: # 20% threshold
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides | | 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?** **Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average - Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval - Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period" - Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()` **Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:** **Code Extract:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
``` ```
**Why Linear Scaling?** **Why Linear Scaling?**
- Simple and predictable - Simple and predictable
- No abrupt behavior changes - No abrupt behavior changes
- Easy to reason about for users and developers - Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive) - Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?** **Why 25% Minimum?**
- Below this, min_distance loses semantic meaning - Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed - Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario - Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants ### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:** **Defined in `coordinator/period_handlers/core.py`:**
```python ```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
``` ```
**Defined in `const.py`:** **Defined in `const.py`:**
```python ```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled) DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection) DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances **Goal:** Find practical time windows for running appliances
**Constraints:** **Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h) - Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead) - Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average" - User wants genuinely cheap times, not just "slightly below average"
**Defaults:** **Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use - **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times - **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones - **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:** **User behavior:**
- Automations trigger actions (turn on devices) - Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money - Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives) - Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction **Goal:** Alert users to expensive periods for consumption reduction
**Constraints:** **Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding) - Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy - Early warning more valuable than perfect accuracy
- User can manually decide whether to react - User can manually decide whether to react
**Defaults:** **Defaults:**
- **30 min minimum** - Catches shorter expensive spikes - **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection - **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods - **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:** **User behavior:**
- Notifications/alerts (informational) - Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost - Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes) - Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:** **Peak Price Volatility:**
Price curves tend to have: Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening) - **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical) - **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times - **Higher variance** in peak times than cheap times
**Example day:** **Example day:**
``` ```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
``` ```
**Implication:** **Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief) - Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes - Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection - Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered #### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)** **Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex - Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes - Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?" - User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)** **Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices - No guidance on best practices
- Users would need to experiment to find good values - Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter - Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)** **Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options - **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price - **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case - Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%) **Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:** **Rationale:**
- Above 50%, period detection becomes unreliable - Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals) - Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50% - Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met) - **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:** **Warning Message:**
``` ```
Flex XX% exceeds maximum safe value! Capping at 50%. Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation. Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%) **Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:** **Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold - Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable" - At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation - **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended) #### With Relaxation Enabled (Recommended)
**Optimal:** 10-20% **Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ... - Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work - Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25% **Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side" - INFO log: "Base flex is on the high side"
**High Warning:** > 30% **High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!" - WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20% - Recommendation: Lower to 15-20%
#### Without Relaxation #### Without Relaxation
**Optimal:** 20-35% **Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start - No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback - Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50% **Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits) - Above this, period detection degrades (see Hard Limits)
--- ---
@ -397,6 +436,7 @@ These three mechanisms handle pathological price situations where standard filte
**Problem:** When all prices are nearly identical (e.g. 28–32 ct, CV=5.4%), requiring 2 distinct "best price" windows is geometrically impossible. Even after exhausting all 11 relaxation phases, only 1 period exists because there is no second cheap cluster. **Problem:** When all prices are nearly identical (e.g. 28–32 ct, CV=5.4%), requiring 2 distinct "best price" windows is geometrically impossible. Even after exhausting all 11 relaxation phases, only 1 period exists because there is no second cheap cluster.
**Solution:** Before the baseline counting loop, compute per-day effective min_periods: **Solution:** Before the baseline counting loop, compute per-day effective min_periods:
```python ```python
if day_cv <= 10%: if day_cv <= 10%:
day_effective_min[day] = 1 # Flat day: 1 period is enough day_effective_min[day] = 1 # Flat day: 1 period is enough
@ -417,13 +457,14 @@ else:
**Problem:** On solar surplus days (avg 2–5 ct/kWh), a percentage-based min_distance like 5% means only 0.1 ct absolute separation is required. The filter either accepts almost the entire day (if ref_price is 2 ct, 5% = 0.1 ct – nearly everything qualifies) or blocks everything (if the spread is within that 0.1 ct band). **Problem:** On solar surplus days (avg 2–5 ct/kWh), a percentage-based min_distance like 5% means only 0.1 ct absolute separation is required. The filter either accepts almost the entire day (if ref_price is 2 ct, 5% = 0.1 ct – nearly everything qualifies) or blocks everything (if the spread is within that 0.1 ct band).
**Solution:** Linear scaling toward zero as avg_price approaches zero: **Solution:** Linear scaling toward zero as avg_price approaches zero:
``` ```
scale_factor = avg_price / LOW_PRICE_AVG_THRESHOLD scale_factor = avg_price / LOW_PRICE_AVG_THRESHOLD
adjusted_min_distance = original_min_distance × scale_factor adjusted_min_distance = original_min_distance × scale_factor
``` ```
| avg_price | scale | Effect on 5% min_distance | | avg_price | scale | Effect on 5% min_distance |
|---|---|---| | ------------------ | ----- | ------------------------- |
| ≥ 10 ct (0.10 EUR) | 100% | 5% (full distance) | | ≥ 10 ct (0.10 EUR) | 100% | 5% (full distance) |
| 5 ct (0.05 EUR) | 50% | 2.5% | | 5 ct (0.05 EUR) | 50% | 2.5% |
| 2 ct (0.02 EUR) | 20% | 1% | | 2 ct (0.02 EUR) | 20% | 1% |
@ -435,11 +476,12 @@ adjusted_min_distance = original_min_distance × scale_factor
**Trigger:** Period mean price < `LOW_PRICE_QUALITY_BYPASS_THRESHOLD` (0.10 EUR) **Trigger:** Period mean price < `LOW_PRICE_QUALITY_BYPASS_THRESHOLD` (0.10 EUR)
**Problem:** A period at 0.5–4 ct has high *relative* variation (CV ≈ 70–80%), but the absolute differences are fractions of a cent. The quality gate (CV ≤ `PERIOD_MAX_CV`) with a relative metric would wrongly reject this as a "heterogeneous" period. **Problem:** A period at 0.5–4 ct has high _relative_ variation (CV ≈ 70–80%), but the absolute differences are fractions of a cent. The quality gate (CV ≤ `PERIOD_MAX_CV`) with a relative metric would wrongly reject this as a "heterogeneous" period.
**Distinguishes from flat normal days:** A flat day at 33–36 ct also has low absolute range, but mean is 34.5 ct (>> 0.10 EUR threshold). The bypass only applies when the mean itself is below the threshold – i.e. the day is genuinely cheap in absolute terms. **Distinguishes from flat normal days:** A flat day at 33–36 ct also has low absolute range, but mean is 34.5 ct (>> 0.10 EUR threshold). The bypass only applies when the mean itself is below the threshold – i.e. the day is genuinely cheap in absolute terms.
**Solution:** Short-circuit the quality gate check: **Solution:** Short-circuit the quality gate check:
```python ```python
period_mean = sum(period_prices) / len(period_prices) period_mean = sum(period_prices) / len(period_prices)
if period_mean < LOW_PRICE_QUALITY_BYPASS_THRESHOLD: if period_mean < LOW_PRICE_QUALITY_BYPASS_THRESHOLD:
@ -470,6 +512,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach ### Multi-Phase Approach
**Each day processed independently:** **Each day processed independently:**
1. Calculate baseline periods with user's config 1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop 2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations 3. Try progressively relaxed filter combinations
@ -493,6 +536,7 @@ for attempt in range(max_relaxation_attempts):
``` ```
**Constants:** **Constants:**
```python ```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20% FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -522,6 +566,7 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Historical Context (Pre-November 2025):** **Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex: The algorithm previously used percentage-based increments that scaled with base flex:
```python ```python
increment = base_flex × (step_pct / 100) # REMOVED increment = base_flex × (step_pct / 100) # REMOVED
``` ```
@ -529,6 +574,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point. This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:** **Warning Messages:**
```python ```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30% if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning( _LOGGER.warning(
@ -547,12 +593,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy ### Filter Combination Strategy
**Per Flex level, try in order:** **Per Flex level, try in order:**
1. Original Level filter 1. Original Level filter
2. Level filter = "any" (disabled) 2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations) **Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):** **Example Flow (target=2 periods/day):**
``` ```
Day 2025-11-19: Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2) 1. Baseline flex=15%: Found 1 period (need 2)
@ -567,6 +615,7 @@ Day 2025-11-19:
### Key Files and Functions ### Key Files and Functions
**Period Calculation Entry Point:** **Period Calculation Entry Point:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods( def calculate_periods(
@ -577,6 +626,7 @@ def calculate_periods(
``` ```
**Flex + Distance Filtering:** **Flex + Distance Filtering:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
def check_interval_criteria( def check_interval_criteria(
@ -586,6 +636,7 @@ def check_interval_criteria(
``` ```
**Relaxation Orchestration:** **Relaxation Orchestration:**
```python ```python
# coordinator/period_handlers/relaxation.py # coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict] def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -616,6 +667,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Rejects asymmetric outliers (threshold: 1.5 std dev) - Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks) - Preserves legitimate price shifts (morning/evening peaks)
- Algorithm: - Algorithm:
```python ```python
residual = abs(actual - predicted) residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev symmetry_threshold = 1.5 × std_dev
@ -638,6 +690,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes) - Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:** **Constants:**
```python ```python
# coordinator/period_handlers/outlier_filtering.py # coordinator/period_handlers/outlier_filtering.py
@ -648,18 +701,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
``` ```
**Data Integrity:** **Data Integrity:**
- Original prices stored in `_original_price` field - Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices - All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic - Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome - Smart counting: Only counts smoothing that changed period outcome
**Performance:** **Performance:**
- Single pass through price data - Single pass through price data
- O(n) complexity with small context window - O(n) complexity with small context window
- No iterative refinement needed - No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals - Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:** **Example Debug Output:**
``` ```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -699,6 +755,7 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips ## Debugging Tips
**Enable DEBUG logging:** **Enable DEBUG logging:**
```yaml ```yaml
# configuration.yaml # configuration.yaml
logger: logger:
@ -708,6 +765,7 @@ logger:
``` ```
**Key log messages to watch:** **Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion 1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering 2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded 3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -720,17 +778,20 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation ### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 40 best_price_flex: 40
enable_relaxation_best: true enable_relaxation_best: true
``` ```
**Problem:** **Problem:**
- Base Flex 40% already very permissive - Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...) - Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns - Quickly approaches 50% cap with diminishing returns
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 15 # Let relaxation increase it best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true enable_relaxation_best: true
@ -739,16 +800,19 @@ enable_relaxation_best: true
### ❌ Anti-Pattern 2: Zero Min_Distance ### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_min_distance_from_avg: 0 best_price_min_distance_from_avg: 0
``` ```
**Problem:** **Problem:**
- "Flat days" (little price variation) accept all intervals - "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap") - Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times - May create periods during barely-below-average times
**Solution:** **Solution:**
```yaml ```yaml
best_price_min_distance_from_avg: 5 # Use default 5% best_price_min_distance_from_avg: 5 # Use default 5%
``` ```
@ -756,16 +820,19 @@ best_price_min_distance_from_avg: 5 # Use default 5%
### ❌ Anti-Pattern 3: Conflicting Flex + Distance ### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 45 best_price_flex: 45
best_price_min_distance_from_avg: 10 best_price_min_distance_from_avg: 10
``` ```
**Problem:** **Problem:**
- Distance filter dominates, making Flex irrelevant - Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal - Dynamic scaling helps but still suboptimal
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 20 best_price_flex: 20
best_price_min_distance_from_avg: 5 best_price_min_distance_from_avg: 5
@ -781,11 +848,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods - Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient) - Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range - Min_Distance 5%: Effective throughout range
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -799,11 +868,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners) - Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify - Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price" - Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -825,6 +896,7 @@ Relaxation would exhaust all 11 phases trying to find a second period. All price
`_compute_day_effective_min()` detects CV ≤ 10% and sets `day_effective_min = 1` for this day. The result is accepted after finding the single cheapest cluster. `_compute_day_effective_min()` detects CV ≤ 10% and sets `day_effective_min = 1` for this day. The result is accepted after finding the single cheapest cluster.
**Expected Logs:** **Expected Logs:**
``` ```
DEBUG: Day 2025-11-11: flat price profile (CV=5.4% ≤ 10.0%) → min_periods relaxed to 1 DEBUG: Day 2025-11-11: flat price profile (CV=5.4% ≤ 10.0%) → min_periods relaxed to 1
INFO: Adaptive min_periods: 1 flat day(s) (CV ≤ 10%) need only 1 period instead of 2 INFO: Adaptive min_periods: 1 flat day(s) (CV ≤ 10%) need only 1 period instead of 2
@ -832,6 +904,7 @@ INFO: Day 2025-11-11: Baseline satisfied (1 period, effective minimum is 1)
``` ```
**Sensor Attributes:** **Sensor Attributes:**
```yaml ```yaml
min_periods_configured: 2 # User's setting min_periods_configured: 2 # User's setting
flat_days_detected: 1 # Explains why only 1 period found flat_days_detected: 1 # Explains why only 1 period found
@ -847,18 +920,20 @@ Peak price always runs full relaxation. On a flat day, the integration still nee
**Configuration:** `min_periods_best: 2`, 5% min_distance **Configuration:** `min_periods_best: 2`, 5% min_distance
**Problems without fixes:** **Problems without fixes:**
1. **min_distance conflict:** 5% of 2.1 ct = 0.105 ct minimum distance. Only prices ≤ 1.995 ct qualify. The daily minimum is 0.5 ct – well within range. But the *relative* threshold becomes meaninglessly tiny: the entire day could qualify.
1. **min_distance conflict:** 5% of 2.1 ct = 0.105 ct minimum distance. Only prices ≤ 1.995 ct qualify. The daily minimum is 0.5 ct – well within range. But the _relative_ threshold becomes meaninglessly tiny: the entire day could qualify.
2. **CV quality gate:** Prices 0.5–4.2 ct show high relative variation (CV ≈ 70-80%), but the absolute differences are fractions of a cent. The quality gate would wrongly reject valid periods. 2. **CV quality gate:** Prices 0.5–4.2 ct show high relative variation (CV ≈ 70-80%), but the absolute differences are fractions of a cent. The quality gate would wrongly reject valid periods.
**Implemented behavior:** **Implemented behavior:**
*`LOW_PRICE_AVG_THRESHOLD = 0.10 EUR` (level_filtering.py):* _`LOW_PRICE_AVG_THRESHOLD = 0.10 EUR` (level_filtering.py):_
When `avg_price < 0.10 EUR`, min_distance is scaled linearly to 0. At avg=2.1 ct (0.021 EUR), scale ≈ 21% → min_distance effectively 1%. Prevents the distance filter from blocking the entire day or accepting the entire day. When `avg_price < 0.10 EUR`, min_distance is scaled linearly to 0. At avg=2.1 ct (0.021 EUR), scale ≈ 21% → min_distance effectively 1%. Prevents the distance filter from blocking the entire day or accepting the entire day.
*`LOW_PRICE_QUALITY_BYPASS_THRESHOLD = 0.10 EUR` (relaxation.py):* _`LOW_PRICE_QUALITY_BYPASS_THRESHOLD = 0.10 EUR` (relaxation.py):_
When period mean < 0.10 EUR, the CV quality gate is bypassed entirely. A period at 0.5–2 ct with CV=60% is practically homogeneous from a cost perspective. When period mean < 0.10 EUR, the CV quality gate is bypassed entirely. A period at 0.5–2 ct with CV=60% is practically homogeneous from a cost perspective.
**Expected Logs:** **Expected Logs:**
``` ```
DEBUG: Low-price day (avg=0.021 EUR < 0.10 threshold): min_distance scaled 5% → 1.1% DEBUG: Low-price day (avg=0.021 EUR < 0.10 threshold): min_distance scaled 5% → 1.1%
DEBUG: Period 02:00-05:00: mean=0.009 EUR < bypass threshold → quality gate bypassed DEBUG: Period 02:00-05:00: mean=0.009 EUR < bypass threshold → quality gate bypassed
@ -870,11 +945,13 @@ DEBUG: Period 02:00-05:00: mean=0.009 EUR < bypass threshold → quality gate
**Average:** 18 ct/kWh **Average:** 18 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct) - Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct) - Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive) - Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct) DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction) DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -889,6 +966,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2 **Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2) DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -904,6 +982,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day **Initial State:** Strict filters, very flat day
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2) DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -954,7 +1033,7 @@ When debugging period calculation issues:
**Diagnostic Sensor Attributes Summary:** **Diagnostic Sensor Attributes Summary:**
| Attribute | Type | When shown | Meaning | | Attribute | Type | When shown | Meaning |
|---|---|---|---| | ------------------------ | ------ | ---------------- | ------------------------------------------- |
| `min_periods_configured` | int | Always | User's configured target per day | | `min_periods_configured` | int | Always | User's configured target per day |
| `flat_days_detected` | int | Only when > 0 | Days where CV ≤ 10% reduced target to 1 | | `flat_days_detected` | int | Only when > 0 | Days where CV ≤ 10% reduced target to 1 |
| `relaxation_incomplete` | bool | Only when true | Relaxation exhausted, target not reached | | `relaxation_incomplete` | bool | Only when true | Relaxation exhausted, target not reached |
@ -999,6 +1078,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation **Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:** **Algorithm:**
```python ```python
# Pseudo-code for adaptive flex # Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg variation = (daily_max - daily_min) / daily_avg
@ -1012,11 +1092,13 @@ else: # Normal day
``` ```
**Benefits:** **Benefits:**
- Eliminates need for relaxation on most days - Eliminates need for relaxation on most days
- Self-adjusting to market conditions - Self-adjusting to market conditions
- Better user experience (less configuration needed) - Better user experience (less configuration needed)
**Challenges:** **Challenges:**
- Harder to predict behavior (less transparent) - Harder to predict behavior (less transparent)
- May conflict with user's mental model - May conflict with user's mental model
- Needs extensive testing across different markets - Needs extensive testing across different markets
@ -1028,17 +1110,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback **Concept:** Learn optimal Flex/Distance from user feedback
**Approach:** **Approach:**
- Track which periods user actually uses (automation triggers) - Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal) - Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults - Apply pattern-specific defaults
- Learn per-user preferences over time - Learn per-user preferences over time
**Benefits:** **Benefits:**
- Personalized to user's actual behavior - Personalized to user's actual behavior
- Adapts to local market patterns - Adapts to local market patterns
- Could discover non-obvious patterns - Could discover non-obvious patterns
**Challenges:** **Challenges:**
- Requires user feedback mechanism (not implemented) - Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns) - Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?" - Complexity for users to understand "why this period?"
@ -1051,22 +1136,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously **Concept:** Balance multiple goals simultaneously
**Goals:** **Goals:**
- Period count vs. quality (cheap vs. very cheap) - Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent) - Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered) - Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher) - User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:** **Algorithm:**
- Pareto optimization (find trade-off frontier) - Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences - User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing - Genetic algorithm or simulated annealing
**Benefits:** **Benefits:**
- More sophisticated period selection - More sophisticated period selection
- Better match to user's actual needs - Better match to user's actual needs
- Could handle complex appliance requirements - Could handle complex appliance requirements
**Challenges:** **Challenges:**
- Much more complex to implement - Much more complex to implement
- Harder to explain to users - Harder to explain to users
- Computational cost (may need caching) - Computational cost (may need caching)
@ -1081,14 +1170,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex **Current:** 3% cap may be too aggressive for very low base Flex
**Example:** **Example:**
- Base flex 5% + 3% increment = 8% (60% increase!) - Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase) - Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:** **Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)` - Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%) - This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:** **Why Not Implemented:**
- Very low base flex (`<`10%) unusual - Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation - Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization - Simplicity preferred over edge case optimization
@ -1098,6 +1190,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative **Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve **Alternative:** Non-linear curve
```python ```python
# Example: Exponential scaling # Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20)) scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -1107,6 +1200,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
``` ```
**Why Not Implemented:** **Why Not Implemented:**
- Linear is easier to reason about - Linear is easier to reason about
- No evidence that non-linear is better - No evidence that non-linear is better
- Would need extensive testing - Would need extensive testing
@ -1116,15 +1210,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day **Issue:** May find all periods in one part of day
**Example:** **Example:**
- All 3 "best price" periods between 02:00-08:00 - All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances) - No periods in evening (when user might want to run appliances)
**Possible Solution:** **Possible Solution:**
- Add "spread" parameter (prefer distributed periods) - Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences - Weight periods by time-of-day preferences
- Consider user's typical usage patterns - Consider user's typical usage patterns
**Why Not Implemented:** **Why Not Implemented:**
- Adds complexity - Adds complexity
- Users can work around with multiple automations - Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all) - Different users have different needs (no one-size-fits-all)
@ -1136,6 +1233,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg). **Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:** **Implementation:**
```python ```python
# In period_building.py build_periods(): # In period_building.py build_periods():
for price_data in all_prices: for price_data in all_prices:
@ -1187,6 +1285,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight** **Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split: When days differ significantly, period can split:
``` ```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1) Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2) Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1198,6 +1297,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:** **Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours: Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium - Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer - Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1206,10 +1306,12 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):** **User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful: Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100) - `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre) - `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting: Automations can check volatility before acting:
```yaml ```yaml
condition: condition:
- condition: template - condition: template
@ -1240,6 +1342,7 @@ Low volatility (< 15%) means classification changes are less economically signif
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness. **Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:** **See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes" - User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`) - Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation) - Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

31
docs/developer/docs/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
``` ```
**Key Points:** **Key Points:**
- Must be a **class attribute** (not instance attribute) - Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance - Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component - Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips` **Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that: **Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely - Never change or change very rarely
- Don't provide analytical value in history - Don't provide analytical value in history
- Consume significant database space when recorded every state change - Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures ### 2. Large Nested Structures
**Attributes:** **Attributes:**
- `periods` (binary_sensor) - Array of all period summaries - `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays - `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis - `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown - `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are: **Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive) - Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each) - Create large database rows (2-20 KB each)
- Slow down history queries - Slow down history queries
@ -66,6 +70,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors **Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:** **Example - periods array:**
```json ```json
{ {
"periods": [ "periods": [
@ -76,7 +81,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
"price_mean": 18.5, "price_mean": 18.5,
"price_median": 18.3, "price_median": 18.3,
"price_min": 17.2, "price_min": 17.2,
"price_max": 19.8, "price_max": 19.8
// ... 10+ more attributes × 10-20 periods // ... 10+ more attributes × 10-20 periods
} }
] ]
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status` **Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:** **Reason:**
- Change every update cycle (every 15 minutes or more frequently) - Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value - Don't provide long-term analytical value
- Create state changes even when core values haven't changed - Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max` **Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:** **Reason:**
- Configuration values that rarely change - Configuration values that rarely change
- Wastes space when recorded repeatedly - Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state - Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `timestamp`, `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error` **Attributes:** `timestamp`, `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:** **Reason:**
- `timestamp` is the rounded-quarter reference time used at the moment of the state write — it's stale as soon as the next update fires and has no analytical value in history - `timestamp` is the rounded-quarter reference time used at the moment of the state write — it's stale as soon as the next update fires and has no analytical value in history
- `next_api_poll`, `next_midnight_turnover` etc. are only relevant at the moment of reading; they're superseded by the next update - `next_api_poll`, `next_midnight_turnover` etc. are only relevant at the moment of reading; they're superseded by the next update
- Similar to `entity_picture` in HA core image entities - Similar to `entity_picture` in HA core image entities
@ -129,6 +137,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%` **Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:** **Reason:**
- Detailed technical information not needed for historical analysis - Detailed technical information not needed for historical analysis
- Only useful for debugging during active development - Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis - Boolean `relaxation_active` is kept for high-level analysis
@ -140,6 +149,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `period_count_total`, `period_count_remaining` **Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `period_count_total`, `period_count_remaining`
**Reason:** **Reason:**
- Can be calculated from other attributes - Can be calculated from other attributes
- Redundant information - Redundant information
- Doesn't add analytical value to history - Doesn't add analytical value to history
@ -153,23 +163,28 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value: These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core ### Time-Series Core
- All price values - Core sensor states (the entity's `native_value` is always recorded separately) - All price values - Core sensor states (the entity's `native_value` is always recorded separately)
### Diagnostics & Tracking ### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time - `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns - `updates_today` - Tracking API usage patterns
### Data Completeness ### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics - `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status - `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data ### Period Data
- `start`, `end`, `duration_minutes` - Core period timing - `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics - `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
- `period_position` - Position of current period in the day's sequence - `period_position` - Position of current period in the day's sequence
- `period_count_today`, `period_count_tomorrow` - How many periods per day (useful in automations) - `period_count_today`, `period_count_tomorrow` - How many periods per day (useful in automations)
### High-Level Status ### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation) - `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact ## Expected Database Impact
@ -177,6 +192,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings ### Space Savings
**Per state change:** **Per state change:**
- Before: ~3-8 KB average - Before: ~3-8 KB average
- After: ~0.5-1.5 KB average - After: ~0.5-1.5 KB average
- **Reduction: 60-85%** - **Reduction: 60-85%**
@ -198,6 +214,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact ### Real-World Impact
For a typical installation with: For a typical installation with:
- 80+ sensors - 80+ sensors
- Updates every 15 minutes - Updates every 15 minutes
- ~10 sensors updating every minute - ~10 sensors updating every minute
@ -216,7 +233,7 @@ For a typical installation with:
- Class: `TibberPricesBinarySensor` - Class: `TibberPricesBinarySensor`
- 29 attributes excluded - 29 attributes excluded
## When to Update _unrecorded_attributes ## When to Update \_unrecorded_attributes
### Add to Exclusion List When: ### Add to Exclusion List When:
@ -267,6 +284,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes 4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:** **SQL Query to check attribute presence:**
```sql ```sql
SELECT SELECT
state_id, state_id,
@ -301,7 +319,7 @@ This makes `state_class=TOTAL` on many sensors the primary cause of long-term da
For sensors with `device_class=SensorDeviceClass.MONETARY`, only two `state_class` values are valid: For sensors with `device_class=SensorDeviceClass.MONETARY`, only two `state_class` values are valid:
| `state_class` | Statistics written | Frontend effect | | `state_class` | Statistics written | Frontend effect |
|---|---|---| | ------------- | ------------------------- | ------------------------------------------------- |
| `TOTAL` | ✅ Yes — unbounded growth | Statistics line-chart on entity detail page | | `TOTAL` | ✅ Yes — unbounded growth | Statistics line-chart on entity detail page |
| `None` | ❌ No | States timeline only (History panel, "Show More") | | `None` | ❌ No | States timeline only (History panel, "Show More") |
| `MEASUREMENT` | ❌ Blocked by hassfest | — | | `MEASUREMENT` | ❌ Blocked by hassfest | — |
@ -313,18 +331,20 @@ For sensors with `device_class=SensorDeviceClass.MONETARY`, only two `state_clas
Only 3 of 26 MONETARY sensors keep `state_class=TOTAL` — those where long-term history is genuinely useful: Only 3 of 26 MONETARY sensors keep `state_class=TOTAL` — those where long-term history is genuinely useful:
| Sensor | Reason | | Sensor | Reason |
|---|---| | ----------------------------- | ------------------------------------ |
| `current_interval_price` | Long-term price trend (weeks/months) | | `current_interval_price` | Long-term price trend (weeks/months) |
| `current_interval_price_base` | Required for Energy Dashboard | | `current_interval_price_base` | Required for Energy Dashboard |
| `average_price_today` | Seasonal daily average tracking | | `average_price_today` | Seasonal daily average tracking |
All other 23 MONETARY sensors use `state_class=None`: All other 23 MONETARY sensors use `state_class=None`:
- Forecast/future sensors (`next_avg_*h`) - Forecast/future sensors (`next_avg_*h`)
- Daily snapshots (`lowest/highest_price_today/tomorrow`) - Daily snapshots (`lowest/highest_price_today/tomorrow`)
- Rolling windows (`trailing/leading_24h_*`) - Rolling windows (`trailing/leading_24h_*`)
- Next/previous interval sensors - Next/previous interval sensors
**Effect of `state_class=None`:** **Effect of `state_class=None`:**
- ✅ Short-term state history (States timeline, ~10 days) still works normally - ✅ Short-term state history (States timeline, ~10 days) still works normally
- ✅ Templates, automations, and attributes are unaffected - ✅ Templates, automations, and attributes are unaffected
- ❌ Statistics line-chart removed from entity detail page for these sensors - ❌ Statistics line-chart removed from entity detail page for these sensors
@ -333,6 +353,7 @@ All other 23 MONETARY sensors use `state_class=None`:
### Expected Impact ### Expected Impact
Going from 26 → 3 sensors writing to the statistics tables: Going from 26 → 3 sensors writing to the statistics tables:
- **~88% reduction** in statistics table writes - **~88% reduction** in statistics table writes
- Prevents the primary cause of long-term database bloat - Prevents the primary cause of long-term database bloat
- Existing statistics data is retained (only new writes stop) - Existing statistics data is retained (only new writes stop)
@ -342,7 +363,7 @@ Going from 26 → 3 sensors writing to the statistics tables:
These are two independent mechanisms targeting different tables: These are two independent mechanisms targeting different tables:
| Mechanism | Table affected | Purged? | Controls | | Mechanism | Table affected | Purged? | Controls |
|---|---|---|---| | ------------------------ | ------------------------------------- | ----------- | ----------------------------------------------- |
| `_unrecorded_attributes` | `state_attributes` | ✅ ~10 days | Which attributes are stored per state write | | `_unrecorded_attributes` | `state_attributes` | ✅ ~10 days | Which attributes are stored per state write |
| `state_class=None` | `statistics`, `statistics_short_term` | ❌ Never | Whether long-term statistics are written at all | | `state_class=None` | `statistics`, `statistics_short_term` | ❌ Never | Whether long-term statistics are written at all |

14
docs/developer/docs/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):** **In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built: git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile) - **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup` - **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):** **Manual installation (outside DevContainer):**
**git-cliff** (template-based): **git-cliff** (template-based):
```bash ```bash
# See: https://git-cliff.org/docs/installation # See: https://git-cliff.org/docs/installation
@ -191,7 +193,7 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which ## 🎯 When to Use Which
| Method | Use Case | Pros | Cons | | Method | Use Case | Pros | Cons |
|--------|----------|------|------| | --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script | | **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump | | **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization | | **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
@ -219,6 +221,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. Script bumps manifest.json → commits → creates tag locally 1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together 2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release 3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
``` ```
**What happens:** **What happens:**
1. You push manifest.json change 1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically 2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release 3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. You create and push tag manually 1. You create and push tag manually
2. Release workflow creates release 2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists) 3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features ## 🛡️ Safety Features
### 1. **Version Validation** ### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z). Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags** ### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote) - Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating - Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations** ### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push. Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered** ### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits. Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions** ### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind. Helper script shows how to undo if you change your mind.
--- ---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:** **Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes: Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely - Tag already exists remotely
- Invalid version format in manifest.json - Invalid version format in manifest.json
- manifest.json not in the commit that was pushed - manifest.json not in the commit that was pushed
@ -348,6 +359,7 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips ## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results: 1. **Conventional Commits:** Use proper commit format for best results:
``` ```
feat(scope): Add new feature feat(scope): Add new feature

22
docs/developer/docs/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle. The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:** **Design Principles:**
- **Proactive**: Detect issues before they become critical - **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance - **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve - **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}` **Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:** **When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`) - Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available - Tomorrow's electricity price data is still not available
**When cleared:** **When cleared:**
- Tomorrow's data becomes available - Tomorrow's data becomes available
- Automatically checks on every successful API update - Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work. Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:** **Implementation:**
```python ```python
# In coordinator update cycle # In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"]) has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18) - `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}` **Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:** **When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429) - Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD` - Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:** **When cleared:**
- Successful API call completes (no rate limit error) - Successful API call completes (no rate limit error)
- Error counter resets to 0 - Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires. API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:** **Implementation:**
```python ```python
# In error handler # In error handler
is_rate_limit = ( is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors - `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}` **Issue ID:** `home_not_found_{entry_id}`
**When triggered:** **When triggered:**
- Home configured in this integration is no longer present in Tibber account - Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check) - Detected during user data refresh (daily check)
**When cleared:** **When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected) - Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup) - Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed. Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:** **Implementation:**
```python ```python
# After user data update # After user data update
home_exists = self._data_fetcher._check_home_exists(home_id) home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the missing home - `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference - `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration ### Lifecycle Integration
**Coordinator Initialization:** **Coordinator Initialization:**
```python ```python
self._repair_manager = TibberPricesRepairManager( self._repair_manager = TibberPricesRepairManager(
hass=hass, hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
``` ```
**Update Cycle Integration:** **Update Cycle Integration:**
```python ```python
# Success path - check conditions # Success path - check conditions
if result and "priceInfo" in result: if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
``` ```
**Shutdown Cleanup:** **Shutdown Cleanup:**
```python ```python
async def async_shutdown(self) -> None: async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up.""" """Shut down coordinator and clean up."""
@ -196,6 +212,7 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json` - `/translations/sv.json`
**Structure:** **Structure:**
```json ```json
{ {
"issues": { "issues": {
@ -210,10 +227,12 @@ Repairs use Home Assistant's standard translation system. Translations are defin
## Home Assistant Integration ## Home Assistant Integration
Repairs appear in: Repairs appear in:
- **Settings → System → Repairs** (main repairs panel) - **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count) - **Notifications** (bell icon in UI shows repair count)
Repair properties: Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required) - **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical) - **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files - **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears 4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:** **Manual trigger:**
```python ```python
# Temporarily set warning hour to current hour for testing # Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair 3. Successful API call clears the repair
**Manual test:** **Manual test:**
- Reduce API polling interval to trigger rate limiting - Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client - Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file 7. **Document** in this file
**Example template:** **Example template:**
```python ```python
async def check_new_condition(self, *, param: bool) -> None: async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair.""" """Check new condition and create/clear repair."""

24
docs/developer/docs/timer-architecture.md
View file

@ -11,7 +11,7 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes: The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method | | Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------| | ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` | | **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` | | **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` | | **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes` **Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:** **What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator` - HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes - Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time) - **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
``` ```
**Load Distribution:** **Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution - Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API - Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once - Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:** **Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects) - Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early - If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully - Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:** **Why we use HA's timer:**
- Automatic restart after HA restart - Automatic restart after HA restart
- Built-in retry logic for temporary failures - Built-in retry logic for temporary failures
- Standard HA integration pattern - Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll** **Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:** **Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48) - Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes - Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13 - Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
``` ```
**Smart Boundary Tolerance:** **Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance - Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval) - HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval) - HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details - See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:** **Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...) - `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes") - NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates) - If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:** **Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`) - All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`) - Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total - ~50-60 entities out of 120+ total
**Why custom timer:** **Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing - HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals - We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation - Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
``` ```
**Which entities listen:** **Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer - `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer - `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%) - `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total - ~10 entities total
**Why custom timer:** **Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time) - Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates - Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation) - Very lightweight (no data processing, just state recalculation)
**Why NOT every second:** **Why NOT every second:**
- Minute precision sufficient for countdown UX - Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds) - Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates) - Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
``` ```
**Why this pattern:** **Why this pattern:**
- Decouples timer logic from entity logic - Decouples timer logic from entity logic
- One timer can notify many entities efficiently - One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup) - Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API ### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers: If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload - ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload - ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem - ❌ "Thundering herd" problem
With HA's unsynchronized timer: With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ... - ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ... - ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ... - ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data. **Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:** **API fetch only when:**
- Tomorrow data missing/invalid (after 13:00) - Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover) - Cache expired (midnight turnover)
- Explicit user refresh - Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics ## Performance Characteristics
### Timer #1 (DataUpdateCoordinator) ### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized) - **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data) - **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate) - **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise) - **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh) ### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries) - **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities) - **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data - **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates - **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh) ### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute) - **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities) - **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all - **No API calls:** No data processing at all
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary ## Summary
**Three independent timers:** **Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed) 1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always) 2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always) 3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:** **Key insights:**
- Timer #1 unsynchronized = good (load distribution on API) - Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately) - Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX) - Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts) - All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:** **"Listener" terminology:**
- Timer = mechanism that triggers - Timer = mechanism that triggers
- Listener = callback that gets called - Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies - Observer pattern = entities register, coordinator notifies

31
docs/developer/versioned_docs/version-v0.21.0/api-reference.md
View file

@ -56,7 +56,7 @@ query {
Fetches quarter-hourly prices: Fetches quarter-hourly prices:
```graphql ```graphql
query($homeId: ID!) { query ($homeId: ID!) {
viewer { viewer {
home(id: $homeId) { home(id: $homeId) {
currentSubscription { currentSubscription {
@ -76,6 +76,7 @@ query($homeId: ID!) {
``` ```
**Parameters:** **Parameters:**
- `homeId`: Tibber home identifier - `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY` - `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data) - `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits ## Rate Limits
Tibber API rate limits (as of 2024): Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token - **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute - **Burst limit:** 100 requests per minute
Integration stays well below these limits: Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day - Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day - User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home - **Total:** ~100 requests/day per home
@ -106,6 +109,7 @@ Integration stays well below these limits:
``` ```
**Fields:** **Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR) - `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone - `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE) - `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -119,6 +123,7 @@ Integration stays well below these limits:
``` ```
Supported currencies: Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh - `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh - `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh - `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses ### Common Error Responses
**Invalid Token:** **Invalid Token:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Unauthorized", "message": "Unauthorized",
"extensions": { "extensions": {
"code": "UNAUTHENTICATED" "code": "UNAUTHENTICATED"
} }
}] }
]
} }
``` ```
**Rate Limit Exceeded:** **Rate Limit Exceeded:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Too Many Requests", "message": "Too Many Requests",
"extensions": { "extensions": {
"code": "RATE_LIMIT_EXCEEDED" "code": "RATE_LIMIT_EXCEEDED"
} }
}] }
]
} }
``` ```
**Home Not Found:** **Home Not Found:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Home not found", "message": "Home not found",
"extensions": { "extensions": {
"code": "NOT_FOUND" "code": "NOT_FOUND"
} }
}] }
]
} }
``` ```
Integration handles these with: Integration handles these with:
- Exponential backoff retry (3 attempts) - Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors - ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures - ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation ## Data Transformation
Raw API data is enriched with: Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals - **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals - **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average - **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
--- ---
💡 **External Resources:** 💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview) - [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer) - [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token) - [Get API Token](https://developer.tibber.com/settings/access-token)

17
docs/developer/versioned_docs/version-v0.21.0/architecture.md
View file

@ -147,7 +147,7 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance: The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory | | Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------| | ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB | | **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB | | **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB | | **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
@ -196,7 +196,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components ### Core Components
| Component | File | Responsibility | | Component | File | Responsibility |
|-----------|------|----------------| | --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling | | **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance | | **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) | | **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
@ -210,7 +210,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025): The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility | | Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------| | ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators | | **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) | | **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) | | **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
@ -219,6 +219,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities | | **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`): **Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access - `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous) - `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows - `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata - `metadata.py` - Home/metering metadata
**Benefits:** **Benefits:**
- 58% reduction in core.py (2,170 → 909 lines) - 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation) - Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator - Independent testability for each calculator
@ -238,7 +240,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities ### Helper Utilities
| Utility | File | Purpose | | Utility | File | Purpose |
|---------|------|---------| | ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation | | **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations | | **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic | | **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025): Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`): **Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals - `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows - `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg - `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics - `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`): **Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods - Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.) - Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`): **Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access - Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc. - Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching) - Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`): **Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation - Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes - Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()` - Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:** **Benefits:**
- Minimal code duplication across 80+ sensors - Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation) - Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing - Easy to extend: Add sensor → choose pattern → add to routing
@ -334,7 +341,7 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization ### CPU Optimization
| Optimization | Location | Savings | | Optimization | Location | Savings |
|--------------|----------|---------| | ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks | | Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation | | Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations | | Lazy logging | Throughout | ~15% on log-heavy operations |

42
docs/developer/versioned_docs/version-v0.21.0/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts. **Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:** **What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total) - **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query - **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation - **Timestamps**: Last update times for validation
**Lifetime:** **Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time) - **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily) - **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage - **Survives**: HA restarts via persistent Storage
@ -36,6 +38,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:** **Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator): 1. **Midnight turnover** (Timer #2 in coordinator):
```python ```python
# coordinator/day_transitions.py # coordinator/day_transitions.py
def _handle_midnight_turnover() -> None: def _handle_midnight_turnover() -> None:
@ -45,6 +48,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
``` ```
2. **Cache validation on load**: 2. **Cache validation on load**:
```python ```python
# coordinator/cache.py # coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool: def is_cache_valid(cache_data: CacheData) -> bool:
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc. **Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:** **What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names - **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions - **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:** **Lifetime:**
- **Forever** (until HA restart) - **Forever** (until HA restart)
- No invalidation during runtime - No invalidation during runtime
**When populated:** **When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py` - At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once - Lazy loading: If translation missing, attempts file load once
**Access pattern:** **Access pattern:**
```python ```python
# Non-blocking synchronous access from cached data # Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en") description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:** **What is cached:**
### DataTransformer Config Cache ### DataTransformer Config Cache
```python ```python
{ {
"thresholds": {"low": 15, "high": 35}, "thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
### PeriodCalculator Config Cache ### PeriodCalculator Config Cache
```python ```python
{ {
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60}, "best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,10 +128,12 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Lifetime:** **Lifetime:**
- Until `invalidate_config_cache()` is called - Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle - Built once on first use per coordinator update cycle
**Invalidation trigger:** **Invalidation trigger:**
- **Options change** (user reconfigures integration): - **Options change** (user reconfigures integration):
```python ```python
# coordinator/core.py # coordinator/core.py
@ -132,6 +144,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Performance impact:** **Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs - **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs - **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update) - **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed. **Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:** **What is cached:**
```python ```python
{ {
"best_price": { "best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Cache key:** Hash of relevant inputs **Cache key:** Hash of relevant inputs
```python ```python
hash_data = ( hash_data = (
today_signature, # (startsAt, rating_level) for each interval today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
``` ```
**Lifetime:** **Lifetime:**
- Until price data changes (today's intervals modified) - Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters) - Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data) - Recalculated at midnight (new today data)
@ -179,6 +195,7 @@ hash_data = (
**Invalidation triggers:** **Invalidation triggers:**
1. **Config change** (explicit): 1. **Config change** (explicit):
```python ```python
def invalidate_config_cache() -> None: def invalidate_config_cache() -> None:
self._cached_periods = None self._cached_periods = None
@ -193,10 +210,12 @@ hash_data = (
``` ```
**Cache hit rate:** **Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged) - **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00) - **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:** **Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts) - **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup) - **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache) - **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy **Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:** **What is cached:**
```python ```python
{ {
"timestamp": ..., "timestamp": ...,
@ -224,6 +244,7 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks. **Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:** **Current behavior:**
- Caches **only enriched price data** (price + statistics) - Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache) - **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when: - Invalidated when:
@ -232,6 +253,7 @@ hash_data = (
- New update cycle begins - New update cycle begins
**Architecture:** **Architecture:**
- DataTransformer: Handles price enrichment only - DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache) - PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches - Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow ## Cache Invalidation Flow
### User Changes Options (Config Flow) ### User Changes Options (Config Flow)
``` ```
User saves options User saves options
↓ ↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
``` ```
### Midnight Turnover (Day Transition) ### Midnight Turnover (Day Transition)
``` ```
Timer #2 fires at 00:00 Timer #2 fires at 00:00
↓ ↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
``` ```
### Tomorrow Data Arrives (~13:00) ### Tomorrow Data Arrives (~13:00)
``` ```
Coordinator update cycle Coordinator update cycle
↓ ↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
``` ```
**No cache invalidation cascades:** **No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update) - Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch) - Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change) - Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load) - Translation cache is **never invalidated** (read-only after load)
**Thread safety:** **Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop) - All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model) - No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics ## Performance Characteristics
### Typical Operation (No Changes) ### Typical Operation (No Changes)
``` ```
Coordinator Update (every 15 min) Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid) ├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
``` ```
### After Midnight Turnover ### After Midnight Turnover
``` ```
Coordinator Update (00:00) Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day) ├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
``` ```
### After Config Change ### After Config Change
``` ```
Options Update Options Update
├─> Cache invalidation: `<`1ms ├─> Cache invalidation: `<`1ms
@ -382,7 +412,7 @@ Options Update
## Summary Table ## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose | | Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------| | ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls | | **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O | | **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups | | **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
@ -392,12 +422,14 @@ Options Update
**Total memory overhead:** ~116KB per coordinator instance (main + subentries) **Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:** **Benefits:**
- 97% reduction in API calls (from every 15min to once per day) - 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation) - 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check) - 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup) - Zero file I/O during runtime (translations cached at startup)
**Trade-offs:** **Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems) - Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented) - Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues - Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues ## Debugging Cache Issues
### Symptom: Stale data after config change ### Symptom: Stale data after config change
**Check:** **Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log) 1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed? 2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger? 3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init. **Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating ### Symptom: Period calculation not updating
**Check:** **Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()` 1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash` 2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs 3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs. **Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today ### Symptom: Yesterday's prices shown as today
**Check:** **Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py` 1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2) 2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs 3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration. **Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations ### Symptom: Missing translations
**Check:** **Check:**
1. `async_load_translations()` called at startup? 1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`? 2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys 3. Cache population: `_TRANSLATIONS_CACHE` keys

3
docs/developer/versioned_docs/version-v0.21.0/coding-guidelines.md
View file

@ -41,12 +41,14 @@ class TimeService:
``` ```
**When prefix is required:** **When prefix is required:**
- Public classes used across multiple modules - Public classes used across multiple modules
- All exception classes - All exception classes
- All coordinator and entity classes - All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs - Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:** **When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore) - Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`) - Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns - Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:** **Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring: Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md` 1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames 2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module 3. Test thoroughly after each module

17
docs/developer/versioned_docs/version-v0.21.0/contributing.md
View file

@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
``` ```
**Branch naming:** **Branch naming:**
- `feature/` - New features - `feature/` - New features
- `fix/` - Bug fixes - `fix/` - Bug fixes
- `docs/` - Documentation only - `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md). Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:** **Run checks frequently:**
```bash ```bash
./scripts/type-check # Pyright type checking ./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix) ./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
``` ```
Run your test: Run your test:
```bash ```bash
./scripts/test tests/test_your_feature.py -v ./scripts/test tests/test_your_feature.py -v
``` ```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
``` ```
**Commit types:** **Commit types:**
- `feat:` - New feature - `feat:` - New feature
- `fix:` - Bug fix - `fix:` - Bug fix
- `docs:` - Documentation - `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance - `chore:` - Maintenance
**Add scope when relevant:** **Add scope when relevant:**
- `feat(sensors):` - Sensor platform - `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator - `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation - `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max) Title: Short, descriptive (50 chars max)
Description should include: Description should include:
```markdown ```markdown
## What ## What
Brief description of changes Brief description of changes
## Why ## Why
Problem being solved or feature rationale Problem being solved or feature rationale
## How ## How
Implementation approach Implementation approach
## Testing ## Testing
- [ ] Manual testing in Home Assistant - [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated - [ ] Unit tests added/updated
- [ ] Type checking passes - [ ] Type checking passes
- [ ] Linting passes - [ ] Linting passes
## Breaking Changes ## Breaking Changes
(If any - describe migration path) (If any - describe migration path)
## Related Issues ## Related Issues
Closes #123 Closes #123
``` ```
### PR Checklist ### PR Checklist
Before submitting: Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md) - [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`) - [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`) - [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For ### What Reviewers Look For
✅ **Good:** ✅ **Good:**
- Clear, self-explanatory code - Clear, self-explanatory code
- Appropriate comments for complex logic - Appropriate comments for complex logic
- Tests covering edge cases - Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns - Follows existing patterns
❌ **Avoid:** ❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones - Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes - Mixing unrelated changes
- Missing tests for new features - Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On ## Finding Issues to Work On
Good first issues are labeled: Good first issues are labeled:
- `good first issue` - Beginner-friendly - `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions - `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements - `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
--- ---
💡 **Related:** 💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup - [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide - [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests - [Testing](testing.md) - Writing tests

33
docs/developer/versioned_docs/version-v0.21.0/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical? ## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to: Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable - **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases - **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks - **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅ #### 1.1 Listener Cleanup ✅
**What is tested:** **What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`) - Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`) - Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`) - Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners - Binary sensor cleanup removes ALL registered listeners
**Why critical:** **Why critical:**
- Each registered listener holds references to Entity + Coordinator - Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak - Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed - With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()` - `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()` - `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()` - `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅ #### 1.2 Timer Cleanup ✅
**What is tested:** **What is tested:**
- Quarter-hour timer is cancelled and reference cleared - Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared - Minute timer is cancelled and reference cleared
- Both timers are cancelled together - Both timers are cancelled together
- Cleanup works even when timers are `None` - Cleanup works even when timers are `None`
**Why critical:** **Why critical:**
- Uncancelled timers continue running after integration unload - Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks - HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates - Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()` - `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅ #### 1.3 Config Entry Cleanup ✅
**What is tested:** **What is tested:**
- Options update listener is registered via `async_on_unload()` - Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()` - Cleanup function is correctly passed to `async_on_unload()`
**Why critical:** **Why critical:**
- `entry.add_update_listener()` registers permanent callback - `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates - Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))` - Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:** **Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration) - `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()` - `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation #### 2.1 Config Cache Invalidation
**What is tested:** **What is tested:**
- DataTransformer config cache is invalidated on options change - DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated - PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update - Trend calculator cache is cleared on coordinator update
**Why critical:** **Why critical:**
- Stale config → Sensors use old user settings - Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods - Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis - Stale trend cache → Outdated trend analysis
**Code Locations:** **Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()` - `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()` - `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()` - `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal #### 3.1 Persistent Storage Removal
**What is tested:** **What is tested:**
- Storage file is deleted on config entry removal - Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss) - Cache is saved on shutdown (no data loss)
**Why critical:** **Why critical:**
- Without storage removal: Old files remain after uninstallation - Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart - Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}` - Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:** **Code Locations:**
- `__init__.py` → `async_remove_entry()` - `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py` **File:** `tests/test_timer_scheduling.py`
**What is tested:** **What is tested:**
- Quarter-hour timer is registered with correct parameters - Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters - Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer) - Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly - Midnight turnover detection works correctly
**Why critical:** **Why critical:**
- Wrong timer parameters → Entities update at wrong times - Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem - Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py` **File:** `tests/test_sensor_timer_assignment.py`
**What is tested:** **What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys - All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys - All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap) - Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked - Sensor and binary sensor platforms are checked
**Why critical:** **Why critical:**
- Wrong timer assignment → Sensors update at wrong times - Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem - Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management ### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks **Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds) - `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms) - `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:** **Why no tests needed:**
- No long-running tasks (all < 2 seconds) - No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically - HA's event loop handles short tasks automatically
- Task exceptions are already logged - Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup ### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented **Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session) - `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created - No new sessions are created
- HA manages session lifecycle automatically - HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory ### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache **Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total - Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading - Module-level cache without re-loading
- Practically no memory issue - Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity ### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop` **Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days) - Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults - Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors - 80+ sensors access `coordinator.data` without errors
**Structure:** **Structure:**
```python ```python
coordinator.data = { coordinator.data = {
"user_data": {...}, "user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory ### 10. Service Response Memory
**Current Status:** HA's response lifecycle **Current Status:** HA's response lifecycle
- HA automatically frees service responses after return - HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call - ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code - No response accumulation in integration code
@ -208,7 +235,7 @@ coordinator.data = {
### ✅ Implemented Tests (41 total) ### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage | | Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------| | ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% | | Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% | | Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% | | Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
@ -222,7 +249,7 @@ coordinator.data = {
### 📋 Analyzed but Not Implemented (Nice-to-Have) ### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale | | Category | Status | Rationale |
|----------|--------|-----------| | ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) | | Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) | | API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) | | Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
@ -230,6 +257,7 @@ coordinator.data = {
| Service Response Memory | 📋 | HA automatically frees service responses | | Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:** **Legend:**
- ✅ = Fully tested or pattern verified correct - ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues) - 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested ### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests: All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks) - ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks) - ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners) - ✅ Config entry cleanup works (no dangling listeners)

18
docs/developer/versioned_docs/version-v0.21.0/debugging.md
View file

@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages ### Key Log Messages
**Coordinator Updates:** **Coordinator Updates:**
``` ```
[custom_components.tibber_prices.coordinator] Successfully fetched price data [custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data [custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
``` ```
**Period Calculation:** **Period Calculation:**
``` ```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0% [custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods [custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
``` ```
**API Errors:** **API Errors:**
``` ```
[custom_components.tibber_prices.api] API request failed: Unauthorized [custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s [custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -67,6 +70,7 @@ Restart Home Assistant to apply.
### Set Breakpoints ### Set Breakpoints
**Coordinator update:** **Coordinator update:**
```python ```python
# coordinator/core.py # coordinator/core.py
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
``` ```
**Period calculation:** **Period calculation:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]: def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
``` ```
**Flags:** **Flags:**
- `-v` - Verbose output - `-v` - Verbose output
- `-s` - Show print statements - `-s` - Show print statements
- `-k pattern` - Run tests matching pattern - `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns ### Useful Test Patterns
**Print coordinator data:** **Print coordinator data:**
```python ```python
def test_something(coordinator): def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}") print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
``` ```
**Inspect period attributes:** **Inspect period attributes:**
```python ```python
def test_periods(hass, coordinator): def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', []) periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading ### Integration Not Loading
**Check:** **Check:**
```bash ```bash
grep "tibber_prices" config/home-assistant.log grep "tibber_prices" config/home-assistant.log
``` ```
**Common causes:** **Common causes:**
- Syntax error in Python code → Check logs for traceback - Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync` - Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*` - Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating ### Sensors Not Updating
**Check coordinator state:** **Check coordinator state:**
```python ```python
# In Developer Tools > Template # In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }} {{ states.sensor.tibber_home_current_interval_price.last_updated }}
``` ```
**Debug in code:** **Debug in code:**
```python ```python
# Add logging in sensor/core.py # Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s", _LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong ### Period Calculation Wrong
**Enable detailed period logs:** **Enable detailed period logs:**
```python ```python
# coordinator/period_handlers/period_building.py # coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s", _LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
``` ```
**Check filter statistics:** **Check filter statistics:**
``` ```
[period_building] Flex filter blocked: 45 intervals [period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals [period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy ### Remote Debugging with debugpy
Add to coordinator code: Add to coordinator code:
```python ```python
import debugpy import debugpy
debugpy.listen(5678) debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL ### IPython REPL
Install in container: Install in container:
```bash ```bash
uv pip install ipython uv pip install ipython
``` ```
Add breakpoint: Add breakpoint:
```python ```python
from IPython import embed from IPython import embed
embed() # Drops into interactive shell embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
--- ---
💡 **Related:** 💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests - [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment - [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure - [Architecture](architecture.md) - Code structure

1
docs/developer/versioned_docs/version-v0.21.0/intro.md
View file

@ -168,6 +168,7 @@ Documentation is organized in two Docusaurus sites:
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory) - **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:** **Best practices:**
- Use clear examples and code snippets - Use clear examples and code snippets
- Keep docs up-to-date with code changes - Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation - Add new pages to appropriate `sidebars.ts` for navigation

17
docs/developer/versioned_docs/version-v0.21.0/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals ## Performance Goals
Target metrics: Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms) - **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor - **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms) - **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching ### Caching
**1. Persistent Cache** (API data): **1. Persistent Cache** (API data):
```python ```python
# Already implemented in coordinator/cache.py # Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY) store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
``` ```
**2. Translation Cache** (in-memory): **2. Translation Cache** (in-memory):
```python ```python
# Already implemented in const.py # Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {} _TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
``` ```
**3. Config Cache** (invalidated on options change): **3. Config Cache** (invalidated on options change):
```python ```python
class DataTransformer: class DataTransformer:
def __init__(self): def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading ### Lazy Loading
**Load data only when needed:** **Load data only when needed:**
```python ```python
@property @property
def extra_state_attributes(self) -> dict | None: def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations ### Bulk Operations
**Process multiple items at once:** **Process multiple items at once:**
```python ```python
# ❌ Slow - loop with individual operations # ❌ Slow - loop with individual operations
for interval in intervals: for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices ### Async Best Practices
**1. Concurrent API calls:** **1. Concurrent API calls:**
```python ```python
# ❌ Sequential (slow) # ❌ Sequential (slow)
user_data = await fetch_user_data() user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
``` ```
**2. Don't block event loop:** **2. Don't block event loop:**
```python ```python
# ❌ Blocking # ❌ Blocking
result = heavy_computation() # Blocks for seconds result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks ### Avoid Memory Leaks
**1. Clear references:** **1. Clear references:**
```python ```python
class Coordinator: class Coordinator:
async def async_shutdown(self): async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
``` ```
**2. Use weak references for callbacks:** **2. Use weak references for callbacks:**
```python ```python
import weakref import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures ### Efficient Data Structures
**Use appropriate types:** **Use appropriate types:**
```python ```python
# ❌ List for lookups (O(n)) # ❌ List for lookups (O(n))
if timestamp in timestamp_list: if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls ### Minimize API Calls
**Already implemented:** **Already implemented:**
- Cache valid until midnight - Cache valid until midnight
- User data cached for 24h - User data cached for 24h
- Only poll when tomorrow data expected - Only poll when tomorrow data expected
**Monitor API usage:** **Monitor API usage:**
```python ```python
_LOGGER.debug("API call: %s (cache_age=%s)", _LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age) endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates ### Smart Updates
**Only update when needed:** **Only update when needed:**
```python ```python
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
"""Fetch data from API.""" """Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection ### State Class Selection
**Affects long-term statistics storage:** **Affects long-term statistics storage:**
```python ```python
# ❌ MEASUREMENT for prices (stores every change) # ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size ### Attribute Size
**Keep attributes minimal:** **Keep attributes minimal:**
```python ```python
# ❌ Large nested structures (KB per update) # ❌ Large nested structures (KB per update)
attributes = { attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
--- ---
💡 **Related:** 💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers - [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design - [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools - [Debugging](debugging.md) - Profiling tools

98
docs/developer/versioned_docs/version-v0.21.0/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic. **Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:** **Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point - `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering - `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy - `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max. **Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be within flex% ABOVE daily minimum # Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex) in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Flex: 15% - Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15) - Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better. **Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be at least min_distance% BELOW daily average # Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100)) meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Min Distance: 5% - Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95) - Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone - Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:** **Implementation:**
```python ```python
# Sensor classification uses user config # Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10) user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,21 +113,25 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5% #### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:** **Given:**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh - Daily Max: 20 ct/kWh
**Flex Filter (50%):** **Flex Filter (50%):**
``` ```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
``` ```
**Min Distance Filter (5%):** **Min Distance Filter (5%):**
``` ```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
``` ```
**Conflict:** **Conflict:**
- Interval at 14.8 ct/kWh: - Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS) - ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL) - ❌ Distance: 14.8 > 14.25 (FAIL)
@ -132,11 +142,13 @@ Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
### Mathematical Analysis ### Mathematical Analysis
**Conflict condition for Best Price:** **Conflict condition for Best Price:**
``` ```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100) daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
``` ```
**Typical values:** **Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5% - Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25` - Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%) - Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases. **Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:** **Formula:**
```python ```python
if flex > 0.20: # 20% threshold if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20 flex_excess = flex - 0.20
@ -159,7 +172,7 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):** **Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale | | Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------| | ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant | | ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction | | 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction | | 30% | 0.75 | 3.75% | Moderate reduction |
@ -167,6 +180,7 @@ if flex > 0.20: # 20% threshold
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides | | 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?** **Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average - Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval - Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period" - Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()` **Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:** **Code Extract:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
``` ```
**Why Linear Scaling?** **Why Linear Scaling?**
- Simple and predictable - Simple and predictable
- No abrupt behavior changes - No abrupt behavior changes
- Easy to reason about for users and developers - Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive) - Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?** **Why 25% Minimum?**
- Below this, min_distance loses semantic meaning - Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed - Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario - Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants ### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:** **Defined in `coordinator/period_handlers/core.py`:**
```python ```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
``` ```
**Defined in `const.py`:** **Defined in `const.py`:**
```python ```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled) DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection) DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances **Goal:** Find practical time windows for running appliances
**Constraints:** **Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h) - Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead) - Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average" - User wants genuinely cheap times, not just "slightly below average"
**Defaults:** **Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use - **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times - **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones - **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:** **User behavior:**
- Automations trigger actions (turn on devices) - Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money - Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives) - Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction **Goal:** Alert users to expensive periods for consumption reduction
**Constraints:** **Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding) - Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy - Early warning more valuable than perfect accuracy
- User can manually decide whether to react - User can manually decide whether to react
**Defaults:** **Defaults:**
- **30 min minimum** - Catches shorter expensive spikes - **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection - **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods - **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:** **User behavior:**
- Notifications/alerts (informational) - Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost - Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes) - Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:** **Peak Price Volatility:**
Price curves tend to have: Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening) - **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical) - **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times - **Higher variance** in peak times than cheap times
**Example day:** **Example day:**
``` ```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
``` ```
**Implication:** **Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief) - Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes - Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection - Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered #### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)** **Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex - Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes - Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?" - User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)** **Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices - No guidance on best practices
- Users would need to experiment to find good values - Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter - Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)** **Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options - **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price - **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case - Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%) **Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:** **Rationale:**
- Above 50%, period detection becomes unreliable - Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals) - Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50% - Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met) - **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:** **Warning Message:**
``` ```
Flex XX% exceeds maximum safe value! Capping at 50%. Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation. Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%) **Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:** **Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold - Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable" - At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation - **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended) #### With Relaxation Enabled (Recommended)
**Optimal:** 10-20% **Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ... - Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work - Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25% **Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side" - INFO log: "Base flex is on the high side"
**High Warning:** > 30% **High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!" - WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20% - Recommendation: Lower to 15-20%
#### Without Relaxation #### Without Relaxation
**Optimal:** 20-35% **Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start - No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback - Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50% **Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits) - Above this, period detection degrades (see Hard Limits)
--- ---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach ### Multi-Phase Approach
**Each day processed independently:** **Each day processed independently:**
1. Calculate baseline periods with user's config 1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop 2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations 3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
``` ```
**Constants:** **Constants:**
```python ```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20% FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -447,6 +488,7 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Historical Context (Pre-November 2025):** **Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex: The algorithm previously used percentage-based increments that scaled with base flex:
```python ```python
increment = base_flex × (step_pct / 100) # REMOVED increment = base_flex × (step_pct / 100) # REMOVED
``` ```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point. This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:** **Warning Messages:**
```python ```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30% if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning( _LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy ### Filter Combination Strategy
**Per Flex level, try in order:** **Per Flex level, try in order:**
1. Original Level filter 1. Original Level filter
2. Level filter = "any" (disabled) 2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations) **Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):** **Example Flow (target=2 periods/day):**
``` ```
Day 2025-11-19: Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2) 1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions ### Key Files and Functions
**Period Calculation Entry Point:** **Period Calculation Entry Point:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods( def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
``` ```
**Flex + Distance Filtering:** **Flex + Distance Filtering:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
def check_interval_criteria( def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
``` ```
**Relaxation Orchestration:** **Relaxation Orchestration:**
```python ```python
# coordinator/period_handlers/relaxation.py # coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict] def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -541,6 +589,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Rejects asymmetric outliers (threshold: 1.5 std dev) - Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks) - Preserves legitimate price shifts (morning/evening peaks)
- Algorithm: - Algorithm:
```python ```python
residual = abs(actual - predicted) residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev symmetry_threshold = 1.5 × std_dev
@ -563,6 +612,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes) - Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:** **Constants:**
```python ```python
# coordinator/period_handlers/outlier_filtering.py # coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
``` ```
**Data Integrity:** **Data Integrity:**
- Original prices stored in `_original_price` field - Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices - All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic - Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome - Smart counting: Only counts smoothing that changed period outcome
**Performance:** **Performance:**
- Single pass through price data - Single pass through price data
- O(n) complexity with small context window - O(n) complexity with small context window
- No iterative refinement needed - No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals - Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:** **Example Debug Output:**
``` ```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -624,6 +677,7 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips ## Debugging Tips
**Enable DEBUG logging:** **Enable DEBUG logging:**
```yaml ```yaml
# configuration.yaml # configuration.yaml
logger: logger:
@ -633,6 +687,7 @@ logger:
``` ```
**Key log messages to watch:** **Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion 1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering 2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded 3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,17 +700,20 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation ### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 40 best_price_flex: 40
enable_relaxation_best: true enable_relaxation_best: true
``` ```
**Problem:** **Problem:**
- Base Flex 40% already very permissive - Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...) - Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns - Quickly approaches 50% cap with diminishing returns
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 15 # Let relaxation increase it best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true enable_relaxation_best: true
@ -664,16 +722,19 @@ enable_relaxation_best: true
### ❌ Anti-Pattern 2: Zero Min_Distance ### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_min_distance_from_avg: 0 best_price_min_distance_from_avg: 0
``` ```
**Problem:** **Problem:**
- "Flat days" (little price variation) accept all intervals - "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap") - Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times - May create periods during barely-below-average times
**Solution:** **Solution:**
```yaml ```yaml
best_price_min_distance_from_avg: 5 # Use default 5% best_price_min_distance_from_avg: 5 # Use default 5%
``` ```
@ -681,16 +742,19 @@ best_price_min_distance_from_avg: 5 # Use default 5%
### ❌ Anti-Pattern 3: Conflicting Flex + Distance ### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 45 best_price_flex: 45
best_price_min_distance_from_avg: 10 best_price_min_distance_from_avg: 10
``` ```
**Problem:** **Problem:**
- Distance filter dominates, making Flex irrelevant - Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal - Dynamic scaling helps but still suboptimal
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 20 best_price_flex: 20
best_price_min_distance_from_avg: 5 best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods - Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient) - Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range - Min_Distance 5%: Effective throughout range
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners) - Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify - Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price" - Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh **Average:** 18 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct) - Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct) - Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive) - Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct) DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction) DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2 **Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2) DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day **Initial State:** Strict filters, very flat day
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2) DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation **Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:** **Algorithm:**
```python ```python
# Pseudo-code for adaptive flex # Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
``` ```
**Benefits:** **Benefits:**
- Eliminates need for relaxation on most days - Eliminates need for relaxation on most days
- Self-adjusting to market conditions - Self-adjusting to market conditions
- Better user experience (less configuration needed) - Better user experience (less configuration needed)
**Challenges:** **Challenges:**
- Harder to predict behavior (less transparent) - Harder to predict behavior (less transparent)
- May conflict with user's mental model - May conflict with user's mental model
- Needs extensive testing across different markets - Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback **Concept:** Learn optimal Flex/Distance from user feedback
**Approach:** **Approach:**
- Track which periods user actually uses (automation triggers) - Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal) - Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults - Apply pattern-specific defaults
- Learn per-user preferences over time - Learn per-user preferences over time
**Benefits:** **Benefits:**
- Personalized to user's actual behavior - Personalized to user's actual behavior
- Adapts to local market patterns - Adapts to local market patterns
- Could discover non-obvious patterns - Could discover non-obvious patterns
**Challenges:** **Challenges:**
- Requires user feedback mechanism (not implemented) - Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns) - Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?" - Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously **Concept:** Balance multiple goals simultaneously
**Goals:** **Goals:**
- Period count vs. quality (cheap vs. very cheap) - Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent) - Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered) - Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher) - User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:** **Algorithm:**
- Pareto optimization (find trade-off frontier) - Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences - User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing - Genetic algorithm or simulated annealing
**Benefits:** **Benefits:**
- More sophisticated period selection - More sophisticated period selection
- Better match to user's actual needs - Better match to user's actual needs
- Could handle complex appliance requirements - Could handle complex appliance requirements
**Challenges:** **Challenges:**
- Much more complex to implement - Much more complex to implement
- Harder to explain to users - Harder to explain to users
- Computational cost (may need caching) - Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex **Current:** 3% cap may be too aggressive for very low base Flex
**Example:** **Example:**
- Base flex 5% + 3% increment = 8% (60% increase!) - Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase) - Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:** **Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)` - Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%) - This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:** **Why Not Implemented:**
- Very low base flex (`<`10%) unusual - Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation - Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization - Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative **Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve **Alternative:** Non-linear curve
```python ```python
# Example: Exponential scaling # Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20)) scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
``` ```
**Why Not Implemented:** **Why Not Implemented:**
- Linear is easier to reason about - Linear is easier to reason about
- No evidence that non-linear is better - No evidence that non-linear is better
- Would need extensive testing - Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day **Issue:** May find all periods in one part of day
**Example:** **Example:**
- All 3 "best price" periods between 02:00-08:00 - All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances) - No periods in evening (when user might want to run appliances)
**Possible Solution:** **Possible Solution:**
- Add "spread" parameter (prefer distributed periods) - Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences - Weight periods by time-of-day preferences
- Consider user's typical usage patterns - Consider user's typical usage patterns
**Why Not Implemented:** **Why Not Implemented:**
- Adds complexity - Adds complexity
- Users can work around with multiple automations - Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all) - Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg). **Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:** **Implementation:**
```python ```python
# In period_building.py build_periods(): # In period_building.py build_periods():
for price_data in all_prices: for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight** **Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split: When days differ significantly, period can split:
``` ```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1) Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2) Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:** **Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours: Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium - Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer - Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,10 +1154,12 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):** **User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful: Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100) - `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre) - `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting: Automations can check volatility before acting:
```yaml ```yaml
condition: condition:
- condition: template - condition: template
@ -1095,6 +1190,7 @@ Low volatility (< 15%) means classification changes are less economically signif
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness. **Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:** **See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes" - User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`) - Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation) - Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

22
docs/developer/versioned_docs/version-v0.21.0/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
``` ```
**Key Points:** **Key Points:**
- Must be a **class attribute** (not instance attribute) - Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance - Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component - Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips` **Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that: **Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely - Never change or change very rarely
- Don't provide analytical value in history - Don't provide analytical value in history
- Consume significant database space when recorded every state change - Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures ### 2. Large Nested Structures
**Attributes:** **Attributes:**
- `periods` (binary_sensor) - Array of all period summaries - `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays - `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis - `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown - `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are: **Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive) - Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each) - Create large database rows (2-20 KB each)
- Slow down history queries - Slow down history queries
@ -66,6 +70,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors **Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:** **Example - periods array:**
```json ```json
{ {
"periods": [ "periods": [
@ -76,7 +81,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
"price_mean": 18.5, "price_mean": 18.5,
"price_median": 18.3, "price_median": 18.3,
"price_min": 17.2, "price_min": 17.2,
"price_max": 19.8, "price_max": 19.8
// ... 10+ more attributes × 10-20 periods // ... 10+ more attributes × 10-20 periods
} }
] ]
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status` **Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:** **Reason:**
- Change every update cycle (every 15 minutes or more frequently) - Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value - Don't provide long-term analytical value
- Create state changes even when core values haven't changed - Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max` **Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:** **Reason:**
- Configuration values that rarely change - Configuration values that rarely change
- Wastes space when recorded repeatedly - Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state - Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error` **Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:** **Reason:**
- Only relevant at moment of reading - Only relevant at moment of reading
- Won't be valid after some time - Won't be valid after some time
- Similar to `entity_picture` in HA core image entities - Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%` **Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:** **Reason:**
- Detailed technical information not needed for historical analysis - Detailed technical information not needed for historical analysis
- Only useful for debugging during active development - Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis - Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining` **Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:** **Reason:**
- Can be calculated from other attributes - Can be calculated from other attributes
- Redundant information - Redundant information
- Doesn't add analytical value to history - Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value: These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core ### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST) - `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states - All price values - Core sensor states
### Diagnostics & Tracking ### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time - `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns - `updates_today` - Tracking API usage patterns
### Data Completeness ### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics - `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status - `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data ### Period Data
- `start`, `end`, `duration_minutes` - Core period timing - `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics - `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status ### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation) - `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact ## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings ### Space Savings
**Per state change:** **Per state change:**
- Before: ~3-8 KB average - Before: ~3-8 KB average
- After: ~0.5-1.5 KB average - After: ~0.5-1.5 KB average
- **Reduction: 60-85%** - **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact ### Real-World Impact
For a typical installation with: For a typical installation with:
- 80+ sensors - 80+ sensors
- Updates every 15 minutes - Updates every 15 minutes
- ~10 sensors updating every minute - ~10 sensors updating every minute
@ -214,7 +231,7 @@ For a typical installation with:
- Class: `TibberPricesBinarySensor` - Class: `TibberPricesBinarySensor`
- 30 attributes excluded - 30 attributes excluded
## When to Update _unrecorded_attributes ## When to Update \_unrecorded_attributes
### Add to Exclusion List When: ### Add to Exclusion List When:
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes 4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:** **SQL Query to check attribute presence:**
```sql ```sql
SELECT SELECT
state_id, state_id,

14
docs/developer/versioned_docs/version-v0.21.0/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):** **In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built: git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile) - **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup` - **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):** **Manual installation (outside DevContainer):**
**git-cliff** (template-based): **git-cliff** (template-based):
```bash ```bash
# See: https://git-cliff.org/docs/installation # See: https://git-cliff.org/docs/installation
@ -191,7 +193,7 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which ## 🎯 When to Use Which
| Method | Use Case | Pros | Cons | | Method | Use Case | Pros | Cons |
|--------|----------|------|------| | --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script | | **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump | | **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization | | **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
@ -219,6 +221,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. Script bumps manifest.json → commits → creates tag locally 1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together 2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release 3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
``` ```
**What happens:** **What happens:**
1. You push manifest.json change 1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically 2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release 3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. You create and push tag manually 1. You create and push tag manually
2. Release workflow creates release 2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists) 3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features ## 🛡️ Safety Features
### 1. **Version Validation** ### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z). Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags** ### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote) - Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating - Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations** ### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push. Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered** ### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits. Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions** ### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind. Helper script shows how to undo if you change your mind.
--- ---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:** **Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes: Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely - Tag already exists remotely
- Invalid version format in manifest.json - Invalid version format in manifest.json
- manifest.json not in the commit that was pushed - manifest.json not in the commit that was pushed
@ -348,6 +359,7 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips ## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results: 1. **Conventional Commits:** Use proper commit format for best results:
``` ```
feat(scope): Add new feature feat(scope): Add new feature

22
docs/developer/versioned_docs/version-v0.21.0/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle. The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:** **Design Principles:**
- **Proactive**: Detect issues before they become critical - **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance - **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve - **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}` **Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:** **When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`) - Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available - Tomorrow's electricity price data is still not available
**When cleared:** **When cleared:**
- Tomorrow's data becomes available - Tomorrow's data becomes available
- Automatically checks on every successful API update - Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work. Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:** **Implementation:**
```python ```python
# In coordinator update cycle # In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"]) has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18) - `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}` **Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:** **When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429) - Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD` - Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:** **When cleared:**
- Successful API call completes (no rate limit error) - Successful API call completes (no rate limit error)
- Error counter resets to 0 - Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires. API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:** **Implementation:**
```python ```python
# In error handler # In error handler
is_rate_limit = ( is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors - `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}` **Issue ID:** `home_not_found_{entry_id}`
**When triggered:** **When triggered:**
- Home configured in this integration is no longer present in Tibber account - Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check) - Detected during user data refresh (daily check)
**When cleared:** **When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected) - Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup) - Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed. Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:** **Implementation:**
```python ```python
# After user data update # After user data update
home_exists = self._data_fetcher._check_home_exists(home_id) home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the missing home - `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference - `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration ### Lifecycle Integration
**Coordinator Initialization:** **Coordinator Initialization:**
```python ```python
self._repair_manager = TibberPricesRepairManager( self._repair_manager = TibberPricesRepairManager(
hass=hass, hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
``` ```
**Update Cycle Integration:** **Update Cycle Integration:**
```python ```python
# Success path - check conditions # Success path - check conditions
if result and "priceInfo" in result: if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
``` ```
**Shutdown Cleanup:** **Shutdown Cleanup:**
```python ```python
async def async_shutdown(self) -> None: async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up.""" """Shut down coordinator and clean up."""
@ -196,6 +212,7 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json` - `/translations/sv.json`
**Structure:** **Structure:**
```json ```json
{ {
"issues": { "issues": {
@ -210,10 +227,12 @@ Repairs use Home Assistant's standard translation system. Translations are defin
## Home Assistant Integration ## Home Assistant Integration
Repairs appear in: Repairs appear in:
- **Settings → System → Repairs** (main repairs panel) - **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count) - **Notifications** (bell icon in UI shows repair count)
Repair properties: Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required) - **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical) - **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files - **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears 4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:** **Manual trigger:**
```python ```python
# Temporarily set warning hour to current hour for testing # Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair 3. Successful API call clears the repair
**Manual test:** **Manual test:**
- Reduce API polling interval to trigger rate limiting - Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client - Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file 7. **Document** in this file
**Example template:** **Example template:**
```python ```python
async def check_new_condition(self, *, param: bool) -> None: async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair.""" """Check new condition and create/clear repair."""

24
docs/developer/versioned_docs/version-v0.21.0/timer-architecture.md
View file

@ -11,7 +11,7 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes: The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method | | Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------| | ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` | | **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` | | **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` | | **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes` **Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:** **What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator` - HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes - Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time) - **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
``` ```
**Load Distribution:** **Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution - Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API - Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once - Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:** **Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects) - Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early - If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully - Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:** **Why we use HA's timer:**
- Automatic restart after HA restart - Automatic restart after HA restart
- Built-in retry logic for temporary failures - Built-in retry logic for temporary failures
- Standard HA integration pattern - Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll** **Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:** **Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48) - Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes - Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13 - Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
``` ```
**Smart Boundary Tolerance:** **Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance - Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval) - HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval) - HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details - See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:** **Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...) - `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes") - NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates) - If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:** **Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`) - All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`) - Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total - ~50-60 entities out of 120+ total
**Why custom timer:** **Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing - HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals - We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation - Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
``` ```
**Which entities listen:** **Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer - `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer - `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%) - `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total - ~10 entities total
**Why custom timer:** **Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time) - Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates - Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation) - Very lightweight (no data processing, just state recalculation)
**Why NOT every second:** **Why NOT every second:**
- Minute precision sufficient for countdown UX - Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds) - Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates) - Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
``` ```
**Why this pattern:** **Why this pattern:**
- Decouples timer logic from entity logic - Decouples timer logic from entity logic
- One timer can notify many entities efficiently - One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup) - Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API ### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers: If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload - ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload - ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem - ❌ "Thundering herd" problem
With HA's unsynchronized timer: With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ... - ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ... - ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ... - ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data. **Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:** **API fetch only when:**
- Tomorrow data missing/invalid (after 13:00) - Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover) - Cache expired (midnight turnover)
- Explicit user refresh - Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics ## Performance Characteristics
### Timer #1 (DataUpdateCoordinator) ### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized) - **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data) - **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate) - **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise) - **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh) ### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries) - **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities) - **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data - **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates - **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh) ### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute) - **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities) - **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all - **No API calls:** No data processing at all
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary ## Summary
**Three independent timers:** **Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed) 1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always) 2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always) 3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:** **Key insights:**
- Timer #1 unsynchronized = good (load distribution on API) - Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately) - Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX) - Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts) - All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:** **"Listener" terminology:**
- Timer = mechanism that triggers - Timer = mechanism that triggers
- Listener = callback that gets called - Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies - Observer pattern = entities register, coordinator notifies

31
docs/developer/versioned_docs/version-v0.22.0/api-reference.md
View file

@ -56,7 +56,7 @@ query {
Fetches quarter-hourly prices: Fetches quarter-hourly prices:
```graphql ```graphql
query($homeId: ID!) { query ($homeId: ID!) {
viewer { viewer {
home(id: $homeId) { home(id: $homeId) {
currentSubscription { currentSubscription {
@ -76,6 +76,7 @@ query($homeId: ID!) {
``` ```
**Parameters:** **Parameters:**
- `homeId`: Tibber home identifier - `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY` - `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data) - `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits ## Rate Limits
Tibber API rate limits (as of 2024): Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token - **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute - **Burst limit:** 100 requests per minute
Integration stays well below these limits: Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day - Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day - User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home - **Total:** ~100 requests/day per home
@ -106,6 +109,7 @@ Integration stays well below these limits:
``` ```
**Fields:** **Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR) - `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone - `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE) - `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -119,6 +123,7 @@ Integration stays well below these limits:
``` ```
Supported currencies: Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh - `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh - `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh - `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses ### Common Error Responses
**Invalid Token:** **Invalid Token:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Unauthorized", "message": "Unauthorized",
"extensions": { "extensions": {
"code": "UNAUTHENTICATED" "code": "UNAUTHENTICATED"
} }
}] }
]
} }
``` ```
**Rate Limit Exceeded:** **Rate Limit Exceeded:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Too Many Requests", "message": "Too Many Requests",
"extensions": { "extensions": {
"code": "RATE_LIMIT_EXCEEDED" "code": "RATE_LIMIT_EXCEEDED"
} }
}] }
]
} }
``` ```
**Home Not Found:** **Home Not Found:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Home not found", "message": "Home not found",
"extensions": { "extensions": {
"code": "NOT_FOUND" "code": "NOT_FOUND"
} }
}] }
]
} }
``` ```
Integration handles these with: Integration handles these with:
- Exponential backoff retry (3 attempts) - Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors - ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures - ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation ## Data Transformation
Raw API data is enriched with: Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals - **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals - **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average - **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
--- ---
💡 **External Resources:** 💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview) - [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer) - [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token) - [Get API Token](https://developer.tibber.com/settings/access-token)

17
docs/developer/versioned_docs/version-v0.22.0/architecture.md
View file

@ -147,7 +147,7 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance: The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory | | Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------| | ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB | | **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB | | **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB | | **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
@ -196,7 +196,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components ### Core Components
| Component | File | Responsibility | | Component | File | Responsibility |
|-----------|------|----------------| | --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling | | **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance | | **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) | | **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
@ -210,7 +210,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025): The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility | | Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------| | ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators | | **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) | | **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) | | **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
@ -219,6 +219,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities | | **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`): **Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access - `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous) - `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows - `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata - `metadata.py` - Home/metering metadata
**Benefits:** **Benefits:**
- 58% reduction in core.py (2,170 → 909 lines) - 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation) - Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator - Independent testability for each calculator
@ -238,7 +240,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities ### Helper Utilities
| Utility | File | Purpose | | Utility | File | Purpose |
|---------|------|---------| | ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation | | **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations | | **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic | | **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025): Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`): **Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals - `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows - `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg - `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics - `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`): **Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods - Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.) - Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`): **Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access - Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc. - Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching) - Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`): **Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation - Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes - Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()` - Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:** **Benefits:**
- Minimal code duplication across 80+ sensors - Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation) - Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing - Easy to extend: Add sensor → choose pattern → add to routing
@ -334,7 +341,7 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization ### CPU Optimization
| Optimization | Location | Savings | | Optimization | Location | Savings |
|--------------|----------|---------| | ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks | | Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation | | Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations | | Lazy logging | Throughout | ~15% on log-heavy operations |

42
docs/developer/versioned_docs/version-v0.22.0/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts. **Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:** **What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total) - **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query - **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation - **Timestamps**: Last update times for validation
**Lifetime:** **Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time) - **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily) - **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage - **Survives**: HA restarts via persistent Storage
@ -36,6 +38,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:** **Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator): 1. **Midnight turnover** (Timer #2 in coordinator):
```python ```python
# coordinator/day_transitions.py # coordinator/day_transitions.py
def _handle_midnight_turnover() -> None: def _handle_midnight_turnover() -> None:
@ -45,6 +48,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
``` ```
2. **Cache validation on load**: 2. **Cache validation on load**:
```python ```python
# coordinator/cache.py # coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool: def is_cache_valid(cache_data: CacheData) -> bool:
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc. **Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:** **What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names - **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions - **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:** **Lifetime:**
- **Forever** (until HA restart) - **Forever** (until HA restart)
- No invalidation during runtime - No invalidation during runtime
**When populated:** **When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py` - At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once - Lazy loading: If translation missing, attempts file load once
**Access pattern:** **Access pattern:**
```python ```python
# Non-blocking synchronous access from cached data # Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en") description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:** **What is cached:**
### DataTransformer Config Cache ### DataTransformer Config Cache
```python ```python
{ {
"thresholds": {"low": 15, "high": 35}, "thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
### PeriodCalculator Config Cache ### PeriodCalculator Config Cache
```python ```python
{ {
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60}, "best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,10 +128,12 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Lifetime:** **Lifetime:**
- Until `invalidate_config_cache()` is called - Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle - Built once on first use per coordinator update cycle
**Invalidation trigger:** **Invalidation trigger:**
- **Options change** (user reconfigures integration): - **Options change** (user reconfigures integration):
```python ```python
# coordinator/core.py # coordinator/core.py
@ -132,6 +144,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Performance impact:** **Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs - **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs - **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update) - **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed. **Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:** **What is cached:**
```python ```python
{ {
"best_price": { "best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Cache key:** Hash of relevant inputs **Cache key:** Hash of relevant inputs
```python ```python
hash_data = ( hash_data = (
today_signature, # (startsAt, rating_level) for each interval today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
``` ```
**Lifetime:** **Lifetime:**
- Until price data changes (today's intervals modified) - Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters) - Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data) - Recalculated at midnight (new today data)
@ -179,6 +195,7 @@ hash_data = (
**Invalidation triggers:** **Invalidation triggers:**
1. **Config change** (explicit): 1. **Config change** (explicit):
```python ```python
def invalidate_config_cache() -> None: def invalidate_config_cache() -> None:
self._cached_periods = None self._cached_periods = None
@ -193,10 +210,12 @@ hash_data = (
``` ```
**Cache hit rate:** **Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged) - **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00) - **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:** **Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts) - **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup) - **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache) - **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy **Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:** **What is cached:**
```python ```python
{ {
"timestamp": ..., "timestamp": ...,
@ -224,6 +244,7 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks. **Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:** **Current behavior:**
- Caches **only enriched price data** (price + statistics) - Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache) - **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when: - Invalidated when:
@ -232,6 +253,7 @@ hash_data = (
- New update cycle begins - New update cycle begins
**Architecture:** **Architecture:**
- DataTransformer: Handles price enrichment only - DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache) - PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches - Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow ## Cache Invalidation Flow
### User Changes Options (Config Flow) ### User Changes Options (Config Flow)
``` ```
User saves options User saves options
↓ ↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
``` ```
### Midnight Turnover (Day Transition) ### Midnight Turnover (Day Transition)
``` ```
Timer #2 fires at 00:00 Timer #2 fires at 00:00
↓ ↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
``` ```
### Tomorrow Data Arrives (~13:00) ### Tomorrow Data Arrives (~13:00)
``` ```
Coordinator update cycle Coordinator update cycle
↓ ↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
``` ```
**No cache invalidation cascades:** **No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update) - Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch) - Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change) - Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load) - Translation cache is **never invalidated** (read-only after load)
**Thread safety:** **Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop) - All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model) - No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics ## Performance Characteristics
### Typical Operation (No Changes) ### Typical Operation (No Changes)
``` ```
Coordinator Update (every 15 min) Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid) ├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
``` ```
### After Midnight Turnover ### After Midnight Turnover
``` ```
Coordinator Update (00:00) Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day) ├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
``` ```
### After Config Change ### After Config Change
``` ```
Options Update Options Update
├─> Cache invalidation: `<`1ms ├─> Cache invalidation: `<`1ms
@ -382,7 +412,7 @@ Options Update
## Summary Table ## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose | | Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------| | ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls | | **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O | | **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups | | **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
@ -392,12 +422,14 @@ Options Update
**Total memory overhead:** ~116KB per coordinator instance (main + subentries) **Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:** **Benefits:**
- 97% reduction in API calls (from every 15min to once per day) - 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation) - 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check) - 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup) - Zero file I/O during runtime (translations cached at startup)
**Trade-offs:** **Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems) - Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented) - Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues - Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues ## Debugging Cache Issues
### Symptom: Stale data after config change ### Symptom: Stale data after config change
**Check:** **Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log) 1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed? 2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger? 3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init. **Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating ### Symptom: Period calculation not updating
**Check:** **Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()` 1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash` 2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs 3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs. **Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today ### Symptom: Yesterday's prices shown as today
**Check:** **Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py` 1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2) 2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs 3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration. **Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations ### Symptom: Missing translations
**Check:** **Check:**
1. `async_load_translations()` called at startup? 1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`? 2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys 3. Cache population: `_TRANSLATIONS_CACHE` keys

3
docs/developer/versioned_docs/version-v0.22.0/coding-guidelines.md
View file

@ -41,12 +41,14 @@ class TimeService:
``` ```
**When prefix is required:** **When prefix is required:**
- Public classes used across multiple modules - Public classes used across multiple modules
- All exception classes - All exception classes
- All coordinator and entity classes - All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs - Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:** **When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore) - Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`) - Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns - Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:** **Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring: Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md` 1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames 2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module 3. Test thoroughly after each module

17
docs/developer/versioned_docs/version-v0.22.0/contributing.md
View file

@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
``` ```
**Branch naming:** **Branch naming:**
- `feature/` - New features - `feature/` - New features
- `fix/` - Bug fixes - `fix/` - Bug fixes
- `docs/` - Documentation only - `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md). Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:** **Run checks frequently:**
```bash ```bash
./scripts/type-check # Pyright type checking ./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix) ./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
``` ```
Run your test: Run your test:
```bash ```bash
./scripts/test tests/test_your_feature.py -v ./scripts/test tests/test_your_feature.py -v
``` ```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
``` ```
**Commit types:** **Commit types:**
- `feat:` - New feature - `feat:` - New feature
- `fix:` - Bug fix - `fix:` - Bug fix
- `docs:` - Documentation - `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance - `chore:` - Maintenance
**Add scope when relevant:** **Add scope when relevant:**
- `feat(sensors):` - Sensor platform - `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator - `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation - `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max) Title: Short, descriptive (50 chars max)
Description should include: Description should include:
```markdown ```markdown
## What ## What
Brief description of changes Brief description of changes
## Why ## Why
Problem being solved or feature rationale Problem being solved or feature rationale
## How ## How
Implementation approach Implementation approach
## Testing ## Testing
- [ ] Manual testing in Home Assistant - [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated - [ ] Unit tests added/updated
- [ ] Type checking passes - [ ] Type checking passes
- [ ] Linting passes - [ ] Linting passes
## Breaking Changes ## Breaking Changes
(If any - describe migration path) (If any - describe migration path)
## Related Issues ## Related Issues
Closes #123 Closes #123
``` ```
### PR Checklist ### PR Checklist
Before submitting: Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md) - [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`) - [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`) - [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For ### What Reviewers Look For
✅ **Good:** ✅ **Good:**
- Clear, self-explanatory code - Clear, self-explanatory code
- Appropriate comments for complex logic - Appropriate comments for complex logic
- Tests covering edge cases - Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns - Follows existing patterns
❌ **Avoid:** ❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones - Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes - Mixing unrelated changes
- Missing tests for new features - Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On ## Finding Issues to Work On
Good first issues are labeled: Good first issues are labeled:
- `good first issue` - Beginner-friendly - `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions - `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements - `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
--- ---
💡 **Related:** 💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup - [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide - [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests - [Testing](testing.md) - Writing tests

33
docs/developer/versioned_docs/version-v0.22.0/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical? ## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to: Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable - **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases - **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks - **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅ #### 1.1 Listener Cleanup ✅
**What is tested:** **What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`) - Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`) - Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`) - Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners - Binary sensor cleanup removes ALL registered listeners
**Why critical:** **Why critical:**
- Each registered listener holds references to Entity + Coordinator - Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak - Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed - With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()` - `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()` - `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()` - `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅ #### 1.2 Timer Cleanup ✅
**What is tested:** **What is tested:**
- Quarter-hour timer is cancelled and reference cleared - Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared - Minute timer is cancelled and reference cleared
- Both timers are cancelled together - Both timers are cancelled together
- Cleanup works even when timers are `None` - Cleanup works even when timers are `None`
**Why critical:** **Why critical:**
- Uncancelled timers continue running after integration unload - Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks - HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates - Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()` - `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅ #### 1.3 Config Entry Cleanup ✅
**What is tested:** **What is tested:**
- Options update listener is registered via `async_on_unload()` - Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()` - Cleanup function is correctly passed to `async_on_unload()`
**Why critical:** **Why critical:**
- `entry.add_update_listener()` registers permanent callback - `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates - Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))` - Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:** **Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration) - `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()` - `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation #### 2.1 Config Cache Invalidation
**What is tested:** **What is tested:**
- DataTransformer config cache is invalidated on options change - DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated - PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update - Trend calculator cache is cleared on coordinator update
**Why critical:** **Why critical:**
- Stale config → Sensors use old user settings - Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods - Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis - Stale trend cache → Outdated trend analysis
**Code Locations:** **Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()` - `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()` - `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()` - `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal #### 3.1 Persistent Storage Removal
**What is tested:** **What is tested:**
- Storage file is deleted on config entry removal - Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss) - Cache is saved on shutdown (no data loss)
**Why critical:** **Why critical:**
- Without storage removal: Old files remain after uninstallation - Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart - Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}` - Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:** **Code Locations:**
- `__init__.py` → `async_remove_entry()` - `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py` **File:** `tests/test_timer_scheduling.py`
**What is tested:** **What is tested:**
- Quarter-hour timer is registered with correct parameters - Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters - Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer) - Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly - Midnight turnover detection works correctly
**Why critical:** **Why critical:**
- Wrong timer parameters → Entities update at wrong times - Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem - Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py` **File:** `tests/test_sensor_timer_assignment.py`
**What is tested:** **What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys - All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys - All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap) - Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked - Sensor and binary sensor platforms are checked
**Why critical:** **Why critical:**
- Wrong timer assignment → Sensors update at wrong times - Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem - Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management ### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks **Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds) - `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms) - `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:** **Why no tests needed:**
- No long-running tasks (all < 2 seconds) - No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically - HA's event loop handles short tasks automatically
- Task exceptions are already logged - Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup ### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented **Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session) - `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created - No new sessions are created
- HA manages session lifecycle automatically - HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory ### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache **Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total - Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading - Module-level cache without re-loading
- Practically no memory issue - Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity ### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop` **Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days) - Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults - Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors - 80+ sensors access `coordinator.data` without errors
**Structure:** **Structure:**
```python ```python
coordinator.data = { coordinator.data = {
"user_data": {...}, "user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory ### 10. Service Response Memory
**Current Status:** HA's response lifecycle **Current Status:** HA's response lifecycle
- HA automatically frees service responses after return - HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call - ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code - No response accumulation in integration code
@ -208,7 +235,7 @@ coordinator.data = {
### ✅ Implemented Tests (41 total) ### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage | | Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------| | ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% | | Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% | | Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% | | Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
@ -222,7 +249,7 @@ coordinator.data = {
### 📋 Analyzed but Not Implemented (Nice-to-Have) ### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale | | Category | Status | Rationale |
|----------|--------|-----------| | ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) | | Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) | | API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) | | Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
@ -230,6 +257,7 @@ coordinator.data = {
| Service Response Memory | 📋 | HA automatically frees service responses | | Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:** **Legend:**
- ✅ = Fully tested or pattern verified correct - ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues) - 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested ### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests: All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks) - ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks) - ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners) - ✅ Config entry cleanup works (no dangling listeners)

18
docs/developer/versioned_docs/version-v0.22.0/debugging.md
View file

@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages ### Key Log Messages
**Coordinator Updates:** **Coordinator Updates:**
``` ```
[custom_components.tibber_prices.coordinator] Successfully fetched price data [custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data [custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
``` ```
**Period Calculation:** **Period Calculation:**
``` ```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0% [custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods [custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
``` ```
**API Errors:** **API Errors:**
``` ```
[custom_components.tibber_prices.api] API request failed: Unauthorized [custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s [custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -67,6 +70,7 @@ Restart Home Assistant to apply.
### Set Breakpoints ### Set Breakpoints
**Coordinator update:** **Coordinator update:**
```python ```python
# coordinator/core.py # coordinator/core.py
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
``` ```
**Period calculation:** **Period calculation:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]: def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
``` ```
**Flags:** **Flags:**
- `-v` - Verbose output - `-v` - Verbose output
- `-s` - Show print statements - `-s` - Show print statements
- `-k pattern` - Run tests matching pattern - `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns ### Useful Test Patterns
**Print coordinator data:** **Print coordinator data:**
```python ```python
def test_something(coordinator): def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}") print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
``` ```
**Inspect period attributes:** **Inspect period attributes:**
```python ```python
def test_periods(hass, coordinator): def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', []) periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading ### Integration Not Loading
**Check:** **Check:**
```bash ```bash
grep "tibber_prices" config/home-assistant.log grep "tibber_prices" config/home-assistant.log
``` ```
**Common causes:** **Common causes:**
- Syntax error in Python code → Check logs for traceback - Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync` - Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*` - Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating ### Sensors Not Updating
**Check coordinator state:** **Check coordinator state:**
```python ```python
# In Developer Tools > Template # In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }} {{ states.sensor.tibber_home_current_interval_price.last_updated }}
``` ```
**Debug in code:** **Debug in code:**
```python ```python
# Add logging in sensor/core.py # Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s", _LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong ### Period Calculation Wrong
**Enable detailed period logs:** **Enable detailed period logs:**
```python ```python
# coordinator/period_handlers/period_building.py # coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s", _LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
``` ```
**Check filter statistics:** **Check filter statistics:**
``` ```
[period_building] Flex filter blocked: 45 intervals [period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals [period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy ### Remote Debugging with debugpy
Add to coordinator code: Add to coordinator code:
```python ```python
import debugpy import debugpy
debugpy.listen(5678) debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL ### IPython REPL
Install in container: Install in container:
```bash ```bash
uv pip install ipython uv pip install ipython
``` ```
Add breakpoint: Add breakpoint:
```python ```python
from IPython import embed from IPython import embed
embed() # Drops into interactive shell embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
--- ---
💡 **Related:** 💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests - [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment - [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure - [Architecture](architecture.md) - Code structure

1
docs/developer/versioned_docs/version-v0.22.0/intro.md
View file

@ -168,6 +168,7 @@ Documentation is organized in two Docusaurus sites:
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory) - **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:** **Best practices:**
- Use clear examples and code snippets - Use clear examples and code snippets
- Keep docs up-to-date with code changes - Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation - Add new pages to appropriate `sidebars.ts` for navigation

17
docs/developer/versioned_docs/version-v0.22.0/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals ## Performance Goals
Target metrics: Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms) - **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor - **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms) - **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching ### Caching
**1. Persistent Cache** (API data): **1. Persistent Cache** (API data):
```python ```python
# Already implemented in coordinator/cache.py # Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY) store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
``` ```
**2. Translation Cache** (in-memory): **2. Translation Cache** (in-memory):
```python ```python
# Already implemented in const.py # Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {} _TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
``` ```
**3. Config Cache** (invalidated on options change): **3. Config Cache** (invalidated on options change):
```python ```python
class DataTransformer: class DataTransformer:
def __init__(self): def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading ### Lazy Loading
**Load data only when needed:** **Load data only when needed:**
```python ```python
@property @property
def extra_state_attributes(self) -> dict | None: def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations ### Bulk Operations
**Process multiple items at once:** **Process multiple items at once:**
```python ```python
# ❌ Slow - loop with individual operations # ❌ Slow - loop with individual operations
for interval in intervals: for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices ### Async Best Practices
**1. Concurrent API calls:** **1. Concurrent API calls:**
```python ```python
# ❌ Sequential (slow) # ❌ Sequential (slow)
user_data = await fetch_user_data() user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
``` ```
**2. Don't block event loop:** **2. Don't block event loop:**
```python ```python
# ❌ Blocking # ❌ Blocking
result = heavy_computation() # Blocks for seconds result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks ### Avoid Memory Leaks
**1. Clear references:** **1. Clear references:**
```python ```python
class Coordinator: class Coordinator:
async def async_shutdown(self): async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
``` ```
**2. Use weak references for callbacks:** **2. Use weak references for callbacks:**
```python ```python
import weakref import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures ### Efficient Data Structures
**Use appropriate types:** **Use appropriate types:**
```python ```python
# ❌ List for lookups (O(n)) # ❌ List for lookups (O(n))
if timestamp in timestamp_list: if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls ### Minimize API Calls
**Already implemented:** **Already implemented:**
- Cache valid until midnight - Cache valid until midnight
- User data cached for 24h - User data cached for 24h
- Only poll when tomorrow data expected - Only poll when tomorrow data expected
**Monitor API usage:** **Monitor API usage:**
```python ```python
_LOGGER.debug("API call: %s (cache_age=%s)", _LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age) endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates ### Smart Updates
**Only update when needed:** **Only update when needed:**
```python ```python
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
"""Fetch data from API.""" """Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection ### State Class Selection
**Affects long-term statistics storage:** **Affects long-term statistics storage:**
```python ```python
# ❌ MEASUREMENT for prices (stores every change) # ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size ### Attribute Size
**Keep attributes minimal:** **Keep attributes minimal:**
```python ```python
# ❌ Large nested structures (KB per update) # ❌ Large nested structures (KB per update)
attributes = { attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
--- ---
💡 **Related:** 💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers - [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design - [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools - [Debugging](debugging.md) - Profiling tools

98
docs/developer/versioned_docs/version-v0.22.0/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic. **Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:** **Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point - `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering - `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy - `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max. **Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be within flex% ABOVE daily minimum # Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex) in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Flex: 15% - Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15) - Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better. **Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be at least min_distance% BELOW daily average # Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100)) meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Min Distance: 5% - Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95) - Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone - Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:** **Implementation:**
```python ```python
# Sensor classification uses user config # Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10) user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,21 +113,25 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5% #### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:** **Given:**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh - Daily Max: 20 ct/kWh
**Flex Filter (50%):** **Flex Filter (50%):**
``` ```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
``` ```
**Min Distance Filter (5%):** **Min Distance Filter (5%):**
``` ```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
``` ```
**Conflict:** **Conflict:**
- Interval at 14.8 ct/kWh: - Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS) - ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL) - ❌ Distance: 14.8 > 14.25 (FAIL)
@ -132,11 +142,13 @@ Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
### Mathematical Analysis ### Mathematical Analysis
**Conflict condition for Best Price:** **Conflict condition for Best Price:**
``` ```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100) daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
``` ```
**Typical values:** **Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5% - Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25` - Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%) - Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases. **Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:** **Formula:**
```python ```python
if flex > 0.20: # 20% threshold if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20 flex_excess = flex - 0.20
@ -159,7 +172,7 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):** **Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale | | Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------| | ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant | | ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction | | 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction | | 30% | 0.75 | 3.75% | Moderate reduction |
@ -167,6 +180,7 @@ if flex > 0.20: # 20% threshold
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides | | 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?** **Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average - Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval - Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period" - Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()` **Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:** **Code Extract:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
``` ```
**Why Linear Scaling?** **Why Linear Scaling?**
- Simple and predictable - Simple and predictable
- No abrupt behavior changes - No abrupt behavior changes
- Easy to reason about for users and developers - Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive) - Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?** **Why 25% Minimum?**
- Below this, min_distance loses semantic meaning - Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed - Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario - Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants ### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:** **Defined in `coordinator/period_handlers/core.py`:**
```python ```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
``` ```
**Defined in `const.py`:** **Defined in `const.py`:**
```python ```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled) DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection) DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances **Goal:** Find practical time windows for running appliances
**Constraints:** **Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h) - Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead) - Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average" - User wants genuinely cheap times, not just "slightly below average"
**Defaults:** **Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use - **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times - **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones - **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:** **User behavior:**
- Automations trigger actions (turn on devices) - Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money - Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives) - Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction **Goal:** Alert users to expensive periods for consumption reduction
**Constraints:** **Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding) - Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy - Early warning more valuable than perfect accuracy
- User can manually decide whether to react - User can manually decide whether to react
**Defaults:** **Defaults:**
- **30 min minimum** - Catches shorter expensive spikes - **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection - **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods - **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:** **User behavior:**
- Notifications/alerts (informational) - Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost - Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes) - Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:** **Peak Price Volatility:**
Price curves tend to have: Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening) - **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical) - **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times - **Higher variance** in peak times than cheap times
**Example day:** **Example day:**
``` ```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
``` ```
**Implication:** **Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief) - Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes - Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection - Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered #### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)** **Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex - Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes - Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?" - User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)** **Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices - No guidance on best practices
- Users would need to experiment to find good values - Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter - Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)** **Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options - **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price - **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case - Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%) **Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:** **Rationale:**
- Above 50%, period detection becomes unreliable - Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals) - Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50% - Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met) - **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:** **Warning Message:**
``` ```
Flex XX% exceeds maximum safe value! Capping at 50%. Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation. Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%) **Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:** **Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold - Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable" - At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation - **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended) #### With Relaxation Enabled (Recommended)
**Optimal:** 10-20% **Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ... - Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work - Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25% **Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side" - INFO log: "Base flex is on the high side"
**High Warning:** > 30% **High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!" - WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20% - Recommendation: Lower to 15-20%
#### Without Relaxation #### Without Relaxation
**Optimal:** 20-35% **Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start - No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback - Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50% **Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits) - Above this, period detection degrades (see Hard Limits)
--- ---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach ### Multi-Phase Approach
**Each day processed independently:** **Each day processed independently:**
1. Calculate baseline periods with user's config 1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop 2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations 3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
``` ```
**Constants:** **Constants:**
```python ```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20% FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -447,6 +488,7 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Historical Context (Pre-November 2025):** **Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex: The algorithm previously used percentage-based increments that scaled with base flex:
```python ```python
increment = base_flex × (step_pct / 100) # REMOVED increment = base_flex × (step_pct / 100) # REMOVED
``` ```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point. This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:** **Warning Messages:**
```python ```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30% if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning( _LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy ### Filter Combination Strategy
**Per Flex level, try in order:** **Per Flex level, try in order:**
1. Original Level filter 1. Original Level filter
2. Level filter = "any" (disabled) 2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations) **Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):** **Example Flow (target=2 periods/day):**
``` ```
Day 2025-11-19: Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2) 1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions ### Key Files and Functions
**Period Calculation Entry Point:** **Period Calculation Entry Point:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods( def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
``` ```
**Flex + Distance Filtering:** **Flex + Distance Filtering:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
def check_interval_criteria( def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
``` ```
**Relaxation Orchestration:** **Relaxation Orchestration:**
```python ```python
# coordinator/period_handlers/relaxation.py # coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict] def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -541,6 +589,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Rejects asymmetric outliers (threshold: 1.5 std dev) - Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks) - Preserves legitimate price shifts (morning/evening peaks)
- Algorithm: - Algorithm:
```python ```python
residual = abs(actual - predicted) residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev symmetry_threshold = 1.5 × std_dev
@ -563,6 +612,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes) - Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:** **Constants:**
```python ```python
# coordinator/period_handlers/outlier_filtering.py # coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
``` ```
**Data Integrity:** **Data Integrity:**
- Original prices stored in `_original_price` field - Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices - All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic - Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome - Smart counting: Only counts smoothing that changed period outcome
**Performance:** **Performance:**
- Single pass through price data - Single pass through price data
- O(n) complexity with small context window - O(n) complexity with small context window
- No iterative refinement needed - No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals - Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:** **Example Debug Output:**
``` ```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -624,6 +677,7 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips ## Debugging Tips
**Enable DEBUG logging:** **Enable DEBUG logging:**
```yaml ```yaml
# configuration.yaml # configuration.yaml
logger: logger:
@ -633,6 +687,7 @@ logger:
``` ```
**Key log messages to watch:** **Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion 1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering 2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded 3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,17 +700,20 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation ### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 40 best_price_flex: 40
enable_relaxation_best: true enable_relaxation_best: true
``` ```
**Problem:** **Problem:**
- Base Flex 40% already very permissive - Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...) - Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns - Quickly approaches 50% cap with diminishing returns
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 15 # Let relaxation increase it best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true enable_relaxation_best: true
@ -664,16 +722,19 @@ enable_relaxation_best: true
### ❌ Anti-Pattern 2: Zero Min_Distance ### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_min_distance_from_avg: 0 best_price_min_distance_from_avg: 0
``` ```
**Problem:** **Problem:**
- "Flat days" (little price variation) accept all intervals - "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap") - Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times - May create periods during barely-below-average times
**Solution:** **Solution:**
```yaml ```yaml
best_price_min_distance_from_avg: 5 # Use default 5% best_price_min_distance_from_avg: 5 # Use default 5%
``` ```
@ -681,16 +742,19 @@ best_price_min_distance_from_avg: 5 # Use default 5%
### ❌ Anti-Pattern 3: Conflicting Flex + Distance ### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 45 best_price_flex: 45
best_price_min_distance_from_avg: 10 best_price_min_distance_from_avg: 10
``` ```
**Problem:** **Problem:**
- Distance filter dominates, making Flex irrelevant - Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal - Dynamic scaling helps but still suboptimal
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 20 best_price_flex: 20
best_price_min_distance_from_avg: 5 best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods - Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient) - Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range - Min_Distance 5%: Effective throughout range
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners) - Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify - Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price" - Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh **Average:** 18 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct) - Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct) - Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive) - Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct) DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction) DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2 **Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2) DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day **Initial State:** Strict filters, very flat day
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2) DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation **Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:** **Algorithm:**
```python ```python
# Pseudo-code for adaptive flex # Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
``` ```
**Benefits:** **Benefits:**
- Eliminates need for relaxation on most days - Eliminates need for relaxation on most days
- Self-adjusting to market conditions - Self-adjusting to market conditions
- Better user experience (less configuration needed) - Better user experience (less configuration needed)
**Challenges:** **Challenges:**
- Harder to predict behavior (less transparent) - Harder to predict behavior (less transparent)
- May conflict with user's mental model - May conflict with user's mental model
- Needs extensive testing across different markets - Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback **Concept:** Learn optimal Flex/Distance from user feedback
**Approach:** **Approach:**
- Track which periods user actually uses (automation triggers) - Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal) - Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults - Apply pattern-specific defaults
- Learn per-user preferences over time - Learn per-user preferences over time
**Benefits:** **Benefits:**
- Personalized to user's actual behavior - Personalized to user's actual behavior
- Adapts to local market patterns - Adapts to local market patterns
- Could discover non-obvious patterns - Could discover non-obvious patterns
**Challenges:** **Challenges:**
- Requires user feedback mechanism (not implemented) - Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns) - Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?" - Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously **Concept:** Balance multiple goals simultaneously
**Goals:** **Goals:**
- Period count vs. quality (cheap vs. very cheap) - Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent) - Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered) - Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher) - User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:** **Algorithm:**
- Pareto optimization (find trade-off frontier) - Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences - User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing - Genetic algorithm or simulated annealing
**Benefits:** **Benefits:**
- More sophisticated period selection - More sophisticated period selection
- Better match to user's actual needs - Better match to user's actual needs
- Could handle complex appliance requirements - Could handle complex appliance requirements
**Challenges:** **Challenges:**
- Much more complex to implement - Much more complex to implement
- Harder to explain to users - Harder to explain to users
- Computational cost (may need caching) - Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex **Current:** 3% cap may be too aggressive for very low base Flex
**Example:** **Example:**
- Base flex 5% + 3% increment = 8% (60% increase!) - Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase) - Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:** **Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)` - Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%) - This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:** **Why Not Implemented:**
- Very low base flex (`<`10%) unusual - Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation - Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization - Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative **Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve **Alternative:** Non-linear curve
```python ```python
# Example: Exponential scaling # Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20)) scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
``` ```
**Why Not Implemented:** **Why Not Implemented:**
- Linear is easier to reason about - Linear is easier to reason about
- No evidence that non-linear is better - No evidence that non-linear is better
- Would need extensive testing - Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day **Issue:** May find all periods in one part of day
**Example:** **Example:**
- All 3 "best price" periods between 02:00-08:00 - All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances) - No periods in evening (when user might want to run appliances)
**Possible Solution:** **Possible Solution:**
- Add "spread" parameter (prefer distributed periods) - Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences - Weight periods by time-of-day preferences
- Consider user's typical usage patterns - Consider user's typical usage patterns
**Why Not Implemented:** **Why Not Implemented:**
- Adds complexity - Adds complexity
- Users can work around with multiple automations - Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all) - Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg). **Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:** **Implementation:**
```python ```python
# In period_building.py build_periods(): # In period_building.py build_periods():
for price_data in all_prices: for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight** **Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split: When days differ significantly, period can split:
``` ```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1) Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2) Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:** **Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours: Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium - Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer - Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,10 +1154,12 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):** **User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful: Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100) - `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre) - `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting: Automations can check volatility before acting:
```yaml ```yaml
condition: condition:
- condition: template - condition: template
@ -1095,6 +1190,7 @@ Low volatility (< 15%) means classification changes are less economically signif
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness. **Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:** **See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes" - User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`) - Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation) - Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

22
docs/developer/versioned_docs/version-v0.22.0/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
``` ```
**Key Points:** **Key Points:**
- Must be a **class attribute** (not instance attribute) - Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance - Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component - Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips` **Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that: **Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely - Never change or change very rarely
- Don't provide analytical value in history - Don't provide analytical value in history
- Consume significant database space when recorded every state change - Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures ### 2. Large Nested Structures
**Attributes:** **Attributes:**
- `periods` (binary_sensor) - Array of all period summaries - `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays - `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis - `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown - `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are: **Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive) - Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each) - Create large database rows (2-20 KB each)
- Slow down history queries - Slow down history queries
@ -66,6 +70,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors **Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:** **Example - periods array:**
```json ```json
{ {
"periods": [ "periods": [
@ -76,7 +81,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
"price_mean": 18.5, "price_mean": 18.5,
"price_median": 18.3, "price_median": 18.3,
"price_min": 17.2, "price_min": 17.2,
"price_max": 19.8, "price_max": 19.8
// ... 10+ more attributes × 10-20 periods // ... 10+ more attributes × 10-20 periods
} }
] ]
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status` **Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:** **Reason:**
- Change every update cycle (every 15 minutes or more frequently) - Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value - Don't provide long-term analytical value
- Create state changes even when core values haven't changed - Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max` **Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:** **Reason:**
- Configuration values that rarely change - Configuration values that rarely change
- Wastes space when recorded repeatedly - Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state - Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error` **Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:** **Reason:**
- Only relevant at moment of reading - Only relevant at moment of reading
- Won't be valid after some time - Won't be valid after some time
- Similar to `entity_picture` in HA core image entities - Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%` **Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:** **Reason:**
- Detailed technical information not needed for historical analysis - Detailed technical information not needed for historical analysis
- Only useful for debugging during active development - Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis - Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining` **Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:** **Reason:**
- Can be calculated from other attributes - Can be calculated from other attributes
- Redundant information - Redundant information
- Doesn't add analytical value to history - Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value: These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core ### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST) - `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states - All price values - Core sensor states
### Diagnostics & Tracking ### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time - `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns - `updates_today` - Tracking API usage patterns
### Data Completeness ### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics - `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status - `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data ### Period Data
- `start`, `end`, `duration_minutes` - Core period timing - `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics - `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status ### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation) - `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact ## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings ### Space Savings
**Per state change:** **Per state change:**
- Before: ~3-8 KB average - Before: ~3-8 KB average
- After: ~0.5-1.5 KB average - After: ~0.5-1.5 KB average
- **Reduction: 60-85%** - **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact ### Real-World Impact
For a typical installation with: For a typical installation with:
- 80+ sensors - 80+ sensors
- Updates every 15 minutes - Updates every 15 minutes
- ~10 sensors updating every minute - ~10 sensors updating every minute
@ -214,7 +231,7 @@ For a typical installation with:
- Class: `TibberPricesBinarySensor` - Class: `TibberPricesBinarySensor`
- 30 attributes excluded - 30 attributes excluded
## When to Update _unrecorded_attributes ## When to Update \_unrecorded_attributes
### Add to Exclusion List When: ### Add to Exclusion List When:
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes 4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:** **SQL Query to check attribute presence:**
```sql ```sql
SELECT SELECT
state_id, state_id,

14
docs/developer/versioned_docs/version-v0.22.0/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):** **In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built: git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile) - **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup` - **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):** **Manual installation (outside DevContainer):**
**git-cliff** (template-based): **git-cliff** (template-based):
```bash ```bash
# See: https://git-cliff.org/docs/installation # See: https://git-cliff.org/docs/installation
@ -191,7 +193,7 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which ## 🎯 When to Use Which
| Method | Use Case | Pros | Cons | | Method | Use Case | Pros | Cons |
|--------|----------|------|------| | --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script | | **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump | | **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization | | **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
@ -219,6 +221,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. Script bumps manifest.json → commits → creates tag locally 1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together 2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release 3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
``` ```
**What happens:** **What happens:**
1. You push manifest.json change 1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically 2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release 3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. You create and push tag manually 1. You create and push tag manually
2. Release workflow creates release 2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists) 3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features ## 🛡️ Safety Features
### 1. **Version Validation** ### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z). Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags** ### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote) - Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating - Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations** ### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push. Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered** ### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits. Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions** ### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind. Helper script shows how to undo if you change your mind.
--- ---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:** **Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes: Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely - Tag already exists remotely
- Invalid version format in manifest.json - Invalid version format in manifest.json
- manifest.json not in the commit that was pushed - manifest.json not in the commit that was pushed
@ -348,6 +359,7 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips ## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results: 1. **Conventional Commits:** Use proper commit format for best results:
``` ```
feat(scope): Add new feature feat(scope): Add new feature

22
docs/developer/versioned_docs/version-v0.22.0/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle. The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:** **Design Principles:**
- **Proactive**: Detect issues before they become critical - **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance - **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve - **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}` **Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:** **When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`) - Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available - Tomorrow's electricity price data is still not available
**When cleared:** **When cleared:**
- Tomorrow's data becomes available - Tomorrow's data becomes available
- Automatically checks on every successful API update - Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work. Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:** **Implementation:**
```python ```python
# In coordinator update cycle # In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"]) has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18) - `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}` **Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:** **When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429) - Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD` - Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:** **When cleared:**
- Successful API call completes (no rate limit error) - Successful API call completes (no rate limit error)
- Error counter resets to 0 - Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires. API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:** **Implementation:**
```python ```python
# In error handler # In error handler
is_rate_limit = ( is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors - `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}` **Issue ID:** `home_not_found_{entry_id}`
**When triggered:** **When triggered:**
- Home configured in this integration is no longer present in Tibber account - Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check) - Detected during user data refresh (daily check)
**When cleared:** **When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected) - Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup) - Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed. Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:** **Implementation:**
```python ```python
# After user data update # After user data update
home_exists = self._data_fetcher._check_home_exists(home_id) home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the missing home - `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference - `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration ### Lifecycle Integration
**Coordinator Initialization:** **Coordinator Initialization:**
```python ```python
self._repair_manager = TibberPricesRepairManager( self._repair_manager = TibberPricesRepairManager(
hass=hass, hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
``` ```
**Update Cycle Integration:** **Update Cycle Integration:**
```python ```python
# Success path - check conditions # Success path - check conditions
if result and "priceInfo" in result: if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
``` ```
**Shutdown Cleanup:** **Shutdown Cleanup:**
```python ```python
async def async_shutdown(self) -> None: async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up.""" """Shut down coordinator and clean up."""
@ -196,6 +212,7 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json` - `/translations/sv.json`
**Structure:** **Structure:**
```json ```json
{ {
"issues": { "issues": {
@ -210,10 +227,12 @@ Repairs use Home Assistant's standard translation system. Translations are defin
## Home Assistant Integration ## Home Assistant Integration
Repairs appear in: Repairs appear in:
- **Settings → System → Repairs** (main repairs panel) - **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count) - **Notifications** (bell icon in UI shows repair count)
Repair properties: Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required) - **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical) - **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files - **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears 4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:** **Manual trigger:**
```python ```python
# Temporarily set warning hour to current hour for testing # Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair 3. Successful API call clears the repair
**Manual test:** **Manual test:**
- Reduce API polling interval to trigger rate limiting - Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client - Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file 7. **Document** in this file
**Example template:** **Example template:**
```python ```python
async def check_new_condition(self, *, param: bool) -> None: async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair.""" """Check new condition and create/clear repair."""

24
docs/developer/versioned_docs/version-v0.22.0/timer-architecture.md
View file

@ -11,7 +11,7 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes: The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method | | Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------| | ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` | | **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` | | **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` | | **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes` **Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:** **What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator` - HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes - Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time) - **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
``` ```
**Load Distribution:** **Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution - Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API - Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once - Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:** **Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects) - Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early - If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully - Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:** **Why we use HA's timer:**
- Automatic restart after HA restart - Automatic restart after HA restart
- Built-in retry logic for temporary failures - Built-in retry logic for temporary failures
- Standard HA integration pattern - Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll** **Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:** **Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48) - Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes - Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13 - Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
``` ```
**Smart Boundary Tolerance:** **Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance - Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval) - HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval) - HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details - See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:** **Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...) - `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes") - NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates) - If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:** **Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`) - All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`) - Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total - ~50-60 entities out of 120+ total
**Why custom timer:** **Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing - HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals - We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation - Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
``` ```
**Which entities listen:** **Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer - `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer - `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%) - `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total - ~10 entities total
**Why custom timer:** **Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time) - Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates - Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation) - Very lightweight (no data processing, just state recalculation)
**Why NOT every second:** **Why NOT every second:**
- Minute precision sufficient for countdown UX - Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds) - Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates) - Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
``` ```
**Why this pattern:** **Why this pattern:**
- Decouples timer logic from entity logic - Decouples timer logic from entity logic
- One timer can notify many entities efficiently - One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup) - Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API ### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers: If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload - ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload - ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem - ❌ "Thundering herd" problem
With HA's unsynchronized timer: With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ... - ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ... - ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ... - ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data. **Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:** **API fetch only when:**
- Tomorrow data missing/invalid (after 13:00) - Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover) - Cache expired (midnight turnover)
- Explicit user refresh - Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics ## Performance Characteristics
### Timer #1 (DataUpdateCoordinator) ### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized) - **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data) - **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate) - **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise) - **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh) ### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries) - **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities) - **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data - **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates - **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh) ### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute) - **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities) - **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all - **No API calls:** No data processing at all
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary ## Summary
**Three independent timers:** **Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed) 1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always) 2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always) 3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:** **Key insights:**
- Timer #1 unsynchronized = good (load distribution on API) - Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately) - Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX) - Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts) - All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:** **"Listener" terminology:**
- Timer = mechanism that triggers - Timer = mechanism that triggers
- Listener = callback that gets called - Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies - Observer pattern = entities register, coordinator notifies

31
docs/developer/versioned_docs/version-v0.22.1/api-reference.md
View file

@ -56,7 +56,7 @@ query {
Fetches quarter-hourly prices: Fetches quarter-hourly prices:
```graphql ```graphql
query($homeId: ID!) { query ($homeId: ID!) {
viewer { viewer {
home(id: $homeId) { home(id: $homeId) {
currentSubscription { currentSubscription {
@ -76,6 +76,7 @@ query($homeId: ID!) {
``` ```
**Parameters:** **Parameters:**
- `homeId`: Tibber home identifier - `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY` - `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data) - `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits ## Rate Limits
Tibber API rate limits (as of 2024): Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token - **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute - **Burst limit:** 100 requests per minute
Integration stays well below these limits: Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day - Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day - User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home - **Total:** ~100 requests/day per home
@ -106,6 +109,7 @@ Integration stays well below these limits:
``` ```
**Fields:** **Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR) - `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone - `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE) - `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -119,6 +123,7 @@ Integration stays well below these limits:
``` ```
Supported currencies: Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh - `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh - `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh - `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses ### Common Error Responses
**Invalid Token:** **Invalid Token:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Unauthorized", "message": "Unauthorized",
"extensions": { "extensions": {
"code": "UNAUTHENTICATED" "code": "UNAUTHENTICATED"
} }
}] }
]
} }
``` ```
**Rate Limit Exceeded:** **Rate Limit Exceeded:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Too Many Requests", "message": "Too Many Requests",
"extensions": { "extensions": {
"code": "RATE_LIMIT_EXCEEDED" "code": "RATE_LIMIT_EXCEEDED"
} }
}] }
]
} }
``` ```
**Home Not Found:** **Home Not Found:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Home not found", "message": "Home not found",
"extensions": { "extensions": {
"code": "NOT_FOUND" "code": "NOT_FOUND"
} }
}] }
]
} }
``` ```
Integration handles these with: Integration handles these with:
- Exponential backoff retry (3 attempts) - Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors - ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures - ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation ## Data Transformation
Raw API data is enriched with: Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals - **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals - **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average - **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
--- ---
💡 **External Resources:** 💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview) - [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer) - [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token) - [Get API Token](https://developer.tibber.com/settings/access-token)

17
docs/developer/versioned_docs/version-v0.22.1/architecture.md
View file

@ -147,7 +147,7 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance: The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory | | Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------| | ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB | | **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB | | **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB | | **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
@ -196,7 +196,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components ### Core Components
| Component | File | Responsibility | | Component | File | Responsibility |
|-----------|------|----------------| | --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling | | **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance | | **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) | | **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
@ -210,7 +210,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025): The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility | | Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------| | ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators | | **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) | | **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) | | **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
@ -219,6 +219,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities | | **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`): **Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access - `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous) - `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows - `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata - `metadata.py` - Home/metering metadata
**Benefits:** **Benefits:**
- 58% reduction in core.py (2,170 → 909 lines) - 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation) - Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator - Independent testability for each calculator
@ -238,7 +240,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities ### Helper Utilities
| Utility | File | Purpose | | Utility | File | Purpose |
|---------|------|---------| | ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation | | **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations | | **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic | | **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025): Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`): **Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals - `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows - `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg - `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics - `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`): **Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods - Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.) - Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`): **Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access - Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc. - Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching) - Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`): **Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation - Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes - Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()` - Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:** **Benefits:**
- Minimal code duplication across 80+ sensors - Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation) - Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing - Easy to extend: Add sensor → choose pattern → add to routing
@ -334,7 +341,7 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization ### CPU Optimization
| Optimization | Location | Savings | | Optimization | Location | Savings |
|--------------|----------|---------| | ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks | | Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation | | Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations | | Lazy logging | Throughout | ~15% on log-heavy operations |

42
docs/developer/versioned_docs/version-v0.22.1/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts. **Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:** **What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total) - **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query - **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation - **Timestamps**: Last update times for validation
**Lifetime:** **Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time) - **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily) - **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage - **Survives**: HA restarts via persistent Storage
@ -36,6 +38,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:** **Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator): 1. **Midnight turnover** (Timer #2 in coordinator):
```python ```python
# coordinator/day_transitions.py # coordinator/day_transitions.py
def _handle_midnight_turnover() -> None: def _handle_midnight_turnover() -> None:
@ -45,6 +48,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
``` ```
2. **Cache validation on load**: 2. **Cache validation on load**:
```python ```python
# coordinator/cache.py # coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool: def is_cache_valid(cache_data: CacheData) -> bool:
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc. **Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:** **What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names - **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions - **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:** **Lifetime:**
- **Forever** (until HA restart) - **Forever** (until HA restart)
- No invalidation during runtime - No invalidation during runtime
**When populated:** **When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py` - At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once - Lazy loading: If translation missing, attempts file load once
**Access pattern:** **Access pattern:**
```python ```python
# Non-blocking synchronous access from cached data # Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en") description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:** **What is cached:**
### DataTransformer Config Cache ### DataTransformer Config Cache
```python ```python
{ {
"thresholds": {"low": 15, "high": 35}, "thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
### PeriodCalculator Config Cache ### PeriodCalculator Config Cache
```python ```python
{ {
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60}, "best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,10 +128,12 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Lifetime:** **Lifetime:**
- Until `invalidate_config_cache()` is called - Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle - Built once on first use per coordinator update cycle
**Invalidation trigger:** **Invalidation trigger:**
- **Options change** (user reconfigures integration): - **Options change** (user reconfigures integration):
```python ```python
# coordinator/core.py # coordinator/core.py
@ -132,6 +144,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Performance impact:** **Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs - **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs - **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update) - **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed. **Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:** **What is cached:**
```python ```python
{ {
"best_price": { "best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Cache key:** Hash of relevant inputs **Cache key:** Hash of relevant inputs
```python ```python
hash_data = ( hash_data = (
today_signature, # (startsAt, rating_level) for each interval today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
``` ```
**Lifetime:** **Lifetime:**
- Until price data changes (today's intervals modified) - Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters) - Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data) - Recalculated at midnight (new today data)
@ -179,6 +195,7 @@ hash_data = (
**Invalidation triggers:** **Invalidation triggers:**
1. **Config change** (explicit): 1. **Config change** (explicit):
```python ```python
def invalidate_config_cache() -> None: def invalidate_config_cache() -> None:
self._cached_periods = None self._cached_periods = None
@ -193,10 +210,12 @@ hash_data = (
``` ```
**Cache hit rate:** **Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged) - **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00) - **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:** **Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts) - **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup) - **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache) - **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy **Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:** **What is cached:**
```python ```python
{ {
"timestamp": ..., "timestamp": ...,
@ -224,6 +244,7 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks. **Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:** **Current behavior:**
- Caches **only enriched price data** (price + statistics) - Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache) - **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when: - Invalidated when:
@ -232,6 +253,7 @@ hash_data = (
- New update cycle begins - New update cycle begins
**Architecture:** **Architecture:**
- DataTransformer: Handles price enrichment only - DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache) - PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches - Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow ## Cache Invalidation Flow
### User Changes Options (Config Flow) ### User Changes Options (Config Flow)
``` ```
User saves options User saves options
↓ ↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
``` ```
### Midnight Turnover (Day Transition) ### Midnight Turnover (Day Transition)
``` ```
Timer #2 fires at 00:00 Timer #2 fires at 00:00
↓ ↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
``` ```
### Tomorrow Data Arrives (~13:00) ### Tomorrow Data Arrives (~13:00)
``` ```
Coordinator update cycle Coordinator update cycle
↓ ↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
``` ```
**No cache invalidation cascades:** **No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update) - Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch) - Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change) - Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load) - Translation cache is **never invalidated** (read-only after load)
**Thread safety:** **Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop) - All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model) - No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics ## Performance Characteristics
### Typical Operation (No Changes) ### Typical Operation (No Changes)
``` ```
Coordinator Update (every 15 min) Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid) ├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
``` ```
### After Midnight Turnover ### After Midnight Turnover
``` ```
Coordinator Update (00:00) Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day) ├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
``` ```
### After Config Change ### After Config Change
``` ```
Options Update Options Update
├─> Cache invalidation: `<`1ms ├─> Cache invalidation: `<`1ms
@ -382,7 +412,7 @@ Options Update
## Summary Table ## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose | | Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------| | ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls | | **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O | | **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups | | **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
@ -392,12 +422,14 @@ Options Update
**Total memory overhead:** ~116KB per coordinator instance (main + subentries) **Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:** **Benefits:**
- 97% reduction in API calls (from every 15min to once per day) - 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation) - 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check) - 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup) - Zero file I/O during runtime (translations cached at startup)
**Trade-offs:** **Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems) - Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented) - Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues - Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues ## Debugging Cache Issues
### Symptom: Stale data after config change ### Symptom: Stale data after config change
**Check:** **Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log) 1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed? 2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger? 3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init. **Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating ### Symptom: Period calculation not updating
**Check:** **Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()` 1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash` 2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs 3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs. **Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today ### Symptom: Yesterday's prices shown as today
**Check:** **Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py` 1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2) 2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs 3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration. **Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations ### Symptom: Missing translations
**Check:** **Check:**
1. `async_load_translations()` called at startup? 1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`? 2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys 3. Cache population: `_TRANSLATIONS_CACHE` keys

3
docs/developer/versioned_docs/version-v0.22.1/coding-guidelines.md
View file

@ -41,12 +41,14 @@ class TimeService:
``` ```
**When prefix is required:** **When prefix is required:**
- Public classes used across multiple modules - Public classes used across multiple modules
- All exception classes - All exception classes
- All coordinator and entity classes - All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs - Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:** **When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore) - Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`) - Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns - Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:** **Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring: Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md` 1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames 2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module 3. Test thoroughly after each module

17
docs/developer/versioned_docs/version-v0.22.1/contributing.md
View file

@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
``` ```
**Branch naming:** **Branch naming:**
- `feature/` - New features - `feature/` - New features
- `fix/` - Bug fixes - `fix/` - Bug fixes
- `docs/` - Documentation only - `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md). Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:** **Run checks frequently:**
```bash ```bash
./scripts/type-check # Pyright type checking ./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix) ./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
``` ```
Run your test: Run your test:
```bash ```bash
./scripts/test tests/test_your_feature.py -v ./scripts/test tests/test_your_feature.py -v
``` ```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
``` ```
**Commit types:** **Commit types:**
- `feat:` - New feature - `feat:` - New feature
- `fix:` - Bug fix - `fix:` - Bug fix
- `docs:` - Documentation - `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance - `chore:` - Maintenance
**Add scope when relevant:** **Add scope when relevant:**
- `feat(sensors):` - Sensor platform - `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator - `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation - `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max) Title: Short, descriptive (50 chars max)
Description should include: Description should include:
```markdown ```markdown
## What ## What
Brief description of changes Brief description of changes
## Why ## Why
Problem being solved or feature rationale Problem being solved or feature rationale
## How ## How
Implementation approach Implementation approach
## Testing ## Testing
- [ ] Manual testing in Home Assistant - [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated - [ ] Unit tests added/updated
- [ ] Type checking passes - [ ] Type checking passes
- [ ] Linting passes - [ ] Linting passes
## Breaking Changes ## Breaking Changes
(If any - describe migration path) (If any - describe migration path)
## Related Issues ## Related Issues
Closes #123 Closes #123
``` ```
### PR Checklist ### PR Checklist
Before submitting: Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md) - [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`) - [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`) - [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For ### What Reviewers Look For
✅ **Good:** ✅ **Good:**
- Clear, self-explanatory code - Clear, self-explanatory code
- Appropriate comments for complex logic - Appropriate comments for complex logic
- Tests covering edge cases - Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns - Follows existing patterns
❌ **Avoid:** ❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones - Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes - Mixing unrelated changes
- Missing tests for new features - Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On ## Finding Issues to Work On
Good first issues are labeled: Good first issues are labeled:
- `good first issue` - Beginner-friendly - `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions - `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements - `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
--- ---
💡 **Related:** 💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup - [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide - [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests - [Testing](testing.md) - Writing tests

33
docs/developer/versioned_docs/version-v0.22.1/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical? ## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to: Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable - **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases - **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks - **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅ #### 1.1 Listener Cleanup ✅
**What is tested:** **What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`) - Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`) - Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`) - Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners - Binary sensor cleanup removes ALL registered listeners
**Why critical:** **Why critical:**
- Each registered listener holds references to Entity + Coordinator - Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak - Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed - With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()` - `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()` - `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()` - `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅ #### 1.2 Timer Cleanup ✅
**What is tested:** **What is tested:**
- Quarter-hour timer is cancelled and reference cleared - Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared - Minute timer is cancelled and reference cleared
- Both timers are cancelled together - Both timers are cancelled together
- Cleanup works even when timers are `None` - Cleanup works even when timers are `None`
**Why critical:** **Why critical:**
- Uncancelled timers continue running after integration unload - Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks - HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates - Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()` - `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅ #### 1.3 Config Entry Cleanup ✅
**What is tested:** **What is tested:**
- Options update listener is registered via `async_on_unload()` - Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()` - Cleanup function is correctly passed to `async_on_unload()`
**Why critical:** **Why critical:**
- `entry.add_update_listener()` registers permanent callback - `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates - Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))` - Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:** **Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration) - `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()` - `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation #### 2.1 Config Cache Invalidation
**What is tested:** **What is tested:**
- DataTransformer config cache is invalidated on options change - DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated - PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update - Trend calculator cache is cleared on coordinator update
**Why critical:** **Why critical:**
- Stale config → Sensors use old user settings - Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods - Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis - Stale trend cache → Outdated trend analysis
**Code Locations:** **Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()` - `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()` - `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()` - `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal #### 3.1 Persistent Storage Removal
**What is tested:** **What is tested:**
- Storage file is deleted on config entry removal - Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss) - Cache is saved on shutdown (no data loss)
**Why critical:** **Why critical:**
- Without storage removal: Old files remain after uninstallation - Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart - Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}` - Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:** **Code Locations:**
- `__init__.py` → `async_remove_entry()` - `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py` **File:** `tests/test_timer_scheduling.py`
**What is tested:** **What is tested:**
- Quarter-hour timer is registered with correct parameters - Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters - Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer) - Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly - Midnight turnover detection works correctly
**Why critical:** **Why critical:**
- Wrong timer parameters → Entities update at wrong times - Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem - Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py` **File:** `tests/test_sensor_timer_assignment.py`
**What is tested:** **What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys - All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys - All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap) - Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked - Sensor and binary sensor platforms are checked
**Why critical:** **Why critical:**
- Wrong timer assignment → Sensors update at wrong times - Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem - Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management ### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks **Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds) - `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms) - `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:** **Why no tests needed:**
- No long-running tasks (all < 2 seconds) - No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically - HA's event loop handles short tasks automatically
- Task exceptions are already logged - Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup ### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented **Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session) - `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created - No new sessions are created
- HA manages session lifecycle automatically - HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory ### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache **Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total - Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading - Module-level cache without re-loading
- Practically no memory issue - Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity ### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop` **Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days) - Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults - Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors - 80+ sensors access `coordinator.data` without errors
**Structure:** **Structure:**
```python ```python
coordinator.data = { coordinator.data = {
"user_data": {...}, "user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory ### 10. Service Response Memory
**Current Status:** HA's response lifecycle **Current Status:** HA's response lifecycle
- HA automatically frees service responses after return - HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call - ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code - No response accumulation in integration code
@ -208,7 +235,7 @@ coordinator.data = {
### ✅ Implemented Tests (41 total) ### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage | | Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------| | ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% | | Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% | | Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% | | Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
@ -222,7 +249,7 @@ coordinator.data = {
### 📋 Analyzed but Not Implemented (Nice-to-Have) ### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale | | Category | Status | Rationale |
|----------|--------|-----------| | ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) | | Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) | | API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) | | Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
@ -230,6 +257,7 @@ coordinator.data = {
| Service Response Memory | 📋 | HA automatically frees service responses | | Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:** **Legend:**
- ✅ = Fully tested or pattern verified correct - ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues) - 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested ### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests: All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks) - ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks) - ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners) - ✅ Config entry cleanup works (no dangling listeners)

18
docs/developer/versioned_docs/version-v0.22.1/debugging.md
View file

@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages ### Key Log Messages
**Coordinator Updates:** **Coordinator Updates:**
``` ```
[custom_components.tibber_prices.coordinator] Successfully fetched price data [custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data [custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
``` ```
**Period Calculation:** **Period Calculation:**
``` ```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0% [custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods [custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
``` ```
**API Errors:** **API Errors:**
``` ```
[custom_components.tibber_prices.api] API request failed: Unauthorized [custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s [custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -67,6 +70,7 @@ Restart Home Assistant to apply.
### Set Breakpoints ### Set Breakpoints
**Coordinator update:** **Coordinator update:**
```python ```python
# coordinator/core.py # coordinator/core.py
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
``` ```
**Period calculation:** **Period calculation:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]: def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
``` ```
**Flags:** **Flags:**
- `-v` - Verbose output - `-v` - Verbose output
- `-s` - Show print statements - `-s` - Show print statements
- `-k pattern` - Run tests matching pattern - `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns ### Useful Test Patterns
**Print coordinator data:** **Print coordinator data:**
```python ```python
def test_something(coordinator): def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}") print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
``` ```
**Inspect period attributes:** **Inspect period attributes:**
```python ```python
def test_periods(hass, coordinator): def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', []) periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading ### Integration Not Loading
**Check:** **Check:**
```bash ```bash
grep "tibber_prices" config/home-assistant.log grep "tibber_prices" config/home-assistant.log
``` ```
**Common causes:** **Common causes:**
- Syntax error in Python code → Check logs for traceback - Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync` - Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*` - Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating ### Sensors Not Updating
**Check coordinator state:** **Check coordinator state:**
```python ```python
# In Developer Tools > Template # In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }} {{ states.sensor.tibber_home_current_interval_price.last_updated }}
``` ```
**Debug in code:** **Debug in code:**
```python ```python
# Add logging in sensor/core.py # Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s", _LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong ### Period Calculation Wrong
**Enable detailed period logs:** **Enable detailed period logs:**
```python ```python
# coordinator/period_handlers/period_building.py # coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s", _LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
``` ```
**Check filter statistics:** **Check filter statistics:**
``` ```
[period_building] Flex filter blocked: 45 intervals [period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals [period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy ### Remote Debugging with debugpy
Add to coordinator code: Add to coordinator code:
```python ```python
import debugpy import debugpy
debugpy.listen(5678) debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL ### IPython REPL
Install in container: Install in container:
```bash ```bash
uv pip install ipython uv pip install ipython
``` ```
Add breakpoint: Add breakpoint:
```python ```python
from IPython import embed from IPython import embed
embed() # Drops into interactive shell embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
--- ---
💡 **Related:** 💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests - [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment - [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure - [Architecture](architecture.md) - Code structure

1
docs/developer/versioned_docs/version-v0.22.1/intro.md
View file

@ -168,6 +168,7 @@ Documentation is organized in two Docusaurus sites:
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory) - **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:** **Best practices:**
- Use clear examples and code snippets - Use clear examples and code snippets
- Keep docs up-to-date with code changes - Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation - Add new pages to appropriate `sidebars.ts` for navigation

17
docs/developer/versioned_docs/version-v0.22.1/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals ## Performance Goals
Target metrics: Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms) - **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor - **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms) - **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching ### Caching
**1. Persistent Cache** (API data): **1. Persistent Cache** (API data):
```python ```python
# Already implemented in coordinator/cache.py # Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY) store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
``` ```
**2. Translation Cache** (in-memory): **2. Translation Cache** (in-memory):
```python ```python
# Already implemented in const.py # Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {} _TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
``` ```
**3. Config Cache** (invalidated on options change): **3. Config Cache** (invalidated on options change):
```python ```python
class DataTransformer: class DataTransformer:
def __init__(self): def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading ### Lazy Loading
**Load data only when needed:** **Load data only when needed:**
```python ```python
@property @property
def extra_state_attributes(self) -> dict | None: def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations ### Bulk Operations
**Process multiple items at once:** **Process multiple items at once:**
```python ```python
# ❌ Slow - loop with individual operations # ❌ Slow - loop with individual operations
for interval in intervals: for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices ### Async Best Practices
**1. Concurrent API calls:** **1. Concurrent API calls:**
```python ```python
# ❌ Sequential (slow) # ❌ Sequential (slow)
user_data = await fetch_user_data() user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
``` ```
**2. Don't block event loop:** **2. Don't block event loop:**
```python ```python
# ❌ Blocking # ❌ Blocking
result = heavy_computation() # Blocks for seconds result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks ### Avoid Memory Leaks
**1. Clear references:** **1. Clear references:**
```python ```python
class Coordinator: class Coordinator:
async def async_shutdown(self): async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
``` ```
**2. Use weak references for callbacks:** **2. Use weak references for callbacks:**
```python ```python
import weakref import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures ### Efficient Data Structures
**Use appropriate types:** **Use appropriate types:**
```python ```python
# ❌ List for lookups (O(n)) # ❌ List for lookups (O(n))
if timestamp in timestamp_list: if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls ### Minimize API Calls
**Already implemented:** **Already implemented:**
- Cache valid until midnight - Cache valid until midnight
- User data cached for 24h - User data cached for 24h
- Only poll when tomorrow data expected - Only poll when tomorrow data expected
**Monitor API usage:** **Monitor API usage:**
```python ```python
_LOGGER.debug("API call: %s (cache_age=%s)", _LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age) endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates ### Smart Updates
**Only update when needed:** **Only update when needed:**
```python ```python
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
"""Fetch data from API.""" """Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection ### State Class Selection
**Affects long-term statistics storage:** **Affects long-term statistics storage:**
```python ```python
# ❌ MEASUREMENT for prices (stores every change) # ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size ### Attribute Size
**Keep attributes minimal:** **Keep attributes minimal:**
```python ```python
# ❌ Large nested structures (KB per update) # ❌ Large nested structures (KB per update)
attributes = { attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
--- ---
💡 **Related:** 💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers - [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design - [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools - [Debugging](debugging.md) - Profiling tools

98
docs/developer/versioned_docs/version-v0.22.1/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic. **Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:** **Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point - `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering - `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy - `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max. **Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be within flex% ABOVE daily minimum # Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex) in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Flex: 15% - Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15) - Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better. **Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be at least min_distance% BELOW daily average # Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100)) meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Min Distance: 5% - Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95) - Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone - Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:** **Implementation:**
```python ```python
# Sensor classification uses user config # Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10) user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,21 +113,25 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5% #### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:** **Given:**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh - Daily Max: 20 ct/kWh
**Flex Filter (50%):** **Flex Filter (50%):**
``` ```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
``` ```
**Min Distance Filter (5%):** **Min Distance Filter (5%):**
``` ```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
``` ```
**Conflict:** **Conflict:**
- Interval at 14.8 ct/kWh: - Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS) - ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL) - ❌ Distance: 14.8 > 14.25 (FAIL)
@ -132,11 +142,13 @@ Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
### Mathematical Analysis ### Mathematical Analysis
**Conflict condition for Best Price:** **Conflict condition for Best Price:**
``` ```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100) daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
``` ```
**Typical values:** **Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5% - Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25` - Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%) - Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases. **Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:** **Formula:**
```python ```python
if flex > 0.20: # 20% threshold if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20 flex_excess = flex - 0.20
@ -159,7 +172,7 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):** **Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale | | Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------| | ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant | | ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction | | 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction | | 30% | 0.75 | 3.75% | Moderate reduction |
@ -167,6 +180,7 @@ if flex > 0.20: # 20% threshold
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides | | 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?** **Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average - Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval - Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period" - Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()` **Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:** **Code Extract:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
``` ```
**Why Linear Scaling?** **Why Linear Scaling?**
- Simple and predictable - Simple and predictable
- No abrupt behavior changes - No abrupt behavior changes
- Easy to reason about for users and developers - Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive) - Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?** **Why 25% Minimum?**
- Below this, min_distance loses semantic meaning - Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed - Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario - Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants ### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:** **Defined in `coordinator/period_handlers/core.py`:**
```python ```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
``` ```
**Defined in `const.py`:** **Defined in `const.py`:**
```python ```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled) DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection) DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances **Goal:** Find practical time windows for running appliances
**Constraints:** **Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h) - Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead) - Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average" - User wants genuinely cheap times, not just "slightly below average"
**Defaults:** **Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use - **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times - **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones - **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:** **User behavior:**
- Automations trigger actions (turn on devices) - Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money - Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives) - Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction **Goal:** Alert users to expensive periods for consumption reduction
**Constraints:** **Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding) - Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy - Early warning more valuable than perfect accuracy
- User can manually decide whether to react - User can manually decide whether to react
**Defaults:** **Defaults:**
- **30 min minimum** - Catches shorter expensive spikes - **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection - **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods - **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:** **User behavior:**
- Notifications/alerts (informational) - Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost - Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes) - Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:** **Peak Price Volatility:**
Price curves tend to have: Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening) - **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical) - **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times - **Higher variance** in peak times than cheap times
**Example day:** **Example day:**
``` ```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
``` ```
**Implication:** **Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief) - Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes - Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection - Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered #### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)** **Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex - Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes - Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?" - User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)** **Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices - No guidance on best practices
- Users would need to experiment to find good values - Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter - Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)** **Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options - **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price - **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case - Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%) **Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:** **Rationale:**
- Above 50%, period detection becomes unreliable - Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals) - Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50% - Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met) - **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:** **Warning Message:**
``` ```
Flex XX% exceeds maximum safe value! Capping at 50%. Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation. Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%) **Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:** **Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold - Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable" - At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation - **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended) #### With Relaxation Enabled (Recommended)
**Optimal:** 10-20% **Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ... - Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work - Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25% **Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side" - INFO log: "Base flex is on the high side"
**High Warning:** > 30% **High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!" - WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20% - Recommendation: Lower to 15-20%
#### Without Relaxation #### Without Relaxation
**Optimal:** 20-35% **Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start - No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback - Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50% **Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits) - Above this, period detection degrades (see Hard Limits)
--- ---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach ### Multi-Phase Approach
**Each day processed independently:** **Each day processed independently:**
1. Calculate baseline periods with user's config 1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop 2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations 3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
``` ```
**Constants:** **Constants:**
```python ```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20% FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -447,6 +488,7 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Historical Context (Pre-November 2025):** **Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex: The algorithm previously used percentage-based increments that scaled with base flex:
```python ```python
increment = base_flex × (step_pct / 100) # REMOVED increment = base_flex × (step_pct / 100) # REMOVED
``` ```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point. This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:** **Warning Messages:**
```python ```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30% if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning( _LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy ### Filter Combination Strategy
**Per Flex level, try in order:** **Per Flex level, try in order:**
1. Original Level filter 1. Original Level filter
2. Level filter = "any" (disabled) 2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations) **Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):** **Example Flow (target=2 periods/day):**
``` ```
Day 2025-11-19: Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2) 1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions ### Key Files and Functions
**Period Calculation Entry Point:** **Period Calculation Entry Point:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods( def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
``` ```
**Flex + Distance Filtering:** **Flex + Distance Filtering:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
def check_interval_criteria( def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
``` ```
**Relaxation Orchestration:** **Relaxation Orchestration:**
```python ```python
# coordinator/period_handlers/relaxation.py # coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict] def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -541,6 +589,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Rejects asymmetric outliers (threshold: 1.5 std dev) - Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks) - Preserves legitimate price shifts (morning/evening peaks)
- Algorithm: - Algorithm:
```python ```python
residual = abs(actual - predicted) residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev symmetry_threshold = 1.5 × std_dev
@ -563,6 +612,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes) - Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:** **Constants:**
```python ```python
# coordinator/period_handlers/outlier_filtering.py # coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
``` ```
**Data Integrity:** **Data Integrity:**
- Original prices stored in `_original_price` field - Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices - All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic - Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome - Smart counting: Only counts smoothing that changed period outcome
**Performance:** **Performance:**
- Single pass through price data - Single pass through price data
- O(n) complexity with small context window - O(n) complexity with small context window
- No iterative refinement needed - No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals - Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:** **Example Debug Output:**
``` ```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -624,6 +677,7 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips ## Debugging Tips
**Enable DEBUG logging:** **Enable DEBUG logging:**
```yaml ```yaml
# configuration.yaml # configuration.yaml
logger: logger:
@ -633,6 +687,7 @@ logger:
``` ```
**Key log messages to watch:** **Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion 1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering 2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded 3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,17 +700,20 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation ### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 40 best_price_flex: 40
enable_relaxation_best: true enable_relaxation_best: true
``` ```
**Problem:** **Problem:**
- Base Flex 40% already very permissive - Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...) - Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns - Quickly approaches 50% cap with diminishing returns
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 15 # Let relaxation increase it best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true enable_relaxation_best: true
@ -664,16 +722,19 @@ enable_relaxation_best: true
### ❌ Anti-Pattern 2: Zero Min_Distance ### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_min_distance_from_avg: 0 best_price_min_distance_from_avg: 0
``` ```
**Problem:** **Problem:**
- "Flat days" (little price variation) accept all intervals - "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap") - Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times - May create periods during barely-below-average times
**Solution:** **Solution:**
```yaml ```yaml
best_price_min_distance_from_avg: 5 # Use default 5% best_price_min_distance_from_avg: 5 # Use default 5%
``` ```
@ -681,16 +742,19 @@ best_price_min_distance_from_avg: 5 # Use default 5%
### ❌ Anti-Pattern 3: Conflicting Flex + Distance ### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 45 best_price_flex: 45
best_price_min_distance_from_avg: 10 best_price_min_distance_from_avg: 10
``` ```
**Problem:** **Problem:**
- Distance filter dominates, making Flex irrelevant - Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal - Dynamic scaling helps but still suboptimal
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 20 best_price_flex: 20
best_price_min_distance_from_avg: 5 best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods - Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient) - Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range - Min_Distance 5%: Effective throughout range
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners) - Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify - Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price" - Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh **Average:** 18 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct) - Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct) - Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive) - Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct) DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction) DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2 **Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2) DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day **Initial State:** Strict filters, very flat day
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2) DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation **Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:** **Algorithm:**
```python ```python
# Pseudo-code for adaptive flex # Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
``` ```
**Benefits:** **Benefits:**
- Eliminates need for relaxation on most days - Eliminates need for relaxation on most days
- Self-adjusting to market conditions - Self-adjusting to market conditions
- Better user experience (less configuration needed) - Better user experience (less configuration needed)
**Challenges:** **Challenges:**
- Harder to predict behavior (less transparent) - Harder to predict behavior (less transparent)
- May conflict with user's mental model - May conflict with user's mental model
- Needs extensive testing across different markets - Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback **Concept:** Learn optimal Flex/Distance from user feedback
**Approach:** **Approach:**
- Track which periods user actually uses (automation triggers) - Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal) - Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults - Apply pattern-specific defaults
- Learn per-user preferences over time - Learn per-user preferences over time
**Benefits:** **Benefits:**
- Personalized to user's actual behavior - Personalized to user's actual behavior
- Adapts to local market patterns - Adapts to local market patterns
- Could discover non-obvious patterns - Could discover non-obvious patterns
**Challenges:** **Challenges:**
- Requires user feedback mechanism (not implemented) - Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns) - Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?" - Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously **Concept:** Balance multiple goals simultaneously
**Goals:** **Goals:**
- Period count vs. quality (cheap vs. very cheap) - Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent) - Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered) - Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher) - User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:** **Algorithm:**
- Pareto optimization (find trade-off frontier) - Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences - User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing - Genetic algorithm or simulated annealing
**Benefits:** **Benefits:**
- More sophisticated period selection - More sophisticated period selection
- Better match to user's actual needs - Better match to user's actual needs
- Could handle complex appliance requirements - Could handle complex appliance requirements
**Challenges:** **Challenges:**
- Much more complex to implement - Much more complex to implement
- Harder to explain to users - Harder to explain to users
- Computational cost (may need caching) - Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex **Current:** 3% cap may be too aggressive for very low base Flex
**Example:** **Example:**
- Base flex 5% + 3% increment = 8% (60% increase!) - Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase) - Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:** **Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)` - Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%) - This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:** **Why Not Implemented:**
- Very low base flex (`<`10%) unusual - Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation - Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization - Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative **Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve **Alternative:** Non-linear curve
```python ```python
# Example: Exponential scaling # Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20)) scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
``` ```
**Why Not Implemented:** **Why Not Implemented:**
- Linear is easier to reason about - Linear is easier to reason about
- No evidence that non-linear is better - No evidence that non-linear is better
- Would need extensive testing - Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day **Issue:** May find all periods in one part of day
**Example:** **Example:**
- All 3 "best price" periods between 02:00-08:00 - All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances) - No periods in evening (when user might want to run appliances)
**Possible Solution:** **Possible Solution:**
- Add "spread" parameter (prefer distributed periods) - Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences - Weight periods by time-of-day preferences
- Consider user's typical usage patterns - Consider user's typical usage patterns
**Why Not Implemented:** **Why Not Implemented:**
- Adds complexity - Adds complexity
- Users can work around with multiple automations - Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all) - Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg). **Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:** **Implementation:**
```python ```python
# In period_building.py build_periods(): # In period_building.py build_periods():
for price_data in all_prices: for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight** **Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split: When days differ significantly, period can split:
``` ```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1) Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2) Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:** **Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours: Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium - Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer - Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,10 +1154,12 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):** **User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful: Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100) - `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre) - `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting: Automations can check volatility before acting:
```yaml ```yaml
condition: condition:
- condition: template - condition: template
@ -1095,6 +1190,7 @@ Low volatility (< 15%) means classification changes are less economically signif
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness. **Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:** **See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes" - User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`) - Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation) - Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

22
docs/developer/versioned_docs/version-v0.22.1/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
``` ```
**Key Points:** **Key Points:**
- Must be a **class attribute** (not instance attribute) - Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance - Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component - Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips` **Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that: **Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely - Never change or change very rarely
- Don't provide analytical value in history - Don't provide analytical value in history
- Consume significant database space when recorded every state change - Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures ### 2. Large Nested Structures
**Attributes:** **Attributes:**
- `periods` (binary_sensor) - Array of all period summaries - `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays - `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis - `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown - `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are: **Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive) - Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each) - Create large database rows (2-20 KB each)
- Slow down history queries - Slow down history queries
@ -66,6 +70,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors **Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:** **Example - periods array:**
```json ```json
{ {
"periods": [ "periods": [
@ -76,7 +81,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
"price_mean": 18.5, "price_mean": 18.5,
"price_median": 18.3, "price_median": 18.3,
"price_min": 17.2, "price_min": 17.2,
"price_max": 19.8, "price_max": 19.8
// ... 10+ more attributes × 10-20 periods // ... 10+ more attributes × 10-20 periods
} }
] ]
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status` **Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:** **Reason:**
- Change every update cycle (every 15 minutes or more frequently) - Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value - Don't provide long-term analytical value
- Create state changes even when core values haven't changed - Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max` **Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:** **Reason:**
- Configuration values that rarely change - Configuration values that rarely change
- Wastes space when recorded repeatedly - Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state - Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error` **Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:** **Reason:**
- Only relevant at moment of reading - Only relevant at moment of reading
- Won't be valid after some time - Won't be valid after some time
- Similar to `entity_picture` in HA core image entities - Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%` **Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:** **Reason:**
- Detailed technical information not needed for historical analysis - Detailed technical information not needed for historical analysis
- Only useful for debugging during active development - Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis - Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining` **Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:** **Reason:**
- Can be calculated from other attributes - Can be calculated from other attributes
- Redundant information - Redundant information
- Doesn't add analytical value to history - Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value: These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core ### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST) - `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states - All price values - Core sensor states
### Diagnostics & Tracking ### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time - `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns - `updates_today` - Tracking API usage patterns
### Data Completeness ### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics - `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status - `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data ### Period Data
- `start`, `end`, `duration_minutes` - Core period timing - `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics - `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status ### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation) - `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact ## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings ### Space Savings
**Per state change:** **Per state change:**
- Before: ~3-8 KB average - Before: ~3-8 KB average
- After: ~0.5-1.5 KB average - After: ~0.5-1.5 KB average
- **Reduction: 60-85%** - **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact ### Real-World Impact
For a typical installation with: For a typical installation with:
- 80+ sensors - 80+ sensors
- Updates every 15 minutes - Updates every 15 minutes
- ~10 sensors updating every minute - ~10 sensors updating every minute
@ -214,7 +231,7 @@ For a typical installation with:
- Class: `TibberPricesBinarySensor` - Class: `TibberPricesBinarySensor`
- 30 attributes excluded - 30 attributes excluded
## When to Update _unrecorded_attributes ## When to Update \_unrecorded_attributes
### Add to Exclusion List When: ### Add to Exclusion List When:
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes 4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:** **SQL Query to check attribute presence:**
```sql ```sql
SELECT SELECT
state_id, state_id,

14
docs/developer/versioned_docs/version-v0.22.1/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):** **In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built: git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile) - **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup` - **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):** **Manual installation (outside DevContainer):**
**git-cliff** (template-based): **git-cliff** (template-based):
```bash ```bash
# See: https://git-cliff.org/docs/installation # See: https://git-cliff.org/docs/installation
@ -191,7 +193,7 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which ## 🎯 When to Use Which
| Method | Use Case | Pros | Cons | | Method | Use Case | Pros | Cons |
|--------|----------|------|------| | --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script | | **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump | | **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization | | **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
@ -219,6 +221,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. Script bumps manifest.json → commits → creates tag locally 1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together 2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release 3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
``` ```
**What happens:** **What happens:**
1. You push manifest.json change 1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically 2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release 3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. You create and push tag manually 1. You create and push tag manually
2. Release workflow creates release 2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists) 3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features ## 🛡️ Safety Features
### 1. **Version Validation** ### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z). Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags** ### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote) - Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating - Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations** ### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push. Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered** ### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits. Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions** ### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind. Helper script shows how to undo if you change your mind.
--- ---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:** **Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes: Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely - Tag already exists remotely
- Invalid version format in manifest.json - Invalid version format in manifest.json
- manifest.json not in the commit that was pushed - manifest.json not in the commit that was pushed
@ -348,6 +359,7 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips ## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results: 1. **Conventional Commits:** Use proper commit format for best results:
``` ```
feat(scope): Add new feature feat(scope): Add new feature

22
docs/developer/versioned_docs/version-v0.22.1/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle. The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:** **Design Principles:**
- **Proactive**: Detect issues before they become critical - **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance - **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve - **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}` **Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:** **When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`) - Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available - Tomorrow's electricity price data is still not available
**When cleared:** **When cleared:**
- Tomorrow's data becomes available - Tomorrow's data becomes available
- Automatically checks on every successful API update - Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work. Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:** **Implementation:**
```python ```python
# In coordinator update cycle # In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"]) has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18) - `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}` **Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:** **When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429) - Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD` - Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:** **When cleared:**
- Successful API call completes (no rate limit error) - Successful API call completes (no rate limit error)
- Error counter resets to 0 - Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires. API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:** **Implementation:**
```python ```python
# In error handler # In error handler
is_rate_limit = ( is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors - `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}` **Issue ID:** `home_not_found_{entry_id}`
**When triggered:** **When triggered:**
- Home configured in this integration is no longer present in Tibber account - Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check) - Detected during user data refresh (daily check)
**When cleared:** **When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected) - Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup) - Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed. Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:** **Implementation:**
```python ```python
# After user data update # After user data update
home_exists = self._data_fetcher._check_home_exists(home_id) home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the missing home - `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference - `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration ### Lifecycle Integration
**Coordinator Initialization:** **Coordinator Initialization:**
```python ```python
self._repair_manager = TibberPricesRepairManager( self._repair_manager = TibberPricesRepairManager(
hass=hass, hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
``` ```
**Update Cycle Integration:** **Update Cycle Integration:**
```python ```python
# Success path - check conditions # Success path - check conditions
if result and "priceInfo" in result: if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
``` ```
**Shutdown Cleanup:** **Shutdown Cleanup:**
```python ```python
async def async_shutdown(self) -> None: async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up.""" """Shut down coordinator and clean up."""
@ -196,6 +212,7 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json` - `/translations/sv.json`
**Structure:** **Structure:**
```json ```json
{ {
"issues": { "issues": {
@ -210,10 +227,12 @@ Repairs use Home Assistant's standard translation system. Translations are defin
## Home Assistant Integration ## Home Assistant Integration
Repairs appear in: Repairs appear in:
- **Settings → System → Repairs** (main repairs panel) - **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count) - **Notifications** (bell icon in UI shows repair count)
Repair properties: Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required) - **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical) - **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files - **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears 4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:** **Manual trigger:**
```python ```python
# Temporarily set warning hour to current hour for testing # Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair 3. Successful API call clears the repair
**Manual test:** **Manual test:**
- Reduce API polling interval to trigger rate limiting - Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client - Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file 7. **Document** in this file
**Example template:** **Example template:**
```python ```python
async def check_new_condition(self, *, param: bool) -> None: async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair.""" """Check new condition and create/clear repair."""

24
docs/developer/versioned_docs/version-v0.22.1/timer-architecture.md
View file

@ -11,7 +11,7 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes: The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method | | Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------| | ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` | | **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` | | **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` | | **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes` **Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:** **What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator` - HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes - Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time) - **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
``` ```
**Load Distribution:** **Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution - Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API - Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once - Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:** **Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects) - Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early - If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully - Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:** **Why we use HA's timer:**
- Automatic restart after HA restart - Automatic restart after HA restart
- Built-in retry logic for temporary failures - Built-in retry logic for temporary failures
- Standard HA integration pattern - Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll** **Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:** **Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48) - Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes - Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13 - Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
``` ```
**Smart Boundary Tolerance:** **Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance - Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval) - HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval) - HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details - See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:** **Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...) - `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes") - NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates) - If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:** **Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`) - All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`) - Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total - ~50-60 entities out of 120+ total
**Why custom timer:** **Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing - HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals - We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation - Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
``` ```
**Which entities listen:** **Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer - `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer - `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%) - `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total - ~10 entities total
**Why custom timer:** **Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time) - Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates - Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation) - Very lightweight (no data processing, just state recalculation)
**Why NOT every second:** **Why NOT every second:**
- Minute precision sufficient for countdown UX - Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds) - Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates) - Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
``` ```
**Why this pattern:** **Why this pattern:**
- Decouples timer logic from entity logic - Decouples timer logic from entity logic
- One timer can notify many entities efficiently - One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup) - Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API ### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers: If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload - ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload - ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem - ❌ "Thundering herd" problem
With HA's unsynchronized timer: With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ... - ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ... - ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ... - ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data. **Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:** **API fetch only when:**
- Tomorrow data missing/invalid (after 13:00) - Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover) - Cache expired (midnight turnover)
- Explicit user refresh - Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics ## Performance Characteristics
### Timer #1 (DataUpdateCoordinator) ### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized) - **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data) - **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate) - **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise) - **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh) ### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries) - **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities) - **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data - **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates - **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh) ### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute) - **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities) - **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all - **No API calls:** No data processing at all
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary ## Summary
**Three independent timers:** **Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed) 1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always) 2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always) 3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:** **Key insights:**
- Timer #1 unsynchronized = good (load distribution on API) - Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately) - Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX) - Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts) - All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:** **"Listener" terminology:**
- Timer = mechanism that triggers - Timer = mechanism that triggers
- Listener = callback that gets called - Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies - Observer pattern = entities register, coordinator notifies

31
docs/developer/versioned_docs/version-v0.23.0/api-reference.md
View file

@ -56,7 +56,7 @@ query {
Fetches quarter-hourly prices: Fetches quarter-hourly prices:
```graphql ```graphql
query($homeId: ID!) { query ($homeId: ID!) {
viewer { viewer {
home(id: $homeId) { home(id: $homeId) {
currentSubscription { currentSubscription {
@ -76,6 +76,7 @@ query($homeId: ID!) {
``` ```
**Parameters:** **Parameters:**
- `homeId`: Tibber home identifier - `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY` - `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data) - `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits ## Rate Limits
Tibber API rate limits (as of 2024): Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token - **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute - **Burst limit:** 100 requests per minute
Integration stays well below these limits: Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day - Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day - User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home - **Total:** ~100 requests/day per home
@ -106,6 +109,7 @@ Integration stays well below these limits:
``` ```
**Fields:** **Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR) - `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone - `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE) - `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -119,6 +123,7 @@ Integration stays well below these limits:
``` ```
Supported currencies: Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh - `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh - `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh - `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses ### Common Error Responses
**Invalid Token:** **Invalid Token:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Unauthorized", "message": "Unauthorized",
"extensions": { "extensions": {
"code": "UNAUTHENTICATED" "code": "UNAUTHENTICATED"
} }
}] }
]
} }
``` ```
**Rate Limit Exceeded:** **Rate Limit Exceeded:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Too Many Requests", "message": "Too Many Requests",
"extensions": { "extensions": {
"code": "RATE_LIMIT_EXCEEDED" "code": "RATE_LIMIT_EXCEEDED"
} }
}] }
]
} }
``` ```
**Home Not Found:** **Home Not Found:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Home not found", "message": "Home not found",
"extensions": { "extensions": {
"code": "NOT_FOUND" "code": "NOT_FOUND"
} }
}] }
]
} }
``` ```
Integration handles these with: Integration handles these with:
- Exponential backoff retry (3 attempts) - Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors - ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures - ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation ## Data Transformation
Raw API data is enriched with: Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals - **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals - **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average - **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
--- ---
💡 **External Resources:** 💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview) - [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer) - [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token) - [Get API Token](https://developer.tibber.com/settings/access-token)

17
docs/developer/versioned_docs/version-v0.23.0/architecture.md
View file

@ -147,7 +147,7 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance: The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory | | Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------| | ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB | | **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB | | **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB | | **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
@ -196,7 +196,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components ### Core Components
| Component | File | Responsibility | | Component | File | Responsibility |
|-----------|------|----------------| | --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling | | **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance | | **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) | | **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
@ -210,7 +210,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025): The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility | | Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------| | ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators | | **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) | | **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) | | **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
@ -219,6 +219,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities | | **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`): **Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access - `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous) - `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows - `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata - `metadata.py` - Home/metering metadata
**Benefits:** **Benefits:**
- 58% reduction in core.py (2,170 → 909 lines) - 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation) - Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator - Independent testability for each calculator
@ -238,7 +240,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities ### Helper Utilities
| Utility | File | Purpose | | Utility | File | Purpose |
|---------|------|---------| | ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation | | **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations | | **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic | | **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025): Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`): **Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals - `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows - `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg - `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics - `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`): **Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods - Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.) - Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`): **Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access - Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc. - Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching) - Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`): **Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation - Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes - Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()` - Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:** **Benefits:**
- Minimal code duplication across 80+ sensors - Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation) - Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing - Easy to extend: Add sensor → choose pattern → add to routing
@ -334,7 +341,7 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization ### CPU Optimization
| Optimization | Location | Savings | | Optimization | Location | Savings |
|--------------|----------|---------| | ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks | | Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation | | Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations | | Lazy logging | Throughout | ~15% on log-heavy operations |

42
docs/developer/versioned_docs/version-v0.23.0/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts. **Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:** **What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total) - **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query - **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation - **Timestamps**: Last update times for validation
**Lifetime:** **Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time) - **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily) - **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage - **Survives**: HA restarts via persistent Storage
@ -36,6 +38,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:** **Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator): 1. **Midnight turnover** (Timer #2 in coordinator):
```python ```python
# coordinator/day_transitions.py # coordinator/day_transitions.py
def _handle_midnight_turnover() -> None: def _handle_midnight_turnover() -> None:
@ -45,6 +48,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
``` ```
2. **Cache validation on load**: 2. **Cache validation on load**:
```python ```python
# coordinator/cache.py # coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool: def is_cache_valid(cache_data: CacheData) -> bool:
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc. **Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:** **What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names - **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions - **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:** **Lifetime:**
- **Forever** (until HA restart) - **Forever** (until HA restart)
- No invalidation during runtime - No invalidation during runtime
**When populated:** **When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py` - At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once - Lazy loading: If translation missing, attempts file load once
**Access pattern:** **Access pattern:**
```python ```python
# Non-blocking synchronous access from cached data # Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en") description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:** **What is cached:**
### DataTransformer Config Cache ### DataTransformer Config Cache
```python ```python
{ {
"thresholds": {"low": 15, "high": 35}, "thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
### PeriodCalculator Config Cache ### PeriodCalculator Config Cache
```python ```python
{ {
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60}, "best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,10 +128,12 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Lifetime:** **Lifetime:**
- Until `invalidate_config_cache()` is called - Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle - Built once on first use per coordinator update cycle
**Invalidation trigger:** **Invalidation trigger:**
- **Options change** (user reconfigures integration): - **Options change** (user reconfigures integration):
```python ```python
# coordinator/core.py # coordinator/core.py
@ -132,6 +144,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Performance impact:** **Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs - **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs - **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update) - **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed. **Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:** **What is cached:**
```python ```python
{ {
"best_price": { "best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Cache key:** Hash of relevant inputs **Cache key:** Hash of relevant inputs
```python ```python
hash_data = ( hash_data = (
today_signature, # (startsAt, rating_level) for each interval today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
``` ```
**Lifetime:** **Lifetime:**
- Until price data changes (today's intervals modified) - Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters) - Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data) - Recalculated at midnight (new today data)
@ -179,6 +195,7 @@ hash_data = (
**Invalidation triggers:** **Invalidation triggers:**
1. **Config change** (explicit): 1. **Config change** (explicit):
```python ```python
def invalidate_config_cache() -> None: def invalidate_config_cache() -> None:
self._cached_periods = None self._cached_periods = None
@ -193,10 +210,12 @@ hash_data = (
``` ```
**Cache hit rate:** **Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged) - **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00) - **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:** **Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts) - **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup) - **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache) - **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy **Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:** **What is cached:**
```python ```python
{ {
"timestamp": ..., "timestamp": ...,
@ -224,6 +244,7 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks. **Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:** **Current behavior:**
- Caches **only enriched price data** (price + statistics) - Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache) - **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when: - Invalidated when:
@ -232,6 +253,7 @@ hash_data = (
- New update cycle begins - New update cycle begins
**Architecture:** **Architecture:**
- DataTransformer: Handles price enrichment only - DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache) - PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches - Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow ## Cache Invalidation Flow
### User Changes Options (Config Flow) ### User Changes Options (Config Flow)
``` ```
User saves options User saves options
↓ ↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
``` ```
### Midnight Turnover (Day Transition) ### Midnight Turnover (Day Transition)
``` ```
Timer #2 fires at 00:00 Timer #2 fires at 00:00
↓ ↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
``` ```
### Tomorrow Data Arrives (~13:00) ### Tomorrow Data Arrives (~13:00)
``` ```
Coordinator update cycle Coordinator update cycle
↓ ↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
``` ```
**No cache invalidation cascades:** **No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update) - Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch) - Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change) - Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load) - Translation cache is **never invalidated** (read-only after load)
**Thread safety:** **Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop) - All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model) - No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics ## Performance Characteristics
### Typical Operation (No Changes) ### Typical Operation (No Changes)
``` ```
Coordinator Update (every 15 min) Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid) ├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
``` ```
### After Midnight Turnover ### After Midnight Turnover
``` ```
Coordinator Update (00:00) Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day) ├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
``` ```
### After Config Change ### After Config Change
``` ```
Options Update Options Update
├─> Cache invalidation: `<`1ms ├─> Cache invalidation: `<`1ms
@ -382,7 +412,7 @@ Options Update
## Summary Table ## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose | | Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------| | ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls | | **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O | | **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups | | **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
@ -392,12 +422,14 @@ Options Update
**Total memory overhead:** ~116KB per coordinator instance (main + subentries) **Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:** **Benefits:**
- 97% reduction in API calls (from every 15min to once per day) - 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation) - 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check) - 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup) - Zero file I/O during runtime (translations cached at startup)
**Trade-offs:** **Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems) - Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented) - Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues - Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues ## Debugging Cache Issues
### Symptom: Stale data after config change ### Symptom: Stale data after config change
**Check:** **Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log) 1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed? 2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger? 3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init. **Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating ### Symptom: Period calculation not updating
**Check:** **Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()` 1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash` 2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs 3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs. **Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today ### Symptom: Yesterday's prices shown as today
**Check:** **Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py` 1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2) 2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs 3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration. **Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations ### Symptom: Missing translations
**Check:** **Check:**
1. `async_load_translations()` called at startup? 1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`? 2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys 3. Cache population: `_TRANSLATIONS_CACHE` keys

3
docs/developer/versioned_docs/version-v0.23.0/coding-guidelines.md
View file

@ -41,12 +41,14 @@ class TimeService:
``` ```
**When prefix is required:** **When prefix is required:**
- Public classes used across multiple modules - Public classes used across multiple modules
- All exception classes - All exception classes
- All coordinator and entity classes - All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs - Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:** **When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore) - Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`) - Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns - Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:** **Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring: Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md` 1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames 2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module 3. Test thoroughly after each module

17
docs/developer/versioned_docs/version-v0.23.0/contributing.md
View file

@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
``` ```
**Branch naming:** **Branch naming:**
- `feature/` - New features - `feature/` - New features
- `fix/` - Bug fixes - `fix/` - Bug fixes
- `docs/` - Documentation only - `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md). Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:** **Run checks frequently:**
```bash ```bash
./scripts/type-check # Pyright type checking ./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix) ./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
``` ```
Run your test: Run your test:
```bash ```bash
./scripts/test tests/test_your_feature.py -v ./scripts/test tests/test_your_feature.py -v
``` ```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
``` ```
**Commit types:** **Commit types:**
- `feat:` - New feature - `feat:` - New feature
- `fix:` - Bug fix - `fix:` - Bug fix
- `docs:` - Documentation - `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance - `chore:` - Maintenance
**Add scope when relevant:** **Add scope when relevant:**
- `feat(sensors):` - Sensor platform - `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator - `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation - `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max) Title: Short, descriptive (50 chars max)
Description should include: Description should include:
```markdown ```markdown
## What ## What
Brief description of changes Brief description of changes
## Why ## Why
Problem being solved or feature rationale Problem being solved or feature rationale
## How ## How
Implementation approach Implementation approach
## Testing ## Testing
- [ ] Manual testing in Home Assistant - [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated - [ ] Unit tests added/updated
- [ ] Type checking passes - [ ] Type checking passes
- [ ] Linting passes - [ ] Linting passes
## Breaking Changes ## Breaking Changes
(If any - describe migration path) (If any - describe migration path)
## Related Issues ## Related Issues
Closes #123 Closes #123
``` ```
### PR Checklist ### PR Checklist
Before submitting: Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md) - [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`) - [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`) - [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For ### What Reviewers Look For
✅ **Good:** ✅ **Good:**
- Clear, self-explanatory code - Clear, self-explanatory code
- Appropriate comments for complex logic - Appropriate comments for complex logic
- Tests covering edge cases - Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns - Follows existing patterns
❌ **Avoid:** ❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones - Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes - Mixing unrelated changes
- Missing tests for new features - Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On ## Finding Issues to Work On
Good first issues are labeled: Good first issues are labeled:
- `good first issue` - Beginner-friendly - `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions - `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements - `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
--- ---
💡 **Related:** 💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup - [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide - [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests - [Testing](testing.md) - Writing tests

33
docs/developer/versioned_docs/version-v0.23.0/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical? ## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to: Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable - **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases - **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks - **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅ #### 1.1 Listener Cleanup ✅
**What is tested:** **What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`) - Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`) - Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`) - Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners - Binary sensor cleanup removes ALL registered listeners
**Why critical:** **Why critical:**
- Each registered listener holds references to Entity + Coordinator - Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak - Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed - With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()` - `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()` - `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()` - `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅ #### 1.2 Timer Cleanup ✅
**What is tested:** **What is tested:**
- Quarter-hour timer is cancelled and reference cleared - Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared - Minute timer is cancelled and reference cleared
- Both timers are cancelled together - Both timers are cancelled together
- Cleanup works even when timers are `None` - Cleanup works even when timers are `None`
**Why critical:** **Why critical:**
- Uncancelled timers continue running after integration unload - Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks - HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates - Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()` - `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅ #### 1.3 Config Entry Cleanup ✅
**What is tested:** **What is tested:**
- Options update listener is registered via `async_on_unload()` - Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()` - Cleanup function is correctly passed to `async_on_unload()`
**Why critical:** **Why critical:**
- `entry.add_update_listener()` registers permanent callback - `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates - Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))` - Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:** **Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration) - `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()` - `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation #### 2.1 Config Cache Invalidation
**What is tested:** **What is tested:**
- DataTransformer config cache is invalidated on options change - DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated - PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update - Trend calculator cache is cleared on coordinator update
**Why critical:** **Why critical:**
- Stale config → Sensors use old user settings - Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods - Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis - Stale trend cache → Outdated trend analysis
**Code Locations:** **Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()` - `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()` - `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()` - `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal #### 3.1 Persistent Storage Removal
**What is tested:** **What is tested:**
- Storage file is deleted on config entry removal - Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss) - Cache is saved on shutdown (no data loss)
**Why critical:** **Why critical:**
- Without storage removal: Old files remain after uninstallation - Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart - Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}` - Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:** **Code Locations:**
- `__init__.py` → `async_remove_entry()` - `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py` **File:** `tests/test_timer_scheduling.py`
**What is tested:** **What is tested:**
- Quarter-hour timer is registered with correct parameters - Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters - Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer) - Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly - Midnight turnover detection works correctly
**Why critical:** **Why critical:**
- Wrong timer parameters → Entities update at wrong times - Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem - Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py` **File:** `tests/test_sensor_timer_assignment.py`
**What is tested:** **What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys - All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys - All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap) - Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked - Sensor and binary sensor platforms are checked
**Why critical:** **Why critical:**
- Wrong timer assignment → Sensors update at wrong times - Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem - Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management ### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks **Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds) - `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms) - `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:** **Why no tests needed:**
- No long-running tasks (all < 2 seconds) - No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically - HA's event loop handles short tasks automatically
- Task exceptions are already logged - Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup ### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented **Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session) - `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created - No new sessions are created
- HA manages session lifecycle automatically - HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory ### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache **Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total - Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading - Module-level cache without re-loading
- Practically no memory issue - Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity ### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop` **Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days) - Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults - Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors - 80+ sensors access `coordinator.data` without errors
**Structure:** **Structure:**
```python ```python
coordinator.data = { coordinator.data = {
"user_data": {...}, "user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory ### 10. Service Response Memory
**Current Status:** HA's response lifecycle **Current Status:** HA's response lifecycle
- HA automatically frees service responses after return - HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call - ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code - No response accumulation in integration code
@ -208,7 +235,7 @@ coordinator.data = {
### ✅ Implemented Tests (41 total) ### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage | | Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------| | ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% | | Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% | | Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% | | Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
@ -222,7 +249,7 @@ coordinator.data = {
### 📋 Analyzed but Not Implemented (Nice-to-Have) ### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale | | Category | Status | Rationale |
|----------|--------|-----------| | ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) | | Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) | | API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) | | Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
@ -230,6 +257,7 @@ coordinator.data = {
| Service Response Memory | 📋 | HA automatically frees service responses | | Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:** **Legend:**
- ✅ = Fully tested or pattern verified correct - ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues) - 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested ### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests: All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks) - ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks) - ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners) - ✅ Config entry cleanup works (no dangling listeners)

18
docs/developer/versioned_docs/version-v0.23.0/debugging.md
View file

@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages ### Key Log Messages
**Coordinator Updates:** **Coordinator Updates:**
``` ```
[custom_components.tibber_prices.coordinator] Successfully fetched price data [custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data [custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
``` ```
**Period Calculation:** **Period Calculation:**
``` ```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0% [custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods [custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
``` ```
**API Errors:** **API Errors:**
``` ```
[custom_components.tibber_prices.api] API request failed: Unauthorized [custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s [custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -67,6 +70,7 @@ Restart Home Assistant to apply.
### Set Breakpoints ### Set Breakpoints
**Coordinator update:** **Coordinator update:**
```python ```python
# coordinator/core.py # coordinator/core.py
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
``` ```
**Period calculation:** **Period calculation:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]: def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
``` ```
**Flags:** **Flags:**
- `-v` - Verbose output - `-v` - Verbose output
- `-s` - Show print statements - `-s` - Show print statements
- `-k pattern` - Run tests matching pattern - `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns ### Useful Test Patterns
**Print coordinator data:** **Print coordinator data:**
```python ```python
def test_something(coordinator): def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}") print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
``` ```
**Inspect period attributes:** **Inspect period attributes:**
```python ```python
def test_periods(hass, coordinator): def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', []) periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading ### Integration Not Loading
**Check:** **Check:**
```bash ```bash
grep "tibber_prices" config/home-assistant.log grep "tibber_prices" config/home-assistant.log
``` ```
**Common causes:** **Common causes:**
- Syntax error in Python code → Check logs for traceback - Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync` - Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*` - Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating ### Sensors Not Updating
**Check coordinator state:** **Check coordinator state:**
```python ```python
# In Developer Tools > Template # In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }} {{ states.sensor.tibber_home_current_interval_price.last_updated }}
``` ```
**Debug in code:** **Debug in code:**
```python ```python
# Add logging in sensor/core.py # Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s", _LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong ### Period Calculation Wrong
**Enable detailed period logs:** **Enable detailed period logs:**
```python ```python
# coordinator/period_handlers/period_building.py # coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s", _LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
``` ```
**Check filter statistics:** **Check filter statistics:**
``` ```
[period_building] Flex filter blocked: 45 intervals [period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals [period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy ### Remote Debugging with debugpy
Add to coordinator code: Add to coordinator code:
```python ```python
import debugpy import debugpy
debugpy.listen(5678) debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL ### IPython REPL
Install in container: Install in container:
```bash ```bash
uv pip install ipython uv pip install ipython
``` ```
Add breakpoint: Add breakpoint:
```python ```python
from IPython import embed from IPython import embed
embed() # Drops into interactive shell embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
--- ---
💡 **Related:** 💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests - [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment - [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure - [Architecture](architecture.md) - Code structure

1
docs/developer/versioned_docs/version-v0.23.0/intro.md
View file

@ -168,6 +168,7 @@ Documentation is organized in two Docusaurus sites:
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory) - **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:** **Best practices:**
- Use clear examples and code snippets - Use clear examples and code snippets
- Keep docs up-to-date with code changes - Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation - Add new pages to appropriate `sidebars.ts` for navigation

17
docs/developer/versioned_docs/version-v0.23.0/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals ## Performance Goals
Target metrics: Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms) - **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor - **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms) - **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching ### Caching
**1. Persistent Cache** (API data): **1. Persistent Cache** (API data):
```python ```python
# Already implemented in coordinator/cache.py # Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY) store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
``` ```
**2. Translation Cache** (in-memory): **2. Translation Cache** (in-memory):
```python ```python
# Already implemented in const.py # Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {} _TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
``` ```
**3. Config Cache** (invalidated on options change): **3. Config Cache** (invalidated on options change):
```python ```python
class DataTransformer: class DataTransformer:
def __init__(self): def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading ### Lazy Loading
**Load data only when needed:** **Load data only when needed:**
```python ```python
@property @property
def extra_state_attributes(self) -> dict | None: def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations ### Bulk Operations
**Process multiple items at once:** **Process multiple items at once:**
```python ```python
# ❌ Slow - loop with individual operations # ❌ Slow - loop with individual operations
for interval in intervals: for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices ### Async Best Practices
**1. Concurrent API calls:** **1. Concurrent API calls:**
```python ```python
# ❌ Sequential (slow) # ❌ Sequential (slow)
user_data = await fetch_user_data() user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
``` ```
**2. Don't block event loop:** **2. Don't block event loop:**
```python ```python
# ❌ Blocking # ❌ Blocking
result = heavy_computation() # Blocks for seconds result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks ### Avoid Memory Leaks
**1. Clear references:** **1. Clear references:**
```python ```python
class Coordinator: class Coordinator:
async def async_shutdown(self): async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
``` ```
**2. Use weak references for callbacks:** **2. Use weak references for callbacks:**
```python ```python
import weakref import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures ### Efficient Data Structures
**Use appropriate types:** **Use appropriate types:**
```python ```python
# ❌ List for lookups (O(n)) # ❌ List for lookups (O(n))
if timestamp in timestamp_list: if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls ### Minimize API Calls
**Already implemented:** **Already implemented:**
- Cache valid until midnight - Cache valid until midnight
- User data cached for 24h - User data cached for 24h
- Only poll when tomorrow data expected - Only poll when tomorrow data expected
**Monitor API usage:** **Monitor API usage:**
```python ```python
_LOGGER.debug("API call: %s (cache_age=%s)", _LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age) endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates ### Smart Updates
**Only update when needed:** **Only update when needed:**
```python ```python
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
"""Fetch data from API.""" """Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection ### State Class Selection
**Affects long-term statistics storage:** **Affects long-term statistics storage:**
```python ```python
# ❌ MEASUREMENT for prices (stores every change) # ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size ### Attribute Size
**Keep attributes minimal:** **Keep attributes minimal:**
```python ```python
# ❌ Large nested structures (KB per update) # ❌ Large nested structures (KB per update)
attributes = { attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
--- ---
💡 **Related:** 💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers - [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design - [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools - [Debugging](debugging.md) - Profiling tools

98
docs/developer/versioned_docs/version-v0.23.0/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic. **Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:** **Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point - `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering - `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy - `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max. **Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be within flex% ABOVE daily minimum # Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex) in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Flex: 15% - Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15) - Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better. **Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be at least min_distance% BELOW daily average # Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100)) meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Min Distance: 5% - Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95) - Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone - Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:** **Implementation:**
```python ```python
# Sensor classification uses user config # Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10) user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,21 +113,25 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5% #### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:** **Given:**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh - Daily Max: 20 ct/kWh
**Flex Filter (50%):** **Flex Filter (50%):**
``` ```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
``` ```
**Min Distance Filter (5%):** **Min Distance Filter (5%):**
``` ```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
``` ```
**Conflict:** **Conflict:**
- Interval at 14.8 ct/kWh: - Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS) - ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL) - ❌ Distance: 14.8 > 14.25 (FAIL)
@ -132,11 +142,13 @@ Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
### Mathematical Analysis ### Mathematical Analysis
**Conflict condition for Best Price:** **Conflict condition for Best Price:**
``` ```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100) daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
``` ```
**Typical values:** **Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5% - Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25` - Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%) - Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases. **Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:** **Formula:**
```python ```python
if flex > 0.20: # 20% threshold if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20 flex_excess = flex - 0.20
@ -159,7 +172,7 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):** **Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale | | Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------| | ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant | | ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction | | 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction | | 30% | 0.75 | 3.75% | Moderate reduction |
@ -167,6 +180,7 @@ if flex > 0.20: # 20% threshold
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides | | 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?** **Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average - Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval - Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period" - Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()` **Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:** **Code Extract:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
``` ```
**Why Linear Scaling?** **Why Linear Scaling?**
- Simple and predictable - Simple and predictable
- No abrupt behavior changes - No abrupt behavior changes
- Easy to reason about for users and developers - Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive) - Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?** **Why 25% Minimum?**
- Below this, min_distance loses semantic meaning - Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed - Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario - Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants ### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:** **Defined in `coordinator/period_handlers/core.py`:**
```python ```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
``` ```
**Defined in `const.py`:** **Defined in `const.py`:**
```python ```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled) DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection) DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances **Goal:** Find practical time windows for running appliances
**Constraints:** **Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h) - Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead) - Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average" - User wants genuinely cheap times, not just "slightly below average"
**Defaults:** **Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use - **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times - **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones - **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:** **User behavior:**
- Automations trigger actions (turn on devices) - Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money - Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives) - Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction **Goal:** Alert users to expensive periods for consumption reduction
**Constraints:** **Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding) - Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy - Early warning more valuable than perfect accuracy
- User can manually decide whether to react - User can manually decide whether to react
**Defaults:** **Defaults:**
- **30 min minimum** - Catches shorter expensive spikes - **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection - **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods - **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:** **User behavior:**
- Notifications/alerts (informational) - Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost - Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes) - Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:** **Peak Price Volatility:**
Price curves tend to have: Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening) - **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical) - **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times - **Higher variance** in peak times than cheap times
**Example day:** **Example day:**
``` ```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
``` ```
**Implication:** **Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief) - Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes - Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection - Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered #### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)** **Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex - Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes - Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?" - User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)** **Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices - No guidance on best practices
- Users would need to experiment to find good values - Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter - Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)** **Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options - **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price - **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case - Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%) **Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:** **Rationale:**
- Above 50%, period detection becomes unreliable - Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals) - Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50% - Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met) - **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:** **Warning Message:**
``` ```
Flex XX% exceeds maximum safe value! Capping at 50%. Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation. Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%) **Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:** **Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold - Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable" - At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation - **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended) #### With Relaxation Enabled (Recommended)
**Optimal:** 10-20% **Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ... - Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work - Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25% **Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side" - INFO log: "Base flex is on the high side"
**High Warning:** > 30% **High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!" - WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20% - Recommendation: Lower to 15-20%
#### Without Relaxation #### Without Relaxation
**Optimal:** 20-35% **Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start - No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback - Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50% **Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits) - Above this, period detection degrades (see Hard Limits)
--- ---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach ### Multi-Phase Approach
**Each day processed independently:** **Each day processed independently:**
1. Calculate baseline periods with user's config 1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop 2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations 3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
``` ```
**Constants:** **Constants:**
```python ```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20% FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -447,6 +488,7 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Historical Context (Pre-November 2025):** **Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex: The algorithm previously used percentage-based increments that scaled with base flex:
```python ```python
increment = base_flex × (step_pct / 100) # REMOVED increment = base_flex × (step_pct / 100) # REMOVED
``` ```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point. This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:** **Warning Messages:**
```python ```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30% if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning( _LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy ### Filter Combination Strategy
**Per Flex level, try in order:** **Per Flex level, try in order:**
1. Original Level filter 1. Original Level filter
2. Level filter = "any" (disabled) 2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations) **Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):** **Example Flow (target=2 periods/day):**
``` ```
Day 2025-11-19: Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2) 1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions ### Key Files and Functions
**Period Calculation Entry Point:** **Period Calculation Entry Point:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods( def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
``` ```
**Flex + Distance Filtering:** **Flex + Distance Filtering:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
def check_interval_criteria( def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
``` ```
**Relaxation Orchestration:** **Relaxation Orchestration:**
```python ```python
# coordinator/period_handlers/relaxation.py # coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict] def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -541,6 +589,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Rejects asymmetric outliers (threshold: 1.5 std dev) - Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks) - Preserves legitimate price shifts (morning/evening peaks)
- Algorithm: - Algorithm:
```python ```python
residual = abs(actual - predicted) residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev symmetry_threshold = 1.5 × std_dev
@ -563,6 +612,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes) - Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:** **Constants:**
```python ```python
# coordinator/period_handlers/outlier_filtering.py # coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
``` ```
**Data Integrity:** **Data Integrity:**
- Original prices stored in `_original_price` field - Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices - All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic - Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome - Smart counting: Only counts smoothing that changed period outcome
**Performance:** **Performance:**
- Single pass through price data - Single pass through price data
- O(n) complexity with small context window - O(n) complexity with small context window
- No iterative refinement needed - No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals - Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:** **Example Debug Output:**
``` ```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -624,6 +677,7 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips ## Debugging Tips
**Enable DEBUG logging:** **Enable DEBUG logging:**
```yaml ```yaml
# configuration.yaml # configuration.yaml
logger: logger:
@ -633,6 +687,7 @@ logger:
``` ```
**Key log messages to watch:** **Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion 1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering 2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded 3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,17 +700,20 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation ### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 40 best_price_flex: 40
enable_relaxation_best: true enable_relaxation_best: true
``` ```
**Problem:** **Problem:**
- Base Flex 40% already very permissive - Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...) - Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns - Quickly approaches 50% cap with diminishing returns
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 15 # Let relaxation increase it best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true enable_relaxation_best: true
@ -664,16 +722,19 @@ enable_relaxation_best: true
### ❌ Anti-Pattern 2: Zero Min_Distance ### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_min_distance_from_avg: 0 best_price_min_distance_from_avg: 0
``` ```
**Problem:** **Problem:**
- "Flat days" (little price variation) accept all intervals - "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap") - Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times - May create periods during barely-below-average times
**Solution:** **Solution:**
```yaml ```yaml
best_price_min_distance_from_avg: 5 # Use default 5% best_price_min_distance_from_avg: 5 # Use default 5%
``` ```
@ -681,16 +742,19 @@ best_price_min_distance_from_avg: 5 # Use default 5%
### ❌ Anti-Pattern 3: Conflicting Flex + Distance ### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 45 best_price_flex: 45
best_price_min_distance_from_avg: 10 best_price_min_distance_from_avg: 10
``` ```
**Problem:** **Problem:**
- Distance filter dominates, making Flex irrelevant - Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal - Dynamic scaling helps but still suboptimal
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 20 best_price_flex: 20
best_price_min_distance_from_avg: 5 best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods - Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient) - Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range - Min_Distance 5%: Effective throughout range
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners) - Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify - Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price" - Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh **Average:** 18 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct) - Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct) - Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive) - Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct) DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction) DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2 **Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2) DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day **Initial State:** Strict filters, very flat day
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2) DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation **Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:** **Algorithm:**
```python ```python
# Pseudo-code for adaptive flex # Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
``` ```
**Benefits:** **Benefits:**
- Eliminates need for relaxation on most days - Eliminates need for relaxation on most days
- Self-adjusting to market conditions - Self-adjusting to market conditions
- Better user experience (less configuration needed) - Better user experience (less configuration needed)
**Challenges:** **Challenges:**
- Harder to predict behavior (less transparent) - Harder to predict behavior (less transparent)
- May conflict with user's mental model - May conflict with user's mental model
- Needs extensive testing across different markets - Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback **Concept:** Learn optimal Flex/Distance from user feedback
**Approach:** **Approach:**
- Track which periods user actually uses (automation triggers) - Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal) - Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults - Apply pattern-specific defaults
- Learn per-user preferences over time - Learn per-user preferences over time
**Benefits:** **Benefits:**
- Personalized to user's actual behavior - Personalized to user's actual behavior
- Adapts to local market patterns - Adapts to local market patterns
- Could discover non-obvious patterns - Could discover non-obvious patterns
**Challenges:** **Challenges:**
- Requires user feedback mechanism (not implemented) - Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns) - Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?" - Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously **Concept:** Balance multiple goals simultaneously
**Goals:** **Goals:**
- Period count vs. quality (cheap vs. very cheap) - Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent) - Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered) - Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher) - User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:** **Algorithm:**
- Pareto optimization (find trade-off frontier) - Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences - User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing - Genetic algorithm or simulated annealing
**Benefits:** **Benefits:**
- More sophisticated period selection - More sophisticated period selection
- Better match to user's actual needs - Better match to user's actual needs
- Could handle complex appliance requirements - Could handle complex appliance requirements
**Challenges:** **Challenges:**
- Much more complex to implement - Much more complex to implement
- Harder to explain to users - Harder to explain to users
- Computational cost (may need caching) - Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex **Current:** 3% cap may be too aggressive for very low base Flex
**Example:** **Example:**
- Base flex 5% + 3% increment = 8% (60% increase!) - Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase) - Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:** **Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)` - Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%) - This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:** **Why Not Implemented:**
- Very low base flex (`<`10%) unusual - Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation - Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization - Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative **Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve **Alternative:** Non-linear curve
```python ```python
# Example: Exponential scaling # Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20)) scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
``` ```
**Why Not Implemented:** **Why Not Implemented:**
- Linear is easier to reason about - Linear is easier to reason about
- No evidence that non-linear is better - No evidence that non-linear is better
- Would need extensive testing - Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day **Issue:** May find all periods in one part of day
**Example:** **Example:**
- All 3 "best price" periods between 02:00-08:00 - All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances) - No periods in evening (when user might want to run appliances)
**Possible Solution:** **Possible Solution:**
- Add "spread" parameter (prefer distributed periods) - Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences - Weight periods by time-of-day preferences
- Consider user's typical usage patterns - Consider user's typical usage patterns
**Why Not Implemented:** **Why Not Implemented:**
- Adds complexity - Adds complexity
- Users can work around with multiple automations - Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all) - Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg). **Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:** **Implementation:**
```python ```python
# In period_building.py build_periods(): # In period_building.py build_periods():
for price_data in all_prices: for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight** **Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split: When days differ significantly, period can split:
``` ```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1) Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2) Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:** **Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours: Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium - Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer - Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,10 +1154,12 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):** **User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful: Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100) - `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre) - `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting: Automations can check volatility before acting:
```yaml ```yaml
condition: condition:
- condition: template - condition: template
@ -1095,6 +1190,7 @@ Low volatility (< 15%) means classification changes are less economically signif
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness. **Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:** **See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes" - User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`) - Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation) - Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

22
docs/developer/versioned_docs/version-v0.23.0/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
``` ```
**Key Points:** **Key Points:**
- Must be a **class attribute** (not instance attribute) - Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance - Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component - Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips` **Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that: **Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely - Never change or change very rarely
- Don't provide analytical value in history - Don't provide analytical value in history
- Consume significant database space when recorded every state change - Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures ### 2. Large Nested Structures
**Attributes:** **Attributes:**
- `periods` (binary_sensor) - Array of all period summaries - `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays - `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis - `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown - `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are: **Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive) - Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each) - Create large database rows (2-20 KB each)
- Slow down history queries - Slow down history queries
@ -66,6 +70,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors **Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:** **Example - periods array:**
```json ```json
{ {
"periods": [ "periods": [
@ -76,7 +81,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
"price_mean": 18.5, "price_mean": 18.5,
"price_median": 18.3, "price_median": 18.3,
"price_min": 17.2, "price_min": 17.2,
"price_max": 19.8, "price_max": 19.8
// ... 10+ more attributes × 10-20 periods // ... 10+ more attributes × 10-20 periods
} }
] ]
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status` **Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:** **Reason:**
- Change every update cycle (every 15 minutes or more frequently) - Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value - Don't provide long-term analytical value
- Create state changes even when core values haven't changed - Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max` **Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:** **Reason:**
- Configuration values that rarely change - Configuration values that rarely change
- Wastes space when recorded repeatedly - Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state - Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error` **Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:** **Reason:**
- Only relevant at moment of reading - Only relevant at moment of reading
- Won't be valid after some time - Won't be valid after some time
- Similar to `entity_picture` in HA core image entities - Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%` **Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:** **Reason:**
- Detailed technical information not needed for historical analysis - Detailed technical information not needed for historical analysis
- Only useful for debugging during active development - Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis - Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining` **Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:** **Reason:**
- Can be calculated from other attributes - Can be calculated from other attributes
- Redundant information - Redundant information
- Doesn't add analytical value to history - Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value: These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core ### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST) - `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states - All price values - Core sensor states
### Diagnostics & Tracking ### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time - `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns - `updates_today` - Tracking API usage patterns
### Data Completeness ### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics - `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status - `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data ### Period Data
- `start`, `end`, `duration_minutes` - Core period timing - `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics - `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status ### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation) - `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact ## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings ### Space Savings
**Per state change:** **Per state change:**
- Before: ~3-8 KB average - Before: ~3-8 KB average
- After: ~0.5-1.5 KB average - After: ~0.5-1.5 KB average
- **Reduction: 60-85%** - **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact ### Real-World Impact
For a typical installation with: For a typical installation with:
- 80+ sensors - 80+ sensors
- Updates every 15 minutes - Updates every 15 minutes
- ~10 sensors updating every minute - ~10 sensors updating every minute
@ -214,7 +231,7 @@ For a typical installation with:
- Class: `TibberPricesBinarySensor` - Class: `TibberPricesBinarySensor`
- 30 attributes excluded - 30 attributes excluded
## When to Update _unrecorded_attributes ## When to Update \_unrecorded_attributes
### Add to Exclusion List When: ### Add to Exclusion List When:
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes 4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:** **SQL Query to check attribute presence:**
```sql ```sql
SELECT SELECT
state_id, state_id,

14
docs/developer/versioned_docs/version-v0.23.0/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):** **In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built: git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile) - **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup` - **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):** **Manual installation (outside DevContainer):**
**git-cliff** (template-based): **git-cliff** (template-based):
```bash ```bash
# See: https://git-cliff.org/docs/installation # See: https://git-cliff.org/docs/installation
@ -191,7 +193,7 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which ## 🎯 When to Use Which
| Method | Use Case | Pros | Cons | | Method | Use Case | Pros | Cons |
|--------|----------|------|------| | --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script | | **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump | | **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization | | **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
@ -219,6 +221,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. Script bumps manifest.json → commits → creates tag locally 1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together 2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release 3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
``` ```
**What happens:** **What happens:**
1. You push manifest.json change 1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically 2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release 3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. You create and push tag manually 1. You create and push tag manually
2. Release workflow creates release 2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists) 3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features ## 🛡️ Safety Features
### 1. **Version Validation** ### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z). Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags** ### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote) - Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating - Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations** ### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push. Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered** ### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits. Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions** ### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind. Helper script shows how to undo if you change your mind.
--- ---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:** **Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes: Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely - Tag already exists remotely
- Invalid version format in manifest.json - Invalid version format in manifest.json
- manifest.json not in the commit that was pushed - manifest.json not in the commit that was pushed
@ -348,6 +359,7 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips ## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results: 1. **Conventional Commits:** Use proper commit format for best results:
``` ```
feat(scope): Add new feature feat(scope): Add new feature

22
docs/developer/versioned_docs/version-v0.23.0/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle. The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:** **Design Principles:**
- **Proactive**: Detect issues before they become critical - **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance - **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve - **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}` **Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:** **When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`) - Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available - Tomorrow's electricity price data is still not available
**When cleared:** **When cleared:**
- Tomorrow's data becomes available - Tomorrow's data becomes available
- Automatically checks on every successful API update - Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work. Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:** **Implementation:**
```python ```python
# In coordinator update cycle # In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"]) has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18) - `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}` **Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:** **When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429) - Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD` - Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:** **When cleared:**
- Successful API call completes (no rate limit error) - Successful API call completes (no rate limit error)
- Error counter resets to 0 - Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires. API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:** **Implementation:**
```python ```python
# In error handler # In error handler
is_rate_limit = ( is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors - `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}` **Issue ID:** `home_not_found_{entry_id}`
**When triggered:** **When triggered:**
- Home configured in this integration is no longer present in Tibber account - Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check) - Detected during user data refresh (daily check)
**When cleared:** **When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected) - Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup) - Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed. Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:** **Implementation:**
```python ```python
# After user data update # After user data update
home_exists = self._data_fetcher._check_home_exists(home_id) home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the missing home - `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference - `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration ### Lifecycle Integration
**Coordinator Initialization:** **Coordinator Initialization:**
```python ```python
self._repair_manager = TibberPricesRepairManager( self._repair_manager = TibberPricesRepairManager(
hass=hass, hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
``` ```
**Update Cycle Integration:** **Update Cycle Integration:**
```python ```python
# Success path - check conditions # Success path - check conditions
if result and "priceInfo" in result: if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
``` ```
**Shutdown Cleanup:** **Shutdown Cleanup:**
```python ```python
async def async_shutdown(self) -> None: async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up.""" """Shut down coordinator and clean up."""
@ -196,6 +212,7 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json` - `/translations/sv.json`
**Structure:** **Structure:**
```json ```json
{ {
"issues": { "issues": {
@ -210,10 +227,12 @@ Repairs use Home Assistant's standard translation system. Translations are defin
## Home Assistant Integration ## Home Assistant Integration
Repairs appear in: Repairs appear in:
- **Settings → System → Repairs** (main repairs panel) - **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count) - **Notifications** (bell icon in UI shows repair count)
Repair properties: Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required) - **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical) - **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files - **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears 4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:** **Manual trigger:**
```python ```python
# Temporarily set warning hour to current hour for testing # Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair 3. Successful API call clears the repair
**Manual test:** **Manual test:**
- Reduce API polling interval to trigger rate limiting - Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client - Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file 7. **Document** in this file
**Example template:** **Example template:**
```python ```python
async def check_new_condition(self, *, param: bool) -> None: async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair.""" """Check new condition and create/clear repair."""

24
docs/developer/versioned_docs/version-v0.23.0/timer-architecture.md
View file

@ -11,7 +11,7 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes: The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method | | Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------| | ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` | | **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` | | **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` | | **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes` **Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:** **What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator` - HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes - Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time) - **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
``` ```
**Load Distribution:** **Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution - Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API - Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once - Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:** **Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects) - Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early - If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully - Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:** **Why we use HA's timer:**
- Automatic restart after HA restart - Automatic restart after HA restart
- Built-in retry logic for temporary failures - Built-in retry logic for temporary failures
- Standard HA integration pattern - Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll** **Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:** **Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48) - Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes - Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13 - Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
``` ```
**Smart Boundary Tolerance:** **Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance - Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval) - HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval) - HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details - See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:** **Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...) - `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes") - NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates) - If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:** **Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`) - All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`) - Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total - ~50-60 entities out of 120+ total
**Why custom timer:** **Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing - HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals - We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation - Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
``` ```
**Which entities listen:** **Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer - `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer - `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%) - `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total - ~10 entities total
**Why custom timer:** **Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time) - Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates - Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation) - Very lightweight (no data processing, just state recalculation)
**Why NOT every second:** **Why NOT every second:**
- Minute precision sufficient for countdown UX - Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds) - Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates) - Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
``` ```
**Why this pattern:** **Why this pattern:**
- Decouples timer logic from entity logic - Decouples timer logic from entity logic
- One timer can notify many entities efficiently - One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup) - Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API ### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers: If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload - ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload - ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem - ❌ "Thundering herd" problem
With HA's unsynchronized timer: With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ... - ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ... - ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ... - ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data. **Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:** **API fetch only when:**
- Tomorrow data missing/invalid (after 13:00) - Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover) - Cache expired (midnight turnover)
- Explicit user refresh - Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics ## Performance Characteristics
### Timer #1 (DataUpdateCoordinator) ### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized) - **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data) - **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate) - **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise) - **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh) ### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries) - **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities) - **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data - **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates - **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh) ### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute) - **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities) - **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all - **No API calls:** No data processing at all
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary ## Summary
**Three independent timers:** **Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed) 1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always) 2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always) 3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:** **Key insights:**
- Timer #1 unsynchronized = good (load distribution on API) - Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately) - Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX) - Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts) - All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:** **"Listener" terminology:**
- Timer = mechanism that triggers - Timer = mechanism that triggers
- Listener = callback that gets called - Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies - Observer pattern = entities register, coordinator notifies

31
docs/developer/versioned_docs/version-v0.23.1/api-reference.md
View file

@ -56,7 +56,7 @@ query {
Fetches quarter-hourly prices: Fetches quarter-hourly prices:
```graphql ```graphql
query($homeId: ID!) { query ($homeId: ID!) {
viewer { viewer {
home(id: $homeId) { home(id: $homeId) {
currentSubscription { currentSubscription {
@ -76,6 +76,7 @@ query($homeId: ID!) {
``` ```
**Parameters:** **Parameters:**
- `homeId`: Tibber home identifier - `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY` - `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data) - `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits ## Rate Limits
Tibber API rate limits (as of 2024): Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token - **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute - **Burst limit:** 100 requests per minute
Integration stays well below these limits: Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day - Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day - User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home - **Total:** ~100 requests/day per home
@ -106,6 +109,7 @@ Integration stays well below these limits:
``` ```
**Fields:** **Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR) - `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone - `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE) - `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -119,6 +123,7 @@ Integration stays well below these limits:
``` ```
Supported currencies: Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh - `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh - `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh - `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses ### Common Error Responses
**Invalid Token:** **Invalid Token:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Unauthorized", "message": "Unauthorized",
"extensions": { "extensions": {
"code": "UNAUTHENTICATED" "code": "UNAUTHENTICATED"
} }
}] }
]
} }
``` ```
**Rate Limit Exceeded:** **Rate Limit Exceeded:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Too Many Requests", "message": "Too Many Requests",
"extensions": { "extensions": {
"code": "RATE_LIMIT_EXCEEDED" "code": "RATE_LIMIT_EXCEEDED"
} }
}] }
]
} }
``` ```
**Home Not Found:** **Home Not Found:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Home not found", "message": "Home not found",
"extensions": { "extensions": {
"code": "NOT_FOUND" "code": "NOT_FOUND"
} }
}] }
]
} }
``` ```
Integration handles these with: Integration handles these with:
- Exponential backoff retry (3 attempts) - Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors - ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures - ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation ## Data Transformation
Raw API data is enriched with: Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals - **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals - **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average - **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
--- ---
💡 **External Resources:** 💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview) - [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer) - [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token) - [Get API Token](https://developer.tibber.com/settings/access-token)

17
docs/developer/versioned_docs/version-v0.23.1/architecture.md
View file

@ -147,7 +147,7 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance: The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory | | Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------| | ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB | | **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB | | **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB | | **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
@ -196,7 +196,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components ### Core Components
| Component | File | Responsibility | | Component | File | Responsibility |
|-----------|------|----------------| | --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling | | **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance | | **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) | | **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
@ -210,7 +210,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025): The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility | | Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------| | ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators | | **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) | | **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) | | **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
@ -219,6 +219,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities | | **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`): **Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access - `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous) - `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows - `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata - `metadata.py` - Home/metering metadata
**Benefits:** **Benefits:**
- 58% reduction in core.py (2,170 → 909 lines) - 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation) - Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator - Independent testability for each calculator
@ -238,7 +240,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities ### Helper Utilities
| Utility | File | Purpose | | Utility | File | Purpose |
|---------|------|---------| | ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation | | **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations | | **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic | | **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025): Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`): **Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals - `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows - `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg - `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics - `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`): **Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods - Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.) - Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`): **Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access - Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc. - Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching) - Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`): **Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation - Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes - Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()` - Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:** **Benefits:**
- Minimal code duplication across 80+ sensors - Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation) - Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing - Easy to extend: Add sensor → choose pattern → add to routing
@ -334,7 +341,7 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization ### CPU Optimization
| Optimization | Location | Savings | | Optimization | Location | Savings |
|--------------|----------|---------| | ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks | | Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation | | Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations | | Lazy logging | Throughout | ~15% on log-heavy operations |

42
docs/developer/versioned_docs/version-v0.23.1/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts. **Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:** **What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total) - **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query - **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation - **Timestamps**: Last update times for validation
**Lifetime:** **Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time) - **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily) - **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage - **Survives**: HA restarts via persistent Storage
@ -36,6 +38,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:** **Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator): 1. **Midnight turnover** (Timer #2 in coordinator):
```python ```python
# coordinator/day_transitions.py # coordinator/day_transitions.py
def _handle_midnight_turnover() -> None: def _handle_midnight_turnover() -> None:
@ -45,6 +48,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
``` ```
2. **Cache validation on load**: 2. **Cache validation on load**:
```python ```python
# coordinator/cache.py # coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool: def is_cache_valid(cache_data: CacheData) -> bool:
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc. **Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:** **What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names - **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions - **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:** **Lifetime:**
- **Forever** (until HA restart) - **Forever** (until HA restart)
- No invalidation during runtime - No invalidation during runtime
**When populated:** **When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py` - At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once - Lazy loading: If translation missing, attempts file load once
**Access pattern:** **Access pattern:**
```python ```python
# Non-blocking synchronous access from cached data # Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en") description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:** **What is cached:**
### DataTransformer Config Cache ### DataTransformer Config Cache
```python ```python
{ {
"thresholds": {"low": 15, "high": 35}, "thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
### PeriodCalculator Config Cache ### PeriodCalculator Config Cache
```python ```python
{ {
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60}, "best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,10 +128,12 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Lifetime:** **Lifetime:**
- Until `invalidate_config_cache()` is called - Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle - Built once on first use per coordinator update cycle
**Invalidation trigger:** **Invalidation trigger:**
- **Options change** (user reconfigures integration): - **Options change** (user reconfigures integration):
```python ```python
# coordinator/core.py # coordinator/core.py
@ -132,6 +144,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Performance impact:** **Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs - **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs - **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update) - **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed. **Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:** **What is cached:**
```python ```python
{ {
"best_price": { "best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Cache key:** Hash of relevant inputs **Cache key:** Hash of relevant inputs
```python ```python
hash_data = ( hash_data = (
today_signature, # (startsAt, rating_level) for each interval today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
``` ```
**Lifetime:** **Lifetime:**
- Until price data changes (today's intervals modified) - Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters) - Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data) - Recalculated at midnight (new today data)
@ -179,6 +195,7 @@ hash_data = (
**Invalidation triggers:** **Invalidation triggers:**
1. **Config change** (explicit): 1. **Config change** (explicit):
```python ```python
def invalidate_config_cache() -> None: def invalidate_config_cache() -> None:
self._cached_periods = None self._cached_periods = None
@ -193,10 +210,12 @@ hash_data = (
``` ```
**Cache hit rate:** **Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged) - **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00) - **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:** **Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts) - **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup) - **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache) - **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy **Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:** **What is cached:**
```python ```python
{ {
"timestamp": ..., "timestamp": ...,
@ -224,6 +244,7 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks. **Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:** **Current behavior:**
- Caches **only enriched price data** (price + statistics) - Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache) - **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when: - Invalidated when:
@ -232,6 +253,7 @@ hash_data = (
- New update cycle begins - New update cycle begins
**Architecture:** **Architecture:**
- DataTransformer: Handles price enrichment only - DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache) - PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches - Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow ## Cache Invalidation Flow
### User Changes Options (Config Flow) ### User Changes Options (Config Flow)
``` ```
User saves options User saves options
↓ ↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
``` ```
### Midnight Turnover (Day Transition) ### Midnight Turnover (Day Transition)
``` ```
Timer #2 fires at 00:00 Timer #2 fires at 00:00
↓ ↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
``` ```
### Tomorrow Data Arrives (~13:00) ### Tomorrow Data Arrives (~13:00)
``` ```
Coordinator update cycle Coordinator update cycle
↓ ↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
``` ```
**No cache invalidation cascades:** **No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update) - Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch) - Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change) - Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load) - Translation cache is **never invalidated** (read-only after load)
**Thread safety:** **Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop) - All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model) - No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics ## Performance Characteristics
### Typical Operation (No Changes) ### Typical Operation (No Changes)
``` ```
Coordinator Update (every 15 min) Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid) ├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
``` ```
### After Midnight Turnover ### After Midnight Turnover
``` ```
Coordinator Update (00:00) Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day) ├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
``` ```
### After Config Change ### After Config Change
``` ```
Options Update Options Update
├─> Cache invalidation: `<`1ms ├─> Cache invalidation: `<`1ms
@ -382,7 +412,7 @@ Options Update
## Summary Table ## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose | | Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------| | ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls | | **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O | | **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups | | **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
@ -392,12 +422,14 @@ Options Update
**Total memory overhead:** ~116KB per coordinator instance (main + subentries) **Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:** **Benefits:**
- 97% reduction in API calls (from every 15min to once per day) - 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation) - 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check) - 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup) - Zero file I/O during runtime (translations cached at startup)
**Trade-offs:** **Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems) - Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented) - Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues - Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues ## Debugging Cache Issues
### Symptom: Stale data after config change ### Symptom: Stale data after config change
**Check:** **Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log) 1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed? 2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger? 3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init. **Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating ### Symptom: Period calculation not updating
**Check:** **Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()` 1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash` 2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs 3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs. **Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today ### Symptom: Yesterday's prices shown as today
**Check:** **Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py` 1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2) 2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs 3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration. **Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations ### Symptom: Missing translations
**Check:** **Check:**
1. `async_load_translations()` called at startup? 1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`? 2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys 3. Cache population: `_TRANSLATIONS_CACHE` keys

3
docs/developer/versioned_docs/version-v0.23.1/coding-guidelines.md
View file

@ -41,12 +41,14 @@ class TimeService:
``` ```
**When prefix is required:** **When prefix is required:**
- Public classes used across multiple modules - Public classes used across multiple modules
- All exception classes - All exception classes
- All coordinator and entity classes - All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs - Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:** **When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore) - Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`) - Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns - Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:** **Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring: Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md` 1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames 2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module 3. Test thoroughly after each module

17
docs/developer/versioned_docs/version-v0.23.1/contributing.md
View file

@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
``` ```
**Branch naming:** **Branch naming:**
- `feature/` - New features - `feature/` - New features
- `fix/` - Bug fixes - `fix/` - Bug fixes
- `docs/` - Documentation only - `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md). Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:** **Run checks frequently:**
```bash ```bash
./scripts/type-check # Pyright type checking ./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix) ./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
``` ```
Run your test: Run your test:
```bash ```bash
./scripts/test tests/test_your_feature.py -v ./scripts/test tests/test_your_feature.py -v
``` ```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
``` ```
**Commit types:** **Commit types:**
- `feat:` - New feature - `feat:` - New feature
- `fix:` - Bug fix - `fix:` - Bug fix
- `docs:` - Documentation - `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance - `chore:` - Maintenance
**Add scope when relevant:** **Add scope when relevant:**
- `feat(sensors):` - Sensor platform - `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator - `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation - `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max) Title: Short, descriptive (50 chars max)
Description should include: Description should include:
```markdown ```markdown
## What ## What
Brief description of changes Brief description of changes
## Why ## Why
Problem being solved or feature rationale Problem being solved or feature rationale
## How ## How
Implementation approach Implementation approach
## Testing ## Testing
- [ ] Manual testing in Home Assistant - [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated - [ ] Unit tests added/updated
- [ ] Type checking passes - [ ] Type checking passes
- [ ] Linting passes - [ ] Linting passes
## Breaking Changes ## Breaking Changes
(If any - describe migration path) (If any - describe migration path)
## Related Issues ## Related Issues
Closes #123 Closes #123
``` ```
### PR Checklist ### PR Checklist
Before submitting: Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md) - [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`) - [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`) - [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For ### What Reviewers Look For
✅ **Good:** ✅ **Good:**
- Clear, self-explanatory code - Clear, self-explanatory code
- Appropriate comments for complex logic - Appropriate comments for complex logic
- Tests covering edge cases - Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns - Follows existing patterns
❌ **Avoid:** ❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones - Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes - Mixing unrelated changes
- Missing tests for new features - Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On ## Finding Issues to Work On
Good first issues are labeled: Good first issues are labeled:
- `good first issue` - Beginner-friendly - `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions - `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements - `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
--- ---
💡 **Related:** 💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup - [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide - [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests - [Testing](testing.md) - Writing tests

33
docs/developer/versioned_docs/version-v0.23.1/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical? ## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to: Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable - **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases - **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks - **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅ #### 1.1 Listener Cleanup ✅
**What is tested:** **What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`) - Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`) - Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`) - Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners - Binary sensor cleanup removes ALL registered listeners
**Why critical:** **Why critical:**
- Each registered listener holds references to Entity + Coordinator - Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak - Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed - With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()` - `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()` - `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()` - `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅ #### 1.2 Timer Cleanup ✅
**What is tested:** **What is tested:**
- Quarter-hour timer is cancelled and reference cleared - Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared - Minute timer is cancelled and reference cleared
- Both timers are cancelled together - Both timers are cancelled together
- Cleanup works even when timers are `None` - Cleanup works even when timers are `None`
**Why critical:** **Why critical:**
- Uncancelled timers continue running after integration unload - Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks - HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates - Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()` - `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅ #### 1.3 Config Entry Cleanup ✅
**What is tested:** **What is tested:**
- Options update listener is registered via `async_on_unload()` - Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()` - Cleanup function is correctly passed to `async_on_unload()`
**Why critical:** **Why critical:**
- `entry.add_update_listener()` registers permanent callback - `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates - Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))` - Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:** **Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration) - `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()` - `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation #### 2.1 Config Cache Invalidation
**What is tested:** **What is tested:**
- DataTransformer config cache is invalidated on options change - DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated - PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update - Trend calculator cache is cleared on coordinator update
**Why critical:** **Why critical:**
- Stale config → Sensors use old user settings - Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods - Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis - Stale trend cache → Outdated trend analysis
**Code Locations:** **Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()` - `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()` - `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()` - `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal #### 3.1 Persistent Storage Removal
**What is tested:** **What is tested:**
- Storage file is deleted on config entry removal - Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss) - Cache is saved on shutdown (no data loss)
**Why critical:** **Why critical:**
- Without storage removal: Old files remain after uninstallation - Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart - Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}` - Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:** **Code Locations:**
- `__init__.py` → `async_remove_entry()` - `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py` **File:** `tests/test_timer_scheduling.py`
**What is tested:** **What is tested:**
- Quarter-hour timer is registered with correct parameters - Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters - Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer) - Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly - Midnight turnover detection works correctly
**Why critical:** **Why critical:**
- Wrong timer parameters → Entities update at wrong times - Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem - Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py` **File:** `tests/test_sensor_timer_assignment.py`
**What is tested:** **What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys - All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys - All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap) - Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked - Sensor and binary sensor platforms are checked
**Why critical:** **Why critical:**
- Wrong timer assignment → Sensors update at wrong times - Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem - Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management ### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks **Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds) - `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms) - `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:** **Why no tests needed:**
- No long-running tasks (all < 2 seconds) - No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically - HA's event loop handles short tasks automatically
- Task exceptions are already logged - Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup ### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented **Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session) - `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created - No new sessions are created
- HA manages session lifecycle automatically - HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory ### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache **Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total - Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading - Module-level cache without re-loading
- Practically no memory issue - Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity ### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop` **Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days) - Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults - Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors - 80+ sensors access `coordinator.data` without errors
**Structure:** **Structure:**
```python ```python
coordinator.data = { coordinator.data = {
"user_data": {...}, "user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory ### 10. Service Response Memory
**Current Status:** HA's response lifecycle **Current Status:** HA's response lifecycle
- HA automatically frees service responses after return - HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call - ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code - No response accumulation in integration code
@ -208,7 +235,7 @@ coordinator.data = {
### ✅ Implemented Tests (41 total) ### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage | | Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------| | ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% | | Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% | | Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% | | Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
@ -222,7 +249,7 @@ coordinator.data = {
### 📋 Analyzed but Not Implemented (Nice-to-Have) ### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale | | Category | Status | Rationale |
|----------|--------|-----------| | ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) | | Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) | | API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) | | Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
@ -230,6 +257,7 @@ coordinator.data = {
| Service Response Memory | 📋 | HA automatically frees service responses | | Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:** **Legend:**
- ✅ = Fully tested or pattern verified correct - ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues) - 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested ### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests: All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks) - ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks) - ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners) - ✅ Config entry cleanup works (no dangling listeners)

18
docs/developer/versioned_docs/version-v0.23.1/debugging.md
View file

@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages ### Key Log Messages
**Coordinator Updates:** **Coordinator Updates:**
``` ```
[custom_components.tibber_prices.coordinator] Successfully fetched price data [custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data [custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
``` ```
**Period Calculation:** **Period Calculation:**
``` ```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0% [custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods [custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
``` ```
**API Errors:** **API Errors:**
``` ```
[custom_components.tibber_prices.api] API request failed: Unauthorized [custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s [custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -67,6 +70,7 @@ Restart Home Assistant to apply.
### Set Breakpoints ### Set Breakpoints
**Coordinator update:** **Coordinator update:**
```python ```python
# coordinator/core.py # coordinator/core.py
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
``` ```
**Period calculation:** **Period calculation:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]: def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
``` ```
**Flags:** **Flags:**
- `-v` - Verbose output - `-v` - Verbose output
- `-s` - Show print statements - `-s` - Show print statements
- `-k pattern` - Run tests matching pattern - `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns ### Useful Test Patterns
**Print coordinator data:** **Print coordinator data:**
```python ```python
def test_something(coordinator): def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}") print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
``` ```
**Inspect period attributes:** **Inspect period attributes:**
```python ```python
def test_periods(hass, coordinator): def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', []) periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading ### Integration Not Loading
**Check:** **Check:**
```bash ```bash
grep "tibber_prices" config/home-assistant.log grep "tibber_prices" config/home-assistant.log
``` ```
**Common causes:** **Common causes:**
- Syntax error in Python code → Check logs for traceback - Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync` - Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*` - Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating ### Sensors Not Updating
**Check coordinator state:** **Check coordinator state:**
```python ```python
# In Developer Tools > Template # In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }} {{ states.sensor.tibber_home_current_interval_price.last_updated }}
``` ```
**Debug in code:** **Debug in code:**
```python ```python
# Add logging in sensor/core.py # Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s", _LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong ### Period Calculation Wrong
**Enable detailed period logs:** **Enable detailed period logs:**
```python ```python
# coordinator/period_handlers/period_building.py # coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s", _LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
``` ```
**Check filter statistics:** **Check filter statistics:**
``` ```
[period_building] Flex filter blocked: 45 intervals [period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals [period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy ### Remote Debugging with debugpy
Add to coordinator code: Add to coordinator code:
```python ```python
import debugpy import debugpy
debugpy.listen(5678) debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL ### IPython REPL
Install in container: Install in container:
```bash ```bash
uv pip install ipython uv pip install ipython
``` ```
Add breakpoint: Add breakpoint:
```python ```python
from IPython import embed from IPython import embed
embed() # Drops into interactive shell embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
--- ---
💡 **Related:** 💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests - [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment - [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure - [Architecture](architecture.md) - Code structure

1
docs/developer/versioned_docs/version-v0.23.1/intro.md
View file

@ -168,6 +168,7 @@ Documentation is organized in two Docusaurus sites:
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory) - **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:** **Best practices:**
- Use clear examples and code snippets - Use clear examples and code snippets
- Keep docs up-to-date with code changes - Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation - Add new pages to appropriate `sidebars.ts` for navigation

17
docs/developer/versioned_docs/version-v0.23.1/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals ## Performance Goals
Target metrics: Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms) - **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor - **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms) - **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching ### Caching
**1. Persistent Cache** (API data): **1. Persistent Cache** (API data):
```python ```python
# Already implemented in coordinator/cache.py # Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY) store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
``` ```
**2. Translation Cache** (in-memory): **2. Translation Cache** (in-memory):
```python ```python
# Already implemented in const.py # Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {} _TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
``` ```
**3. Config Cache** (invalidated on options change): **3. Config Cache** (invalidated on options change):
```python ```python
class DataTransformer: class DataTransformer:
def __init__(self): def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading ### Lazy Loading
**Load data only when needed:** **Load data only when needed:**
```python ```python
@property @property
def extra_state_attributes(self) -> dict | None: def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations ### Bulk Operations
**Process multiple items at once:** **Process multiple items at once:**
```python ```python
# ❌ Slow - loop with individual operations # ❌ Slow - loop with individual operations
for interval in intervals: for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices ### Async Best Practices
**1. Concurrent API calls:** **1. Concurrent API calls:**
```python ```python
# ❌ Sequential (slow) # ❌ Sequential (slow)
user_data = await fetch_user_data() user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
``` ```
**2. Don't block event loop:** **2. Don't block event loop:**
```python ```python
# ❌ Blocking # ❌ Blocking
result = heavy_computation() # Blocks for seconds result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks ### Avoid Memory Leaks
**1. Clear references:** **1. Clear references:**
```python ```python
class Coordinator: class Coordinator:
async def async_shutdown(self): async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
``` ```
**2. Use weak references for callbacks:** **2. Use weak references for callbacks:**
```python ```python
import weakref import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures ### Efficient Data Structures
**Use appropriate types:** **Use appropriate types:**
```python ```python
# ❌ List for lookups (O(n)) # ❌ List for lookups (O(n))
if timestamp in timestamp_list: if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls ### Minimize API Calls
**Already implemented:** **Already implemented:**
- Cache valid until midnight - Cache valid until midnight
- User data cached for 24h - User data cached for 24h
- Only poll when tomorrow data expected - Only poll when tomorrow data expected
**Monitor API usage:** **Monitor API usage:**
```python ```python
_LOGGER.debug("API call: %s (cache_age=%s)", _LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age) endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates ### Smart Updates
**Only update when needed:** **Only update when needed:**
```python ```python
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
"""Fetch data from API.""" """Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection ### State Class Selection
**Affects long-term statistics storage:** **Affects long-term statistics storage:**
```python ```python
# ❌ MEASUREMENT for prices (stores every change) # ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size ### Attribute Size
**Keep attributes minimal:** **Keep attributes minimal:**
```python ```python
# ❌ Large nested structures (KB per update) # ❌ Large nested structures (KB per update)
attributes = { attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
--- ---
💡 **Related:** 💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers - [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design - [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools - [Debugging](debugging.md) - Profiling tools

98
docs/developer/versioned_docs/version-v0.23.1/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic. **Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:** **Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point - `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering - `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy - `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max. **Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be within flex% ABOVE daily minimum # Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex) in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Flex: 15% - Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15) - Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better. **Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be at least min_distance% BELOW daily average # Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100)) meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Min Distance: 5% - Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95) - Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone - Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:** **Implementation:**
```python ```python
# Sensor classification uses user config # Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10) user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,21 +113,25 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5% #### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:** **Given:**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh - Daily Max: 20 ct/kWh
**Flex Filter (50%):** **Flex Filter (50%):**
``` ```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
``` ```
**Min Distance Filter (5%):** **Min Distance Filter (5%):**
``` ```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
``` ```
**Conflict:** **Conflict:**
- Interval at 14.8 ct/kWh: - Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS) - ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL) - ❌ Distance: 14.8 > 14.25 (FAIL)
@ -132,11 +142,13 @@ Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
### Mathematical Analysis ### Mathematical Analysis
**Conflict condition for Best Price:** **Conflict condition for Best Price:**
``` ```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100) daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
``` ```
**Typical values:** **Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5% - Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25` - Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%) - Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases. **Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:** **Formula:**
```python ```python
if flex > 0.20: # 20% threshold if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20 flex_excess = flex - 0.20
@ -159,7 +172,7 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):** **Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale | | Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------| | ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant | | ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction | | 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction | | 30% | 0.75 | 3.75% | Moderate reduction |
@ -167,6 +180,7 @@ if flex > 0.20: # 20% threshold
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides | | 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?** **Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average - Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval - Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period" - Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()` **Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:** **Code Extract:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
``` ```
**Why Linear Scaling?** **Why Linear Scaling?**
- Simple and predictable - Simple and predictable
- No abrupt behavior changes - No abrupt behavior changes
- Easy to reason about for users and developers - Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive) - Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?** **Why 25% Minimum?**
- Below this, min_distance loses semantic meaning - Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed - Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario - Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants ### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:** **Defined in `coordinator/period_handlers/core.py`:**
```python ```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
``` ```
**Defined in `const.py`:** **Defined in `const.py`:**
```python ```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled) DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection) DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances **Goal:** Find practical time windows for running appliances
**Constraints:** **Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h) - Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead) - Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average" - User wants genuinely cheap times, not just "slightly below average"
**Defaults:** **Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use - **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times - **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones - **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:** **User behavior:**
- Automations trigger actions (turn on devices) - Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money - Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives) - Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction **Goal:** Alert users to expensive periods for consumption reduction
**Constraints:** **Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding) - Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy - Early warning more valuable than perfect accuracy
- User can manually decide whether to react - User can manually decide whether to react
**Defaults:** **Defaults:**
- **30 min minimum** - Catches shorter expensive spikes - **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection - **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods - **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:** **User behavior:**
- Notifications/alerts (informational) - Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost - Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes) - Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:** **Peak Price Volatility:**
Price curves tend to have: Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening) - **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical) - **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times - **Higher variance** in peak times than cheap times
**Example day:** **Example day:**
``` ```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
``` ```
**Implication:** **Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief) - Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes - Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection - Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered #### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)** **Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex - Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes - Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?" - User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)** **Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices - No guidance on best practices
- Users would need to experiment to find good values - Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter - Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)** **Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options - **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price - **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case - Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%) **Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:** **Rationale:**
- Above 50%, period detection becomes unreliable - Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals) - Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50% - Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met) - **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:** **Warning Message:**
``` ```
Flex XX% exceeds maximum safe value! Capping at 50%. Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation. Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%) **Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:** **Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold - Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable" - At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation - **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended) #### With Relaxation Enabled (Recommended)
**Optimal:** 10-20% **Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ... - Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work - Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25% **Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side" - INFO log: "Base flex is on the high side"
**High Warning:** > 30% **High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!" - WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20% - Recommendation: Lower to 15-20%
#### Without Relaxation #### Without Relaxation
**Optimal:** 20-35% **Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start - No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback - Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50% **Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits) - Above this, period detection degrades (see Hard Limits)
--- ---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach ### Multi-Phase Approach
**Each day processed independently:** **Each day processed independently:**
1. Calculate baseline periods with user's config 1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop 2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations 3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
``` ```
**Constants:** **Constants:**
```python ```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20% FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -447,6 +488,7 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Historical Context (Pre-November 2025):** **Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex: The algorithm previously used percentage-based increments that scaled with base flex:
```python ```python
increment = base_flex × (step_pct / 100) # REMOVED increment = base_flex × (step_pct / 100) # REMOVED
``` ```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point. This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:** **Warning Messages:**
```python ```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30% if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning( _LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy ### Filter Combination Strategy
**Per Flex level, try in order:** **Per Flex level, try in order:**
1. Original Level filter 1. Original Level filter
2. Level filter = "any" (disabled) 2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations) **Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):** **Example Flow (target=2 periods/day):**
``` ```
Day 2025-11-19: Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2) 1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions ### Key Files and Functions
**Period Calculation Entry Point:** **Period Calculation Entry Point:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods( def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
``` ```
**Flex + Distance Filtering:** **Flex + Distance Filtering:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
def check_interval_criteria( def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
``` ```
**Relaxation Orchestration:** **Relaxation Orchestration:**
```python ```python
# coordinator/period_handlers/relaxation.py # coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict] def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -541,6 +589,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Rejects asymmetric outliers (threshold: 1.5 std dev) - Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks) - Preserves legitimate price shifts (morning/evening peaks)
- Algorithm: - Algorithm:
```python ```python
residual = abs(actual - predicted) residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev symmetry_threshold = 1.5 × std_dev
@ -563,6 +612,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes) - Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:** **Constants:**
```python ```python
# coordinator/period_handlers/outlier_filtering.py # coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
``` ```
**Data Integrity:** **Data Integrity:**
- Original prices stored in `_original_price` field - Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices - All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic - Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome - Smart counting: Only counts smoothing that changed period outcome
**Performance:** **Performance:**
- Single pass through price data - Single pass through price data
- O(n) complexity with small context window - O(n) complexity with small context window
- No iterative refinement needed - No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals - Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:** **Example Debug Output:**
``` ```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -624,6 +677,7 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips ## Debugging Tips
**Enable DEBUG logging:** **Enable DEBUG logging:**
```yaml ```yaml
# configuration.yaml # configuration.yaml
logger: logger:
@ -633,6 +687,7 @@ logger:
``` ```
**Key log messages to watch:** **Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion 1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering 2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded 3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,17 +700,20 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation ### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 40 best_price_flex: 40
enable_relaxation_best: true enable_relaxation_best: true
``` ```
**Problem:** **Problem:**
- Base Flex 40% already very permissive - Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...) - Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns - Quickly approaches 50% cap with diminishing returns
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 15 # Let relaxation increase it best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true enable_relaxation_best: true
@ -664,16 +722,19 @@ enable_relaxation_best: true
### ❌ Anti-Pattern 2: Zero Min_Distance ### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_min_distance_from_avg: 0 best_price_min_distance_from_avg: 0
``` ```
**Problem:** **Problem:**
- "Flat days" (little price variation) accept all intervals - "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap") - Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times - May create periods during barely-below-average times
**Solution:** **Solution:**
```yaml ```yaml
best_price_min_distance_from_avg: 5 # Use default 5% best_price_min_distance_from_avg: 5 # Use default 5%
``` ```
@ -681,16 +742,19 @@ best_price_min_distance_from_avg: 5 # Use default 5%
### ❌ Anti-Pattern 3: Conflicting Flex + Distance ### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 45 best_price_flex: 45
best_price_min_distance_from_avg: 10 best_price_min_distance_from_avg: 10
``` ```
**Problem:** **Problem:**
- Distance filter dominates, making Flex irrelevant - Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal - Dynamic scaling helps but still suboptimal
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 20 best_price_flex: 20
best_price_min_distance_from_avg: 5 best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods - Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient) - Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range - Min_Distance 5%: Effective throughout range
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners) - Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify - Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price" - Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh **Average:** 18 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct) - Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct) - Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive) - Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct) DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction) DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2 **Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2) DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day **Initial State:** Strict filters, very flat day
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2) DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation **Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:** **Algorithm:**
```python ```python
# Pseudo-code for adaptive flex # Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
``` ```
**Benefits:** **Benefits:**
- Eliminates need for relaxation on most days - Eliminates need for relaxation on most days
- Self-adjusting to market conditions - Self-adjusting to market conditions
- Better user experience (less configuration needed) - Better user experience (less configuration needed)
**Challenges:** **Challenges:**
- Harder to predict behavior (less transparent) - Harder to predict behavior (less transparent)
- May conflict with user's mental model - May conflict with user's mental model
- Needs extensive testing across different markets - Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback **Concept:** Learn optimal Flex/Distance from user feedback
**Approach:** **Approach:**
- Track which periods user actually uses (automation triggers) - Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal) - Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults - Apply pattern-specific defaults
- Learn per-user preferences over time - Learn per-user preferences over time
**Benefits:** **Benefits:**
- Personalized to user's actual behavior - Personalized to user's actual behavior
- Adapts to local market patterns - Adapts to local market patterns
- Could discover non-obvious patterns - Could discover non-obvious patterns
**Challenges:** **Challenges:**
- Requires user feedback mechanism (not implemented) - Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns) - Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?" - Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously **Concept:** Balance multiple goals simultaneously
**Goals:** **Goals:**
- Period count vs. quality (cheap vs. very cheap) - Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent) - Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered) - Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher) - User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:** **Algorithm:**
- Pareto optimization (find trade-off frontier) - Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences - User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing - Genetic algorithm or simulated annealing
**Benefits:** **Benefits:**
- More sophisticated period selection - More sophisticated period selection
- Better match to user's actual needs - Better match to user's actual needs
- Could handle complex appliance requirements - Could handle complex appliance requirements
**Challenges:** **Challenges:**
- Much more complex to implement - Much more complex to implement
- Harder to explain to users - Harder to explain to users
- Computational cost (may need caching) - Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex **Current:** 3% cap may be too aggressive for very low base Flex
**Example:** **Example:**
- Base flex 5% + 3% increment = 8% (60% increase!) - Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase) - Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:** **Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)` - Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%) - This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:** **Why Not Implemented:**
- Very low base flex (`<`10%) unusual - Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation - Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization - Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative **Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve **Alternative:** Non-linear curve
```python ```python
# Example: Exponential scaling # Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20)) scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
``` ```
**Why Not Implemented:** **Why Not Implemented:**
- Linear is easier to reason about - Linear is easier to reason about
- No evidence that non-linear is better - No evidence that non-linear is better
- Would need extensive testing - Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day **Issue:** May find all periods in one part of day
**Example:** **Example:**
- All 3 "best price" periods between 02:00-08:00 - All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances) - No periods in evening (when user might want to run appliances)
**Possible Solution:** **Possible Solution:**
- Add "spread" parameter (prefer distributed periods) - Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences - Weight periods by time-of-day preferences
- Consider user's typical usage patterns - Consider user's typical usage patterns
**Why Not Implemented:** **Why Not Implemented:**
- Adds complexity - Adds complexity
- Users can work around with multiple automations - Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all) - Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg). **Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:** **Implementation:**
```python ```python
# In period_building.py build_periods(): # In period_building.py build_periods():
for price_data in all_prices: for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight** **Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split: When days differ significantly, period can split:
``` ```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1) Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2) Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:** **Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours: Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium - Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer - Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,10 +1154,12 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):** **User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful: Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100) - `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre) - `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting: Automations can check volatility before acting:
```yaml ```yaml
condition: condition:
- condition: template - condition: template
@ -1095,6 +1190,7 @@ Low volatility (< 15%) means classification changes are less economically signif
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness. **Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:** **See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes" - User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`) - Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation) - Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

22
docs/developer/versioned_docs/version-v0.23.1/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
``` ```
**Key Points:** **Key Points:**
- Must be a **class attribute** (not instance attribute) - Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance - Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component - Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips` **Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that: **Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely - Never change or change very rarely
- Don't provide analytical value in history - Don't provide analytical value in history
- Consume significant database space when recorded every state change - Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures ### 2. Large Nested Structures
**Attributes:** **Attributes:**
- `periods` (binary_sensor) - Array of all period summaries - `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays - `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis - `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown - `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are: **Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive) - Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each) - Create large database rows (2-20 KB each)
- Slow down history queries - Slow down history queries
@ -66,6 +70,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors **Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:** **Example - periods array:**
```json ```json
{ {
"periods": [ "periods": [
@ -76,7 +81,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
"price_mean": 18.5, "price_mean": 18.5,
"price_median": 18.3, "price_median": 18.3,
"price_min": 17.2, "price_min": 17.2,
"price_max": 19.8, "price_max": 19.8
// ... 10+ more attributes × 10-20 periods // ... 10+ more attributes × 10-20 periods
} }
] ]
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status` **Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:** **Reason:**
- Change every update cycle (every 15 minutes or more frequently) - Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value - Don't provide long-term analytical value
- Create state changes even when core values haven't changed - Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max` **Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:** **Reason:**
- Configuration values that rarely change - Configuration values that rarely change
- Wastes space when recorded repeatedly - Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state - Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error` **Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:** **Reason:**
- Only relevant at moment of reading - Only relevant at moment of reading
- Won't be valid after some time - Won't be valid after some time
- Similar to `entity_picture` in HA core image entities - Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%` **Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:** **Reason:**
- Detailed technical information not needed for historical analysis - Detailed technical information not needed for historical analysis
- Only useful for debugging during active development - Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis - Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining` **Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:** **Reason:**
- Can be calculated from other attributes - Can be calculated from other attributes
- Redundant information - Redundant information
- Doesn't add analytical value to history - Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value: These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core ### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST) - `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states - All price values - Core sensor states
### Diagnostics & Tracking ### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time - `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns - `updates_today` - Tracking API usage patterns
### Data Completeness ### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics - `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status - `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data ### Period Data
- `start`, `end`, `duration_minutes` - Core period timing - `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics - `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status ### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation) - `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact ## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings ### Space Savings
**Per state change:** **Per state change:**
- Before: ~3-8 KB average - Before: ~3-8 KB average
- After: ~0.5-1.5 KB average - After: ~0.5-1.5 KB average
- **Reduction: 60-85%** - **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact ### Real-World Impact
For a typical installation with: For a typical installation with:
- 80+ sensors - 80+ sensors
- Updates every 15 minutes - Updates every 15 minutes
- ~10 sensors updating every minute - ~10 sensors updating every minute
@ -214,7 +231,7 @@ For a typical installation with:
- Class: `TibberPricesBinarySensor` - Class: `TibberPricesBinarySensor`
- 30 attributes excluded - 30 attributes excluded
## When to Update _unrecorded_attributes ## When to Update \_unrecorded_attributes
### Add to Exclusion List When: ### Add to Exclusion List When:
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes 4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:** **SQL Query to check attribute presence:**
```sql ```sql
SELECT SELECT
state_id, state_id,

14
docs/developer/versioned_docs/version-v0.23.1/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):** **In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built: git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile) - **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup` - **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):** **Manual installation (outside DevContainer):**
**git-cliff** (template-based): **git-cliff** (template-based):
```bash ```bash
# See: https://git-cliff.org/docs/installation # See: https://git-cliff.org/docs/installation
@ -191,7 +193,7 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which ## 🎯 When to Use Which
| Method | Use Case | Pros | Cons | | Method | Use Case | Pros | Cons |
|--------|----------|------|------| | --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script | | **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump | | **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization | | **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
@ -219,6 +221,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. Script bumps manifest.json → commits → creates tag locally 1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together 2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release 3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
``` ```
**What happens:** **What happens:**
1. You push manifest.json change 1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically 2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release 3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. You create and push tag manually 1. You create and push tag manually
2. Release workflow creates release 2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists) 3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features ## 🛡️ Safety Features
### 1. **Version Validation** ### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z). Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags** ### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote) - Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating - Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations** ### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push. Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered** ### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits. Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions** ### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind. Helper script shows how to undo if you change your mind.
--- ---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:** **Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes: Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely - Tag already exists remotely
- Invalid version format in manifest.json - Invalid version format in manifest.json
- manifest.json not in the commit that was pushed - manifest.json not in the commit that was pushed
@ -348,6 +359,7 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips ## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results: 1. **Conventional Commits:** Use proper commit format for best results:
``` ```
feat(scope): Add new feature feat(scope): Add new feature

22
docs/developer/versioned_docs/version-v0.23.1/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle. The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:** **Design Principles:**
- **Proactive**: Detect issues before they become critical - **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance - **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve - **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}` **Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:** **When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`) - Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available - Tomorrow's electricity price data is still not available
**When cleared:** **When cleared:**
- Tomorrow's data becomes available - Tomorrow's data becomes available
- Automatically checks on every successful API update - Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work. Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:** **Implementation:**
```python ```python
# In coordinator update cycle # In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"]) has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18) - `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}` **Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:** **When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429) - Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD` - Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:** **When cleared:**
- Successful API call completes (no rate limit error) - Successful API call completes (no rate limit error)
- Error counter resets to 0 - Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires. API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:** **Implementation:**
```python ```python
# In error handler # In error handler
is_rate_limit = ( is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors - `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}` **Issue ID:** `home_not_found_{entry_id}`
**When triggered:** **When triggered:**
- Home configured in this integration is no longer present in Tibber account - Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check) - Detected during user data refresh (daily check)
**When cleared:** **When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected) - Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup) - Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed. Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:** **Implementation:**
```python ```python
# After user data update # After user data update
home_exists = self._data_fetcher._check_home_exists(home_id) home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the missing home - `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference - `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration ### Lifecycle Integration
**Coordinator Initialization:** **Coordinator Initialization:**
```python ```python
self._repair_manager = TibberPricesRepairManager( self._repair_manager = TibberPricesRepairManager(
hass=hass, hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
``` ```
**Update Cycle Integration:** **Update Cycle Integration:**
```python ```python
# Success path - check conditions # Success path - check conditions
if result and "priceInfo" in result: if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
``` ```
**Shutdown Cleanup:** **Shutdown Cleanup:**
```python ```python
async def async_shutdown(self) -> None: async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up.""" """Shut down coordinator and clean up."""
@ -196,6 +212,7 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json` - `/translations/sv.json`
**Structure:** **Structure:**
```json ```json
{ {
"issues": { "issues": {
@ -210,10 +227,12 @@ Repairs use Home Assistant's standard translation system. Translations are defin
## Home Assistant Integration ## Home Assistant Integration
Repairs appear in: Repairs appear in:
- **Settings → System → Repairs** (main repairs panel) - **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count) - **Notifications** (bell icon in UI shows repair count)
Repair properties: Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required) - **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical) - **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files - **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears 4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:** **Manual trigger:**
```python ```python
# Temporarily set warning hour to current hour for testing # Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair 3. Successful API call clears the repair
**Manual test:** **Manual test:**
- Reduce API polling interval to trigger rate limiting - Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client - Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file 7. **Document** in this file
**Example template:** **Example template:**
```python ```python
async def check_new_condition(self, *, param: bool) -> None: async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair.""" """Check new condition and create/clear repair."""

24
docs/developer/versioned_docs/version-v0.23.1/timer-architecture.md
View file

@ -11,7 +11,7 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes: The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method | | Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------| | ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` | | **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` | | **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` | | **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes` **Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:** **What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator` - HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes - Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time) - **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
``` ```
**Load Distribution:** **Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution - Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API - Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once - Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:** **Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects) - Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early - If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully - Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:** **Why we use HA's timer:**
- Automatic restart after HA restart - Automatic restart after HA restart
- Built-in retry logic for temporary failures - Built-in retry logic for temporary failures
- Standard HA integration pattern - Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll** **Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:** **Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48) - Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes - Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13 - Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
``` ```
**Smart Boundary Tolerance:** **Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance - Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval) - HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval) - HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details - See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:** **Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...) - `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes") - NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates) - If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:** **Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`) - All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`) - Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total - ~50-60 entities out of 120+ total
**Why custom timer:** **Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing - HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals - We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation - Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
``` ```
**Which entities listen:** **Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer - `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer - `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%) - `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total - ~10 entities total
**Why custom timer:** **Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time) - Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates - Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation) - Very lightweight (no data processing, just state recalculation)
**Why NOT every second:** **Why NOT every second:**
- Minute precision sufficient for countdown UX - Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds) - Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates) - Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
``` ```
**Why this pattern:** **Why this pattern:**
- Decouples timer logic from entity logic - Decouples timer logic from entity logic
- One timer can notify many entities efficiently - One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup) - Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API ### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers: If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload - ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload - ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem - ❌ "Thundering herd" problem
With HA's unsynchronized timer: With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ... - ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ... - ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ... - ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data. **Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:** **API fetch only when:**
- Tomorrow data missing/invalid (after 13:00) - Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover) - Cache expired (midnight turnover)
- Explicit user refresh - Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics ## Performance Characteristics
### Timer #1 (DataUpdateCoordinator) ### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized) - **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data) - **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate) - **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise) - **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh) ### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries) - **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities) - **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data - **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates - **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh) ### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute) - **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities) - **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all - **No API calls:** No data processing at all
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary ## Summary
**Three independent timers:** **Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed) 1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always) 2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always) 3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:** **Key insights:**
- Timer #1 unsynchronized = good (load distribution on API) - Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately) - Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX) - Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts) - All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:** **"Listener" terminology:**
- Timer = mechanism that triggers - Timer = mechanism that triggers
- Listener = callback that gets called - Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies - Observer pattern = entities register, coordinator notifies

31
docs/developer/versioned_docs/version-v0.24.0/api-reference.md
View file

@ -56,7 +56,7 @@ query {
Fetches quarter-hourly prices: Fetches quarter-hourly prices:
```graphql ```graphql
query($homeId: ID!) { query ($homeId: ID!) {
viewer { viewer {
home(id: $homeId) { home(id: $homeId) {
currentSubscription { currentSubscription {
@ -76,6 +76,7 @@ query($homeId: ID!) {
``` ```
**Parameters:** **Parameters:**
- `homeId`: Tibber home identifier - `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY` - `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data) - `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits ## Rate Limits
Tibber API rate limits (as of 2024): Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token - **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute - **Burst limit:** 100 requests per minute
Integration stays well below these limits: Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day - Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day - User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home - **Total:** ~100 requests/day per home
@ -106,6 +109,7 @@ Integration stays well below these limits:
``` ```
**Fields:** **Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR) - `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone - `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE) - `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -119,6 +123,7 @@ Integration stays well below these limits:
``` ```
Supported currencies: Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh - `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh - `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh - `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses ### Common Error Responses
**Invalid Token:** **Invalid Token:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Unauthorized", "message": "Unauthorized",
"extensions": { "extensions": {
"code": "UNAUTHENTICATED" "code": "UNAUTHENTICATED"
} }
}] }
]
} }
``` ```
**Rate Limit Exceeded:** **Rate Limit Exceeded:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Too Many Requests", "message": "Too Many Requests",
"extensions": { "extensions": {
"code": "RATE_LIMIT_EXCEEDED" "code": "RATE_LIMIT_EXCEEDED"
} }
}] }
]
} }
``` ```
**Home Not Found:** **Home Not Found:**
```json ```json
{ {
"errors": [{ "errors": [
{
"message": "Home not found", "message": "Home not found",
"extensions": { "extensions": {
"code": "NOT_FOUND" "code": "NOT_FOUND"
} }
}] }
]
} }
``` ```
Integration handles these with: Integration handles these with:
- Exponential backoff retry (3 attempts) - Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors - ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures - ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation ## Data Transformation
Raw API data is enriched with: Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals - **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals - **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average - **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
--- ---
💡 **External Resources:** 💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview) - [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer) - [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token) - [Get API Token](https://developer.tibber.com/settings/access-token)

17
docs/developer/versioned_docs/version-v0.24.0/architecture.md
View file

@ -147,7 +147,7 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance: The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory | | Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------| | ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB | | **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB | | **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB | | **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
@ -196,7 +196,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components ### Core Components
| Component | File | Responsibility | | Component | File | Responsibility |
|-----------|------|----------------| | --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling | | **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance | | **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) | | **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
@ -210,7 +210,7 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025): The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility | | Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------| | ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators | | **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) | | **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) | | **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
@ -219,6 +219,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities | | **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`): **Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access - `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous) - `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows - `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata - `metadata.py` - Home/metering metadata
**Benefits:** **Benefits:**
- 58% reduction in core.py (2,170 → 909 lines) - 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation) - Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator - Independent testability for each calculator
@ -238,7 +240,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities ### Helper Utilities
| Utility | File | Purpose | | Utility | File | Purpose |
|---------|------|---------| | ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation | | **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations | | **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic | | **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025): Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`): **Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals - `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows - `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg - `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics - `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`): **Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods - Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.) - Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`): **Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access - Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc. - Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching) - Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`): **Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation - Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes - Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()` - Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:** **Benefits:**
- Minimal code duplication across 80+ sensors - Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation) - Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing - Easy to extend: Add sensor → choose pattern → add to routing
@ -334,7 +341,7 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization ### CPU Optimization
| Optimization | Location | Savings | | Optimization | Location | Savings |
|--------------|----------|---------| | ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks | | Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation | | Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations | | Lazy logging | Throughout | ~15% on log-heavy operations |

42
docs/developer/versioned_docs/version-v0.24.0/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts. **Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:** **What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total) - **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query - **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation - **Timestamps**: Last update times for validation
**Lifetime:** **Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time) - **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily) - **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage - **Survives**: HA restarts via persistent Storage
@ -36,6 +38,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:** **Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator): 1. **Midnight turnover** (Timer #2 in coordinator):
```python ```python
# coordinator/day_transitions.py # coordinator/day_transitions.py
def _handle_midnight_turnover() -> None: def _handle_midnight_turnover() -> None:
@ -45,6 +48,7 @@ The integration uses **4 distinct caching layers** with different purposes and l
``` ```
2. **Cache validation on load**: 2. **Cache validation on load**:
```python ```python
# coordinator/cache.py # coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool: def is_cache_valid(cache_data: CacheData) -> bool:
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc. **Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:** **What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names - **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions - **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:** **Lifetime:**
- **Forever** (until HA restart) - **Forever** (until HA restart)
- No invalidation during runtime - No invalidation during runtime
**When populated:** **When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py` - At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once - Lazy loading: If translation missing, attempts file load once
**Access pattern:** **Access pattern:**
```python ```python
# Non-blocking synchronous access from cached data # Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en") description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:** **What is cached:**
### DataTransformer Config Cache ### DataTransformer Config Cache
```python ```python
{ {
"thresholds": {"low": 15, "high": 35}, "thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
### PeriodCalculator Config Cache ### PeriodCalculator Config Cache
```python ```python
{ {
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60}, "best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,10 +128,12 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Lifetime:** **Lifetime:**
- Until `invalidate_config_cache()` is called - Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle - Built once on first use per coordinator update cycle
**Invalidation trigger:** **Invalidation trigger:**
- **Options change** (user reconfigures integration): - **Options change** (user reconfigures integration):
```python ```python
# coordinator/core.py # coordinator/core.py
@ -132,6 +144,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Performance impact:** **Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs - **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs - **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update) - **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed. **Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:** **What is cached:**
```python ```python
{ {
"best_price": { "best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
``` ```
**Cache key:** Hash of relevant inputs **Cache key:** Hash of relevant inputs
```python ```python
hash_data = ( hash_data = (
today_signature, # (startsAt, rating_level) for each interval today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
``` ```
**Lifetime:** **Lifetime:**
- Until price data changes (today's intervals modified) - Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters) - Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data) - Recalculated at midnight (new today data)
@ -179,6 +195,7 @@ hash_data = (
**Invalidation triggers:** **Invalidation triggers:**
1. **Config change** (explicit): 1. **Config change** (explicit):
```python ```python
def invalidate_config_cache() -> None: def invalidate_config_cache() -> None:
self._cached_periods = None self._cached_periods = None
@ -193,10 +210,12 @@ hash_data = (
``` ```
**Cache hit rate:** **Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged) - **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00) - **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:** **Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts) - **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup) - **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache) - **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy **Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:** **What is cached:**
```python ```python
{ {
"timestamp": ..., "timestamp": ...,
@ -224,6 +244,7 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks. **Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:** **Current behavior:**
- Caches **only enriched price data** (price + statistics) - Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache) - **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when: - Invalidated when:
@ -232,6 +253,7 @@ hash_data = (
- New update cycle begins - New update cycle begins
**Architecture:** **Architecture:**
- DataTransformer: Handles price enrichment only - DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache) - PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches - Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow ## Cache Invalidation Flow
### User Changes Options (Config Flow) ### User Changes Options (Config Flow)
``` ```
User saves options User saves options
↓ ↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
``` ```
### Midnight Turnover (Day Transition) ### Midnight Turnover (Day Transition)
``` ```
Timer #2 fires at 00:00 Timer #2 fires at 00:00
↓ ↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
``` ```
### Tomorrow Data Arrives (~13:00) ### Tomorrow Data Arrives (~13:00)
``` ```
Coordinator update cycle Coordinator update cycle
↓ ↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
``` ```
**No cache invalidation cascades:** **No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update) - Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch) - Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change) - Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load) - Translation cache is **never invalidated** (read-only after load)
**Thread safety:** **Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop) - All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model) - No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics ## Performance Characteristics
### Typical Operation (No Changes) ### Typical Operation (No Changes)
``` ```
Coordinator Update (every 15 min) Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid) ├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
``` ```
### After Midnight Turnover ### After Midnight Turnover
``` ```
Coordinator Update (00:00) Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day) ├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
``` ```
### After Config Change ### After Config Change
``` ```
Options Update Options Update
├─> Cache invalidation: `<`1ms ├─> Cache invalidation: `<`1ms
@ -382,7 +412,7 @@ Options Update
## Summary Table ## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose | | Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------| | ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls | | **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O | | **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups | | **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
@ -392,12 +422,14 @@ Options Update
**Total memory overhead:** ~116KB per coordinator instance (main + subentries) **Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:** **Benefits:**
- 97% reduction in API calls (from every 15min to once per day) - 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation) - 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check) - 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup) - Zero file I/O during runtime (translations cached at startup)
**Trade-offs:** **Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems) - Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented) - Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues - Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues ## Debugging Cache Issues
### Symptom: Stale data after config change ### Symptom: Stale data after config change
**Check:** **Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log) 1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed? 2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger? 3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init. **Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating ### Symptom: Period calculation not updating
**Check:** **Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()` 1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash` 2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs 3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs. **Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today ### Symptom: Yesterday's prices shown as today
**Check:** **Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py` 1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2) 2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs 3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration. **Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations ### Symptom: Missing translations
**Check:** **Check:**
1. `async_load_translations()` called at startup? 1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`? 2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys 3. Cache population: `_TRANSLATIONS_CACHE` keys

3
docs/developer/versioned_docs/version-v0.24.0/coding-guidelines.md
View file

@ -41,12 +41,14 @@ class TimeService:
``` ```
**When prefix is required:** **When prefix is required:**
- Public classes used across multiple modules - Public classes used across multiple modules
- All exception classes - All exception classes
- All coordinator and entity classes - All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs - Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:** **When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore) - Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`) - Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns - Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:** **Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring: Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md` 1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames 2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module 3. Test thoroughly after each module

17
docs/developer/versioned_docs/version-v0.24.0/contributing.md
View file

@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
``` ```
**Branch naming:** **Branch naming:**
- `feature/` - New features - `feature/` - New features
- `fix/` - Bug fixes - `fix/` - Bug fixes
- `docs/` - Documentation only - `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md). Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:** **Run checks frequently:**
```bash ```bash
./scripts/type-check # Pyright type checking ./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix) ./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
``` ```
Run your test: Run your test:
```bash ```bash
./scripts/test tests/test_your_feature.py -v ./scripts/test tests/test_your_feature.py -v
``` ```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
``` ```
**Commit types:** **Commit types:**
- `feat:` - New feature - `feat:` - New feature
- `fix:` - Bug fix - `fix:` - Bug fix
- `docs:` - Documentation - `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance - `chore:` - Maintenance
**Add scope when relevant:** **Add scope when relevant:**
- `feat(sensors):` - Sensor platform - `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator - `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation - `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max) Title: Short, descriptive (50 chars max)
Description should include: Description should include:
```markdown ```markdown
## What ## What
Brief description of changes Brief description of changes
## Why ## Why
Problem being solved or feature rationale Problem being solved or feature rationale
## How ## How
Implementation approach Implementation approach
## Testing ## Testing
- [ ] Manual testing in Home Assistant - [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated - [ ] Unit tests added/updated
- [ ] Type checking passes - [ ] Type checking passes
- [ ] Linting passes - [ ] Linting passes
## Breaking Changes ## Breaking Changes
(If any - describe migration path) (If any - describe migration path)
## Related Issues ## Related Issues
Closes #123 Closes #123
``` ```
### PR Checklist ### PR Checklist
Before submitting: Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md) - [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`) - [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`) - [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For ### What Reviewers Look For
✅ **Good:** ✅ **Good:**
- Clear, self-explanatory code - Clear, self-explanatory code
- Appropriate comments for complex logic - Appropriate comments for complex logic
- Tests covering edge cases - Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns - Follows existing patterns
❌ **Avoid:** ❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones - Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes - Mixing unrelated changes
- Missing tests for new features - Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On ## Finding Issues to Work On
Good first issues are labeled: Good first issues are labeled:
- `good first issue` - Beginner-friendly - `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions - `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements - `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
--- ---
💡 **Related:** 💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup - [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide - [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests - [Testing](testing.md) - Writing tests

33
docs/developer/versioned_docs/version-v0.24.0/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical? ## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to: Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable - **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases - **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks - **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅ #### 1.1 Listener Cleanup ✅
**What is tested:** **What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`) - Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`) - Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`) - Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners - Binary sensor cleanup removes ALL registered listeners
**Why critical:** **Why critical:**
- Each registered listener holds references to Entity + Coordinator - Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak - Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed - With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()` - `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()` - `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()` - `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅ #### 1.2 Timer Cleanup ✅
**What is tested:** **What is tested:**
- Quarter-hour timer is cancelled and reference cleared - Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared - Minute timer is cancelled and reference cleared
- Both timers are cancelled together - Both timers are cancelled together
- Cleanup works even when timers are `None` - Cleanup works even when timers are `None`
**Why critical:** **Why critical:**
- Uncancelled timers continue running after integration unload - Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks - HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates - Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:** **Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()` - `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅ #### 1.3 Config Entry Cleanup ✅
**What is tested:** **What is tested:**
- Options update listener is registered via `async_on_unload()` - Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()` - Cleanup function is correctly passed to `async_on_unload()`
**Why critical:** **Why critical:**
- `entry.add_update_listener()` registers permanent callback - `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates - Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))` - Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:** **Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration) - `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()` - `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation #### 2.1 Config Cache Invalidation
**What is tested:** **What is tested:**
- DataTransformer config cache is invalidated on options change - DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated - PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update - Trend calculator cache is cleared on coordinator update
**Why critical:** **Why critical:**
- Stale config → Sensors use old user settings - Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods - Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis - Stale trend cache → Outdated trend analysis
**Code Locations:** **Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()` - `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()` - `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()` - `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal #### 3.1 Persistent Storage Removal
**What is tested:** **What is tested:**
- Storage file is deleted on config entry removal - Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss) - Cache is saved on shutdown (no data loss)
**Why critical:** **Why critical:**
- Without storage removal: Old files remain after uninstallation - Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart - Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}` - Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:** **Code Locations:**
- `__init__.py` → `async_remove_entry()` - `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()` - `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py` **File:** `tests/test_timer_scheduling.py`
**What is tested:** **What is tested:**
- Quarter-hour timer is registered with correct parameters - Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters - Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer) - Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly - Midnight turnover detection works correctly
**Why critical:** **Why critical:**
- Wrong timer parameters → Entities update at wrong times - Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem - Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py` **File:** `tests/test_sensor_timer_assignment.py`
**What is tested:** **What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys - All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys - All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap) - Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked - Sensor and binary sensor platforms are checked
**Why critical:** **Why critical:**
- Wrong timer assignment → Sensors update at wrong times - Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem - Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management ### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks **Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds) - `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms) - `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:** **Why no tests needed:**
- No long-running tasks (all < 2 seconds) - No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically - HA's event loop handles short tasks automatically
- Task exceptions are already logged - Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup ### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented **Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session) - `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created - No new sessions are created
- HA manages session lifecycle automatically - HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory ### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache **Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total - Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading - Module-level cache without re-loading
- Practically no memory issue - Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity ### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop` **Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days) - Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults - Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors - 80+ sensors access `coordinator.data` without errors
**Structure:** **Structure:**
```python ```python
coordinator.data = { coordinator.data = {
"user_data": {...}, "user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory ### 10. Service Response Memory
**Current Status:** HA's response lifecycle **Current Status:** HA's response lifecycle
- HA automatically frees service responses after return - HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call - ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code - No response accumulation in integration code
@ -208,7 +235,7 @@ coordinator.data = {
### ✅ Implemented Tests (41 total) ### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage | | Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------| | ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% | | Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% | | Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% | | Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
@ -222,7 +249,7 @@ coordinator.data = {
### 📋 Analyzed but Not Implemented (Nice-to-Have) ### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale | | Category | Status | Rationale |
|----------|--------|-----------| | ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) | | Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) | | API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) | | Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
@ -230,6 +257,7 @@ coordinator.data = {
| Service Response Memory | 📋 | HA automatically frees service responses | | Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:** **Legend:**
- ✅ = Fully tested or pattern verified correct - ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues) - 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested ### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests: All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks) - ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks) - ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners) - ✅ Config entry cleanup works (no dangling listeners)

18
docs/developer/versioned_docs/version-v0.24.0/debugging.md
View file

@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages ### Key Log Messages
**Coordinator Updates:** **Coordinator Updates:**
``` ```
[custom_components.tibber_prices.coordinator] Successfully fetched price data [custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data [custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
``` ```
**Period Calculation:** **Period Calculation:**
``` ```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0% [custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods [custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
``` ```
**API Errors:** **API Errors:**
``` ```
[custom_components.tibber_prices.api] API request failed: Unauthorized [custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s [custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -67,6 +70,7 @@ Restart Home Assistant to apply.
### Set Breakpoints ### Set Breakpoints
**Coordinator update:** **Coordinator update:**
```python ```python
# coordinator/core.py # coordinator/core.py
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
``` ```
**Period calculation:** **Period calculation:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]: def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
``` ```
**Flags:** **Flags:**
- `-v` - Verbose output - `-v` - Verbose output
- `-s` - Show print statements - `-s` - Show print statements
- `-k pattern` - Run tests matching pattern - `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns ### Useful Test Patterns
**Print coordinator data:** **Print coordinator data:**
```python ```python
def test_something(coordinator): def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}") print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
``` ```
**Inspect period attributes:** **Inspect period attributes:**
```python ```python
def test_periods(hass, coordinator): def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', []) periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading ### Integration Not Loading
**Check:** **Check:**
```bash ```bash
grep "tibber_prices" config/home-assistant.log grep "tibber_prices" config/home-assistant.log
``` ```
**Common causes:** **Common causes:**
- Syntax error in Python code → Check logs for traceback - Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync` - Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*` - Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating ### Sensors Not Updating
**Check coordinator state:** **Check coordinator state:**
```python ```python
# In Developer Tools > Template # In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }} {{ states.sensor.tibber_home_current_interval_price.last_updated }}
``` ```
**Debug in code:** **Debug in code:**
```python ```python
# Add logging in sensor/core.py # Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s", _LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong ### Period Calculation Wrong
**Enable detailed period logs:** **Enable detailed period logs:**
```python ```python
# coordinator/period_handlers/period_building.py # coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s", _LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
``` ```
**Check filter statistics:** **Check filter statistics:**
``` ```
[period_building] Flex filter blocked: 45 intervals [period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals [period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy ### Remote Debugging with debugpy
Add to coordinator code: Add to coordinator code:
```python ```python
import debugpy import debugpy
debugpy.listen(5678) debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL ### IPython REPL
Install in container: Install in container:
```bash ```bash
uv pip install ipython uv pip install ipython
``` ```
Add breakpoint: Add breakpoint:
```python ```python
from IPython import embed from IPython import embed
embed() # Drops into interactive shell embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
--- ---
💡 **Related:** 💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests - [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment - [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure - [Architecture](architecture.md) - Code structure

1
docs/developer/versioned_docs/version-v0.24.0/intro.md
View file

@ -168,6 +168,7 @@ Documentation is organized in two Docusaurus sites:
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory) - **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:** **Best practices:**
- Use clear examples and code snippets - Use clear examples and code snippets
- Keep docs up-to-date with code changes - Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation - Add new pages to appropriate `sidebars.ts` for navigation

17
docs/developer/versioned_docs/version-v0.24.0/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals ## Performance Goals
Target metrics: Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms) - **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor - **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms) - **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching ### Caching
**1. Persistent Cache** (API data): **1. Persistent Cache** (API data):
```python ```python
# Already implemented in coordinator/cache.py # Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY) store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
``` ```
**2. Translation Cache** (in-memory): **2. Translation Cache** (in-memory):
```python ```python
# Already implemented in const.py # Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {} _TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
``` ```
**3. Config Cache** (invalidated on options change): **3. Config Cache** (invalidated on options change):
```python ```python
class DataTransformer: class DataTransformer:
def __init__(self): def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading ### Lazy Loading
**Load data only when needed:** **Load data only when needed:**
```python ```python
@property @property
def extra_state_attributes(self) -> dict | None: def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations ### Bulk Operations
**Process multiple items at once:** **Process multiple items at once:**
```python ```python
# ❌ Slow - loop with individual operations # ❌ Slow - loop with individual operations
for interval in intervals: for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices ### Async Best Practices
**1. Concurrent API calls:** **1. Concurrent API calls:**
```python ```python
# ❌ Sequential (slow) # ❌ Sequential (slow)
user_data = await fetch_user_data() user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
``` ```
**2. Don't block event loop:** **2. Don't block event loop:**
```python ```python
# ❌ Blocking # ❌ Blocking
result = heavy_computation() # Blocks for seconds result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks ### Avoid Memory Leaks
**1. Clear references:** **1. Clear references:**
```python ```python
class Coordinator: class Coordinator:
async def async_shutdown(self): async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
``` ```
**2. Use weak references for callbacks:** **2. Use weak references for callbacks:**
```python ```python
import weakref import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures ### Efficient Data Structures
**Use appropriate types:** **Use appropriate types:**
```python ```python
# ❌ List for lookups (O(n)) # ❌ List for lookups (O(n))
if timestamp in timestamp_list: if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls ### Minimize API Calls
**Already implemented:** **Already implemented:**
- Cache valid until midnight - Cache valid until midnight
- User data cached for 24h - User data cached for 24h
- Only poll when tomorrow data expected - Only poll when tomorrow data expected
**Monitor API usage:** **Monitor API usage:**
```python ```python
_LOGGER.debug("API call: %s (cache_age=%s)", _LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age) endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates ### Smart Updates
**Only update when needed:** **Only update when needed:**
```python ```python
async def _async_update_data(self) -> dict: async def _async_update_data(self) -> dict:
"""Fetch data from API.""" """Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection ### State Class Selection
**Affects long-term statistics storage:** **Affects long-term statistics storage:**
```python ```python
# ❌ MEASUREMENT for prices (stores every change) # ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size ### Attribute Size
**Keep attributes minimal:** **Keep attributes minimal:**
```python ```python
# ❌ Large nested structures (KB per update) # ❌ Large nested structures (KB per update)
attributes = { attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
--- ---
💡 **Related:** 💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers - [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design - [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools - [Debugging](debugging.md) - Profiling tools

98
docs/developer/versioned_docs/version-v0.24.0/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic. **Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:** **Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point - `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering - `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy - `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max. **Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be within flex% ABOVE daily minimum # Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex) in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Flex: 15% - Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15) - Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better. **Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:** **Logic:**
```python ```python
# Best Price: Price must be at least min_distance% BELOW daily average # Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100)) meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
``` ```
**Example (Best Price):** **Example (Best Price):**
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Min Distance: 5% - Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95) - Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone - Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:** **Implementation:**
```python ```python
# Sensor classification uses user config # Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10) user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,21 +113,25 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5% #### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:** **Given:**
- Daily Min: 10 ct/kWh - Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh - Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh - Daily Max: 20 ct/kWh
**Flex Filter (50%):** **Flex Filter (50%):**
``` ```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
``` ```
**Min Distance Filter (5%):** **Min Distance Filter (5%):**
``` ```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
``` ```
**Conflict:** **Conflict:**
- Interval at 14.8 ct/kWh: - Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS) - ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL) - ❌ Distance: 14.8 > 14.25 (FAIL)
@ -132,11 +142,13 @@ Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
### Mathematical Analysis ### Mathematical Analysis
**Conflict condition for Best Price:** **Conflict condition for Best Price:**
``` ```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100) daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
``` ```
**Typical values:** **Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5% - Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25` - Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%) - Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases. **Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:** **Formula:**
```python ```python
if flex > 0.20: # 20% threshold if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20 flex_excess = flex - 0.20
@ -159,7 +172,7 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):** **Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale | | Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------| | ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant | | ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction | | 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction | | 30% | 0.75 | 3.75% | Moderate reduction |
@ -167,6 +180,7 @@ if flex > 0.20: # 20% threshold
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides | | 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?** **Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average - Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval - Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period" - Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()` **Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:** **Code Extract:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
``` ```
**Why Linear Scaling?** **Why Linear Scaling?**
- Simple and predictable - Simple and predictable
- No abrupt behavior changes - No abrupt behavior changes
- Easy to reason about for users and developers - Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive) - Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?** **Why 25% Minimum?**
- Below this, min_distance loses semantic meaning - Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed - Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario - Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants ### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:** **Defined in `coordinator/period_handlers/core.py`:**
```python ```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
``` ```
**Defined in `const.py`:** **Defined in `const.py`:**
```python ```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled) DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection) DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances **Goal:** Find practical time windows for running appliances
**Constraints:** **Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h) - Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead) - Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average" - User wants genuinely cheap times, not just "slightly below average"
**Defaults:** **Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use - **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times - **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones - **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:** **User behavior:**
- Automations trigger actions (turn on devices) - Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money - Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives) - Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction **Goal:** Alert users to expensive periods for consumption reduction
**Constraints:** **Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding) - Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy - Early warning more valuable than perfect accuracy
- User can manually decide whether to react - User can manually decide whether to react
**Defaults:** **Defaults:**
- **30 min minimum** - Catches shorter expensive spikes - **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection - **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods - **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:** **User behavior:**
- Notifications/alerts (informational) - Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost - Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes) - Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:** **Peak Price Volatility:**
Price curves tend to have: Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening) - **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical) - **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times - **Higher variance** in peak times than cheap times
**Example day:** **Example day:**
``` ```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
``` ```
**Implication:** **Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief) - Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes - Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection - Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered #### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)** **Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex - Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes - Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?" - User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)** **Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices - No guidance on best practices
- Users would need to experiment to find good values - Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter - Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)** **Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options - **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price - **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case - Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%) **Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:** **Rationale:**
- Above 50%, period detection becomes unreliable - Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals) - Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50% - Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met) - **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:** **Warning Message:**
``` ```
Flex XX% exceeds maximum safe value! Capping at 50%. Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation. Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%) **Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:** **Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold - Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable" - At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation - **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended) #### With Relaxation Enabled (Recommended)
**Optimal:** 10-20% **Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ... - Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work - Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25% **Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side" - INFO log: "Base flex is on the high side"
**High Warning:** > 30% **High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!" - WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20% - Recommendation: Lower to 15-20%
#### Without Relaxation #### Without Relaxation
**Optimal:** 20-35% **Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start - No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback - Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50% **Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits) - Above this, period detection degrades (see Hard Limits)
--- ---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach ### Multi-Phase Approach
**Each day processed independently:** **Each day processed independently:**
1. Calculate baseline periods with user's config 1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop 2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations 3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
``` ```
**Constants:** **Constants:**
```python ```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20% FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -447,6 +488,7 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Historical Context (Pre-November 2025):** **Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex: The algorithm previously used percentage-based increments that scaled with base flex:
```python ```python
increment = base_flex × (step_pct / 100) # REMOVED increment = base_flex × (step_pct / 100) # REMOVED
``` ```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point. This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:** **Warning Messages:**
```python ```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30% if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning( _LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy ### Filter Combination Strategy
**Per Flex level, try in order:** **Per Flex level, try in order:**
1. Original Level filter 1. Original Level filter
2. Level filter = "any" (disabled) 2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations) **Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):** **Example Flow (target=2 periods/day):**
``` ```
Day 2025-11-19: Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2) 1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions ### Key Files and Functions
**Period Calculation Entry Point:** **Period Calculation Entry Point:**
```python ```python
# coordinator/period_handlers/core.py # coordinator/period_handlers/core.py
def calculate_periods( def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
``` ```
**Flex + Distance Filtering:** **Flex + Distance Filtering:**
```python ```python
# coordinator/period_handlers/level_filtering.py # coordinator/period_handlers/level_filtering.py
def check_interval_criteria( def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
``` ```
**Relaxation Orchestration:** **Relaxation Orchestration:**
```python ```python
# coordinator/period_handlers/relaxation.py # coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict] def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -541,6 +589,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Rejects asymmetric outliers (threshold: 1.5 std dev) - Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks) - Preserves legitimate price shifts (morning/evening peaks)
- Algorithm: - Algorithm:
```python ```python
residual = abs(actual - predicted) residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev symmetry_threshold = 1.5 × std_dev
@ -563,6 +612,7 @@ def relax_single_day(...) -> tuple[dict, dict]
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes) - Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:** **Constants:**
```python ```python
# coordinator/period_handlers/outlier_filtering.py # coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
``` ```
**Data Integrity:** **Data Integrity:**
- Original prices stored in `_original_price` field - Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices - All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic - Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome - Smart counting: Only counts smoothing that changed period outcome
**Performance:** **Performance:**
- Single pass through price data - Single pass through price data
- O(n) complexity with small context window - O(n) complexity with small context window
- No iterative refinement needed - No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals - Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:** **Example Debug Output:**
``` ```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -624,6 +677,7 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips ## Debugging Tips
**Enable DEBUG logging:** **Enable DEBUG logging:**
```yaml ```yaml
# configuration.yaml # configuration.yaml
logger: logger:
@ -633,6 +687,7 @@ logger:
``` ```
**Key log messages to watch:** **Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion 1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering 2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded 3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,17 +700,20 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation ### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 40 best_price_flex: 40
enable_relaxation_best: true enable_relaxation_best: true
``` ```
**Problem:** **Problem:**
- Base Flex 40% already very permissive - Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...) - Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns - Quickly approaches 50% cap with diminishing returns
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 15 # Let relaxation increase it best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true enable_relaxation_best: true
@ -664,16 +722,19 @@ enable_relaxation_best: true
### ❌ Anti-Pattern 2: Zero Min_Distance ### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_min_distance_from_avg: 0 best_price_min_distance_from_avg: 0
``` ```
**Problem:** **Problem:**
- "Flat days" (little price variation) accept all intervals - "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap") - Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times - May create periods during barely-below-average times
**Solution:** **Solution:**
```yaml ```yaml
best_price_min_distance_from_avg: 5 # Use default 5% best_price_min_distance_from_avg: 5 # Use default 5%
``` ```
@ -681,16 +742,19 @@ best_price_min_distance_from_avg: 5 # Use default 5%
### ❌ Anti-Pattern 3: Conflicting Flex + Distance ### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:** **Configuration:**
```yaml ```yaml
best_price_flex: 45 best_price_flex: 45
best_price_min_distance_from_avg: 10 best_price_min_distance_from_avg: 10
``` ```
**Problem:** **Problem:**
- Distance filter dominates, making Flex irrelevant - Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal - Dynamic scaling helps but still suboptimal
**Solution:** **Solution:**
```yaml ```yaml
best_price_flex: 20 best_price_flex: 20
best_price_min_distance_from_avg: 5 best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods - Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient) - Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range - Min_Distance 5%: Effective throughout range
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh **Average:** 15 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners) - Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify - Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price" - Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Filter statistics: 96 intervals checked DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh **Average:** 18 ct/kWh
**Expected Behavior:** **Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct) - Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct) - Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive) - Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:** **Debug Checks:**
``` ```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct) DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction) DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2 **Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2) DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day **Initial State:** Strict filters, very flat day
**Expected Flow:** **Expected Flow:**
``` ```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0% INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2) DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation **Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:** **Algorithm:**
```python ```python
# Pseudo-code for adaptive flex # Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
``` ```
**Benefits:** **Benefits:**
- Eliminates need for relaxation on most days - Eliminates need for relaxation on most days
- Self-adjusting to market conditions - Self-adjusting to market conditions
- Better user experience (less configuration needed) - Better user experience (less configuration needed)
**Challenges:** **Challenges:**
- Harder to predict behavior (less transparent) - Harder to predict behavior (less transparent)
- May conflict with user's mental model - May conflict with user's mental model
- Needs extensive testing across different markets - Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback **Concept:** Learn optimal Flex/Distance from user feedback
**Approach:** **Approach:**
- Track which periods user actually uses (automation triggers) - Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal) - Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults - Apply pattern-specific defaults
- Learn per-user preferences over time - Learn per-user preferences over time
**Benefits:** **Benefits:**
- Personalized to user's actual behavior - Personalized to user's actual behavior
- Adapts to local market patterns - Adapts to local market patterns
- Could discover non-obvious patterns - Could discover non-obvious patterns
**Challenges:** **Challenges:**
- Requires user feedback mechanism (not implemented) - Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns) - Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?" - Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously **Concept:** Balance multiple goals simultaneously
**Goals:** **Goals:**
- Period count vs. quality (cheap vs. very cheap) - Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent) - Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered) - Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher) - User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:** **Algorithm:**
- Pareto optimization (find trade-off frontier) - Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences - User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing - Genetic algorithm or simulated annealing
**Benefits:** **Benefits:**
- More sophisticated period selection - More sophisticated period selection
- Better match to user's actual needs - Better match to user's actual needs
- Could handle complex appliance requirements - Could handle complex appliance requirements
**Challenges:** **Challenges:**
- Much more complex to implement - Much more complex to implement
- Harder to explain to users - Harder to explain to users
- Computational cost (may need caching) - Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex **Current:** 3% cap may be too aggressive for very low base Flex
**Example:** **Example:**
- Base flex 5% + 3% increment = 8% (60% increase!) - Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase) - Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:** **Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)` - Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%) - This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:** **Why Not Implemented:**
- Very low base flex (`<`10%) unusual - Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation - Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization - Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative **Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve **Alternative:** Non-linear curve
```python ```python
# Example: Exponential scaling # Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20)) scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
``` ```
**Why Not Implemented:** **Why Not Implemented:**
- Linear is easier to reason about - Linear is easier to reason about
- No evidence that non-linear is better - No evidence that non-linear is better
- Would need extensive testing - Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day **Issue:** May find all periods in one part of day
**Example:** **Example:**
- All 3 "best price" periods between 02:00-08:00 - All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances) - No periods in evening (when user might want to run appliances)
**Possible Solution:** **Possible Solution:**
- Add "spread" parameter (prefer distributed periods) - Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences - Weight periods by time-of-day preferences
- Consider user's typical usage patterns - Consider user's typical usage patterns
**Why Not Implemented:** **Why Not Implemented:**
- Adds complexity - Adds complexity
- Users can work around with multiple automations - Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all) - Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg). **Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:** **Implementation:**
```python ```python
# In period_building.py build_periods(): # In period_building.py build_periods():
for price_data in all_prices: for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight** **Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split: When days differ significantly, period can split:
``` ```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1) Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2) Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:** **Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours: Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium - Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer - Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,10 +1154,12 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):** **User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful: Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100) - `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre) - `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting: Automations can check volatility before acting:
```yaml ```yaml
condition: condition:
- condition: template - condition: template
@ -1095,6 +1190,7 @@ Low volatility (< 15%) means classification changes are less economically signif
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness. **Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:** **See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes" - User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`) - Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation) - Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

22
docs/developer/versioned_docs/version-v0.24.0/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
``` ```
**Key Points:** **Key Points:**
- Must be a **class attribute** (not instance attribute) - Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance - Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component - Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips` **Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that: **Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely - Never change or change very rarely
- Don't provide analytical value in history - Don't provide analytical value in history
- Consume significant database space when recorded every state change - Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures ### 2. Large Nested Structures
**Attributes:** **Attributes:**
- `periods` (binary_sensor) - Array of all period summaries - `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays - `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis - `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown - `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are: **Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive) - Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each) - Create large database rows (2-20 KB each)
- Slow down history queries - Slow down history queries
@ -66,6 +70,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors **Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:** **Example - periods array:**
```json ```json
{ {
"periods": [ "periods": [
@ -76,7 +81,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
"price_mean": 18.5, "price_mean": 18.5,
"price_median": 18.3, "price_median": 18.3,
"price_min": 17.2, "price_min": 17.2,
"price_max": 19.8, "price_max": 19.8
// ... 10+ more attributes × 10-20 periods // ... 10+ more attributes × 10-20 periods
} }
] ]
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status` **Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:** **Reason:**
- Change every update cycle (every 15 minutes or more frequently) - Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value - Don't provide long-term analytical value
- Create state changes even when core values haven't changed - Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max` **Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:** **Reason:**
- Configuration values that rarely change - Configuration values that rarely change
- Wastes space when recorded repeatedly - Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state - Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error` **Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:** **Reason:**
- Only relevant at moment of reading - Only relevant at moment of reading
- Won't be valid after some time - Won't be valid after some time
- Similar to `entity_picture` in HA core image entities - Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%` **Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:** **Reason:**
- Detailed technical information not needed for historical analysis - Detailed technical information not needed for historical analysis
- Only useful for debugging during active development - Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis - Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining` **Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:** **Reason:**
- Can be calculated from other attributes - Can be calculated from other attributes
- Redundant information - Redundant information
- Doesn't add analytical value to history - Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value: These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core ### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST) - `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states - All price values - Core sensor states
### Diagnostics & Tracking ### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time - `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns - `updates_today` - Tracking API usage patterns
### Data Completeness ### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics - `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status - `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data ### Period Data
- `start`, `end`, `duration_minutes` - Core period timing - `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics - `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status ### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation) - `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact ## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings ### Space Savings
**Per state change:** **Per state change:**
- Before: ~3-8 KB average - Before: ~3-8 KB average
- After: ~0.5-1.5 KB average - After: ~0.5-1.5 KB average
- **Reduction: 60-85%** - **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact ### Real-World Impact
For a typical installation with: For a typical installation with:
- 80+ sensors - 80+ sensors
- Updates every 15 minutes - Updates every 15 minutes
- ~10 sensors updating every minute - ~10 sensors updating every minute
@ -214,7 +231,7 @@ For a typical installation with:
- Class: `TibberPricesBinarySensor` - Class: `TibberPricesBinarySensor`
- 30 attributes excluded - 30 attributes excluded
## When to Update _unrecorded_attributes ## When to Update \_unrecorded_attributes
### Add to Exclusion List When: ### Add to Exclusion List When:
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes 4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:** **SQL Query to check attribute presence:**
```sql ```sql
SELECT SELECT
state_id, state_id,

14
docs/developer/versioned_docs/version-v0.24.0/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):** **In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built: git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile) - **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup` - **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):** **Manual installation (outside DevContainer):**
**git-cliff** (template-based): **git-cliff** (template-based):
```bash ```bash
# See: https://git-cliff.org/docs/installation # See: https://git-cliff.org/docs/installation
@ -191,7 +193,7 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which ## 🎯 When to Use Which
| Method | Use Case | Pros | Cons | | Method | Use Case | Pros | Cons |
|--------|----------|------|------| | --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script | | **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump | | **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization | | **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
@ -219,6 +221,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. Script bumps manifest.json → commits → creates tag locally 1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together 2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release 3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
``` ```
**What happens:** **What happens:**
1. You push manifest.json change 1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically 2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release 3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
``` ```
**What happens:** **What happens:**
1. You create and push tag manually 1. You create and push tag manually
2. Release workflow creates release 2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists) 3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features ## 🛡️ Safety Features
### 1. **Version Validation** ### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z). Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags** ### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote) - Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating - Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations** ### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push. Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered** ### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits. Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions** ### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind. Helper script shows how to undo if you change your mind.
--- ---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:** **Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes: Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely - Tag already exists remotely
- Invalid version format in manifest.json - Invalid version format in manifest.json
- manifest.json not in the commit that was pushed - manifest.json not in the commit that was pushed
@ -348,6 +359,7 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips ## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results: 1. **Conventional Commits:** Use proper commit format for best results:
``` ```
feat(scope): Add new feature feat(scope): Add new feature

22
docs/developer/versioned_docs/version-v0.24.0/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle. The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:** **Design Principles:**
- **Proactive**: Detect issues before they become critical - **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance - **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve - **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}` **Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:** **When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`) - Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available - Tomorrow's electricity price data is still not available
**When cleared:** **When cleared:**
- Tomorrow's data becomes available - Tomorrow's data becomes available
- Automatically checks on every successful API update - Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work. Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:** **Implementation:**
```python ```python
# In coordinator update cycle # In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"]) has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18) - `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}` **Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:** **When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429) - Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD` - Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:** **When cleared:**
- Successful API call completes (no rate limit error) - Successful API call completes (no rate limit error)
- Error counter resets to 0 - Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires. API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:** **Implementation:**
```python ```python
# In error handler # In error handler
is_rate_limit = ( is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the affected home - `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors - `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}` **Issue ID:** `home_not_found_{entry_id}`
**When triggered:** **When triggered:**
- Home configured in this integration is no longer present in Tibber account - Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check) - Detected during user data refresh (daily check)
**When cleared:** **When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected) - Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup) - Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed. Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:** **Implementation:**
```python ```python
# After user data update # After user data update
home_exists = self._data_fetcher._check_home_exists(home_id) home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
``` ```
**Translation placeholders:** **Translation placeholders:**
- `home_name`: Name of the missing home - `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference - `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration ### Lifecycle Integration
**Coordinator Initialization:** **Coordinator Initialization:**
```python ```python
self._repair_manager = TibberPricesRepairManager( self._repair_manager = TibberPricesRepairManager(
hass=hass, hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
``` ```
**Update Cycle Integration:** **Update Cycle Integration:**
```python ```python
# Success path - check conditions # Success path - check conditions
if result and "priceInfo" in result: if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
``` ```
**Shutdown Cleanup:** **Shutdown Cleanup:**
```python ```python
async def async_shutdown(self) -> None: async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up.""" """Shut down coordinator and clean up."""
@ -196,6 +212,7 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json` - `/translations/sv.json`
**Structure:** **Structure:**
```json ```json
{ {
"issues": { "issues": {
@ -210,10 +227,12 @@ Repairs use Home Assistant's standard translation system. Translations are defin
## Home Assistant Integration ## Home Assistant Integration
Repairs appear in: Repairs appear in:
- **Settings → System → Repairs** (main repairs panel) - **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count) - **Notifications** (bell icon in UI shows repair count)
Repair properties: Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required) - **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical) - **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files - **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears 4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:** **Manual trigger:**
```python ```python
# Temporarily set warning hour to current hour for testing # Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair 3. Successful API call clears the repair
**Manual test:** **Manual test:**
- Reduce API polling interval to trigger rate limiting - Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client - Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file 7. **Document** in this file
**Example template:** **Example template:**
```python ```python
async def check_new_condition(self, *, param: bool) -> None: async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair.""" """Check new condition and create/clear repair."""

Some files were not shown because too many files have changed in this diff Show more