jpawlowski/hass.tibber_prices
1
0
Fork 0
mirror of https://github.com/jpawlowski/hass.tibber_prices.git synced 2026-05-28 18:43:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

chore(style): normalize Markdown list indentation across all docs

Browse source 
Convert four-space-indented list items (`-   item`) to standard two-space
(`- item`) in AGENTS.md, CONTRIBUTING.md, README.md, and all Docusaurus
documentation pages (developer and user, including versioned snapshots).
No content changes.

Release-Notes: skip
This commit is contained in:
Julian Pawlowski 2026-04-12 14:15:31 +00:00
parent e163a47d57
commit aa9a1200b8
339 changed files with 16987 additions and 12955 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1043
AGENTS.md
View file

File diff suppressed because it is too large Load diff

34
CONTRIBUTING.md
View file

@ -18,14 +18,14 @@ For detailed developer documentation, see [docs/development/](docs/development/)
1. **Fork the repository** on GitHub
2. **Clone your fork**:
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
3. **Open in DevContainer** (recommended):
- Open in VS Code
- Click "Reopen in Container" when prompted
- Or manually: `Ctrl+Shift+P` → "Dev Containers: Reopen in Container"
- Open in VS Code
- Click "Reopen in Container" when prompted
- Or manually: `Ctrl+Shift+P` → "Dev Containers: Reopen in Container"
See [Development Setup](docs/development/setup.md) for detailed instructions.
@ -73,14 +73,17 @@ Impact: <user-visible effects>
**Types:** `feat`, `fix`, `docs`, `refactor`, `chore`, `test`
For full commit-message rules (including release-note skip trailers for internal/unreleased fixes), see:
- `.github/instructions/commit-messages.instructions.md`
Important trailers for commits that should NOT appear in release notes:
- `Release-Notes: skip`
- `User-Impact: none`
- `Released-Bug: no`
**Example:**
```bash
git commit -m "feat(sensors): add daily average price sensor
@ -97,9 +100,9 @@ See `.github/instructions/commit-messages.instructions.md` for detailed commit-m
1. **Push your branch** to your fork
2. **Create a Pull Request** on GitHub with:
- Clear title describing the change
- Detailed description with context
- Reference related issues (`Fixes #123`)
- Clear title describing the change
- Detailed description with context
- Reference related issues (`Fixes #123`)
3. **Wait for review** and address feedback
### PR Requirements
@ -119,6 +122,7 @@ See `.github/instructions/commit-messages.instructions.md` for detailed commit-m
- **Python version**: 3.13+
Always run before committing:
```bash
./scripts/lint
```
@ -137,13 +141,14 @@ See [Coding Guidelines](docs/developer/docs/coding-guidelines.md) for complete d
Documentation is organized in two Docusaurus sites:
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation via `docs/user/sidebars.ts`
- Markdown files in `docs/user/docs/*.md`
- Navigation via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation via `docs/developer/sidebars.ts`
- Markdown files in `docs/developer/docs/*.md`
- Navigation via `docs/developer/sidebars.ts`
**When adding new documentation:**
1. Place file in appropriate `docs/*/docs/` directory
2. Add to corresponding `sidebars.ts` for navigation
3. Update translations when changing `translations/en.json` (update ALL language files)
@ -153,6 +158,7 @@ Documentation is organized in two Docusaurus sites:
Report bugs via [GitHub Issues](../../issues/new/choose).
**Great bug reports include:**
- Quick summary and background
- Steps to reproduce (be specific!)
- Expected vs. actual behavior

88
README.md
View file

@ -22,8 +22,8 @@
**[📚 Complete Documentation](https://jpawlowski.github.io/hass.tibber_prices/)** — Installation, guides, examples, and full sensor reference:
- **[👤 User Documentation](https://jpawlowski.github.io/hass.tibber_prices/user/)** — Setup, sensors, automations, dashboards
- **[🔧 Developer Documentation](https://jpawlowski.github.io/hass.tibber_prices/developer/)** — Architecture, contributing, development
- **[👤 User Documentation](https://jpawlowski.github.io/hass.tibber_prices/user/)** — Setup, sensors, automations, dashboards
- **[🔧 Developer Documentation](https://jpawlowski.github.io/hass.tibber_prices/developer/)** — Architecture, contributing, development
**Quick Links:**
[Installation](https://jpawlowski.github.io/hass.tibber_prices/user/installation) · [Sensor Reference](https://jpawlowski.github.io/hass.tibber_prices/user/sensor-reference) · [Charts](https://jpawlowski.github.io/hass.tibber_prices/user/chart-examples) · [Automations](https://jpawlowski.github.io/hass.tibber_prices/user/automation-examples) · [FAQ](https://jpawlowski.github.io/hass.tibber_prices/user/faq) · [Changelog](https://github.com/jpawlowski/hass.tibber_prices/releases)
@ -34,39 +34,39 @@ Most Tibber integrations give you a single price sensor. This one gives you a **
### 🔮 Know What's Coming
- **Quarter-hourly precision** — 15-minute interval prices, not just hourly averages
- **Price forecasts** — See average prices for the next 1h, 2h, 3h, ... up to 12h ahead
- **Trend analysis** — Know if prices are rising, falling, or stable — and when the next trend change happens
- **Price trajectory** — Detect turning points before they happen (first-half vs second-half window comparison)
- **Price outlook** — Instantly see if the next hours will be cheaper or more expensive than now
- **Quarter-hourly precision** — 15-minute interval prices, not just hourly averages
- **Price forecasts** — See average prices for the next 1h, 2h, 3h, ... up to 12h ahead
- **Trend analysis** — Know if prices are rising, falling, or stable — and when the next trend change happens
- **Price trajectory** — Detect turning points before they happen (first-half vs second-half window comparison)
- **Price outlook** — Instantly see if the next hours will be cheaper or more expensive than now
### ⚡ Automate Smartly
- **Best Price & Peak Price Periods** — Intelligent binary sensors that detect the cheapest and most expensive periods of the day, with configurable flexibility, relaxation strategies, and gap tolerance ([how it works](https://jpawlowski.github.io/hass.tibber_prices/user/period-calculation))
- **Period timing sensors** — Duration, end time, remaining minutes, progress percentage, and countdown to next period — everything you need for advanced automations
- **Runtime configuration** — Adjust period detection parameters on the fly via switches and number entities, without restarting — perfect for automations that adapt to your schedule
- **5-level price classification** — VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE from Tibber's API
- **3-level price ratings** — LOW, NORMAL, HIGH based on 24h trailing average comparison
- **Best Price & Peak Price Periods** — Intelligent binary sensors that detect the cheapest and most expensive periods of the day, with configurable flexibility, relaxation strategies, and gap tolerance ([how it works](https://jpawlowski.github.io/hass.tibber_prices/user/period-calculation))
- **Period timing sensors** — Duration, end time, remaining minutes, progress percentage, and countdown to next period — everything you need for advanced automations
- **Runtime configuration** — Adjust period detection parameters on the fly via switches and number entities, without restarting — perfect for automations that adapt to your schedule
- **5-level price classification** — VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE from Tibber's API
- **3-level price ratings** — LOW, NORMAL, HIGH based on 24h trailing average comparison
### 📊 Visualize Beautifully
- **Auto-generated ApexCharts** — One action call generates a complete chart configuration with dynamic Y-axis scaling and color-coded price levels ([see examples](https://jpawlowski.github.io/hass.tibber_prices/user/chart-examples))
- **Dynamic icons & colors** — Every sensor adapts its icon and color to the current price state — cheap prices glow green, expensive ones turn red ([icon guide](https://jpawlowski.github.io/hass.tibber_prices/user/dynamic-icons))
- **Chart data export** — Flexible data API with filtering, resolution control, and multiple output formats for any visualization card
- **Auto-generated ApexCharts** — One action call generates a complete chart configuration with dynamic Y-axis scaling and color-coded price levels ([see examples](https://jpawlowski.github.io/hass.tibber_prices/user/chart-examples))
- **Dynamic icons & colors** — Every sensor adapts its icon and color to the current price state — cheap prices glow green, expensive ones turn red ([icon guide](https://jpawlowski.github.io/hass.tibber_prices/user/dynamic-icons))
- **Chart data export** — Flexible data API with filtering, resolution control, and multiple output formats for any visualization card
### 📈 Understand Your Market
- **Volatility analysis** — Know if today's prices are stable or wild (low/moderate/high/very_high)
- **Daily & rolling statistics** — Min, max, average, median for today, tomorrow, trailing 24h, and leading 24h
- **Energy & tax breakdown** — See spot price vs. tax components as sensor attributes
- **Multi-currency support** — EUR, NOK, SEK, DKK, USD, GBP with configurable base/subunit display (€ vs ct, kr vs øre)
- **Volatility analysis** — Know if today's prices are stable or wild (low/moderate/high/very_high)
- **Daily & rolling statistics** — Min, max, average, median for today, tomorrow, trailing 24h, and leading 24h
- **Energy & tax breakdown** — See spot price vs. tax components as sensor attributes
- **Multi-currency support** — EUR, NOK, SEK, DKK, USD, GBP with configurable base/subunit display (€ vs ct, kr vs øre)
### 🛡️ Built for Reliability
- **Intelligent caching** — Multi-layer caching minimizes API calls, survives HA restarts, auto-invalidates at midnight
- **High-performance interval pool** — O(1) timestamp lookups, gap detection, auto-fetching of missing data
- **Quarter-hour precision updates** — Sensors refresh at :00/:15/:30/:45 boundaries, independent of API polling
- **Official API only** — Uses Tibber's [`priceInfo`](https://developer.tibber.com/docs/reference#priceinfo) and [`priceInfoRange`](https://developer.tibber.com/docs/reference#subscription) endpoints. All ratings and statistics are calculated locally.
- **Intelligent caching** — Multi-layer caching minimizes API calls, survives HA restarts, auto-invalidates at midnight
- **High-performance interval pool** — O(1) timestamp lookups, gap detection, auto-fetching of missing data
- **Quarter-hour precision updates** — Sensors refresh at :00/:15/:30/:45 boundaries, independent of API polling
- **Official API only** — Uses Tibber's [`priceInfo`](https://developer.tibber.com/docs/reference#priceinfo) and [`priceInfoRange`](https://developer.tibber.com/docs/reference#subscription) endpoints. All ratings and statistics are calculated locally.
## 🚀 Quick Start
@ -91,9 +91,9 @@ Or manually: **Settings** → **Devices & Services** → **+ Add Integration**
### Step 3: Done!
- **100+ sensors** are now available (key sensors enabled by default, advanced ones ready to enable)
- Explore entities in **Settings** → **Devices & Services** → **Tibber Price Information & Ratings**
- Start building automations, dashboards, and energy-saving workflows
- **100+ sensors** are now available (key sensors enabled by default, advanced ones ready to enable)
- Explore entities in **Settings** → **Devices & Services** → **Tibber Price Information & Ratings**
- Start building automations, dashboards, and energy-saving workflows
📖 **[Full Installation Guide →](https://jpawlowski.github.io/hass.tibber_prices/user/installation)**
@ -103,18 +103,18 @@ The integration provides **100+ entities** across sensors, binary sensors, switc
<img src="https://raw.githubusercontent.com/jpawlowski/hass.tibber_prices/main/docs/user/static/img/entities-overview.jpg" width="400" alt="Entity list showing dynamic icons for different price states">
| Category | Highlights | Count |
|----------|-----------|-------|
| **💰 Prices** | Current, next & previous interval price + rolling hour averages | 6+ |
| **📊 Statistics** | Daily min/max/avg for today & tomorrow, 24h trailing & leading windows | 12+ |
| **🔮 Forecasts** | Next 1h–12h average prices, price outlook & trajectory sensors | 20+ |
| **📈 Trends** | Current trend direction, next trend change time & countdown | 3 |
| **📉 Volatility** | Today, tomorrow, next 24h & combined volatility levels | 4 |
| **🏷️ Levels & Ratings** | 5-level (API) and 3-level (computed) classification per interval, hour & day | 12+ |
| **⏰ Period Timing** | Best/peak: end time, duration, remaining, progress, next start | 10+ |
| **🔌 Binary Sensors** | Best price period, peak price period, tomorrow data available, API connection | 4+ |
| **🎛️ Runtime Config** | Switches & numbers to adjust period detection live — no restart needed | 14 |
| **🔧 Diagnostics** | Data lifecycle status, home metadata, grid info, subscription status | 15+ |
| Category | Highlights | Count |
| ----------------------- | ----------------------------------------------------------------------------- | ----- |
| **💰 Prices** | Current, next & previous interval price + rolling hour averages | 6+ |
| **📊 Statistics** | Daily min/max/avg for today & tomorrow, 24h trailing & leading windows | 12+ |
| **🔮 Forecasts** | Next 1h–12h average prices, price outlook & trajectory sensors | 20+ |
| **📈 Trends** | Current trend direction, next trend change time & countdown | 3 |
| **📉 Volatility** | Today, tomorrow, next 24h & combined volatility levels | 4 |
| **🏷️ Levels & Ratings** | 5-level (API) and 3-level (computed) classification per interval, hour & day | 12+ |
| **⏰ Period Timing** | Best/peak: end time, duration, remaining, progress, next start | 10+ |
| **🔌 Binary Sensors** | Best price period, peak price period, tomorrow data available, API connection | 4+ |
| **🎛️ Runtime Config** | Switches & numbers to adjust period detection live — no restart needed | 14 |
| **🔧 Diagnostics** | Data lifecycle status, home metadata, grid info, subscription status | 15+ |
> **Every sensor includes rich attributes** — timestamps, detailed descriptions, and context data. Enable **Extended Descriptions** in the integration options to get `long_description` and `usage_tips` on every entity.
@ -168,17 +168,17 @@ Generate beautiful price charts with a single action call — dynamic Y-axis, co
## ❓ Help & Support
- 📖 **[FAQ](https://jpawlowski.github.io/hass.tibber_prices/user/faq)** — Common questions answered
- 🔧 **[Troubleshooting](https://jpawlowski.github.io/hass.tibber_prices/user/troubleshooting)** — Solving common issues
- 🐛 **[Report an Issue](https://github.com/jpawlowski/hass.tibber_prices/issues/new)** — Found a bug? Let us know
- 📖 **[FAQ](https://jpawlowski.github.io/hass.tibber_prices/user/faq)** — Common questions answered
- 🔧 **[Troubleshooting](https://jpawlowski.github.io/hass.tibber_prices/user/troubleshooting)** — Solving common issues
- 🐛 **[Report an Issue](https://github.com/jpawlowski/hass.tibber_prices/issues/new)** — Found a bug? Let us know
## 🤝 Contributing
Contributions are welcome! See the [Contributing Guidelines](CONTRIBUTING.md) and [Developer Documentation](https://jpawlowski.github.io/hass.tibber_prices/developer/) to get started.
- **[Developer Setup](https://jpawlowski.github.io/hass.tibber_prices/developer/setup)** — DevContainer-based development environment
- **[Architecture Guide](https://jpawlowski.github.io/hass.tibber_prices/developer/architecture)** — Understand the codebase
- **[Release Management](https://jpawlowski.github.io/hass.tibber_prices/developer/release-management)** — Release process and versioning
- **[Developer Setup](https://jpawlowski.github.io/hass.tibber_prices/developer/setup)** — DevContainer-based development environment
- **[Architecture Guide](https://jpawlowski.github.io/hass.tibber_prices/developer/architecture)** — Understand the codebase
- **[Release Management](https://jpawlowski.github.io/hass.tibber_prices/developer/release-management)** — Release process and versioning
## 🤖 Development Note

131
docs/developer/docs/api-reference.md
View file

@ -22,30 +22,30 @@ Fetches home information and metadata:
```graphql
query {
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
}
```
@ -56,26 +56,27 @@ query {
Fetches quarter-hourly prices:
```graphql
query($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
query ($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
}
}
}
}
}
}
}
}
}
}
```
**Parameters:**
- `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits
Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute
Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home
@ -99,13 +102,14 @@ Integration stays well below these limits:
```json
{
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
}
```
**Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -114,11 +118,12 @@ Integration stays well below these limits:
```json
{
"currency": "EUR"
"currency": "EUR"
}
```
Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses
**Invalid Token:**
```json
{
"errors": [{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}]
"errors": [
{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}
]
}
```
**Rate Limit Exceeded:**
```json
{
"errors": [{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}]
"errors": [
{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}
]
}
```
**Home Not Found:**
```json
{
"errors": [{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}]
"errors": [
{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}
]
}
```
Integration handles these with:
- Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation
Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
---
💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token)

143
docs/developer/docs/architecture.md
View file

@ -100,43 +100,43 @@ flowchart TB
### Flow Description
1. **Setup** (`__init__.py`)
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
2. **Data Fetch** (every 15 minutes)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
3. **Price Enrichment**
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
4. **Period Calculation**
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
5. **Entity Updates**
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
6. **Service Calls**
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
---
@ -146,13 +146,13 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------|
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
| Layer | Location | Lifetime | Invalidation | Memory |
| ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
**Total cache overhead:** ~126KB per coordinator instance (main entry + subentries)
@ -195,30 +195,31 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components
| Component | File | Responsibility |
|-----------|------|----------------|
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
| Component | File | Responsibility |
| --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
### Sensor Architecture (Calculator Pattern)
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------|
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
| Component | Files | Lines | Responsibility |
| ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata
**Benefits:**
- 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator
@ -237,12 +239,12 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities
| Utility | File | Purpose |
|---------|------|---------|
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
| Utility | File | Purpose |
| ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
---
@ -283,12 +285,12 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
- **API polling**: Every 15 minutes (coordinator fetch cycle)
- **Entity updates**: On 00/15/30/45-minute boundaries via `coordinator/listeners.py`
- **Timer scheduling**: Uses `async_track_utc_time_change(minute=[0, 15, 30, 45], second=0)`
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- **Absolute time tracking**: Timer plans for **all future boundaries** (not relative delays)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- **Result**: Current price sensors update without waiting for next API poll
### 4. Calculator Pattern (Sensor Platform)
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:**
- Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing
@ -333,12 +340,12 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization
| Optimization | Location | Savings |
|--------------|----------|---------|
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
| Optimization | Location | Savings |
| ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
### Memory Usage

132
docs/developer/docs/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation
**Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage
@ -36,29 +38,31 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator):
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
2. **Cache validation on load**:
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
3. **Tomorrow data check** (after 13:00):
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
**Why this cache matters:** Reduces API load on Tibber (~192 intervals per fetch), speeds up HA restarts, enables offline operation until cache expires.
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:**
- **Forever** (until HA restart)
- No invalidation during runtime
**When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once
**Access pattern:**
```python
# Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:**
### DataTransformer Config Cache
```python
{
"thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
### PeriodCalculator Config Cache
```python
{
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,20 +128,23 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Lifetime:**
- Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle
**Invalidation trigger:**
- **Options change** (user reconfigures integration):
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
**Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:**
```python
{
"best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Cache key:** Hash of relevant inputs
```python
hash_data = (
today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
```
**Lifetime:**
- Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data)
@ -179,24 +195,27 @@ hash_data = (
**Invalidation triggers:**
1. **Config change** (explicit):
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
2. **Price data change** (automatic via hash mismatch):
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
**Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:**
```python
{
"timestamp": ...,
@ -224,14 +244,16 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:**
- Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when:
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
**Architecture:**
- DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow
### User Changes Options (Config Flow)
```
User saves options
↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
```
### Midnight Turnover (Day Transition)
```
Timer #2 fires at 00:00
↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
```
### Tomorrow Data Arrives (~13:00)
```
Coordinator update cycle
↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
```
**No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load)
**Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics
### Typical Operation (No Changes)
```
Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
```
### After Midnight Turnover
```
Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
```
### After Config Change
```
Options Update
├─> Cache invalidation: `<`1ms
@ -381,23 +411,25 @@ Options Update
## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------|
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
| Cache Type | Lifetime | Size | Invalidation | Purpose |
| ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
**Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:**
- 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup)
**Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues
### Symptom: Stale data after config change
**Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating
**Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today
**Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations
**Check:**
1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys

11
docs/developer/docs/coding-guidelines.md
View file

@ -8,10 +8,10 @@ comments: false
## Code Style
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
Run before committing:
@ -41,12 +41,14 @@ class TimeService:
```
**When prefix is required:**
- Public classes used across multiple modules
- All exception classes
- All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module

25
docs/developer/docs/contributing.md
View file

@ -14,10 +14,10 @@ Welcome! This guide helps you contribute to the Tibber Prices integration.
1. Fork the repository on GitHub
2. Clone your fork:
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
3. Open in VS Code
4. Click "Reopen in Container" when prompted
@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
```
**Branch naming:**
- `feature/` - New features
- `fix/` - Bug fixes
- `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:**
```bash
./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
```
Run your test:
```bash
./scripts/test tests/test_your_feature.py -v
```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
```
**Commit types:**
- `feat:` - New feature
- `fix:` - Bug fix
- `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance
**Add scope when relevant:**
- `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max)
Description should include:
```markdown
## What
Brief description of changes
## Why
Problem being solved or feature rationale
## How
Implementation approach
## Testing
- [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated
- [ ] Type checking passes
- [ ] Linting passes
## Breaking Changes
(If any - describe migration path)
## Related Issues
Closes #123
```
### PR Checklist
Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For
✅ **Good:**
- Clear, self-explanatory code
- Appropriate comments for complex logic
- Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns
❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes
- Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On
Good first issues are labeled:
- `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
---
💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests

65
docs/developer/docs/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅
**What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners
**Why critical:**
- Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅
**What is tested:**
- Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared
- Both timers are cancelled together
- Cleanup works even when timers are `None`
**Why critical:**
- Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅
**What is tested:**
- Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()`
**Why critical:**
- `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation
**What is tested:**
- DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update
**Why critical:**
- Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis
**Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal
**What is tested:**
- Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss)
**Why critical:**
- Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:**
- `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py`
**What is tested:**
- Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly
**Why critical:**
- Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py`
**What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked
**Why critical:**
- Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:**
- No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically
- Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created
- HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading
- Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors
**Structure:**
```python
coordinator.data = {
"user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory
**Current Status:** HA's response lifecycle
- HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code
@ -207,29 +234,30 @@ coordinator.data = {
### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------|
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
| Category | Status | Tests | File | Coverage |
| ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale |
|----------|--------|-----------|
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
| Category | Status | Rationale |
| ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:**
- ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners)

52
docs/developer/docs/debugging.md
View file

@ -10,9 +10,9 @@ Add to `configuration.yaml`:
```yaml
logger:
default: info
logs:
custom_components.tibber_prices: debug
default: info
logs:
custom_components.tibber_prices: debug
```
Restart Home Assistant to apply.
@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages
**Coordinator Updates:**
```
[custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
```
**Period Calculation:**
```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
```
**API Errors:**
```
[custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -47,26 +50,27 @@ Restart Home Assistant to apply.
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
}
```
### Set Breakpoints
**Coordinator update:**
```python
# coordinator/core.py
async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
```
**Period calculation:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
```
**Flags:**
- `-v` - Verbose output
- `-s` - Show print statements
- `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns
**Print coordinator data:**
```python
def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
```
**Inspect period attributes:**
```python
def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading
**Check:**
```bash
grep "tibber_prices" config/home-assistant.log
```
**Common causes:**
- Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating
**Check coordinator state:**
```python
# In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }}
```
**Debug in code:**
```python
# Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong
**Enable detailed period logs:**
```python
# coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
```
**Check filter statistics:**
```
[period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy
Add to coordinator code:
```python
import debugpy
debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL
Install in container:
```bash
uv pip install ipython
```
Add breakpoint:
```python
from IPython import embed
embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
---
💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure

123
docs/developer/docs/intro.md
View file

@ -8,25 +8,25 @@ This is an independent, community-maintained custom integration for Home Assista
## 📚 Developer Guides
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
## 🤖 AI Documentation
The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md). This file serves as long-term memory for AI assistants and contains:
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
**Important:** When proposing changes to patterns or conventions, always update [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) to keep AI guidance consistent.
@ -34,32 +34,32 @@ The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlow
This integration is developed with extensive AI assistance (GitHub Copilot, Claude, and other AI tools). The AI handles:
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
**Quality Assurance:**
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
**Benefits:**
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
**Limitations:**
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
If you're working with AI tools on this project, the [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) file provides the context and patterns that ensure consistency.
@ -80,15 +80,15 @@ If you're working with AI tools on this project, the [`AGENTS.md`](https://githu
The project includes several helper scripts in `./scripts/`:
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
## 📦 Project Structure
@ -121,23 +121,23 @@ custom_components/tibber_prices/
**DataUpdateCoordinator Pattern:**
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
**Price Data Enrichment:**
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
**Translation System:**
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
## 🧪 Testing
@ -159,18 +159,19 @@ pytest --cov=custom_components.tibber_prices tests/
Documentation is organized in two Docusaurus sites:
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:**
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
## 🤝 Contributing

17
docs/developer/docs/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals
Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching
**1. Persistent Cache** (API data):
```python
# Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
```
**2. Translation Cache** (in-memory):
```python
# Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
```
**3. Config Cache** (invalidated on options change):
```python
class DataTransformer:
def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading
**Load data only when needed:**
```python
@property
def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations
**Process multiple items at once:**
```python
# ❌ Slow - loop with individual operations
for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices
**1. Concurrent API calls:**
```python
# ❌ Sequential (slow)
user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
```
**2. Don't block event loop:**
```python
# ❌ Blocking
result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks
**1. Clear references:**
```python
class Coordinator:
async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
```
**2. Use weak references for callbacks:**
```python
import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures
**Use appropriate types:**
```python
# ❌ List for lookups (O(n))
if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls
**Already implemented:**
- Cache valid until midnight
- User data cached for 24h
- Only poll when tomorrow data expected
**Monitor API usage:**
```python
_LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates
**Only update when needed:**
```python
async def _async_update_data(self) -> dict:
"""Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection
**Affects long-term statistics storage:**
```python
# ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size
**Keep attributes minimal:**
```python
# ❌ Large nested structures (KB per update)
attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
---
💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools

371
docs/developer/docs/period-calculation-theory.md
View file

File diff suppressed because it is too large Load diff

103
docs/developer/docs/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
```
**Key Points:**
- Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely
- Don't provide analytical value in history
- Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures
**Attributes:**
- `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each)
- Slow down history queries
@ -66,20 +70,21 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:**
```json
{
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8,
// ... 10+ more attributes × 10-20 periods
}
]
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8
// ... 10+ more attributes × 10-20 periods
}
]
}
```
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:**
- Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value
- Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:**
- Configuration values that rarely change
- Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `timestamp`, `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:**
- `timestamp` is the rounded-quarter reference time used at the moment of the state write — it's stale as soon as the next update fires and has no analytical value in history
- `next_api_poll`, `next_midnight_turnover` etc. are only relevant at the moment of reading; they're superseded by the next update
- Similar to `entity_picture` in HA core image entities
@ -129,6 +137,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:**
- Detailed technical information not needed for historical analysis
- Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis
@ -140,6 +149,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `period_count_total`, `period_count_remaining`
**Reason:**
- Can be calculated from other attributes
- Redundant information
- Doesn't add analytical value to history
@ -153,23 +163,28 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core
- All price values - Core sensor states (the entity's `native_value` is always recorded separately)
### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns
### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data
- `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
- `period_position` - Position of current period in the day's sequence
- `period_count_today`, `period_count_tomorrow` - How many periods per day (useful in automations)
### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact
@ -177,6 +192,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings
**Per state change:**
- Before: ~3-8 KB average
- After: ~0.5-1.5 KB average
- **Reduction: 60-85%**
@ -198,6 +214,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact
For a typical installation with:
- 80+ sensors
- Updates every 15 minutes
- ~10 sensors updating every minute
@ -209,14 +226,14 @@ For a typical installation with:
## Implementation Files
- **Sensor Platform**: `custom_components/tibber_prices/sensor/core.py`
- Class: `TibberPricesSensor`
- 46 attributes excluded
- Class: `TibberPricesSensor`
- 46 attributes excluded
- **Binary Sensor Platform**: `custom_components/tibber_prices/binary_sensor/core.py`
- Class: `TibberPricesBinarySensor`
- 29 attributes excluded
- Class: `TibberPricesBinarySensor`
- 29 attributes excluded
## When to Update _unrecorded_attributes
## When to Update \_unrecorded_attributes
### Add to Exclusion List When:
@ -238,24 +255,24 @@ For a typical installation with:
When adding a new attribute, ask:
1. **Will this be useful in history queries 1 week from now?**
- No → Exclude
- Yes → Keep
- No → Exclude
- Yes → Keep
2. **Can this be calculated from other recorded attributes?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
3. **Is this primarily for current UI display?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
4. **Does this change frequently without indicating state change?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
5. **Is this larger than 100 bytes and not essential for analysis?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
## Testing
@ -267,6 +284,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:**
```sql
SELECT
state_id,
@ -300,11 +318,11 @@ This makes `state_class=TOTAL` on many sensors the primary cause of long-term da
For sensors with `device_class=SensorDeviceClass.MONETARY`, only two `state_class` values are valid:
| `state_class` | Statistics written | Frontend effect |
|---|---|---|
| `TOTAL` | ✅ Yes — unbounded growth | Statistics line-chart on entity detail page |
| `None` | ❌ No | States timeline only (History panel, "Show More") |
| `MEASUREMENT` | ❌ Blocked by hassfest | — |
| `state_class` | Statistics written | Frontend effect |
| ------------- | ------------------------- | ------------------------------------------------- |
| `TOTAL` | ✅ Yes — unbounded growth | Statistics line-chart on entity detail page |
| `None` | ❌ No | States timeline only (History panel, "Show More") |
| `MEASUREMENT` | ❌ Blocked by hassfest | — |
`MEASUREMENT` causes a hassfest validation error for MONETARY sensors, leaving only `TOTAL` or `None`.
@ -312,19 +330,21 @@ For sensors with `device_class=SensorDeviceClass.MONETARY`, only two `state_clas
Only 3 of 26 MONETARY sensors keep `state_class=TOTAL` — those where long-term history is genuinely useful:
| Sensor | Reason |
|---|---|
| `current_interval_price` | Long-term price trend (weeks/months) |
| `current_interval_price_base` | Required for Energy Dashboard |
| `average_price_today` | Seasonal daily average tracking |
| Sensor | Reason |
| ----------------------------- | ------------------------------------ |
| `current_interval_price` | Long-term price trend (weeks/months) |
| `current_interval_price_base` | Required for Energy Dashboard |
| `average_price_today` | Seasonal daily average tracking |
All other 23 MONETARY sensors use `state_class=None`:
- Forecast/future sensors (`next_avg_*h`)
- Daily snapshots (`lowest/highest_price_today/tomorrow`)
- Rolling windows (`trailing/leading_24h_*`)
- Next/previous interval sensors
**Effect of `state_class=None`:**
- ✅ Short-term state history (States timeline, ~10 days) still works normally
- ✅ Templates, automations, and attributes are unaffected
- ❌ Statistics line-chart removed from entity detail page for these sensors
@ -333,6 +353,7 @@ All other 23 MONETARY sensors use `state_class=None`:
### Expected Impact
Going from 26 → 3 sensors writing to the statistics tables:
- **~88% reduction** in statistics table writes
- Prevents the primary cause of long-term database bloat
- Existing statistics data is retained (only new writes stop)
@ -341,10 +362,10 @@ Going from 26 → 3 sensors writing to the statistics tables:
These are two independent mechanisms targeting different tables:
| Mechanism | Table affected | Purged? | Controls |
|---|---|---|---|
| `_unrecorded_attributes` | `state_attributes` | ✅ ~10 days | Which attributes are stored per state write |
| `state_class=None` | `statistics`, `statistics_short_term` | ❌ Never | Whether long-term statistics are written at all |
| Mechanism | Table affected | Purged? | Controls |
| ------------------------ | ------------------------------------- | ----------- | ----------------------------------------------- |
| `_unrecorded_attributes` | `state_attributes` | ✅ ~10 days | Which attributes are stored per state write |
| `state_class=None` | `statistics`, `statistics_short_term` | ❌ Never | Whether long-term statistics are written at all |
Both optimizations work together. `_unrecorded_attributes` reduces the size of each state write; `state_class=None` eliminates an entire category of unbounded writes.

200
docs/developer/docs/refactoring-guide.md
View file

@ -8,22 +8,22 @@ Not every code change needs a detailed plan. Create a refactoring plan when:
🔴 **Major changes requiring planning:**
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
🟡 **Medium changes that might benefit from planning:**
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
🟢 **Small changes - no planning needed:**
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
## The Planning Process
@ -51,34 +51,34 @@ Every planning document should include:
## Problem Statement
- What's the issue?
- Why does it need fixing?
- Current pain points
- What's the issue?
- Why does it need fixing?
- Current pain points
## Proposed Solution
- High-level approach
- File structure (before/after)
- Module responsibilities
- High-level approach
- File structure (before/after)
- Module responsibilities
## Migration Strategy
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
## Risks & Mitigation
- What could go wrong?
- How to prevent it?
- Rollback strategy
- What could go wrong?
- How to prevent it?
- Rollback strategy
## Success Criteria
- Measurable improvements
- Testing requirements
- Verification steps
- Measurable improvements
- Testing requirements
- Verification steps
```
See `planning/README.md` for detailed template explanation.
@ -87,19 +87,19 @@ See `planning/README.md` for detailed template explanation.
Since `planning/` is git-ignored:
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
### 4. Implementation Phase
Once plan is approved:
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
### 5. After Completion
@ -134,13 +134,13 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Before:**
- `sensor.py` - 2,574 lines, hard to navigate
- `sensor.py` - 2,574 lines, hard to navigate
**After:**
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
**Process:**
@ -153,10 +153,10 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Key learnings:**
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
**Note:** The complete module splitting plan was documented during implementation but has been superseded by the actual code structure.
@ -166,11 +166,11 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
Breaking refactorings into phases:
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
### Phase Structure
@ -191,8 +191,8 @@ Each phase should:
**File Lifecycle**:
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
**Steps**:
@ -205,10 +205,10 @@ Each phase should:
**Success criteria**:
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
```
## Testing Strategy
@ -238,13 +238,13 @@ Minimum testing checklist:
After completing all phases:
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
## Common Pitfalls
@ -286,21 +286,21 @@ This project uses AI heavily (GitHub Copilot, Claude). The planning process supp
**AI reads from:**
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
**AI updates:**
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
**Why separate AGENTS.md and docs/development/?**
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) section "Planning Major Refactorings" for AI-specific guidance.
@ -308,16 +308,16 @@ See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENT
### Planning Directory
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
### Example Plans
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
### Helper Scripts
@ -341,21 +341,21 @@ Simple rule: If you can't describe the entire change in 3 sentences, create a pl
Good plan level:
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
Too detailed:
- Exact code snippets for every change
- Line-by-line instructions
- Exact code snippets for every change
- Line-by-line instructions
Too vague:
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
### Q: What if the plan changes during implementation?
@ -363,9 +363,9 @@ Too vague:
If you discover:
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
Document WHY the plan changed (helps future refactorings).
@ -373,9 +373,9 @@ Document WHY the plan changed (helps future refactorings).
**A:** No! Use judgment:
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
### Q: How do I know when a refactoring is successful?
@ -383,12 +383,12 @@ Document WHY the plan changed (helps future refactorings).
Typical criteria:
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
If you can't tick all boxes, the refactoring isn't done.
@ -409,6 +409,6 @@ If you can't tick all boxes, the refactoring isn't done.
**Next steps:**
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans

36
docs/developer/docs/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):**
**git-cliff** (template-based):
```bash
# See: https://git-cliff.org/docs/installation
@ -190,13 +192,13 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which
| Method | Use Case | Pros | Cons |
|--------|----------|------|------|
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
| Method | Use Case | Pros | Cons |
| --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
---
@ -219,6 +221,7 @@ git push origin main v0.3.0
```
**What happens:**
1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
```
**What happens:**
1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
```
**What happens:**
1. You create and push tag manually
2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features
### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind.
---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely
- Invalid version format in manifest.json
- manifest.json not in the commit that was pushed
@ -348,13 +359,14 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results:
```
feat(scope): Add new feature
Detailed description of what changed.
```
feat(scope): Add new feature
Impact: Users can now do X and Y.
```
Detailed description of what changed.
Impact: Users can now do X and Y.
```
2. **Impact Section:** Add `Impact:` in commit body for user-friendly descriptions

32
docs/developer/docs/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:**
- **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available
**When cleared:**
- Tomorrow's data becomes available
- Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:**
```python
# In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:**
- Successful API call completes (no rate limit error)
- Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:**
```python
# In error handler
is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}`
**When triggered:**
- Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check)
**When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:**
```python
# After user data update
home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
```
**Translation placeholders:**
- `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration
**Coordinator Initialization:**
```python
self._repair_manager = TibberPricesRepairManager(
hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
```
**Update Cycle Integration:**
```python
# Success path - check conditions
if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
```
**Shutdown Cleanup:**
```python
async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up."""
@ -196,24 +212,27 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json`
**Structure:**
```json
{
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
}
}
}
}
```
## Home Assistant Integration
Repairs appear in:
- **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count)
Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:**
```python
# Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair
**Manual test:**
- Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file
**Example template:**
```python
async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair."""

16
docs/developer/docs/setup.md
View file

@ -4,9 +4,9 @@
## Prerequisites
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
## Quick Setup
@ -26,11 +26,11 @@ code .
The DevContainer includes:
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
## Running the Integration

16
docs/developer/docs/testing.md
View file

@ -13,10 +13,10 @@ Before running tests or committing changes, validate the integration structure:
This lightweight script checks:
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
**Note:** Full hassfest validation runs in GitHub Actions on push.
@ -42,10 +42,10 @@ pytest --cov=custom_components.tibber_prices tests/
Then test in Home Assistant UI:
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
## Test Guidelines

44
docs/developer/docs/timer-architecture.md
View file

@ -10,11 +10,11 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------|
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
| Timer | Type | Interval | Purpose | Trigger Method |
| ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
**Key principle:** Timer #1 (HA) controls **data fetching**, Timer #2 controls **entity updates**, Timer #3 controls **timing displays**.
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
```
**Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:**
- Automatic restart after HA restart
- Built-in retry logic for temporary failures
- Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
```
**Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total
**Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
```
**Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total
**Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation)
**Why NOT every second:**
- Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
```
**Why this pattern:**
- Decouples timer logic from entity logic
- One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem
With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:**
- Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover)
- Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics
### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all
@ -393,16 +412,16 @@ _LOGGER.setLevel(logging.DEBUG)
### Common Issues
1. **Timer #2 not triggering:**
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
2. **Double updates at midnight:**
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
3. **API overload:**
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
---
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary
**Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:**
- Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:**
- Timer = mechanism that triggers
- Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies

131
docs/developer/versioned_docs/version-v0.21.0/api-reference.md
View file

@ -22,30 +22,30 @@ Fetches home information and metadata:
```graphql
query {
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
}
```
@ -56,26 +56,27 @@ query {
Fetches quarter-hourly prices:
```graphql
query($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
query ($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
}
}
}
}
}
}
}
}
}
}
```
**Parameters:**
- `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits
Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute
Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home
@ -99,13 +102,14 @@ Integration stays well below these limits:
```json
{
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
}
```
**Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -114,11 +118,12 @@ Integration stays well below these limits:
```json
{
"currency": "EUR"
"currency": "EUR"
}
```
Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses
**Invalid Token:**
```json
{
"errors": [{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}]
"errors": [
{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}
]
}
```
**Rate Limit Exceeded:**
```json
{
"errors": [{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}]
"errors": [
{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}
]
}
```
**Home Not Found:**
```json
{
"errors": [{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}]
"errors": [
{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}
]
}
```
Integration handles these with:
- Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation
Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
---
💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token)

143
docs/developer/versioned_docs/version-v0.21.0/architecture.md
View file

@ -100,43 +100,43 @@ flowchart TB
### Flow Description
1. **Setup** (`__init__.py`)
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
2. **Data Fetch** (every 15 minutes)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
3. **Price Enrichment**
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
4. **Period Calculation**
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
5. **Entity Updates**
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
6. **Service Calls**
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
---
@ -146,13 +146,13 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------|
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
| Layer | Location | Lifetime | Invalidation | Memory |
| ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
**Total cache overhead:** ~126KB per coordinator instance (main entry + subentries)
@ -195,30 +195,31 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components
| Component | File | Responsibility |
|-----------|------|----------------|
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
| Component | File | Responsibility |
| --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
### Sensor Architecture (Calculator Pattern)
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------|
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
| Component | Files | Lines | Responsibility |
| ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata
**Benefits:**
- 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator
@ -237,12 +239,12 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities
| Utility | File | Purpose |
|---------|------|---------|
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
| Utility | File | Purpose |
| ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
---
@ -283,12 +285,12 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
- **API polling**: Every 15 minutes (coordinator fetch cycle)
- **Entity updates**: On 00/15/30/45-minute boundaries via `coordinator/listeners.py`
- **Timer scheduling**: Uses `async_track_utc_time_change(minute=[0, 15, 30, 45], second=0)`
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- **Absolute time tracking**: Timer plans for **all future boundaries** (not relative delays)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- **Result**: Current price sensors update without waiting for next API poll
### 4. Calculator Pattern (Sensor Platform)
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:**
- Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing
@ -333,12 +340,12 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization
| Optimization | Location | Savings |
|--------------|----------|---------|
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
| Optimization | Location | Savings |
| ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
### Memory Usage

132
docs/developer/versioned_docs/version-v0.21.0/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation
**Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage
@ -36,29 +38,31 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator):
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
2. **Cache validation on load**:
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
3. **Tomorrow data check** (after 13:00):
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
**Why this cache matters:** Reduces API load on Tibber (~192 intervals per fetch), speeds up HA restarts, enables offline operation until cache expires.
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:**
- **Forever** (until HA restart)
- No invalidation during runtime
**When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once
**Access pattern:**
```python
# Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:**
### DataTransformer Config Cache
```python
{
"thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
### PeriodCalculator Config Cache
```python
{
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,20 +128,23 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Lifetime:**
- Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle
**Invalidation trigger:**
- **Options change** (user reconfigures integration):
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
**Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:**
```python
{
"best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Cache key:** Hash of relevant inputs
```python
hash_data = (
today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
```
**Lifetime:**
- Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data)
@ -179,24 +195,27 @@ hash_data = (
**Invalidation triggers:**
1. **Config change** (explicit):
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
2. **Price data change** (automatic via hash mismatch):
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
**Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:**
```python
{
"timestamp": ...,
@ -224,14 +244,16 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:**
- Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when:
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
**Architecture:**
- DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow
### User Changes Options (Config Flow)
```
User saves options
↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
```
### Midnight Turnover (Day Transition)
```
Timer #2 fires at 00:00
↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
```
### Tomorrow Data Arrives (~13:00)
```
Coordinator update cycle
↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
```
**No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load)
**Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics
### Typical Operation (No Changes)
```
Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
```
### After Midnight Turnover
```
Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
```
### After Config Change
```
Options Update
├─> Cache invalidation: `<`1ms
@ -381,23 +411,25 @@ Options Update
## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------|
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
| Cache Type | Lifetime | Size | Invalidation | Purpose |
| ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
**Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:**
- 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup)
**Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues
### Symptom: Stale data after config change
**Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating
**Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today
**Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations
**Check:**
1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys

11
docs/developer/versioned_docs/version-v0.21.0/coding-guidelines.md
View file

@ -8,10 +8,10 @@ comments: false
## Code Style
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
Run before committing:
@ -41,12 +41,14 @@ class TimeService:
```
**When prefix is required:**
- Public classes used across multiple modules
- All exception classes
- All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module

25
docs/developer/versioned_docs/version-v0.21.0/contributing.md
View file

@ -14,10 +14,10 @@ Welcome! This guide helps you contribute to the Tibber Prices integration.
1. Fork the repository on GitHub
2. Clone your fork:
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
3. Open in VS Code
4. Click "Reopen in Container" when prompted
@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
```
**Branch naming:**
- `feature/` - New features
- `fix/` - Bug fixes
- `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:**
```bash
./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
```
Run your test:
```bash
./scripts/test tests/test_your_feature.py -v
```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
```
**Commit types:**
- `feat:` - New feature
- `fix:` - Bug fix
- `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance
**Add scope when relevant:**
- `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max)
Description should include:
```markdown
## What
Brief description of changes
## Why
Problem being solved or feature rationale
## How
Implementation approach
## Testing
- [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated
- [ ] Type checking passes
- [ ] Linting passes
## Breaking Changes
(If any - describe migration path)
## Related Issues
Closes #123
```
### PR Checklist
Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For
✅ **Good:**
- Clear, self-explanatory code
- Appropriate comments for complex logic
- Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns
❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes
- Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On
Good first issues are labeled:
- `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
---
💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests

65
docs/developer/versioned_docs/version-v0.21.0/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅
**What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners
**Why critical:**
- Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅
**What is tested:**
- Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared
- Both timers are cancelled together
- Cleanup works even when timers are `None`
**Why critical:**
- Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅
**What is tested:**
- Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()`
**Why critical:**
- `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation
**What is tested:**
- DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update
**Why critical:**
- Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis
**Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal
**What is tested:**
- Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss)
**Why critical:**
- Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:**
- `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py`
**What is tested:**
- Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly
**Why critical:**
- Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py`
**What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked
**Why critical:**
- Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:**
- No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically
- Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created
- HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading
- Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors
**Structure:**
```python
coordinator.data = {
"user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory
**Current Status:** HA's response lifecycle
- HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code
@ -207,29 +234,30 @@ coordinator.data = {
### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------|
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
| Category | Status | Tests | File | Coverage |
| ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale |
|----------|--------|-----------|
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
| Category | Status | Rationale |
| ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:**
- ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners)

52
docs/developer/versioned_docs/version-v0.21.0/debugging.md
View file

@ -10,9 +10,9 @@ Add to `configuration.yaml`:
```yaml
logger:
default: info
logs:
custom_components.tibber_prices: debug
default: info
logs:
custom_components.tibber_prices: debug
```
Restart Home Assistant to apply.
@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages
**Coordinator Updates:**
```
[custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
```
**Period Calculation:**
```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
```
**API Errors:**
```
[custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -47,26 +50,27 @@ Restart Home Assistant to apply.
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
}
```
### Set Breakpoints
**Coordinator update:**
```python
# coordinator/core.py
async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
```
**Period calculation:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
```
**Flags:**
- `-v` - Verbose output
- `-s` - Show print statements
- `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns
**Print coordinator data:**
```python
def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
```
**Inspect period attributes:**
```python
def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading
**Check:**
```bash
grep "tibber_prices" config/home-assistant.log
```
**Common causes:**
- Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating
**Check coordinator state:**
```python
# In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }}
```
**Debug in code:**
```python
# Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong
**Enable detailed period logs:**
```python
# coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
```
**Check filter statistics:**
```
[period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy
Add to coordinator code:
```python
import debugpy
debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL
Install in container:
```bash
uv pip install ipython
```
Add breakpoint:
```python
from IPython import embed
embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
---
💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure

123
docs/developer/versioned_docs/version-v0.21.0/intro.md
View file

@ -8,25 +8,25 @@ This is an independent, community-maintained custom integration for Home Assista
## 📚 Developer Guides
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
## 🤖 AI Documentation
The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md). This file serves as long-term memory for AI assistants and contains:
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
**Important:** When proposing changes to patterns or conventions, always update [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) to keep AI guidance consistent.
@ -34,32 +34,32 @@ The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlow
This integration is developed with extensive AI assistance (GitHub Copilot, Claude, and other AI tools). The AI handles:
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
**Quality Assurance:**
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
**Benefits:**
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
**Limitations:**
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
If you're working with AI tools on this project, the [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) file provides the context and patterns that ensure consistency.
@ -80,15 +80,15 @@ If you're working with AI tools on this project, the [`AGENTS.md`](https://githu
The project includes several helper scripts in `./scripts/`:
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
## 📦 Project Structure
@ -121,23 +121,23 @@ custom_components/tibber_prices/
**DataUpdateCoordinator Pattern:**
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
**Price Data Enrichment:**
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
**Translation System:**
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
## 🧪 Testing
@ -159,18 +159,19 @@ pytest --cov=custom_components.tibber_prices tests/
Documentation is organized in two Docusaurus sites:
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:**
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
## 🤝 Contributing

17
docs/developer/versioned_docs/version-v0.21.0/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals
Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching
**1. Persistent Cache** (API data):
```python
# Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
```
**2. Translation Cache** (in-memory):
```python
# Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
```
**3. Config Cache** (invalidated on options change):
```python
class DataTransformer:
def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading
**Load data only when needed:**
```python
@property
def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations
**Process multiple items at once:**
```python
# ❌ Slow - loop with individual operations
for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices
**1. Concurrent API calls:**
```python
# ❌ Sequential (slow)
user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
```
**2. Don't block event loop:**
```python
# ❌ Blocking
result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks
**1. Clear references:**
```python
class Coordinator:
async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
```
**2. Use weak references for callbacks:**
```python
import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures
**Use appropriate types:**
```python
# ❌ List for lookups (O(n))
if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls
**Already implemented:**
- Cache valid until midnight
- User data cached for 24h
- Only poll when tomorrow data expected
**Monitor API usage:**
```python
_LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates
**Only update when needed:**
```python
async def _async_update_data(self) -> dict:
"""Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection
**Affects long-term statistics storage:**
```python
# ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size
**Keep attributes minimal:**
```python
# ❌ Large nested structures (KB per update)
attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
---
💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools

314
docs/developer/versioned_docs/version-v0.21.0/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:**
```python
# Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
```
**Example (Best Price):**
- Daily Min: 10 ct/kWh
- Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:**
```python
# Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
```
**Example (Best Price):**
- Daily Avg: 15 ct/kWh
- Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -65,17 +70,17 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
The integration maintains **two independent sets** of volatility thresholds:
1. **Sensor Thresholds** (user-configurable via `CONF_VOLATILITY_*_THRESHOLD`)
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
2. **Period Filter Thresholds** (internal, fixed)
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
**Rationale for Separation:**
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:**
```python
# Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,36 +113,42 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:**
- Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh
**Flex Filter (50%):**
```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
```
**Min Distance Filter (5%):**
```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
```
**Conflict:**
- Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
**The Issue:** At high Flex values, Min_Distance becomes the dominant filter and blocks intervals that Flex would permit. This defeats the purpose of having high Flex.
### Mathematical Analysis
**Conflict condition for Best Price:**
```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
```
**Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:**
```python
if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20
@ -158,15 +171,16 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------|
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
| ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:**
```python
# coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
```
**Why Linear Scaling?**
- Simple and predictable
- No abrupt behavior changes
- Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?**
- Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:**
```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
```
**Defined in `const.py`:**
```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances
**Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average"
**Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:**
- Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction
**Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy
- User can manually decide whether to react
**Defaults:**
- **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:**
- Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:**
Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times
**Example day:**
```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
```
**Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices
- Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:**
- Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:**
```
Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended)
**Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side"
**High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20%
#### Without Relaxation
**Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits)
---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach
**Each day processed independently:**
1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
```
**Constants:**
```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -427,26 +468,27 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Design Decisions:**
1. **Why 3% fixed increment?**
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
2. **Why hard-coded, not configurable?**
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
3. **Why warn at 25% base flex?**
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
**Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex:
```python
increment = base_flex × (step_pct / 100) # REMOVED
```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:**
```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy
**Per Flex level, try in order:**
1. Original Level filter
2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):**
```
Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions
**Period Calculation Entry Point:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
```
**Flex + Distance Filtering:**
```python
# coordinator/period_handlers/level_filtering.py
def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
```
**Relaxation Orchestration:**
```python
# coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -526,43 +574,45 @@ def relax_single_day(...) -> tuple[dict, dict]
**Algorithm Details:**
1. **Linear Regression Prediction:**
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
2. **Confidence Intervals:**
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
3. **Symmetry Check:**
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
4. **Enhanced Zigzag Detection:**
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:**
```python
# coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
```
**Data Integrity:**
- Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome
**Performance:**
- Single pass through price data
- O(n) complexity with small context window
- No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:**
```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -598,19 +651,19 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
**Why This Approach?**
1. **Linear regression over moving average:**
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
2. **Symmetry check over fixed threshold:**
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
3. **Single-pass over iterative:**
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
**Alternative Approaches Considered:**
@ -624,15 +677,17 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips
**Enable DEBUG logging:**
```yaml
# configuration.yaml
logger:
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
```
**Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,52 +700,61 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:**
```yaml
best_price_flex: 40
enable_relaxation_best: true
```
**Problem:**
- Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns
**Solution:**
```yaml
best_price_flex: 15 # Let relaxation increase it
best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true
```
### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:**
```yaml
best_price_min_distance_from_avg: 0
```
**Problem:**
- "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times
**Solution:**
```yaml
best_price_min_distance_from_avg: 5 # Use default 5%
best_price_min_distance_from_avg: 5 # Use default 5%
```
### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:**
```yaml
best_price_flex: 45
best_price_min_distance_from_avg: 10
```
**Problem:**
- Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal
**Solution:**
```yaml
best_price_flex: 20
best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh
**Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:**
```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -790,31 +862,31 @@ INFO: Period calculation completed: 1/2 days reached target
When debugging period calculation issues:
1. **Check Filter Statistics**
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
2. **Check Relaxation Behavior**
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
3. **Check Flex Warnings**
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
4. **Check Min_Distance Scaling**
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
5. **Check Outlier Filtering**
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
---
@ -823,19 +895,19 @@ When debugging period calculation issues:
### Potential Improvements
1. **Adaptive Flex Calculation:**
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
2. **Machine Learning Approach:**
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
3. **Multi-Objective Optimization:**
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
### Known Limitations
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:**
```python
# Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
```
**Benefits:**
- Eliminates need for relaxation on most days
- Self-adjusting to market conditions
- Better user experience (less configuration needed)
**Challenges:**
- Harder to predict behavior (less transparent)
- May conflict with user's mental model
- Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback
**Approach:**
- Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn per-user preferences over time
**Benefits:**
- Personalized to user's actual behavior
- Adapts to local market patterns
- Could discover non-obvious patterns
**Challenges:**
- Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously
**Goals:**
- Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:**
- Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing
**Benefits:**
- More sophisticated period selection
- Better match to user's actual needs
- Could handle complex appliance requirements
**Challenges:**
- Much more complex to implement
- Harder to explain to users
- Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex
**Example:**
- Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:**
- Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve
```python
# Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
```
**Why Not Implemented:**
- Linear is easier to reason about
- No evidence that non-linear is better
- Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day
**Example:**
- All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances)
**Possible Solution:**
- Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences
- Consider user's typical usage patterns
**Why Not Implemented:**
- Adds complexity
- Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:**
```python
# In period_building.py build_periods():
for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split:
```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,15 +1154,17 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting:
```yaml
condition:
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
```
Low volatility (< 15%) means classification changes are less economically significant.
@ -1077,24 +1172,25 @@ Low volatility (< 15%) means classification changes are less economically signif
**Alternative Approaches Rejected:**
1. **Use period start day for all intervals**
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
2. **Adjust flex/distance at midnight**
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
3. **Split at midnight always**
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
4. **Use next day's reference after midnight**
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

72
docs/developer/versioned_docs/version-v0.21.0/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
```
**Key Points:**
- Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely
- Don't provide analytical value in history
- Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures
**Attributes:**
- `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each)
- Slow down history queries
@ -66,20 +70,21 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:**
```json
{
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8,
// ... 10+ more attributes × 10-20 periods
}
]
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8
// ... 10+ more attributes × 10-20 periods
}
]
}
```
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:**
- Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value
- Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:**
- Configuration values that rarely change
- Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:**
- Only relevant at moment of reading
- Won't be valid after some time
- Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:**
- Detailed technical information not needed for historical analysis
- Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:**
- Can be calculated from other attributes
- Redundant information
- Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states
### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns
### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data
- `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings
**Per state change:**
- Before: ~3-8 KB average
- After: ~0.5-1.5 KB average
- **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact
For a typical installation with:
- 80+ sensors
- Updates every 15 minutes
- ~10 sensors updating every minute
@ -207,14 +224,14 @@ For a typical installation with:
## Implementation Files
- **Sensor Platform**: `custom_components/tibber_prices/sensor/core.py`
- Class: `TibberPricesSensor`
- 47 attributes excluded
- Class: `TibberPricesSensor`
- 47 attributes excluded
- **Binary Sensor Platform**: `custom_components/tibber_prices/binary_sensor/core.py`
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
## When to Update _unrecorded_attributes
## When to Update \_unrecorded_attributes
### Add to Exclusion List When:
@ -236,24 +253,24 @@ For a typical installation with:
When adding a new attribute, ask:
1. **Will this be useful in history queries 1 week from now?**
- No → Exclude
- Yes → Keep
- No → Exclude
- Yes → Keep
2. **Can this be calculated from other recorded attributes?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
3. **Is this primarily for current UI display?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
4. **Does this change frequently without indicating state change?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
5. **Is this larger than 100 bytes and not essential for analysis?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
## Testing
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:**
```sql
SELECT
state_id,

200
docs/developer/versioned_docs/version-v0.21.0/refactoring-guide.md
View file

@ -8,22 +8,22 @@ Not every code change needs a detailed plan. Create a refactoring plan when:
🔴 **Major changes requiring planning:**
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
🟡 **Medium changes that might benefit from planning:**
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
🟢 **Small changes - no planning needed:**
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
## The Planning Process
@ -51,34 +51,34 @@ Every planning document should include:
## Problem Statement
- What's the issue?
- Why does it need fixing?
- Current pain points
- What's the issue?
- Why does it need fixing?
- Current pain points
## Proposed Solution
- High-level approach
- File structure (before/after)
- Module responsibilities
- High-level approach
- File structure (before/after)
- Module responsibilities
## Migration Strategy
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
## Risks & Mitigation
- What could go wrong?
- How to prevent it?
- Rollback strategy
- What could go wrong?
- How to prevent it?
- Rollback strategy
## Success Criteria
- Measurable improvements
- Testing requirements
- Verification steps
- Measurable improvements
- Testing requirements
- Verification steps
```
See `planning/README.md` for detailed template explanation.
@ -87,19 +87,19 @@ See `planning/README.md` for detailed template explanation.
Since `planning/` is git-ignored:
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
### 4. Implementation Phase
Once plan is approved:
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
### 5. After Completion
@ -134,13 +134,13 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Before:**
- `sensor.py` - 2,574 lines, hard to navigate
- `sensor.py` - 2,574 lines, hard to navigate
**After:**
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
**Process:**
@ -153,10 +153,10 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Key learnings:**
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
**Note:** The complete module splitting plan was documented during implementation but has been superseded by the actual code structure.
@ -166,11 +166,11 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
Breaking refactorings into phases:
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
### Phase Structure
@ -191,8 +191,8 @@ Each phase should:
**File Lifecycle**:
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
**Steps**:
@ -205,10 +205,10 @@ Each phase should:
**Success criteria**:
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
```
## Testing Strategy
@ -238,13 +238,13 @@ Minimum testing checklist:
After completing all phases:
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
## Common Pitfalls
@ -286,21 +286,21 @@ This project uses AI heavily (GitHub Copilot, Claude). The planning process supp
**AI reads from:**
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
**AI updates:**
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
**Why separate AGENTS.md and docs/development/?**
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) section "Planning Major Refactorings" for AI-specific guidance.
@ -308,16 +308,16 @@ See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENT
### Planning Directory
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
### Example Plans
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
### Helper Scripts
@ -341,21 +341,21 @@ Simple rule: If you can't describe the entire change in 3 sentences, create a pl
Good plan level:
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
Too detailed:
- Exact code snippets for every change
- Line-by-line instructions
- Exact code snippets for every change
- Line-by-line instructions
Too vague:
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
### Q: What if the plan changes during implementation?
@ -363,9 +363,9 @@ Too vague:
If you discover:
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
Document WHY the plan changed (helps future refactorings).
@ -373,9 +373,9 @@ Document WHY the plan changed (helps future refactorings).
**A:** No! Use judgment:
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
### Q: How do I know when a refactoring is successful?
@ -383,12 +383,12 @@ Document WHY the plan changed (helps future refactorings).
Typical criteria:
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
If you can't tick all boxes, the refactoring isn't done.
@ -409,6 +409,6 @@ If you can't tick all boxes, the refactoring isn't done.
**Next steps:**
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans

36
docs/developer/versioned_docs/version-v0.21.0/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):**
**git-cliff** (template-based):
```bash
# See: https://git-cliff.org/docs/installation
@ -190,13 +192,13 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which
| Method | Use Case | Pros | Cons |
|--------|----------|------|------|
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
| Method | Use Case | Pros | Cons |
| --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
---
@ -219,6 +221,7 @@ git push origin main v0.3.0
```
**What happens:**
1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
```
**What happens:**
1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
```
**What happens:**
1. You create and push tag manually
2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features
### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind.
---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely
- Invalid version format in manifest.json
- manifest.json not in the commit that was pushed
@ -348,13 +359,14 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results:
```
feat(scope): Add new feature
Detailed description of what changed.
```
feat(scope): Add new feature
Impact: Users can now do X and Y.
```
Detailed description of what changed.
Impact: Users can now do X and Y.
```
2. **Impact Section:** Add `Impact:` in commit body for user-friendly descriptions

32
docs/developer/versioned_docs/version-v0.21.0/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:**
- **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available
**When cleared:**
- Tomorrow's data becomes available
- Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:**
```python
# In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:**
- Successful API call completes (no rate limit error)
- Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:**
```python
# In error handler
is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}`
**When triggered:**
- Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check)
**When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:**
```python
# After user data update
home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
```
**Translation placeholders:**
- `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration
**Coordinator Initialization:**
```python
self._repair_manager = TibberPricesRepairManager(
hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
```
**Update Cycle Integration:**
```python
# Success path - check conditions
if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
```
**Shutdown Cleanup:**
```python
async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up."""
@ -196,24 +212,27 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json`
**Structure:**
```json
{
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
}
}
}
}
```
## Home Assistant Integration
Repairs appear in:
- **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count)
Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:**
```python
# Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair
**Manual test:**
- Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file
**Example template:**
```python
async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair."""

16
docs/developer/versioned_docs/version-v0.21.0/setup.md
View file

@ -4,9 +4,9 @@
## Prerequisites
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
## Quick Setup
@ -26,11 +26,11 @@ code .
The DevContainer includes:
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
## Running the Integration

16
docs/developer/versioned_docs/version-v0.21.0/testing.md
View file

@ -13,10 +13,10 @@ Before running tests or committing changes, validate the integration structure:
This lightweight script checks:
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
**Note:** Full hassfest validation runs in GitHub Actions on push.
@ -42,10 +42,10 @@ pytest --cov=custom_components.tibber_prices tests/
Then test in Home Assistant UI:
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
## Test Guidelines

44
docs/developer/versioned_docs/version-v0.21.0/timer-architecture.md
View file

@ -10,11 +10,11 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------|
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
| Timer | Type | Interval | Purpose | Trigger Method |
| ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
**Key principle:** Timer #1 (HA) controls **data fetching**, Timer #2 controls **entity updates**, Timer #3 controls **timing displays**.
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
```
**Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:**
- Automatic restart after HA restart
- Built-in retry logic for temporary failures
- Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
```
**Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total
**Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
```
**Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total
**Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation)
**Why NOT every second:**
- Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
```
**Why this pattern:**
- Decouples timer logic from entity logic
- One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem
With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:**
- Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover)
- Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics
### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all
@ -393,16 +412,16 @@ _LOGGER.setLevel(logging.DEBUG)
### Common Issues
1. **Timer #2 not triggering:**
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
2. **Double updates at midnight:**
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
3. **API overload:**
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
---
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary
**Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:**
- Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:**
- Timer = mechanism that triggers
- Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies

131
docs/developer/versioned_docs/version-v0.22.0/api-reference.md
View file

@ -22,30 +22,30 @@ Fetches home information and metadata:
```graphql
query {
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
}
```
@ -56,26 +56,27 @@ query {
Fetches quarter-hourly prices:
```graphql
query($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
query ($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
}
}
}
}
}
}
}
}
}
}
```
**Parameters:**
- `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits
Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute
Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home
@ -99,13 +102,14 @@ Integration stays well below these limits:
```json
{
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
}
```
**Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -114,11 +118,12 @@ Integration stays well below these limits:
```json
{
"currency": "EUR"
"currency": "EUR"
}
```
Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses
**Invalid Token:**
```json
{
"errors": [{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}]
"errors": [
{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}
]
}
```
**Rate Limit Exceeded:**
```json
{
"errors": [{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}]
"errors": [
{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}
]
}
```
**Home Not Found:**
```json
{
"errors": [{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}]
"errors": [
{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}
]
}
```
Integration handles these with:
- Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation
Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
---
💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token)

143
docs/developer/versioned_docs/version-v0.22.0/architecture.md
View file

@ -100,43 +100,43 @@ flowchart TB
### Flow Description
1. **Setup** (`__init__.py`)
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
2. **Data Fetch** (every 15 minutes)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
3. **Price Enrichment**
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
4. **Period Calculation**
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
5. **Entity Updates**
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
6. **Service Calls**
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
---
@ -146,13 +146,13 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------|
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
| Layer | Location | Lifetime | Invalidation | Memory |
| ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
**Total cache overhead:** ~126KB per coordinator instance (main entry + subentries)
@ -195,30 +195,31 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components
| Component | File | Responsibility |
|-----------|------|----------------|
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
| Component | File | Responsibility |
| --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
### Sensor Architecture (Calculator Pattern)
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------|
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
| Component | Files | Lines | Responsibility |
| ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata
**Benefits:**
- 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator
@ -237,12 +239,12 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities
| Utility | File | Purpose |
|---------|------|---------|
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
| Utility | File | Purpose |
| ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
---
@ -283,12 +285,12 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
- **API polling**: Every 15 minutes (coordinator fetch cycle)
- **Entity updates**: On 00/15/30/45-minute boundaries via `coordinator/listeners.py`
- **Timer scheduling**: Uses `async_track_utc_time_change(minute=[0, 15, 30, 45], second=0)`
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- **Absolute time tracking**: Timer plans for **all future boundaries** (not relative delays)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- **Result**: Current price sensors update without waiting for next API poll
### 4. Calculator Pattern (Sensor Platform)
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:**
- Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing
@ -333,12 +340,12 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization
| Optimization | Location | Savings |
|--------------|----------|---------|
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
| Optimization | Location | Savings |
| ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
### Memory Usage

132
docs/developer/versioned_docs/version-v0.22.0/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation
**Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage
@ -36,29 +38,31 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator):
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
2. **Cache validation on load**:
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
3. **Tomorrow data check** (after 13:00):
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
**Why this cache matters:** Reduces API load on Tibber (~192 intervals per fetch), speeds up HA restarts, enables offline operation until cache expires.
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:**
- **Forever** (until HA restart)
- No invalidation during runtime
**When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once
**Access pattern:**
```python
# Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:**
### DataTransformer Config Cache
```python
{
"thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
### PeriodCalculator Config Cache
```python
{
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,20 +128,23 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Lifetime:**
- Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle
**Invalidation trigger:**
- **Options change** (user reconfigures integration):
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
**Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:**
```python
{
"best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Cache key:** Hash of relevant inputs
```python
hash_data = (
today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
```
**Lifetime:**
- Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data)
@ -179,24 +195,27 @@ hash_data = (
**Invalidation triggers:**
1. **Config change** (explicit):
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
2. **Price data change** (automatic via hash mismatch):
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
**Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:**
```python
{
"timestamp": ...,
@ -224,14 +244,16 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:**
- Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when:
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
**Architecture:**
- DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow
### User Changes Options (Config Flow)
```
User saves options
↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
```
### Midnight Turnover (Day Transition)
```
Timer #2 fires at 00:00
↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
```
### Tomorrow Data Arrives (~13:00)
```
Coordinator update cycle
↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
```
**No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load)
**Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics
### Typical Operation (No Changes)
```
Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
```
### After Midnight Turnover
```
Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
```
### After Config Change
```
Options Update
├─> Cache invalidation: `<`1ms
@ -381,23 +411,25 @@ Options Update
## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------|
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
| Cache Type | Lifetime | Size | Invalidation | Purpose |
| ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
**Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:**
- 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup)
**Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues
### Symptom: Stale data after config change
**Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating
**Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today
**Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations
**Check:**
1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys

11
docs/developer/versioned_docs/version-v0.22.0/coding-guidelines.md
View file

@ -8,10 +8,10 @@ comments: false
## Code Style
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
Run before committing:
@ -41,12 +41,14 @@ class TimeService:
```
**When prefix is required:**
- Public classes used across multiple modules
- All exception classes
- All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module

25
docs/developer/versioned_docs/version-v0.22.0/contributing.md
View file

@ -14,10 +14,10 @@ Welcome! This guide helps you contribute to the Tibber Prices integration.
1. Fork the repository on GitHub
2. Clone your fork:
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
3. Open in VS Code
4. Click "Reopen in Container" when prompted
@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
```
**Branch naming:**
- `feature/` - New features
- `fix/` - Bug fixes
- `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:**
```bash
./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
```
Run your test:
```bash
./scripts/test tests/test_your_feature.py -v
```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
```
**Commit types:**
- `feat:` - New feature
- `fix:` - Bug fix
- `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance
**Add scope when relevant:**
- `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max)
Description should include:
```markdown
## What
Brief description of changes
## Why
Problem being solved or feature rationale
## How
Implementation approach
## Testing
- [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated
- [ ] Type checking passes
- [ ] Linting passes
## Breaking Changes
(If any - describe migration path)
## Related Issues
Closes #123
```
### PR Checklist
Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For
✅ **Good:**
- Clear, self-explanatory code
- Appropriate comments for complex logic
- Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns
❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes
- Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On
Good first issues are labeled:
- `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
---
💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests

65
docs/developer/versioned_docs/version-v0.22.0/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅
**What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners
**Why critical:**
- Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅
**What is tested:**
- Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared
- Both timers are cancelled together
- Cleanup works even when timers are `None`
**Why critical:**
- Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅
**What is tested:**
- Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()`
**Why critical:**
- `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation
**What is tested:**
- DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update
**Why critical:**
- Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis
**Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal
**What is tested:**
- Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss)
**Why critical:**
- Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:**
- `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py`
**What is tested:**
- Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly
**Why critical:**
- Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py`
**What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked
**Why critical:**
- Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:**
- No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically
- Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created
- HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading
- Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors
**Structure:**
```python
coordinator.data = {
"user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory
**Current Status:** HA's response lifecycle
- HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code
@ -207,29 +234,30 @@ coordinator.data = {
### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------|
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
| Category | Status | Tests | File | Coverage |
| ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale |
|----------|--------|-----------|
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
| Category | Status | Rationale |
| ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:**
- ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners)

52
docs/developer/versioned_docs/version-v0.22.0/debugging.md
View file

@ -10,9 +10,9 @@ Add to `configuration.yaml`:
```yaml
logger:
default: info
logs:
custom_components.tibber_prices: debug
default: info
logs:
custom_components.tibber_prices: debug
```
Restart Home Assistant to apply.
@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages
**Coordinator Updates:**
```
[custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
```
**Period Calculation:**
```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
```
**API Errors:**
```
[custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -47,26 +50,27 @@ Restart Home Assistant to apply.
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
}
```
### Set Breakpoints
**Coordinator update:**
```python
# coordinator/core.py
async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
```
**Period calculation:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
```
**Flags:**
- `-v` - Verbose output
- `-s` - Show print statements
- `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns
**Print coordinator data:**
```python
def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
```
**Inspect period attributes:**
```python
def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading
**Check:**
```bash
grep "tibber_prices" config/home-assistant.log
```
**Common causes:**
- Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating
**Check coordinator state:**
```python
# In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }}
```
**Debug in code:**
```python
# Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong
**Enable detailed period logs:**
```python
# coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
```
**Check filter statistics:**
```
[period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy
Add to coordinator code:
```python
import debugpy
debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL
Install in container:
```bash
uv pip install ipython
```
Add breakpoint:
```python
from IPython import embed
embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
---
💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure

123
docs/developer/versioned_docs/version-v0.22.0/intro.md
View file

@ -8,25 +8,25 @@ This is an independent, community-maintained custom integration for Home Assista
## 📚 Developer Guides
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
## 🤖 AI Documentation
The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md). This file serves as long-term memory for AI assistants and contains:
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
**Important:** When proposing changes to patterns or conventions, always update [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) to keep AI guidance consistent.
@ -34,32 +34,32 @@ The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlow
This integration is developed with extensive AI assistance (GitHub Copilot, Claude, and other AI tools). The AI handles:
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
**Quality Assurance:**
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
**Benefits:**
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
**Limitations:**
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
If you're working with AI tools on this project, the [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) file provides the context and patterns that ensure consistency.
@ -80,15 +80,15 @@ If you're working with AI tools on this project, the [`AGENTS.md`](https://githu
The project includes several helper scripts in `./scripts/`:
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
## 📦 Project Structure
@ -121,23 +121,23 @@ custom_components/tibber_prices/
**DataUpdateCoordinator Pattern:**
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
**Price Data Enrichment:**
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
**Translation System:**
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
## 🧪 Testing
@ -159,18 +159,19 @@ pytest --cov=custom_components.tibber_prices tests/
Documentation is organized in two Docusaurus sites:
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:**
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
## 🤝 Contributing

17
docs/developer/versioned_docs/version-v0.22.0/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals
Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching
**1. Persistent Cache** (API data):
```python
# Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
```
**2. Translation Cache** (in-memory):
```python
# Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
```
**3. Config Cache** (invalidated on options change):
```python
class DataTransformer:
def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading
**Load data only when needed:**
```python
@property
def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations
**Process multiple items at once:**
```python
# ❌ Slow - loop with individual operations
for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices
**1. Concurrent API calls:**
```python
# ❌ Sequential (slow)
user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
```
**2. Don't block event loop:**
```python
# ❌ Blocking
result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks
**1. Clear references:**
```python
class Coordinator:
async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
```
**2. Use weak references for callbacks:**
```python
import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures
**Use appropriate types:**
```python
# ❌ List for lookups (O(n))
if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls
**Already implemented:**
- Cache valid until midnight
- User data cached for 24h
- Only poll when tomorrow data expected
**Monitor API usage:**
```python
_LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates
**Only update when needed:**
```python
async def _async_update_data(self) -> dict:
"""Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection
**Affects long-term statistics storage:**
```python
# ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size
**Keep attributes minimal:**
```python
# ❌ Large nested structures (KB per update)
attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
---
💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools

314
docs/developer/versioned_docs/version-v0.22.0/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:**
```python
# Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
```
**Example (Best Price):**
- Daily Min: 10 ct/kWh
- Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:**
```python
# Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
```
**Example (Best Price):**
- Daily Avg: 15 ct/kWh
- Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -65,17 +70,17 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
The integration maintains **two independent sets** of volatility thresholds:
1. **Sensor Thresholds** (user-configurable via `CONF_VOLATILITY_*_THRESHOLD`)
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
2. **Period Filter Thresholds** (internal, fixed)
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
**Rationale for Separation:**
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:**
```python
# Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,36 +113,42 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:**
- Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh
**Flex Filter (50%):**
```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
```
**Min Distance Filter (5%):**
```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
```
**Conflict:**
- Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
**The Issue:** At high Flex values, Min_Distance becomes the dominant filter and blocks intervals that Flex would permit. This defeats the purpose of having high Flex.
### Mathematical Analysis
**Conflict condition for Best Price:**
```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
```
**Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:**
```python
if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20
@ -158,15 +171,16 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------|
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
| ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:**
```python
# coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
```
**Why Linear Scaling?**
- Simple and predictable
- No abrupt behavior changes
- Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?**
- Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:**
```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
```
**Defined in `const.py`:**
```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances
**Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average"
**Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:**
- Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction
**Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy
- User can manually decide whether to react
**Defaults:**
- **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:**
- Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:**
Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times
**Example day:**
```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
```
**Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices
- Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:**
- Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:**
```
Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended)
**Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side"
**High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20%
#### Without Relaxation
**Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits)
---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach
**Each day processed independently:**
1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
```
**Constants:**
```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -427,26 +468,27 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Design Decisions:**
1. **Why 3% fixed increment?**
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
2. **Why hard-coded, not configurable?**
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
3. **Why warn at 25% base flex?**
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
**Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex:
```python
increment = base_flex × (step_pct / 100) # REMOVED
```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:**
```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy
**Per Flex level, try in order:**
1. Original Level filter
2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):**
```
Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions
**Period Calculation Entry Point:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
```
**Flex + Distance Filtering:**
```python
# coordinator/period_handlers/level_filtering.py
def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
```
**Relaxation Orchestration:**
```python
# coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -526,43 +574,45 @@ def relax_single_day(...) -> tuple[dict, dict]
**Algorithm Details:**
1. **Linear Regression Prediction:**
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
2. **Confidence Intervals:**
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
3. **Symmetry Check:**
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
4. **Enhanced Zigzag Detection:**
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:**
```python
# coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
```
**Data Integrity:**
- Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome
**Performance:**
- Single pass through price data
- O(n) complexity with small context window
- No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:**
```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -598,19 +651,19 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
**Why This Approach?**
1. **Linear regression over moving average:**
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
2. **Symmetry check over fixed threshold:**
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
3. **Single-pass over iterative:**
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
**Alternative Approaches Considered:**
@ -624,15 +677,17 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips
**Enable DEBUG logging:**
```yaml
# configuration.yaml
logger:
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
```
**Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,52 +700,61 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:**
```yaml
best_price_flex: 40
enable_relaxation_best: true
```
**Problem:**
- Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns
**Solution:**
```yaml
best_price_flex: 15 # Let relaxation increase it
best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true
```
### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:**
```yaml
best_price_min_distance_from_avg: 0
```
**Problem:**
- "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times
**Solution:**
```yaml
best_price_min_distance_from_avg: 5 # Use default 5%
best_price_min_distance_from_avg: 5 # Use default 5%
```
### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:**
```yaml
best_price_flex: 45
best_price_min_distance_from_avg: 10
```
**Problem:**
- Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal
**Solution:**
```yaml
best_price_flex: 20
best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh
**Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:**
```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -790,31 +862,31 @@ INFO: Period calculation completed: 1/2 days reached target
When debugging period calculation issues:
1. **Check Filter Statistics**
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
2. **Check Relaxation Behavior**
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
3. **Check Flex Warnings**
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
4. **Check Min_Distance Scaling**
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
5. **Check Outlier Filtering**
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
---
@ -823,19 +895,19 @@ When debugging period calculation issues:
### Potential Improvements
1. **Adaptive Flex Calculation:**
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
2. **Machine Learning Approach:**
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
3. **Multi-Objective Optimization:**
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
### Known Limitations
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:**
```python
# Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
```
**Benefits:**
- Eliminates need for relaxation on most days
- Self-adjusting to market conditions
- Better user experience (less configuration needed)
**Challenges:**
- Harder to predict behavior (less transparent)
- May conflict with user's mental model
- Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback
**Approach:**
- Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn per-user preferences over time
**Benefits:**
- Personalized to user's actual behavior
- Adapts to local market patterns
- Could discover non-obvious patterns
**Challenges:**
- Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously
**Goals:**
- Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:**
- Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing
**Benefits:**
- More sophisticated period selection
- Better match to user's actual needs
- Could handle complex appliance requirements
**Challenges:**
- Much more complex to implement
- Harder to explain to users
- Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex
**Example:**
- Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:**
- Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve
```python
# Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
```
**Why Not Implemented:**
- Linear is easier to reason about
- No evidence that non-linear is better
- Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day
**Example:**
- All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances)
**Possible Solution:**
- Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences
- Consider user's typical usage patterns
**Why Not Implemented:**
- Adds complexity
- Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:**
```python
# In period_building.py build_periods():
for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split:
```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,15 +1154,17 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting:
```yaml
condition:
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
```
Low volatility (< 15%) means classification changes are less economically significant.
@ -1077,24 +1172,25 @@ Low volatility (< 15%) means classification changes are less economically signif
**Alternative Approaches Rejected:**
1. **Use period start day for all intervals**
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
2. **Adjust flex/distance at midnight**
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
3. **Split at midnight always**
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
4. **Use next day's reference after midnight**
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

72
docs/developer/versioned_docs/version-v0.22.0/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
```
**Key Points:**
- Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely
- Don't provide analytical value in history
- Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures
**Attributes:**
- `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each)
- Slow down history queries
@ -66,20 +70,21 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:**
```json
{
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8,
// ... 10+ more attributes × 10-20 periods
}
]
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8
// ... 10+ more attributes × 10-20 periods
}
]
}
```
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:**
- Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value
- Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:**
- Configuration values that rarely change
- Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:**
- Only relevant at moment of reading
- Won't be valid after some time
- Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:**
- Detailed technical information not needed for historical analysis
- Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:**
- Can be calculated from other attributes
- Redundant information
- Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states
### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns
### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data
- `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings
**Per state change:**
- Before: ~3-8 KB average
- After: ~0.5-1.5 KB average
- **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact
For a typical installation with:
- 80+ sensors
- Updates every 15 minutes
- ~10 sensors updating every minute
@ -207,14 +224,14 @@ For a typical installation with:
## Implementation Files
- **Sensor Platform**: `custom_components/tibber_prices/sensor/core.py`
- Class: `TibberPricesSensor`
- 47 attributes excluded
- Class: `TibberPricesSensor`
- 47 attributes excluded
- **Binary Sensor Platform**: `custom_components/tibber_prices/binary_sensor/core.py`
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
## When to Update _unrecorded_attributes
## When to Update \_unrecorded_attributes
### Add to Exclusion List When:
@ -236,24 +253,24 @@ For a typical installation with:
When adding a new attribute, ask:
1. **Will this be useful in history queries 1 week from now?**
- No → Exclude
- Yes → Keep
- No → Exclude
- Yes → Keep
2. **Can this be calculated from other recorded attributes?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
3. **Is this primarily for current UI display?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
4. **Does this change frequently without indicating state change?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
5. **Is this larger than 100 bytes and not essential for analysis?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
## Testing
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:**
```sql
SELECT
state_id,

200
docs/developer/versioned_docs/version-v0.22.0/refactoring-guide.md
View file

@ -8,22 +8,22 @@ Not every code change needs a detailed plan. Create a refactoring plan when:
🔴 **Major changes requiring planning:**
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
🟡 **Medium changes that might benefit from planning:**
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
🟢 **Small changes - no planning needed:**
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
## The Planning Process
@ -51,34 +51,34 @@ Every planning document should include:
## Problem Statement
- What's the issue?
- Why does it need fixing?
- Current pain points
- What's the issue?
- Why does it need fixing?
- Current pain points
## Proposed Solution
- High-level approach
- File structure (before/after)
- Module responsibilities
- High-level approach
- File structure (before/after)
- Module responsibilities
## Migration Strategy
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
## Risks & Mitigation
- What could go wrong?
- How to prevent it?
- Rollback strategy
- What could go wrong?
- How to prevent it?
- Rollback strategy
## Success Criteria
- Measurable improvements
- Testing requirements
- Verification steps
- Measurable improvements
- Testing requirements
- Verification steps
```
See `planning/README.md` for detailed template explanation.
@ -87,19 +87,19 @@ See `planning/README.md` for detailed template explanation.
Since `planning/` is git-ignored:
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
### 4. Implementation Phase
Once plan is approved:
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
### 5. After Completion
@ -134,13 +134,13 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Before:**
- `sensor.py` - 2,574 lines, hard to navigate
- `sensor.py` - 2,574 lines, hard to navigate
**After:**
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
**Process:**
@ -153,10 +153,10 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Key learnings:**
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
**Note:** The complete module splitting plan was documented during implementation but has been superseded by the actual code structure.
@ -166,11 +166,11 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
Breaking refactorings into phases:
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
### Phase Structure
@ -191,8 +191,8 @@ Each phase should:
**File Lifecycle**:
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
**Steps**:
@ -205,10 +205,10 @@ Each phase should:
**Success criteria**:
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
```
## Testing Strategy
@ -238,13 +238,13 @@ Minimum testing checklist:
After completing all phases:
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
## Common Pitfalls
@ -286,21 +286,21 @@ This project uses AI heavily (GitHub Copilot, Claude). The planning process supp
**AI reads from:**
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
**AI updates:**
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
**Why separate AGENTS.md and docs/development/?**
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) section "Planning Major Refactorings" for AI-specific guidance.
@ -308,16 +308,16 @@ See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENT
### Planning Directory
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
### Example Plans
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
### Helper Scripts
@ -341,21 +341,21 @@ Simple rule: If you can't describe the entire change in 3 sentences, create a pl
Good plan level:
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
Too detailed:
- Exact code snippets for every change
- Line-by-line instructions
- Exact code snippets for every change
- Line-by-line instructions
Too vague:
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
### Q: What if the plan changes during implementation?
@ -363,9 +363,9 @@ Too vague:
If you discover:
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
Document WHY the plan changed (helps future refactorings).
@ -373,9 +373,9 @@ Document WHY the plan changed (helps future refactorings).
**A:** No! Use judgment:
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
### Q: How do I know when a refactoring is successful?
@ -383,12 +383,12 @@ Document WHY the plan changed (helps future refactorings).
Typical criteria:
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
If you can't tick all boxes, the refactoring isn't done.
@ -409,6 +409,6 @@ If you can't tick all boxes, the refactoring isn't done.
**Next steps:**
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans

36
docs/developer/versioned_docs/version-v0.22.0/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):**
**git-cliff** (template-based):
```bash
# See: https://git-cliff.org/docs/installation
@ -190,13 +192,13 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which
| Method | Use Case | Pros | Cons |
|--------|----------|------|------|
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
| Method | Use Case | Pros | Cons |
| --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
---
@ -219,6 +221,7 @@ git push origin main v0.3.0
```
**What happens:**
1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
```
**What happens:**
1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
```
**What happens:**
1. You create and push tag manually
2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features
### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind.
---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely
- Invalid version format in manifest.json
- manifest.json not in the commit that was pushed
@ -348,13 +359,14 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results:
```
feat(scope): Add new feature
Detailed description of what changed.
```
feat(scope): Add new feature
Impact: Users can now do X and Y.
```
Detailed description of what changed.
Impact: Users can now do X and Y.
```
2. **Impact Section:** Add `Impact:` in commit body for user-friendly descriptions

32
docs/developer/versioned_docs/version-v0.22.0/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:**
- **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available
**When cleared:**
- Tomorrow's data becomes available
- Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:**
```python
# In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:**
- Successful API call completes (no rate limit error)
- Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:**
```python
# In error handler
is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}`
**When triggered:**
- Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check)
**When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:**
```python
# After user data update
home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
```
**Translation placeholders:**
- `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration
**Coordinator Initialization:**
```python
self._repair_manager = TibberPricesRepairManager(
hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
```
**Update Cycle Integration:**
```python
# Success path - check conditions
if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
```
**Shutdown Cleanup:**
```python
async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up."""
@ -196,24 +212,27 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json`
**Structure:**
```json
{
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
}
}
}
}
```
## Home Assistant Integration
Repairs appear in:
- **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count)
Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:**
```python
# Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair
**Manual test:**
- Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file
**Example template:**
```python
async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair."""

16
docs/developer/versioned_docs/version-v0.22.0/setup.md
View file

@ -4,9 +4,9 @@
## Prerequisites
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
## Quick Setup
@ -26,11 +26,11 @@ code .
The DevContainer includes:
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
## Running the Integration

16
docs/developer/versioned_docs/version-v0.22.0/testing.md
View file

@ -13,10 +13,10 @@ Before running tests or committing changes, validate the integration structure:
This lightweight script checks:
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
**Note:** Full hassfest validation runs in GitHub Actions on push.
@ -42,10 +42,10 @@ pytest --cov=custom_components.tibber_prices tests/
Then test in Home Assistant UI:
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
## Test Guidelines

44
docs/developer/versioned_docs/version-v0.22.0/timer-architecture.md
View file

@ -10,11 +10,11 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------|
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
| Timer | Type | Interval | Purpose | Trigger Method |
| ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
**Key principle:** Timer #1 (HA) controls **data fetching**, Timer #2 controls **entity updates**, Timer #3 controls **timing displays**.
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
```
**Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:**
- Automatic restart after HA restart
- Built-in retry logic for temporary failures
- Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
```
**Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total
**Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
```
**Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total
**Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation)
**Why NOT every second:**
- Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
```
**Why this pattern:**
- Decouples timer logic from entity logic
- One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem
With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:**
- Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover)
- Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics
### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all
@ -393,16 +412,16 @@ _LOGGER.setLevel(logging.DEBUG)
### Common Issues
1. **Timer #2 not triggering:**
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
2. **Double updates at midnight:**
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
3. **API overload:**
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
---
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary
**Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:**
- Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:**
- Timer = mechanism that triggers
- Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies

131
docs/developer/versioned_docs/version-v0.22.1/api-reference.md
View file

@ -22,30 +22,30 @@ Fetches home information and metadata:
```graphql
query {
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
}
```
@ -56,26 +56,27 @@ query {
Fetches quarter-hourly prices:
```graphql
query($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
query ($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
}
}
}
}
}
}
}
}
}
}
```
**Parameters:**
- `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits
Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute
Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home
@ -99,13 +102,14 @@ Integration stays well below these limits:
```json
{
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
}
```
**Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -114,11 +118,12 @@ Integration stays well below these limits:
```json
{
"currency": "EUR"
"currency": "EUR"
}
```
Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses
**Invalid Token:**
```json
{
"errors": [{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}]
"errors": [
{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}
]
}
```
**Rate Limit Exceeded:**
```json
{
"errors": [{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}]
"errors": [
{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}
]
}
```
**Home Not Found:**
```json
{
"errors": [{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}]
"errors": [
{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}
]
}
```
Integration handles these with:
- Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation
Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
---
💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token)

143
docs/developer/versioned_docs/version-v0.22.1/architecture.md
View file

@ -100,43 +100,43 @@ flowchart TB
### Flow Description
1. **Setup** (`__init__.py`)
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
2. **Data Fetch** (every 15 minutes)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
3. **Price Enrichment**
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
4. **Period Calculation**
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
5. **Entity Updates**
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
6. **Service Calls**
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
---
@ -146,13 +146,13 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------|
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
| Layer | Location | Lifetime | Invalidation | Memory |
| ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
**Total cache overhead:** ~126KB per coordinator instance (main entry + subentries)
@ -195,30 +195,31 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components
| Component | File | Responsibility |
|-----------|------|----------------|
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
| Component | File | Responsibility |
| --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
### Sensor Architecture (Calculator Pattern)
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------|
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
| Component | Files | Lines | Responsibility |
| ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata
**Benefits:**
- 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator
@ -237,12 +239,12 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities
| Utility | File | Purpose |
|---------|------|---------|
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
| Utility | File | Purpose |
| ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
---
@ -283,12 +285,12 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
- **API polling**: Every 15 minutes (coordinator fetch cycle)
- **Entity updates**: On 00/15/30/45-minute boundaries via `coordinator/listeners.py`
- **Timer scheduling**: Uses `async_track_utc_time_change(minute=[0, 15, 30, 45], second=0)`
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- **Absolute time tracking**: Timer plans for **all future boundaries** (not relative delays)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- **Result**: Current price sensors update without waiting for next API poll
### 4. Calculator Pattern (Sensor Platform)
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:**
- Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing
@ -333,12 +340,12 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization
| Optimization | Location | Savings |
|--------------|----------|---------|
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
| Optimization | Location | Savings |
| ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
### Memory Usage

132
docs/developer/versioned_docs/version-v0.22.1/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation
**Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage
@ -36,29 +38,31 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator):
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
2. **Cache validation on load**:
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
3. **Tomorrow data check** (after 13:00):
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
**Why this cache matters:** Reduces API load on Tibber (~192 intervals per fetch), speeds up HA restarts, enables offline operation until cache expires.
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:**
- **Forever** (until HA restart)
- No invalidation during runtime
**When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once
**Access pattern:**
```python
# Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:**
### DataTransformer Config Cache
```python
{
"thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
### PeriodCalculator Config Cache
```python
{
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,20 +128,23 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Lifetime:**
- Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle
**Invalidation trigger:**
- **Options change** (user reconfigures integration):
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
**Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:**
```python
{
"best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Cache key:** Hash of relevant inputs
```python
hash_data = (
today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
```
**Lifetime:**
- Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data)
@ -179,24 +195,27 @@ hash_data = (
**Invalidation triggers:**
1. **Config change** (explicit):
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
2. **Price data change** (automatic via hash mismatch):
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
**Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:**
```python
{
"timestamp": ...,
@ -224,14 +244,16 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:**
- Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when:
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
**Architecture:**
- DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow
### User Changes Options (Config Flow)
```
User saves options
↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
```
### Midnight Turnover (Day Transition)
```
Timer #2 fires at 00:00
↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
```
### Tomorrow Data Arrives (~13:00)
```
Coordinator update cycle
↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
```
**No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load)
**Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics
### Typical Operation (No Changes)
```
Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
```
### After Midnight Turnover
```
Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
```
### After Config Change
```
Options Update
├─> Cache invalidation: `<`1ms
@ -381,23 +411,25 @@ Options Update
## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------|
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
| Cache Type | Lifetime | Size | Invalidation | Purpose |
| ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
**Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:**
- 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup)
**Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues
### Symptom: Stale data after config change
**Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating
**Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today
**Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations
**Check:**
1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys

11
docs/developer/versioned_docs/version-v0.22.1/coding-guidelines.md
View file

@ -8,10 +8,10 @@ comments: false
## Code Style
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
Run before committing:
@ -41,12 +41,14 @@ class TimeService:
```
**When prefix is required:**
- Public classes used across multiple modules
- All exception classes
- All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module

25
docs/developer/versioned_docs/version-v0.22.1/contributing.md
View file

@ -14,10 +14,10 @@ Welcome! This guide helps you contribute to the Tibber Prices integration.
1. Fork the repository on GitHub
2. Clone your fork:
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
3. Open in VS Code
4. Click "Reopen in Container" when prompted
@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
```
**Branch naming:**
- `feature/` - New features
- `fix/` - Bug fixes
- `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:**
```bash
./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
```
Run your test:
```bash
./scripts/test tests/test_your_feature.py -v
```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
```
**Commit types:**
- `feat:` - New feature
- `fix:` - Bug fix
- `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance
**Add scope when relevant:**
- `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max)
Description should include:
```markdown
## What
Brief description of changes
## Why
Problem being solved or feature rationale
## How
Implementation approach
## Testing
- [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated
- [ ] Type checking passes
- [ ] Linting passes
## Breaking Changes
(If any - describe migration path)
## Related Issues
Closes #123
```
### PR Checklist
Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For
✅ **Good:**
- Clear, self-explanatory code
- Appropriate comments for complex logic
- Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns
❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes
- Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On
Good first issues are labeled:
- `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
---
💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests

65
docs/developer/versioned_docs/version-v0.22.1/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅
**What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners
**Why critical:**
- Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅
**What is tested:**
- Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared
- Both timers are cancelled together
- Cleanup works even when timers are `None`
**Why critical:**
- Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅
**What is tested:**
- Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()`
**Why critical:**
- `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation
**What is tested:**
- DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update
**Why critical:**
- Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis
**Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal
**What is tested:**
- Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss)
**Why critical:**
- Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:**
- `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py`
**What is tested:**
- Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly
**Why critical:**
- Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py`
**What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked
**Why critical:**
- Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:**
- No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically
- Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created
- HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading
- Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors
**Structure:**
```python
coordinator.data = {
"user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory
**Current Status:** HA's response lifecycle
- HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code
@ -207,29 +234,30 @@ coordinator.data = {
### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------|
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
| Category | Status | Tests | File | Coverage |
| ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale |
|----------|--------|-----------|
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
| Category | Status | Rationale |
| ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:**
- ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners)

52
docs/developer/versioned_docs/version-v0.22.1/debugging.md
View file

@ -10,9 +10,9 @@ Add to `configuration.yaml`:
```yaml
logger:
default: info
logs:
custom_components.tibber_prices: debug
default: info
logs:
custom_components.tibber_prices: debug
```
Restart Home Assistant to apply.
@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages
**Coordinator Updates:**
```
[custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
```
**Period Calculation:**
```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
```
**API Errors:**
```
[custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -47,26 +50,27 @@ Restart Home Assistant to apply.
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
}
```
### Set Breakpoints
**Coordinator update:**
```python
# coordinator/core.py
async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
```
**Period calculation:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
```
**Flags:**
- `-v` - Verbose output
- `-s` - Show print statements
- `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns
**Print coordinator data:**
```python
def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
```
**Inspect period attributes:**
```python
def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading
**Check:**
```bash
grep "tibber_prices" config/home-assistant.log
```
**Common causes:**
- Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating
**Check coordinator state:**
```python
# In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }}
```
**Debug in code:**
```python
# Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong
**Enable detailed period logs:**
```python
# coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
```
**Check filter statistics:**
```
[period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy
Add to coordinator code:
```python
import debugpy
debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL
Install in container:
```bash
uv pip install ipython
```
Add breakpoint:
```python
from IPython import embed
embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
---
💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure

123
docs/developer/versioned_docs/version-v0.22.1/intro.md
View file

@ -8,25 +8,25 @@ This is an independent, community-maintained custom integration for Home Assista
## 📚 Developer Guides
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
## 🤖 AI Documentation
The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md). This file serves as long-term memory for AI assistants and contains:
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
**Important:** When proposing changes to patterns or conventions, always update [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) to keep AI guidance consistent.
@ -34,32 +34,32 @@ The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlow
This integration is developed with extensive AI assistance (GitHub Copilot, Claude, and other AI tools). The AI handles:
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
**Quality Assurance:**
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
**Benefits:**
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
**Limitations:**
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
If you're working with AI tools on this project, the [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) file provides the context and patterns that ensure consistency.
@ -80,15 +80,15 @@ If you're working with AI tools on this project, the [`AGENTS.md`](https://githu
The project includes several helper scripts in `./scripts/`:
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
## 📦 Project Structure
@ -121,23 +121,23 @@ custom_components/tibber_prices/
**DataUpdateCoordinator Pattern:**
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
**Price Data Enrichment:**
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
**Translation System:**
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
## 🧪 Testing
@ -159,18 +159,19 @@ pytest --cov=custom_components.tibber_prices tests/
Documentation is organized in two Docusaurus sites:
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:**
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
## 🤝 Contributing

17
docs/developer/versioned_docs/version-v0.22.1/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals
Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching
**1. Persistent Cache** (API data):
```python
# Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
```
**2. Translation Cache** (in-memory):
```python
# Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
```
**3. Config Cache** (invalidated on options change):
```python
class DataTransformer:
def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading
**Load data only when needed:**
```python
@property
def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations
**Process multiple items at once:**
```python
# ❌ Slow - loop with individual operations
for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices
**1. Concurrent API calls:**
```python
# ❌ Sequential (slow)
user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
```
**2. Don't block event loop:**
```python
# ❌ Blocking
result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks
**1. Clear references:**
```python
class Coordinator:
async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
```
**2. Use weak references for callbacks:**
```python
import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures
**Use appropriate types:**
```python
# ❌ List for lookups (O(n))
if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls
**Already implemented:**
- Cache valid until midnight
- User data cached for 24h
- Only poll when tomorrow data expected
**Monitor API usage:**
```python
_LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates
**Only update when needed:**
```python
async def _async_update_data(self) -> dict:
"""Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection
**Affects long-term statistics storage:**
```python
# ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size
**Keep attributes minimal:**
```python
# ❌ Large nested structures (KB per update)
attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
---
💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools

314
docs/developer/versioned_docs/version-v0.22.1/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:**
```python
# Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
```
**Example (Best Price):**
- Daily Min: 10 ct/kWh
- Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:**
```python
# Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
```
**Example (Best Price):**
- Daily Avg: 15 ct/kWh
- Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -65,17 +70,17 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
The integration maintains **two independent sets** of volatility thresholds:
1. **Sensor Thresholds** (user-configurable via `CONF_VOLATILITY_*_THRESHOLD`)
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
2. **Period Filter Thresholds** (internal, fixed)
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
**Rationale for Separation:**
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:**
```python
# Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,36 +113,42 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:**
- Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh
**Flex Filter (50%):**
```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
```
**Min Distance Filter (5%):**
```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
```
**Conflict:**
- Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
**The Issue:** At high Flex values, Min_Distance becomes the dominant filter and blocks intervals that Flex would permit. This defeats the purpose of having high Flex.
### Mathematical Analysis
**Conflict condition for Best Price:**
```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
```
**Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:**
```python
if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20
@ -158,15 +171,16 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------|
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
| ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:**
```python
# coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
```
**Why Linear Scaling?**
- Simple and predictable
- No abrupt behavior changes
- Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?**
- Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:**
```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
```
**Defined in `const.py`:**
```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances
**Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average"
**Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:**
- Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction
**Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy
- User can manually decide whether to react
**Defaults:**
- **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:**
- Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:**
Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times
**Example day:**
```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
```
**Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices
- Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:**
- Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:**
```
Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended)
**Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side"
**High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20%
#### Without Relaxation
**Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits)
---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach
**Each day processed independently:**
1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
```
**Constants:**
```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -427,26 +468,27 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Design Decisions:**
1. **Why 3% fixed increment?**
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
2. **Why hard-coded, not configurable?**
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
3. **Why warn at 25% base flex?**
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
**Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex:
```python
increment = base_flex × (step_pct / 100) # REMOVED
```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:**
```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy
**Per Flex level, try in order:**
1. Original Level filter
2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):**
```
Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions
**Period Calculation Entry Point:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
```
**Flex + Distance Filtering:**
```python
# coordinator/period_handlers/level_filtering.py
def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
```
**Relaxation Orchestration:**
```python
# coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -526,43 +574,45 @@ def relax_single_day(...) -> tuple[dict, dict]
**Algorithm Details:**
1. **Linear Regression Prediction:**
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
2. **Confidence Intervals:**
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
3. **Symmetry Check:**
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
4. **Enhanced Zigzag Detection:**
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:**
```python
# coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
```
**Data Integrity:**
- Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome
**Performance:**
- Single pass through price data
- O(n) complexity with small context window
- No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:**
```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -598,19 +651,19 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
**Why This Approach?**
1. **Linear regression over moving average:**
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
2. **Symmetry check over fixed threshold:**
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
3. **Single-pass over iterative:**
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
**Alternative Approaches Considered:**
@ -624,15 +677,17 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips
**Enable DEBUG logging:**
```yaml
# configuration.yaml
logger:
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
```
**Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,52 +700,61 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:**
```yaml
best_price_flex: 40
enable_relaxation_best: true
```
**Problem:**
- Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns
**Solution:**
```yaml
best_price_flex: 15 # Let relaxation increase it
best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true
```
### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:**
```yaml
best_price_min_distance_from_avg: 0
```
**Problem:**
- "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times
**Solution:**
```yaml
best_price_min_distance_from_avg: 5 # Use default 5%
best_price_min_distance_from_avg: 5 # Use default 5%
```
### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:**
```yaml
best_price_flex: 45
best_price_min_distance_from_avg: 10
```
**Problem:**
- Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal
**Solution:**
```yaml
best_price_flex: 20
best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh
**Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:**
```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -790,31 +862,31 @@ INFO: Period calculation completed: 1/2 days reached target
When debugging period calculation issues:
1. **Check Filter Statistics**
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
2. **Check Relaxation Behavior**
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
3. **Check Flex Warnings**
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
4. **Check Min_Distance Scaling**
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
5. **Check Outlier Filtering**
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
---
@ -823,19 +895,19 @@ When debugging period calculation issues:
### Potential Improvements
1. **Adaptive Flex Calculation:**
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
2. **Machine Learning Approach:**
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
3. **Multi-Objective Optimization:**
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
### Known Limitations
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:**
```python
# Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
```
**Benefits:**
- Eliminates need for relaxation on most days
- Self-adjusting to market conditions
- Better user experience (less configuration needed)
**Challenges:**
- Harder to predict behavior (less transparent)
- May conflict with user's mental model
- Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback
**Approach:**
- Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn per-user preferences over time
**Benefits:**
- Personalized to user's actual behavior
- Adapts to local market patterns
- Could discover non-obvious patterns
**Challenges:**
- Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously
**Goals:**
- Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:**
- Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing
**Benefits:**
- More sophisticated period selection
- Better match to user's actual needs
- Could handle complex appliance requirements
**Challenges:**
- Much more complex to implement
- Harder to explain to users
- Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex
**Example:**
- Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:**
- Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve
```python
# Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
```
**Why Not Implemented:**
- Linear is easier to reason about
- No evidence that non-linear is better
- Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day
**Example:**
- All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances)
**Possible Solution:**
- Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences
- Consider user's typical usage patterns
**Why Not Implemented:**
- Adds complexity
- Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:**
```python
# In period_building.py build_periods():
for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split:
```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,15 +1154,17 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting:
```yaml
condition:
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
```
Low volatility (< 15%) means classification changes are less economically significant.
@ -1077,24 +1172,25 @@ Low volatility (< 15%) means classification changes are less economically signif
**Alternative Approaches Rejected:**
1. **Use period start day for all intervals**
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
2. **Adjust flex/distance at midnight**
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
3. **Split at midnight always**
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
4. **Use next day's reference after midnight**
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

72
docs/developer/versioned_docs/version-v0.22.1/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
```
**Key Points:**
- Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely
- Don't provide analytical value in history
- Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures
**Attributes:**
- `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each)
- Slow down history queries
@ -66,20 +70,21 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:**
```json
{
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8,
// ... 10+ more attributes × 10-20 periods
}
]
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8
// ... 10+ more attributes × 10-20 periods
}
]
}
```
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:**
- Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value
- Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:**
- Configuration values that rarely change
- Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:**
- Only relevant at moment of reading
- Won't be valid after some time
- Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:**
- Detailed technical information not needed for historical analysis
- Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:**
- Can be calculated from other attributes
- Redundant information
- Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states
### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns
### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data
- `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings
**Per state change:**
- Before: ~3-8 KB average
- After: ~0.5-1.5 KB average
- **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact
For a typical installation with:
- 80+ sensors
- Updates every 15 minutes
- ~10 sensors updating every minute
@ -207,14 +224,14 @@ For a typical installation with:
## Implementation Files
- **Sensor Platform**: `custom_components/tibber_prices/sensor/core.py`
- Class: `TibberPricesSensor`
- 47 attributes excluded
- Class: `TibberPricesSensor`
- 47 attributes excluded
- **Binary Sensor Platform**: `custom_components/tibber_prices/binary_sensor/core.py`
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
## When to Update _unrecorded_attributes
## When to Update \_unrecorded_attributes
### Add to Exclusion List When:
@ -236,24 +253,24 @@ For a typical installation with:
When adding a new attribute, ask:
1. **Will this be useful in history queries 1 week from now?**
- No → Exclude
- Yes → Keep
- No → Exclude
- Yes → Keep
2. **Can this be calculated from other recorded attributes?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
3. **Is this primarily for current UI display?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
4. **Does this change frequently without indicating state change?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
5. **Is this larger than 100 bytes and not essential for analysis?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
## Testing
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:**
```sql
SELECT
state_id,

200
docs/developer/versioned_docs/version-v0.22.1/refactoring-guide.md
View file

@ -8,22 +8,22 @@ Not every code change needs a detailed plan. Create a refactoring plan when:
🔴 **Major changes requiring planning:**
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
🟡 **Medium changes that might benefit from planning:**
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
🟢 **Small changes - no planning needed:**
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
## The Planning Process
@ -51,34 +51,34 @@ Every planning document should include:
## Problem Statement
- What's the issue?
- Why does it need fixing?
- Current pain points
- What's the issue?
- Why does it need fixing?
- Current pain points
## Proposed Solution
- High-level approach
- File structure (before/after)
- Module responsibilities
- High-level approach
- File structure (before/after)
- Module responsibilities
## Migration Strategy
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
## Risks & Mitigation
- What could go wrong?
- How to prevent it?
- Rollback strategy
- What could go wrong?
- How to prevent it?
- Rollback strategy
## Success Criteria
- Measurable improvements
- Testing requirements
- Verification steps
- Measurable improvements
- Testing requirements
- Verification steps
```
See `planning/README.md` for detailed template explanation.
@ -87,19 +87,19 @@ See `planning/README.md` for detailed template explanation.
Since `planning/` is git-ignored:
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
### 4. Implementation Phase
Once plan is approved:
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
### 5. After Completion
@ -134,13 +134,13 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Before:**
- `sensor.py` - 2,574 lines, hard to navigate
- `sensor.py` - 2,574 lines, hard to navigate
**After:**
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
**Process:**
@ -153,10 +153,10 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Key learnings:**
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
**Note:** The complete module splitting plan was documented during implementation but has been superseded by the actual code structure.
@ -166,11 +166,11 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
Breaking refactorings into phases:
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
### Phase Structure
@ -191,8 +191,8 @@ Each phase should:
**File Lifecycle**:
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
**Steps**:
@ -205,10 +205,10 @@ Each phase should:
**Success criteria**:
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
```
## Testing Strategy
@ -238,13 +238,13 @@ Minimum testing checklist:
After completing all phases:
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
## Common Pitfalls
@ -286,21 +286,21 @@ This project uses AI heavily (GitHub Copilot, Claude). The planning process supp
**AI reads from:**
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
**AI updates:**
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
**Why separate AGENTS.md and docs/development/?**
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENTS.md) section "Planning Major Refactorings" for AI-specific guidance.
@ -308,16 +308,16 @@ See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/main/AGENT
### Planning Directory
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
### Example Plans
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
### Helper Scripts
@ -341,21 +341,21 @@ Simple rule: If you can't describe the entire change in 3 sentences, create a pl
Good plan level:
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
Too detailed:
- Exact code snippets for every change
- Line-by-line instructions
- Exact code snippets for every change
- Line-by-line instructions
Too vague:
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
### Q: What if the plan changes during implementation?
@ -363,9 +363,9 @@ Too vague:
If you discover:
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
Document WHY the plan changed (helps future refactorings).
@ -373,9 +373,9 @@ Document WHY the plan changed (helps future refactorings).
**A:** No! Use judgment:
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
### Q: How do I know when a refactoring is successful?
@ -383,12 +383,12 @@ Document WHY the plan changed (helps future refactorings).
Typical criteria:
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
If you can't tick all boxes, the refactoring isn't done.
@ -409,6 +409,6 @@ If you can't tick all boxes, the refactoring isn't done.
**Next steps:**
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans

36
docs/developer/versioned_docs/version-v0.22.1/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):**
**git-cliff** (template-based):
```bash
# See: https://git-cliff.org/docs/installation
@ -190,13 +192,13 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which
| Method | Use Case | Pros | Cons |
|--------|----------|------|------|
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
| Method | Use Case | Pros | Cons |
| --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
---
@ -219,6 +221,7 @@ git push origin main v0.3.0
```
**What happens:**
1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
```
**What happens:**
1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
```
**What happens:**
1. You create and push tag manually
2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features
### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind.
---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely
- Invalid version format in manifest.json
- manifest.json not in the commit that was pushed
@ -348,13 +359,14 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results:
```
feat(scope): Add new feature
Detailed description of what changed.
```
feat(scope): Add new feature
Impact: Users can now do X and Y.
```
Detailed description of what changed.
Impact: Users can now do X and Y.
```
2. **Impact Section:** Add `Impact:` in commit body for user-friendly descriptions

32
docs/developer/versioned_docs/version-v0.22.1/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:**
- **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available
**When cleared:**
- Tomorrow's data becomes available
- Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:**
```python
# In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:**
- Successful API call completes (no rate limit error)
- Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:**
```python
# In error handler
is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}`
**When triggered:**
- Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check)
**When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:**
```python
# After user data update
home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
```
**Translation placeholders:**
- `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration
**Coordinator Initialization:**
```python
self._repair_manager = TibberPricesRepairManager(
hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
```
**Update Cycle Integration:**
```python
# Success path - check conditions
if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
```
**Shutdown Cleanup:**
```python
async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up."""
@ -196,24 +212,27 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json`
**Structure:**
```json
{
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
}
}
}
}
```
## Home Assistant Integration
Repairs appear in:
- **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count)
Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:**
```python
# Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair
**Manual test:**
- Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file
**Example template:**
```python
async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair."""

16
docs/developer/versioned_docs/version-v0.22.1/setup.md
View file

@ -4,9 +4,9 @@
## Prerequisites
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
## Quick Setup
@ -26,11 +26,11 @@ code .
The DevContainer includes:
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
## Running the Integration

16
docs/developer/versioned_docs/version-v0.22.1/testing.md
View file

@ -13,10 +13,10 @@ Before running tests or committing changes, validate the integration structure:
This lightweight script checks:
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
**Note:** Full hassfest validation runs in GitHub Actions on push.
@ -42,10 +42,10 @@ pytest --cov=custom_components.tibber_prices tests/
Then test in Home Assistant UI:
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
## Test Guidelines

44
docs/developer/versioned_docs/version-v0.22.1/timer-architecture.md
View file

@ -10,11 +10,11 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------|
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
| Timer | Type | Interval | Purpose | Trigger Method |
| ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
**Key principle:** Timer #1 (HA) controls **data fetching**, Timer #2 controls **entity updates**, Timer #3 controls **timing displays**.
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
```
**Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:**
- Automatic restart after HA restart
- Built-in retry logic for temporary failures
- Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
```
**Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total
**Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
```
**Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total
**Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation)
**Why NOT every second:**
- Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
```
**Why this pattern:**
- Decouples timer logic from entity logic
- One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem
With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:**
- Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover)
- Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics
### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all
@ -393,16 +412,16 @@ _LOGGER.setLevel(logging.DEBUG)
### Common Issues
1. **Timer #2 not triggering:**
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
2. **Double updates at midnight:**
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
3. **API overload:**
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
---
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary
**Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:**
- Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:**
- Timer = mechanism that triggers
- Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies

131
docs/developer/versioned_docs/version-v0.23.0/api-reference.md
View file

@ -22,30 +22,30 @@ Fetches home information and metadata:
```graphql
query {
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
}
```
@ -56,26 +56,27 @@ query {
Fetches quarter-hourly prices:
```graphql
query($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
query ($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
}
}
}
}
}
}
}
}
}
}
```
**Parameters:**
- `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits
Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute
Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home
@ -99,13 +102,14 @@ Integration stays well below these limits:
```json
{
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
}
```
**Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -114,11 +118,12 @@ Integration stays well below these limits:
```json
{
"currency": "EUR"
"currency": "EUR"
}
```
Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses
**Invalid Token:**
```json
{
"errors": [{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}]
"errors": [
{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}
]
}
```
**Rate Limit Exceeded:**
```json
{
"errors": [{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}]
"errors": [
{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}
]
}
```
**Home Not Found:**
```json
{
"errors": [{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}]
"errors": [
{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}
]
}
```
Integration handles these with:
- Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation
Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
---
💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token)

143
docs/developer/versioned_docs/version-v0.23.0/architecture.md
View file

@ -100,43 +100,43 @@ flowchart TB
### Flow Description
1. **Setup** (`__init__.py`)
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
2. **Data Fetch** (every 15 minutes)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
3. **Price Enrichment**
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
4. **Period Calculation**
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
5. **Entity Updates**
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
6. **Service Calls**
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
---
@ -146,13 +146,13 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------|
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
| Layer | Location | Lifetime | Invalidation | Memory |
| ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
**Total cache overhead:** ~126KB per coordinator instance (main entry + subentries)
@ -195,30 +195,31 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components
| Component | File | Responsibility |
|-----------|------|----------------|
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
| Component | File | Responsibility |
| --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
### Sensor Architecture (Calculator Pattern)
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------|
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
| Component | Files | Lines | Responsibility |
| ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata
**Benefits:**
- 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator
@ -237,12 +239,12 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities
| Utility | File | Purpose |
|---------|------|---------|
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
| Utility | File | Purpose |
| ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
---
@ -283,12 +285,12 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
- **API polling**: Every 15 minutes (coordinator fetch cycle)
- **Entity updates**: On 00/15/30/45-minute boundaries via `coordinator/listeners.py`
- **Timer scheduling**: Uses `async_track_utc_time_change(minute=[0, 15, 30, 45], second=0)`
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- **Absolute time tracking**: Timer plans for **all future boundaries** (not relative delays)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- **Result**: Current price sensors update without waiting for next API poll
### 4. Calculator Pattern (Sensor Platform)
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:**
- Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing
@ -333,12 +340,12 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization
| Optimization | Location | Savings |
|--------------|----------|---------|
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
| Optimization | Location | Savings |
| ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
### Memory Usage

132
docs/developer/versioned_docs/version-v0.23.0/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation
**Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage
@ -36,29 +38,31 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator):
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
2. **Cache validation on load**:
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
3. **Tomorrow data check** (after 13:00):
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
**Why this cache matters:** Reduces API load on Tibber (~192 intervals per fetch), speeds up HA restarts, enables offline operation until cache expires.
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:**
- **Forever** (until HA restart)
- No invalidation during runtime
**When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once
**Access pattern:**
```python
# Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:**
### DataTransformer Config Cache
```python
{
"thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
### PeriodCalculator Config Cache
```python
{
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,20 +128,23 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Lifetime:**
- Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle
**Invalidation trigger:**
- **Options change** (user reconfigures integration):
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
**Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:**
```python
{
"best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Cache key:** Hash of relevant inputs
```python
hash_data = (
today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
```
**Lifetime:**
- Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data)
@ -179,24 +195,27 @@ hash_data = (
**Invalidation triggers:**
1. **Config change** (explicit):
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
2. **Price data change** (automatic via hash mismatch):
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
**Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:**
```python
{
"timestamp": ...,
@ -224,14 +244,16 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:**
- Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when:
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
**Architecture:**
- DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow
### User Changes Options (Config Flow)
```
User saves options
↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
```
### Midnight Turnover (Day Transition)
```
Timer #2 fires at 00:00
↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
```
### Tomorrow Data Arrives (~13:00)
```
Coordinator update cycle
↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
```
**No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load)
**Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics
### Typical Operation (No Changes)
```
Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
```
### After Midnight Turnover
```
Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
```
### After Config Change
```
Options Update
├─> Cache invalidation: `<`1ms
@ -381,23 +411,25 @@ Options Update
## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------|
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
| Cache Type | Lifetime | Size | Invalidation | Purpose |
| ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
**Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:**
- 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup)
**Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues
### Symptom: Stale data after config change
**Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating
**Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today
**Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations
**Check:**
1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys

11
docs/developer/versioned_docs/version-v0.23.0/coding-guidelines.md
View file

@ -8,10 +8,10 @@ comments: false
## Code Style
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
Run before committing:
@ -41,12 +41,14 @@ class TimeService:
```
**When prefix is required:**
- Public classes used across multiple modules
- All exception classes
- All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module

25
docs/developer/versioned_docs/version-v0.23.0/contributing.md
View file

@ -14,10 +14,10 @@ Welcome! This guide helps you contribute to the Tibber Prices integration.
1. Fork the repository on GitHub
2. Clone your fork:
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
3. Open in VS Code
4. Click "Reopen in Container" when prompted
@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
```
**Branch naming:**
- `feature/` - New features
- `fix/` - Bug fixes
- `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:**
```bash
./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
```
Run your test:
```bash
./scripts/test tests/test_your_feature.py -v
```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
```
**Commit types:**
- `feat:` - New feature
- `fix:` - Bug fix
- `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance
**Add scope when relevant:**
- `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max)
Description should include:
```markdown
## What
Brief description of changes
## Why
Problem being solved or feature rationale
## How
Implementation approach
## Testing
- [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated
- [ ] Type checking passes
- [ ] Linting passes
## Breaking Changes
(If any - describe migration path)
## Related Issues
Closes #123
```
### PR Checklist
Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For
✅ **Good:**
- Clear, self-explanatory code
- Appropriate comments for complex logic
- Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns
❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes
- Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On
Good first issues are labeled:
- `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
---
💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests

65
docs/developer/versioned_docs/version-v0.23.0/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅
**What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners
**Why critical:**
- Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅
**What is tested:**
- Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared
- Both timers are cancelled together
- Cleanup works even when timers are `None`
**Why critical:**
- Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅
**What is tested:**
- Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()`
**Why critical:**
- `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation
**What is tested:**
- DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update
**Why critical:**
- Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis
**Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal
**What is tested:**
- Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss)
**Why critical:**
- Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:**
- `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py`
**What is tested:**
- Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly
**Why critical:**
- Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py`
**What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked
**Why critical:**
- Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:**
- No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically
- Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created
- HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading
- Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors
**Structure:**
```python
coordinator.data = {
"user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory
**Current Status:** HA's response lifecycle
- HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code
@ -207,29 +234,30 @@ coordinator.data = {
### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------|
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
| Category | Status | Tests | File | Coverage |
| ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale |
|----------|--------|-----------|
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
| Category | Status | Rationale |
| ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:**
- ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners)

52
docs/developer/versioned_docs/version-v0.23.0/debugging.md
View file

@ -10,9 +10,9 @@ Add to `configuration.yaml`:
```yaml
logger:
default: info
logs:
custom_components.tibber_prices: debug
default: info
logs:
custom_components.tibber_prices: debug
```
Restart Home Assistant to apply.
@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages
**Coordinator Updates:**
```
[custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
```
**Period Calculation:**
```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
```
**API Errors:**
```
[custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -47,26 +50,27 @@ Restart Home Assistant to apply.
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
}
```
### Set Breakpoints
**Coordinator update:**
```python
# coordinator/core.py
async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
```
**Period calculation:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
```
**Flags:**
- `-v` - Verbose output
- `-s` - Show print statements
- `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns
**Print coordinator data:**
```python
def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
```
**Inspect period attributes:**
```python
def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading
**Check:**
```bash
grep "tibber_prices" config/home-assistant.log
```
**Common causes:**
- Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating
**Check coordinator state:**
```python
# In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }}
```
**Debug in code:**
```python
# Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong
**Enable detailed period logs:**
```python
# coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
```
**Check filter statistics:**
```
[period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy
Add to coordinator code:
```python
import debugpy
debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL
Install in container:
```bash
uv pip install ipython
```
Add breakpoint:
```python
from IPython import embed
embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
---
💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure

123
docs/developer/versioned_docs/version-v0.23.0/intro.md
View file

@ -8,25 +8,25 @@ This is an independent, community-maintained custom integration for Home Assista
## 📚 Developer Guides
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
## 🤖 AI Documentation
The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.0/AGENTS.md). This file serves as long-term memory for AI assistants and contains:
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
**Important:** When proposing changes to patterns or conventions, always update [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.0/AGENTS.md) to keep AI guidance consistent.
@ -34,32 +34,32 @@ The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlow
This integration is developed with extensive AI assistance (GitHub Copilot, Claude, and other AI tools). The AI handles:
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
**Quality Assurance:**
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
**Benefits:**
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
**Limitations:**
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
If you're working with AI tools on this project, the [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.0/AGENTS.md) file provides the context and patterns that ensure consistency.
@ -80,15 +80,15 @@ If you're working with AI tools on this project, the [`AGENTS.md`](https://githu
The project includes several helper scripts in `./scripts/`:
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
## 📦 Project Structure
@ -121,23 +121,23 @@ custom_components/tibber_prices/
**DataUpdateCoordinator Pattern:**
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
**Price Data Enrichment:**
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
**Translation System:**
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
## 🧪 Testing
@ -159,18 +159,19 @@ pytest --cov=custom_components.tibber_prices tests/
Documentation is organized in two Docusaurus sites:
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:**
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
## 🤝 Contributing

17
docs/developer/versioned_docs/version-v0.23.0/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals
Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching
**1. Persistent Cache** (API data):
```python
# Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
```
**2. Translation Cache** (in-memory):
```python
# Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
```
**3. Config Cache** (invalidated on options change):
```python
class DataTransformer:
def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading
**Load data only when needed:**
```python
@property
def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations
**Process multiple items at once:**
```python
# ❌ Slow - loop with individual operations
for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices
**1. Concurrent API calls:**
```python
# ❌ Sequential (slow)
user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
```
**2. Don't block event loop:**
```python
# ❌ Blocking
result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks
**1. Clear references:**
```python
class Coordinator:
async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
```
**2. Use weak references for callbacks:**
```python
import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures
**Use appropriate types:**
```python
# ❌ List for lookups (O(n))
if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls
**Already implemented:**
- Cache valid until midnight
- User data cached for 24h
- Only poll when tomorrow data expected
**Monitor API usage:**
```python
_LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates
**Only update when needed:**
```python
async def _async_update_data(self) -> dict:
"""Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection
**Affects long-term statistics storage:**
```python
# ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size
**Keep attributes minimal:**
```python
# ❌ Large nested structures (KB per update)
attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
---
💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools

314
docs/developer/versioned_docs/version-v0.23.0/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:**
```python
# Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
```
**Example (Best Price):**
- Daily Min: 10 ct/kWh
- Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:**
```python
# Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
```
**Example (Best Price):**
- Daily Avg: 15 ct/kWh
- Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -65,17 +70,17 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
The integration maintains **two independent sets** of volatility thresholds:
1. **Sensor Thresholds** (user-configurable via `CONF_VOLATILITY_*_THRESHOLD`)
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
2. **Period Filter Thresholds** (internal, fixed)
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
**Rationale for Separation:**
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:**
```python
# Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,36 +113,42 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:**
- Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh
**Flex Filter (50%):**
```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
```
**Min Distance Filter (5%):**
```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
```
**Conflict:**
- Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
**The Issue:** At high Flex values, Min_Distance becomes the dominant filter and blocks intervals that Flex would permit. This defeats the purpose of having high Flex.
### Mathematical Analysis
**Conflict condition for Best Price:**
```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
```
**Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:**
```python
if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20
@ -158,15 +171,16 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------|
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
| ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:**
```python
# coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
```
**Why Linear Scaling?**
- Simple and predictable
- No abrupt behavior changes
- Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?**
- Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:**
```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
```
**Defined in `const.py`:**
```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances
**Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average"
**Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:**
- Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction
**Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy
- User can manually decide whether to react
**Defaults:**
- **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:**
- Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:**
Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times
**Example day:**
```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
```
**Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices
- Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:**
- Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:**
```
Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended)
**Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side"
**High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20%
#### Without Relaxation
**Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits)
---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach
**Each day processed independently:**
1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
```
**Constants:**
```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -427,26 +468,27 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Design Decisions:**
1. **Why 3% fixed increment?**
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
2. **Why hard-coded, not configurable?**
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
3. **Why warn at 25% base flex?**
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
**Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex:
```python
increment = base_flex × (step_pct / 100) # REMOVED
```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:**
```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy
**Per Flex level, try in order:**
1. Original Level filter
2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):**
```
Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions
**Period Calculation Entry Point:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
```
**Flex + Distance Filtering:**
```python
# coordinator/period_handlers/level_filtering.py
def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
```
**Relaxation Orchestration:**
```python
# coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -526,43 +574,45 @@ def relax_single_day(...) -> tuple[dict, dict]
**Algorithm Details:**
1. **Linear Regression Prediction:**
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
2. **Confidence Intervals:**
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
3. **Symmetry Check:**
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
4. **Enhanced Zigzag Detection:**
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:**
```python
# coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
```
**Data Integrity:**
- Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome
**Performance:**
- Single pass through price data
- O(n) complexity with small context window
- No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:**
```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -598,19 +651,19 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
**Why This Approach?**
1. **Linear regression over moving average:**
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
2. **Symmetry check over fixed threshold:**
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
3. **Single-pass over iterative:**
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
**Alternative Approaches Considered:**
@ -624,15 +677,17 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips
**Enable DEBUG logging:**
```yaml
# configuration.yaml
logger:
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
```
**Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,52 +700,61 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:**
```yaml
best_price_flex: 40
enable_relaxation_best: true
```
**Problem:**
- Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns
**Solution:**
```yaml
best_price_flex: 15 # Let relaxation increase it
best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true
```
### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:**
```yaml
best_price_min_distance_from_avg: 0
```
**Problem:**
- "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times
**Solution:**
```yaml
best_price_min_distance_from_avg: 5 # Use default 5%
best_price_min_distance_from_avg: 5 # Use default 5%
```
### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:**
```yaml
best_price_flex: 45
best_price_min_distance_from_avg: 10
```
**Problem:**
- Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal
**Solution:**
```yaml
best_price_flex: 20
best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh
**Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:**
```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -790,31 +862,31 @@ INFO: Period calculation completed: 1/2 days reached target
When debugging period calculation issues:
1. **Check Filter Statistics**
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
2. **Check Relaxation Behavior**
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
3. **Check Flex Warnings**
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
4. **Check Min_Distance Scaling**
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
5. **Check Outlier Filtering**
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
---
@ -823,19 +895,19 @@ When debugging period calculation issues:
### Potential Improvements
1. **Adaptive Flex Calculation:**
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
2. **Machine Learning Approach:**
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
3. **Multi-Objective Optimization:**
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
### Known Limitations
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:**
```python
# Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
```
**Benefits:**
- Eliminates need for relaxation on most days
- Self-adjusting to market conditions
- Better user experience (less configuration needed)
**Challenges:**
- Harder to predict behavior (less transparent)
- May conflict with user's mental model
- Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback
**Approach:**
- Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn per-user preferences over time
**Benefits:**
- Personalized to user's actual behavior
- Adapts to local market patterns
- Could discover non-obvious patterns
**Challenges:**
- Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously
**Goals:**
- Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:**
- Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing
**Benefits:**
- More sophisticated period selection
- Better match to user's actual needs
- Could handle complex appliance requirements
**Challenges:**
- Much more complex to implement
- Harder to explain to users
- Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex
**Example:**
- Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:**
- Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve
```python
# Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
```
**Why Not Implemented:**
- Linear is easier to reason about
- No evidence that non-linear is better
- Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day
**Example:**
- All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances)
**Possible Solution:**
- Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences
- Consider user's typical usage patterns
**Why Not Implemented:**
- Adds complexity
- Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:**
```python
# In period_building.py build_periods():
for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split:
```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,15 +1154,17 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting:
```yaml
condition:
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
```
Low volatility (< 15%) means classification changes are less economically significant.
@ -1077,24 +1172,25 @@ Low volatility (< 15%) means classification changes are less economically signif
**Alternative Approaches Rejected:**
1. **Use period start day for all intervals**
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
2. **Adjust flex/distance at midnight**
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
3. **Split at midnight always**
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
4. **Use next day's reference after midnight**
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

72
docs/developer/versioned_docs/version-v0.23.0/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
```
**Key Points:**
- Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely
- Don't provide analytical value in history
- Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures
**Attributes:**
- `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each)
- Slow down history queries
@ -66,20 +70,21 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:**
```json
{
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8,
// ... 10+ more attributes × 10-20 periods
}
]
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8
// ... 10+ more attributes × 10-20 periods
}
]
}
```
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:**
- Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value
- Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:**
- Configuration values that rarely change
- Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:**
- Only relevant at moment of reading
- Won't be valid after some time
- Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:**
- Detailed technical information not needed for historical analysis
- Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:**
- Can be calculated from other attributes
- Redundant information
- Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states
### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns
### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data
- `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings
**Per state change:**
- Before: ~3-8 KB average
- After: ~0.5-1.5 KB average
- **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact
For a typical installation with:
- 80+ sensors
- Updates every 15 minutes
- ~10 sensors updating every minute
@ -207,14 +224,14 @@ For a typical installation with:
## Implementation Files
- **Sensor Platform**: `custom_components/tibber_prices/sensor/core.py`
- Class: `TibberPricesSensor`
- 47 attributes excluded
- Class: `TibberPricesSensor`
- 47 attributes excluded
- **Binary Sensor Platform**: `custom_components/tibber_prices/binary_sensor/core.py`
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
## When to Update _unrecorded_attributes
## When to Update \_unrecorded_attributes
### Add to Exclusion List When:
@ -236,24 +253,24 @@ For a typical installation with:
When adding a new attribute, ask:
1. **Will this be useful in history queries 1 week from now?**
- No → Exclude
- Yes → Keep
- No → Exclude
- Yes → Keep
2. **Can this be calculated from other recorded attributes?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
3. **Is this primarily for current UI display?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
4. **Does this change frequently without indicating state change?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
5. **Is this larger than 100 bytes and not essential for analysis?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
## Testing
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:**
```sql
SELECT
state_id,

200
docs/developer/versioned_docs/version-v0.23.0/refactoring-guide.md
View file

@ -8,22 +8,22 @@ Not every code change needs a detailed plan. Create a refactoring plan when:
🔴 **Major changes requiring planning:**
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
🟡 **Medium changes that might benefit from planning:**
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
🟢 **Small changes - no planning needed:**
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
## The Planning Process
@ -51,34 +51,34 @@ Every planning document should include:
## Problem Statement
- What's the issue?
- Why does it need fixing?
- Current pain points
- What's the issue?
- Why does it need fixing?
- Current pain points
## Proposed Solution
- High-level approach
- File structure (before/after)
- Module responsibilities
- High-level approach
- File structure (before/after)
- Module responsibilities
## Migration Strategy
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
## Risks & Mitigation
- What could go wrong?
- How to prevent it?
- Rollback strategy
- What could go wrong?
- How to prevent it?
- Rollback strategy
## Success Criteria
- Measurable improvements
- Testing requirements
- Verification steps
- Measurable improvements
- Testing requirements
- Verification steps
```
See `planning/README.md` for detailed template explanation.
@ -87,19 +87,19 @@ See `planning/README.md` for detailed template explanation.
Since `planning/` is git-ignored:
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
### 4. Implementation Phase
Once plan is approved:
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
### 5. After Completion
@ -134,13 +134,13 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Before:**
- `sensor.py` - 2,574 lines, hard to navigate
- `sensor.py` - 2,574 lines, hard to navigate
**After:**
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
**Process:**
@ -153,10 +153,10 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Key learnings:**
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
**Note:** The complete module splitting plan was documented during implementation but has been superseded by the actual code structure.
@ -166,11 +166,11 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
Breaking refactorings into phases:
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
### Phase Structure
@ -191,8 +191,8 @@ Each phase should:
**File Lifecycle**:
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
**Steps**:
@ -205,10 +205,10 @@ Each phase should:
**Success criteria**:
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
```
## Testing Strategy
@ -238,13 +238,13 @@ Minimum testing checklist:
After completing all phases:
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
## Common Pitfalls
@ -286,21 +286,21 @@ This project uses AI heavily (GitHub Copilot, Claude). The planning process supp
**AI reads from:**
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
**AI updates:**
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
**Why separate AGENTS.md and docs/development/?**
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.0/AGENTS.md) section "Planning Major Refactorings" for AI-specific guidance.
@ -308,16 +308,16 @@ See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.0/AG
### Planning Directory
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
### Example Plans
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
### Helper Scripts
@ -341,21 +341,21 @@ Simple rule: If you can't describe the entire change in 3 sentences, create a pl
Good plan level:
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
Too detailed:
- Exact code snippets for every change
- Line-by-line instructions
- Exact code snippets for every change
- Line-by-line instructions
Too vague:
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
### Q: What if the plan changes during implementation?
@ -363,9 +363,9 @@ Too vague:
If you discover:
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
Document WHY the plan changed (helps future refactorings).
@ -373,9 +373,9 @@ Document WHY the plan changed (helps future refactorings).
**A:** No! Use judgment:
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
### Q: How do I know when a refactoring is successful?
@ -383,12 +383,12 @@ Document WHY the plan changed (helps future refactorings).
Typical criteria:
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
If you can't tick all boxes, the refactoring isn't done.
@ -409,6 +409,6 @@ If you can't tick all boxes, the refactoring isn't done.
**Next steps:**
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans

36
docs/developer/versioned_docs/version-v0.23.0/release-management.md
View file

@ -112,6 +112,7 @@ In CI/CD (`$CI` or `$GITHUB_ACTIONS`), AI is automatically disabled.
**In DevContainer (automatic):**
git-cliff is automatically installed when the DevContainer is built:
- **Rust toolchain**: Installed via `ghcr.io/devcontainers/features/rust:1` (minimal profile)
- **git-cliff**: Installed via cargo in `scripts/setup/setup`
@ -120,6 +121,7 @@ Simply rebuild the container (VS Code: "Dev Containers: Rebuild Container") and
**Manual installation (outside DevContainer):**
**git-cliff** (template-based):
```bash
# See: https://git-cliff.org/docs/installation
@ -190,13 +192,13 @@ All methods produce GitHub-flavored Markdown with emoji categories:
## 🎯 When to Use Which
| Method | Use Case | Pros | Cons |
|--------|----------|------|------|
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
| Method | Use Case | Pros | Cons |
| --------------------- | --------------------- | ----------------------------- | ------------------------ |
| **Helper Script** | Normal releases | Foolproof, automatic | Requires script |
| **Auto-Tag Workflow** | Forgot script | Safety net, automatic tagging | Still need manifest bump |
| **GitHub Button** | Manual quick release | Easy, no script | Limited categorization |
| **Local Script** | Testing release notes | Preview before release | Manual process |
| **CI/CD** | After tag push | Fully automatic | Needs tag first |
---
@ -219,6 +221,7 @@ git push origin main v0.3.0
```
**What happens:**
1. Script bumps manifest.json → commits → creates tag locally
2. You push commit + tag together
3. Release workflow sees tag → generates notes → creates release
@ -242,6 +245,7 @@ git push
```
**What happens:**
1. You push manifest.json change
2. Auto-Tag workflow detects change → creates tag automatically
3. Release workflow sees new tag → creates release
@ -263,6 +267,7 @@ git push origin main v0.3.0
```
**What happens:**
1. You create and push tag manually
2. Release workflow creates release
3. Auto-Tag workflow skips (tag already exists)
@ -282,19 +287,24 @@ git push origin main v0.3.0
## 🛡️ Safety Features
### 1. **Version Validation**
Both helper script and auto-tag workflow validate version format (X.Y.Z).
### 2. **No Duplicate Tags**
- Helper script checks if tag exists (local + remote)
- Auto-tag workflow checks if tag exists before creating
### 3. **Atomic Operations**
Helper script creates commit + tag locally. You decide when to push.
### 4. **Version Bumps Filtered**
Release notes automatically exclude `chore(release): bump version` commits.
### 5. **Rollback Instructions**
Helper script shows how to undo if you change your mind.
---
@ -330,6 +340,7 @@ git push -f origin main v0.3.0
**Auto-tag didn't create tag:**
Check workflow runs in GitHub Actions. Common causes:
- Tag already exists remotely
- Invalid version format in manifest.json
- manifest.json not in the commit that was pushed
@ -348,13 +359,14 @@ Check workflow runs in GitHub Actions. Common causes:
## 💡 Tips
1. **Conventional Commits:** Use proper commit format for best results:
```
feat(scope): Add new feature
Detailed description of what changed.
```
feat(scope): Add new feature
Impact: Users can now do X and Y.
```
Detailed description of what changed.
Impact: Users can now do X and Y.
```
2. **Impact Section:** Add `Impact:` in commit body for user-friendly descriptions

32
docs/developer/versioned_docs/version-v0.23.0/repairs-system.md
View file

@ -7,6 +7,7 @@ The Tibber Prices integration includes a proactive repair notification system th
The repairs system is implemented in `coordinator/repairs.py` via the `TibberPricesRepairManager` class, which is instantiated in the coordinator and integrated into the update cycle.
**Design Principles:**
- **Proactive**: Detect issues before they become critical
- **User-friendly**: Clear explanations with actionable guidance
- **Auto-clearing**: Repairs automatically disappear when conditions resolve
@ -19,10 +20,12 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
**Issue ID:** `tomorrow_data_missing_{entry_id}`
**When triggered:**
- Current time is after 18:00 (configurable via `TOMORROW_DATA_WARNING_HOUR`)
- Tomorrow's electricity price data is still not available
**When cleared:**
- Tomorrow's data becomes available
- Automatically checks on every successful API update
@ -30,6 +33,7 @@ The repairs system is implemented in `coordinator/repairs.py` via the `TibberPri
Users cannot plan ahead for tomorrow's electricity usage optimization. Automations relying on tomorrow's prices will not work.
**Implementation:**
```python
# In coordinator update cycle
has_tomorrow_data = self._data_fetcher.has_tomorrow_data(result["priceInfo"])
@ -40,6 +44,7 @@ await self._repair_manager.check_tomorrow_data_availability(
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `warning_hour`: Hour after which warning appears (default: 18)
@ -48,10 +53,12 @@ await self._repair_manager.check_tomorrow_data_availability(
**Issue ID:** `rate_limit_exceeded_{entry_id}`
**When triggered:**
- Integration encounters 3 or more consecutive rate limit errors (HTTP 429)
- Threshold configurable via `RATE_LIMIT_WARNING_THRESHOLD`
**When cleared:**
- Successful API call completes (no rate limit error)
- Error counter resets to 0
@ -59,6 +66,7 @@ await self._repair_manager.check_tomorrow_data_availability(
API requests are being throttled, causing stale data. Updates may be delayed until rate limit expires.
**Implementation:**
```python
# In error handler
is_rate_limit = (
@ -74,6 +82,7 @@ await self._repair_manager.clear_rate_limit_tracking()
```
**Translation placeholders:**
- `home_name`: Name of the affected home
- `error_count`: Number of consecutive rate limit errors
@ -82,10 +91,12 @@ await self._repair_manager.clear_rate_limit_tracking()
**Issue ID:** `home_not_found_{entry_id}`
**When triggered:**
- Home configured in this integration is no longer present in Tibber account
- Detected during user data refresh (daily check)
**When cleared:**
- Home reappears in Tibber account (unlikely - manual cleanup expected)
- Integration entry is removed (shutdown cleanup)
@ -93,6 +104,7 @@ await self._repair_manager.clear_rate_limit_tracking()
Integration cannot fetch data for a non-existent home. User must remove the config entry and re-add if needed.
**Implementation:**
```python
# After user data update
home_exists = self._data_fetcher._check_home_exists(home_id)
@ -103,6 +115,7 @@ else:
```
**Translation placeholders:**
- `home_name`: Name of the missing home
- `entry_id`: Config entry ID for reference
@ -153,6 +166,7 @@ Each repair type maintains internal state to avoid redundant operations:
### Lifecycle Integration
**Coordinator Initialization:**
```python
self._repair_manager = TibberPricesRepairManager(
hass=hass,
@ -162,6 +176,7 @@ self._repair_manager = TibberPricesRepairManager(
```
**Update Cycle Integration:**
```python
# Success path - check conditions
if result and "priceInfo" in result:
@ -178,6 +193,7 @@ if is_rate_limit:
```
**Shutdown Cleanup:**
```python
async def async_shutdown(self) -> None:
"""Shut down coordinator and clean up."""
@ -196,24 +212,27 @@ Repairs use Home Assistant's standard translation system. Translations are defin
- `/translations/sv.json`
**Structure:**
```json
{
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
"issues": {
"tomorrow_data_missing": {
"title": "Tomorrow's price data missing for {home_name}",
"description": "Detailed explanation with multiple paragraphs...\n\nPossible causes:\n- Cause 1\n- Cause 2"
}
}
}
}
```
## Home Assistant Integration
Repairs appear in:
- **Settings → System → Repairs** (main repairs panel)
- **Notifications** (bell icon in UI shows repair count)
Repair properties:
- **`is_fixable=False`**: No automated fix available (user action required)
- **`severity=IssueSeverity.WARNING`**: Yellow warning level (not critical)
- **`translation_key`**: References `issues.{key}` in translation files
@ -228,6 +247,7 @@ Repair properties:
4. When tomorrow data arrives (next API fetch), repair clears
**Manual trigger:**
```python
# Temporarily set warning hour to current hour for testing
TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
@ -240,6 +260,7 @@ TOMORROW_DATA_WARNING_HOUR = datetime.now().hour
3. Successful API call clears the repair
**Manual test:**
- Reduce API polling interval to trigger rate limiting
- Or temporarily return HTTP 429 in API client
@ -263,6 +284,7 @@ To add a new repair type:
7. **Document** in this file
**Example template:**
```python
async def check_new_condition(self, *, param: bool) -> None:
"""Check new condition and create/clear repair."""

16
docs/developer/versioned_docs/version-v0.23.0/setup.md
View file

@ -4,9 +4,9 @@
## Prerequisites
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
- VS Code with Dev Container support
- Docker installed and running
- GitHub account (for Tibber API token)
## Quick Setup
@ -26,11 +26,11 @@ code .
The DevContainer includes:
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
- Python 3.13 with `.venv` at `/home/vscode/.venv/`
- `uv` package manager (fast, modern Python tooling)
- Home Assistant development dependencies
- Ruff linter/formatter
- Git, GitHub CLI, Node.js, Rust toolchain
## Running the Integration

16
docs/developer/versioned_docs/version-v0.23.0/testing.md
View file

@ -13,10 +13,10 @@ Before running tests or committing changes, validate the integration structure:
This lightweight script checks:
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
- ✓ `config_flow.py` exists
- ✓ `manifest.json` is valid JSON with required fields
- ✓ Translation files have valid JSON syntax
- ✓ All Python files compile without syntax errors
**Note:** Full hassfest validation runs in GitHub Actions on push.
@ -42,10 +42,10 @@ pytest --cov=custom_components.tibber_prices tests/
Then test in Home Assistant UI:
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
- Configuration flow
- Sensor states and attributes
- Services
- Translation strings
## Test Guidelines

44
docs/developer/versioned_docs/version-v0.23.0/timer-architecture.md
View file

@ -10,11 +10,11 @@ This document explains the timer/scheduler system in the Tibber Prices integrati
The integration uses **three independent timer mechanisms** for different purposes:
| Timer | Type | Interval | Purpose | Trigger Method |
|-------|------|----------|---------|----------------|
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
| Timer | Type | Interval | Purpose | Trigger Method |
| ------------ | ----------- | ------------------ | -------------------- | ------------------------------- |
| **Timer #1** | HA built-in | 15 minutes | API data updates | `DataUpdateCoordinator` |
| **Timer #2** | Custom | :00, :15, :30, :45 | Entity state refresh | `async_track_utc_time_change()` |
| **Timer #3** | Custom | Every minute | Countdown/progress | `async_track_utc_time_change()` |
**Key principle:** Timer #1 (HA) controls **data fetching**, Timer #2 controls **entity updates**, Timer #3 controls **timing displays**.
@ -27,6 +27,7 @@ The integration uses **three independent timer mechanisms** for different purpos
**Type:** Home Assistant's built-in `DataUpdateCoordinator` with `UPDATE_INTERVAL = 15 minutes`
**What it is:**
- HA provides this timer system automatically when you inherit from `DataUpdateCoordinator`
- Triggers `_async_update_data()` method every 15 minutes
- **Not** synchronized to clock boundaries (each installation has different start time)
@ -53,16 +54,19 @@ async def _async_update_data(self) -> TibberPricesData:
```
**Load Distribution:**
- Each HA installation starts Timer #1 at different times → natural distribution
- Tomorrow data check adds 0-30s random delay → prevents "thundering herd" on Tibber API
- Result: API load spread over ~30 minutes instead of all at once
**Midnight Coordination:**
- Atomic check: `_check_midnight_turnover_needed(now)` compares dates only (no side effects)
- If midnight turnover needed → performs it and returns early
- Timer #2 will see turnover already done and skip gracefully
**Why we use HA's timer:**
- Automatic restart after HA restart
- Built-in retry logic for temporary failures
- Standard HA integration pattern
@ -79,6 +83,7 @@ async def _async_update_data(self) -> TibberPricesData:
**Purpose:** Update time-sensitive entity states at interval boundaries **without waiting for API poll**
**Problem it solves:**
- Timer #1 runs every 15 minutes but NOT synchronized to clock (:03, :18, :33, :48)
- Current price changes at :00, :15, :30, :45 → entities would show stale data for up to 15 minutes
- Example: 14:00 new price, but Timer #1 ran at 13:58 → next update at 14:13 → users see old price until 14:13
@ -100,22 +105,26 @@ async def _handle_quarter_hour_refresh(self, now: datetime) -> None:
```
**Smart Boundary Tolerance:**
- Uses `round_to_nearest_quarter_hour()` with ±2 second tolerance
- HA may schedule timer at 14:59:58 → rounds to 15:00:00 (shows new interval)
- HA restart at 14:59:30 → stays at 14:45:00 (shows current interval)
- See [Architecture](./architecture.md#3-quarter-hour-precision) for details
**Absolute Time Scheduling:**
- `async_track_utc_time_change()` plans for **all future boundaries** (15:00, 15:15, 15:30, ...)
- NOT relative delays ("in 15 minutes")
- If triggered at 14:59:58 → next trigger is 15:15:00, NOT 15:00:00 (prevents double updates)
**Which entities listen:**
- All sensors that depend on "current interval" (e.g., `current_interval_price`, `next_interval_price`)
- Binary sensors that check "is now in period?" (e.g., `best_price_period_active`)
- ~50-60 entities out of 120+ total
**Why custom timer:**
- HA's built-in coordinator doesn't support exact boundary timing
- We need **absolute time** triggers, not periodic intervals
- Allows fast entity updates without expensive data transformation
@ -140,6 +149,7 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
```
**Which entities listen:**
- `best_price_remaining_minutes` - Countdown timer
- `peak_price_remaining_minutes` - Countdown timer
- `best_price_progress` - Progress bar (0-100%)
@ -147,11 +157,13 @@ async def _handle_minute_refresh(self, now: datetime) -> None:
- ~10 entities total
**Why custom timer:**
- Users want smooth countdowns (not jumping 15 minutes at a time)
- Progress bars need minute-by-minute updates
- Very lightweight (no data processing, just state recalculation)
**Why NOT every second:**
- Minute precision sufficient for countdown UX
- Reduces CPU load (60× fewer updates than seconds)
- Home Assistant best practice (avoid sub-minute updates)
@ -194,6 +206,7 @@ class ListenerManager:
```
**Why this pattern:**
- Decouples timer logic from entity logic
- One timer can notify many entities efficiently
- Entities can unregister when removed (cleanup)
@ -279,11 +292,13 @@ class ListenerManager:
### Reason 1: Load Distribution on Tibber API
If all installations used synchronized timers:
- ❌ Everyone fetches at 13:00:00 → Tibber API overload
- ❌ Everyone fetches at 14:00:00 → Tibber API overload
- ❌ "Thundering herd" problem
With HA's unsynchronized timer:
- ✅ Installation A: 13:03:12, 13:18:12, 13:33:12, ...
- ✅ Installation B: 13:07:45, 13:22:45, 13:37:45, ...
- ✅ Installation C: 13:11:28, 13:26:28, 13:41:28, ...
@ -316,6 +331,7 @@ def _should_update_price_data(self) -> str:
**Most Timer #1 cycles:** Fast path (~2ms), no API call, just returns cached data.
**API fetch only when:**
- Tomorrow data missing/invalid (after 13:00)
- Cache expired (midnight turnover)
- Explicit user refresh
@ -339,6 +355,7 @@ def _should_update_price_data(self) -> str:
## Performance Characteristics
### Timer #1 (DataUpdateCoordinator)
- **Triggers:** Every 15 minutes (unsynchronized)
- **Fast path:** ~2ms (cache check, return existing data)
- **Slow path:** ~600ms (API fetch + transform + calculate)
@ -346,12 +363,14 @@ def _should_update_price_data(self) -> str:
- **API calls:** ~1-2 times/day (cached otherwise)
### Timer #2 (Quarter-Hour Refresh)
- **Triggers:** 96 times/day (exact boundaries)
- **Processing:** ~5ms (notify 60 entities)
- **No API calls:** Uses cached/transformed data
- **No transformation:** Just entity state updates
### Timer #3 (Minute Refresh)
- **Triggers:** 1440 times/day (every minute)
- **Processing:** ~1ms (notify 10 entities)
- **No API calls:** No data processing at all
@ -393,16 +412,16 @@ _LOGGER.setLevel(logging.DEBUG)
### Common Issues
1. **Timer #2 not triggering:**
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
- Check: `schedule_quarter_hour_refresh()` called in `__init__`?
- Check: `_quarter_hour_timer_cancel` properly stored?
2. **Double updates at midnight:**
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
- Should NOT happen (atomic coordination)
- Check: Both timers use same date comparison logic?
3. **API overload:**
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
- Check: Random delay working? (0-30s jitter on tomorrow check)
- Check: Cache validation logic correct?
---
@ -417,17 +436,20 @@ _LOGGER.setLevel(logging.DEBUG)
## Summary
**Three independent timers:**
1. **Timer #1** (HA built-in, 15 min, unsynchronized) → Data fetching (when needed)
2. **Timer #2** (Custom, :00/:15/:30/:45) → Entity state updates (always)
3. **Timer #3** (Custom, every minute) → Countdown/progress (always)
**Key insights:**
- Timer #1 unsynchronized = good (load distribution on API)
- Timer #2 synchronized = good (user sees correct data immediately)
- Timer #3 synchronized = good (smooth countdown UX)
- All three coordinate gracefully (atomic midnight checks, no conflicts)
**"Listener" terminology:**
- Timer = mechanism that triggers
- Listener = callback that gets called
- Observer pattern = entities register, coordinator notifies

131
docs/developer/versioned_docs/version-v0.23.1/api-reference.md
View file

@ -22,30 +22,30 @@ Fetches home information and metadata:
```graphql
query {
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
viewer {
homes {
id
appNickname
address {
address1
postalCode
city
country
}
timeZone
currentSubscription {
priceInfo {
current {
currency
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
meteringPointData {
consumptionEan
gridAreaCode
}
}
}
}
```
@ -56,26 +56,27 @@ query {
Fetches quarter-hourly prices:
```graphql
query($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
query ($homeId: ID!) {
viewer {
home(id: $homeId) {
currentSubscription {
priceInfo {
range(resolution: QUARTER_HOURLY, first: 384) {
nodes {
total
startsAt
level
}
}
}
}
}
}
}
}
}
}
```
**Parameters:**
- `homeId`: Tibber home identifier
- `resolution`: Always `QUARTER_HOURLY`
- `first`: 384 intervals (4 days of data)
@ -85,10 +86,12 @@ query($homeId: ID!) {
## Rate Limits
Tibber API rate limits (as of 2024):
- **5000 requests per hour** per token
- **Burst limit:** 100 requests per minute
Integration stays well below these limits:
- Polls every 15 minutes = 96 requests/day
- User data cached for 24h = 1 request/day
- **Total:** ~100 requests/day per home
@ -99,13 +102,14 @@ Integration stays well below these limits:
```json
{
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
"total": 0.2456,
"startsAt": "2024-12-06T14:00:00.000+01:00",
"level": "NORMAL"
}
```
**Fields:**
- `total`: Price including VAT and fees (currency's major unit, e.g., EUR)
- `startsAt`: ISO 8601 timestamp with timezone
- `level`: Tibber's own classification (VERY_CHEAP, CHEAP, NORMAL, EXPENSIVE, VERY_EXPENSIVE)
@ -114,11 +118,12 @@ Integration stays well below these limits:
```json
{
"currency": "EUR"
"currency": "EUR"
}
```
Supported currencies:
- `EUR` (Euro) - displayed as ct/kWh
- `NOK` (Norwegian Krone) - displayed as øre/kWh
- `SEK` (Swedish Krona) - displayed as öre/kWh
@ -128,42 +133,52 @@ Supported currencies:
### Common Error Responses
**Invalid Token:**
```json
{
"errors": [{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}]
"errors": [
{
"message": "Unauthorized",
"extensions": {
"code": "UNAUTHENTICATED"
}
}
]
}
```
**Rate Limit Exceeded:**
```json
{
"errors": [{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}]
"errors": [
{
"message": "Too Many Requests",
"extensions": {
"code": "RATE_LIMIT_EXCEEDED"
}
}
]
}
```
**Home Not Found:**
```json
{
"errors": [{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}]
"errors": [
{
"message": "Home not found",
"extensions": {
"code": "NOT_FOUND"
}
}
]
}
```
Integration handles these with:
- Exponential backoff retry (3 attempts)
- ConfigEntryAuthFailed for auth errors
- ConfigEntryNotReady for temporary failures
@ -171,6 +186,7 @@ Integration handles these with:
## Data Transformation
Raw API data is enriched with:
- **Trailing 24h average** - Calculated from previous intervals
- **Leading 24h average** - Calculated from future intervals
- **Price difference %** - Deviation from average
@ -181,6 +197,7 @@ See `utils/price.py` for enrichment logic.
---
💡 **External Resources:**
- [Tibber API Documentation](https://developer.tibber.com/docs/overview)
- [GraphQL Explorer](https://developer.tibber.com/explorer)
- [Get API Token](https://developer.tibber.com/settings/access-token)

143
docs/developer/versioned_docs/version-v0.23.1/architecture.md
View file

@ -100,43 +100,43 @@ flowchart TB
### Flow Description
1. **Setup** (`__init__.py`)
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
- Integration loads, creates coordinator instance
- Registers entity platforms (sensor, binary_sensor)
- Sets up custom services
2. **Data Fetch** (every 15 minutes)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
- Coordinator triggers update via `api.py`
- API client checks **persistent cache** first (`coordinator/cache.py`)
- If cache valid → return cached data
- If cache stale → query Tibber GraphQL API
- Store fresh data in persistent cache (survives HA restart)
3. **Price Enrichment**
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
- Coordinator passes raw prices to `DataTransformer`
- Transformer checks **transformation cache** (memory)
- If cache valid → return enriched data
- If cache invalid → enrich via `price_utils.py` + `average_utils.py`
- Calculate 24h trailing/leading averages
- Calculate price differences (% from average)
- Assign rating levels (LOW/NORMAL/HIGH)
- Store enriched data in transformation cache
4. **Period Calculation**
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
- Coordinator passes enriched data to `PeriodCalculator`
- Calculator computes **hash** from prices + config
- If hash matches cache → return cached periods
- If hash differs → recalculate best/peak price periods
- Store periods with new hash
5. **Entity Updates**
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
- Coordinator provides complete data (prices + periods)
- Sensors read values via unified handlers
- Binary sensors evaluate period states
- Entities update on quarter-hour boundaries (00/15/30/45)
6. **Service Calls**
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
- Custom services access coordinator data directly
- Return formatted responses (JSON, ApexCharts format)
---
@ -146,13 +146,13 @@ flowchart TB
The integration uses **5 independent caching layers** for optimal performance:
| Layer | Location | Lifetime | Invalidation | Memory |
|-------|----------|----------|--------------|--------|
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
| Layer | Location | Lifetime | Invalidation | Memory |
| ------------------------ | ------------------------------------ | -------------------------------------- | ------------ | ------ |
| **API Cache** | `coordinator/cache.py` | 24h (user)<br/>Until midnight (prices) | Automatic | 50KB |
| **Translation Cache** | `const.py` | Until HA restart | Never | 5KB |
| **Config Cache** | `coordinator/*` | Until config change | Explicit | 1KB |
| **Period Cache** | `coordinator/periods.py` | Until data/config change | Hash-based | 10KB |
| **Transformation Cache** | `coordinator/data_transformation.py` | Until midnight/config | Automatic | 60KB |
**Total cache overhead:** ~126KB per coordinator instance (main entry + subentries)
@ -195,30 +195,31 @@ For detailed cache behavior, see [Caching Strategy](./caching-strategy.md).
### Core Components
| Component | File | Responsibility |
|-----------|------|----------------|
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
| Component | File | Responsibility |
| --------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
| **API Client** | `api.py` | GraphQL queries to Tibber, retry logic, error handling |
| **Coordinator** | `coordinator.py` | Update orchestration, cache management, absolute-time scheduling with boundary tolerance |
| **Data Transformer** | `coordinator/data_transformation.py` | Price enrichment (averages, ratings, differences) |
| **Period Calculator** | `coordinator/periods.py` | Best/peak price period calculation with relaxation |
| **Sensors** | `sensor/` | 80+ entities for prices, levels, ratings, statistics |
| **Binary Sensors** | `binary_sensor/` | Period indicators (best/peak price active) |
| **Services** | `services/` | Custom service endpoints (get_chartdata, get_apexcharts_yaml, refresh_user_data) |
### Sensor Architecture (Calculator Pattern)
The sensor platform uses **Calculator Pattern** for clean separation of concerns (refactored Nov 2025):
| Component | Files | Lines | Responsibility |
|-----------|-------|-------|----------------|
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
| Component | Files | Lines | Responsibility |
| ---------------- | ------------------------- | ----- | ------------------------------------------------------- |
| **Entity Class** | `sensor/core.py` | 909 | Entity lifecycle, coordinator, delegates to calculators |
| **Calculators** | `sensor/calculators/` | 1,838 | Business logic (8 specialized calculators) |
| **Attributes** | `sensor/attributes/` | 1,209 | State presentation (8 specialized modules) |
| **Routing** | `sensor/value_getters.py` | 276 | Centralized sensor → calculator mapping |
| **Chart Export** | `sensor/chart_data.py` | 144 | Service call handling, YAML parsing |
| **Helpers** | `sensor/helpers.py` | 188 | Aggregation functions, utilities |
**Calculator Package** (`sensor/calculators/`):
- `base.py` - Abstract BaseCalculator with coordinator access
- `interval.py` - Single interval calculations (current/next/previous)
- `rolling_hour.py` - 5-interval rolling windows
@ -230,6 +231,7 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
- `metadata.py` - Home/metering metadata
**Benefits:**
- 58% reduction in core.py (2,170 → 909 lines)
- Clear separation: Calculators (logic) vs Attributes (presentation)
- Independent testability for each calculator
@ -237,12 +239,12 @@ The sensor platform uses **Calculator Pattern** for clean separation of concerns
### Helper Utilities
| Utility | File | Purpose |
|---------|------|---------|
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
| Utility | File | Purpose |
| ----------------- | ------------------ | ------------------------------------------------- |
| **Price Utils** | `utils/price.py` | Rating calculation, enrichment, level aggregation |
| **Average Utils** | `utils/average.py` | Trailing/leading 24h average calculations |
| **Entity Utils** | `entity_utils/` | Shared icon/color/attribute logic |
| **Translations** | `const.py` | Translation loading and caching |
---
@ -283,12 +285,12 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
- **API polling**: Every 15 minutes (coordinator fetch cycle)
- **Entity updates**: On 00/15/30/45-minute boundaries via `coordinator/listeners.py`
- **Timer scheduling**: Uses `async_track_utc_time_change(minute=[0, 15, 30, 45], second=0)`
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- HA may trigger ±few milliseconds before/after exact boundary
- Smart boundary tolerance (±2 seconds) handles scheduling jitter in `sensor/helpers.py`
- If HA schedules at 14:59:58 → rounds to 15:00:00 (shows new interval data)
- If HA restarts at 14:59:30 → stays at 14:45:00 (shows current interval data)
- **Absolute time tracking**: Timer plans for **all future boundaries** (not relative delays)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- Prevents double-updates (if triggered at 14:59:58, next trigger is 15:15:00, not 15:00:00)
- **Result**: Current price sensors update without waiting for next API poll
### 4. Calculator Pattern (Sensor Platform)
@ -296,26 +298,31 @@ All quarter-hourly price intervals get augmented via `utils/price.py`:
Sensors organized by **calculation method** (refactored Nov 2025):
**Unified Handler Methods** (`sensor/core.py`):
- `_get_interval_value(offset, type)` - current/next/previous intervals
- `_get_rolling_hour_value(offset, type)` - 5-interval rolling windows
- `_get_daily_stat_value(day, stat_func)` - calendar day min/max/avg
- `_get_24h_window_value(stat_func)` - trailing/leading statistics
**Routing** (`sensor/value_getters.py`):
- Single source of truth mapping 80+ entity keys to calculator methods
- Organized by calculation type (Interval, Rolling Hour, Daily Stats, etc.)
**Calculators** (`sensor/calculators/`):
- Each calculator inherits from `BaseCalculator` with coordinator access
- Focused responsibility: `IntervalCalculator`, `TrendCalculator`, etc.
- Complex logic isolated (e.g., `TrendCalculator` has internal caching)
**Attributes** (`sensor/attributes/`):
- Separate from business logic, handles state presentation
- Builds extra_state_attributes dicts for entity classes
- Unified builders: `build_sensor_attributes()`, `build_extra_state_attributes()`
**Benefits:**
- Minimal code duplication across 80+ sensors
- Clear separation of concerns (calculation vs presentation)
- Easy to extend: Add sensor → choose pattern → add to routing
@ -333,12 +340,12 @@ Sensors organized by **calculation method** (refactored Nov 2025):
### CPU Optimization
| Optimization | Location | Savings |
|--------------|----------|---------|
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
| Optimization | Location | Savings |
| ------------------- | ------------------------ | ---------------------------- |
| Config caching | `coordinator/*` | ~50% on config checks |
| Period caching | `coordinator/periods.py` | ~70% on period recalculation |
| Lazy logging | Throughout | ~15% on log-heavy operations |
| Import optimization | Module structure | ~20% faster loading |
### Memory Usage

132
docs/developer/versioned_docs/version-v0.23.1/caching-strategy.md
View file

@ -24,11 +24,13 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Reduce API calls to Tibber by caching user data and price data between HA restarts.
**What is cached:**
- **Price data** (`price_data`): Day before yesterday/yesterday/today/tomorrow price intervals with enriched fields (384 intervals total)
- **User data** (`user_data`): Homes, subscriptions, features from Tibber GraphQL `viewer` query
- **Timestamps**: Last update times for validation
**Lifetime:**
- **Price data**: Until midnight turnover (cleared daily at 00:00 local time)
- **User data**: 24 hours (refreshed daily)
- **Survives**: HA restarts via persistent Storage
@ -36,29 +38,31 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Invalidation triggers:**
1. **Midnight turnover** (Timer #2 in coordinator):
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
```python
# coordinator/day_transitions.py
def _handle_midnight_turnover() -> None:
self._cached_price_data = None # Force fresh fetch for new day
self._last_price_update = None
await self.store_cache()
```
2. **Cache validation on load**:
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
```python
# coordinator/cache.py
def is_cache_valid(cache_data: CacheData) -> bool:
# Checks if price data is from a previous day
if today_date < local_now.date(): # Yesterday's data
return False
```
3. **Tomorrow data check** (after 13:00):
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
```python
# coordinator/data_fetching.py
if tomorrow_missing or tomorrow_invalid:
return "tomorrow_check" # Update needed
```
**Why this cache matters:** Reduces API load on Tibber (~192 intervals per fetch), speeds up HA restarts, enables offline operation until cache expires.
@ -71,18 +75,22 @@ The integration uses **4 distinct caching layers** with different purposes and l
**Purpose:** Avoid repeated file I/O when accessing entity descriptions, UI strings, etc.
**What is cached:**
- **Standard translations** (`/translations/*.json`): Config flow, selector options, entity names
- **Custom translations** (`/custom_translations/*.json`): Entity descriptions, usage tips, long descriptions
**Lifetime:**
- **Forever** (until HA restart)
- No invalidation during runtime
**When populated:**
- At integration setup: `async_load_translations(hass, "en")` in `__init__.py`
- Lazy loading: If translation missing, attempts file load once
**Access pattern:**
```python
# Non-blocking synchronous access from cached data
description = get_translation("binary_sensor.best_price_period.description", "en")
@ -101,6 +109,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**What is cached:**
### DataTransformer Config Cache
```python
{
"thresholds": {"low": 15, "high": 35},
@ -110,6 +119,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
### PeriodCalculator Config Cache
```python
{
"best": {"flex": 0.15, "min_distance_from_avg": 5.0, "min_period_length": 60},
@ -118,20 +128,23 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Lifetime:**
- Until `invalidate_config_cache()` is called
- Built once on first use per coordinator update cycle
**Invalidation trigger:**
- **Options change** (user reconfigures integration):
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
```python
# coordinator/core.py
async def _handle_options_update(...) -> None:
self._data_transformer.invalidate_config_cache()
self._period_calculator.invalidate_config_cache()
await self.async_request_refresh()
```
**Performance impact:**
- **Before:** ~30 dict lookups + type conversions per update = ~50μs
- **After:** 1 cache check = ~1μs
- **Savings:** ~98% (50μs → 1μs per update)
@ -147,6 +160,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
**Purpose:** Avoid expensive period calculations (~100-500ms) when price data and config haven't changed.
**What is cached:**
```python
{
"best_price": {
@ -161,6 +175,7 @@ description = get_translation("binary_sensor.best_price_period.description", "en
```
**Cache key:** Hash of relevant inputs
```python
hash_data = (
today_signature, # (startsAt, rating_level) for each interval
@ -172,6 +187,7 @@ hash_data = (
```
**Lifetime:**
- Until price data changes (today's intervals modified)
- Until config changes (flex, thresholds, filters)
- Recalculated at midnight (new today data)
@ -179,24 +195,27 @@ hash_data = (
**Invalidation triggers:**
1. **Config change** (explicit):
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
```python
def invalidate_config_cache() -> None:
self._cached_periods = None
self._last_periods_hash = None
```
2. **Price data change** (automatic via hash mismatch):
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
```python
current_hash = self._compute_periods_hash(price_info)
if self._last_periods_hash != current_hash:
# Cache miss - recalculate
```
**Cache hit rate:**
- **High:** During normal operation (coordinator updates every 15min, price data unchanged)
- **Low:** After midnight (new today data) or when tomorrow data arrives (~13:00-14:00)
**Performance impact:**
- **Period calculation:** ~100-500ms (depends on interval count, relaxation attempts)
- **Cache hit:** `<`1ms (hash comparison + dict lookup)
- **Savings:** ~70% of calculation time (most updates hit cache)
@ -212,6 +231,7 @@ hash_data = (
**Status:** ✅ **Clean separation** - enrichment only, no redundancy
**What is cached:**
```python
{
"timestamp": ...,
@ -224,14 +244,16 @@ hash_data = (
**Purpose:** Avoid re-enriching price data when config unchanged between midnight checks.
**Current behavior:**
- Caches **only enriched price data** (price + statistics)
- **Does NOT cache periods** (handled by Period Calculation Cache)
- Invalidated when:
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
- Config changes (thresholds affect enrichment)
- Midnight turnover detected
- New update cycle begins
**Architecture:**
- DataTransformer: Handles price enrichment only
- PeriodCalculator: Handles period calculation only (with hash-based cache)
- Coordinator: Assembles final data on-demand from both caches
@ -243,6 +265,7 @@ hash_data = (
## Cache Invalidation Flow
### User Changes Options (Config Flow)
```
User saves options
↓
@ -267,6 +290,7 @@ Fresh data fetch with new config
```
### Midnight Turnover (Day Transition)
```
Timer #2 fires at 00:00
↓
@ -286,6 +310,7 @@ Fresh API fetch for new day
```
### Tomorrow Data Arrives (~13:00)
```
Coordinator update cycle
↓
@ -327,12 +352,14 @@ API Data Cache (price_data, user_data)
```
**No cache invalidation cascades:**
- Config cache invalidation is **explicit** (on options update)
- Period cache invalidation is **automatic** (via hash mismatch)
- Transformation cache invalidation is **automatic** (on midnight/config change)
- Translation cache is **never invalidated** (read-only after load)
**Thread safety:**
- All caches are accessed from `MainThread` only (Home Assistant event loop)
- No locking needed (single-threaded execution model)
@ -341,6 +368,7 @@ API Data Cache (price_data, user_data)
## Performance Characteristics
### Typical Operation (No Changes)
```
Coordinator Update (every 15 min)
├─> API fetch: SKIP (cache valid)
@ -353,6 +381,7 @@ Total: ~16ms (down from ~600ms without caching)
```
### After Midnight Turnover
```
Coordinator Update (00:00)
├─> API fetch: ~500ms (cache cleared, fetch new day)
@ -365,6 +394,7 @@ Total: ~755ms (expected once per day)
```
### After Config Change
```
Options Update
├─> Cache invalidation: `<`1ms
@ -381,23 +411,25 @@ Options Update
## Summary Table
| Cache Type | Lifetime | Size | Invalidation | Purpose |
|------------|----------|------|--------------|---------|
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
| Cache Type | Lifetime | Size | Invalidation | Purpose |
| ---------------------- | ---------------------------- | ------ | ------------------------- | ------------------------------- |
| **API Data** | Hours to 1 day | ~50KB | Midnight, validation | Reduce API calls |
| **Translations** | Forever (until HA restart) | ~5KB | Never | Avoid file I/O |
| **Config Dicts** | Until options change | `<`1KB | Explicit (options update) | Avoid dict lookups |
| **Period Calculation** | Until data/config change | ~10KB | Auto (hash mismatch) | Avoid CPU-intensive calculation |
| **Transformation** | Until midnight/config change | ~50KB | Auto (midnight/config) | Avoid re-enrichment |
**Total memory overhead:** ~116KB per coordinator instance (main + subentries)
**Benefits:**
- 97% reduction in API calls (from every 15min to once per day)
- 70% reduction in period calculation time (cache hits during normal operation)
- 98% reduction in config access time (30+ lookups → 1 cache check)
- Zero file I/O during runtime (translations cached at startup)
**Trade-offs:**
- Memory usage: ~116KB per home (negligible for modern systems)
- Code complexity: 5 cache invalidation points (well-tested, documented)
- Debugging: Must understand cache lifetime when investigating stale data issues
@ -407,7 +439,9 @@ Options Update
## Debugging Cache Issues
### Symptom: Stale data after config change
**Check:**
1. Is `_handle_options_update()` called? (should see "Options updated" log)
2. Are `invalidate_config_cache()` methods executed?
3. Does `async_request_refresh()` trigger?
@ -415,7 +449,9 @@ Options Update
**Fix:** Ensure `config_entry.add_update_listener()` is registered in coordinator init.
### Symptom: Period calculation not updating
**Check:**
1. Verify hash changes when data changes: `_compute_periods_hash()`
2. Check `_last_periods_hash` vs `current_hash`
3. Look for "Using cached period calculation" vs "Calculating periods" logs
@ -423,7 +459,9 @@ Options Update
**Fix:** Hash function may not include all relevant data. Review `_compute_periods_hash()` inputs.
### Symptom: Yesterday's prices shown as today
**Check:**
1. `is_cache_valid()` logic in `coordinator/cache.py`
2. Midnight turnover execution (Timer #2)
3. Cache clear confirmation in logs
@ -431,7 +469,9 @@ Options Update
**Fix:** Timer may not be firing. Check `_schedule_midnight_turnover()` registration.
### Symptom: Missing translations
**Check:**
1. `async_load_translations()` called at startup?
2. Translation files exist in `/translations/` and `/custom_translations/`?
3. Cache population: `_TRANSLATIONS_CACHE` keys

11
docs/developer/versioned_docs/version-v0.23.1/coding-guidelines.md
View file

@ -8,10 +8,10 @@ comments: false
## Code Style
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
- **Formatter/Linter**: Ruff (replaces Black, Flake8, isort)
- **Max line length**: 120 characters
- **Max complexity**: 25 (McCabe)
- **Target**: Python 3.13
Run before committing:
@ -41,12 +41,14 @@ class TimeService:
```
**When prefix is required:**
- Public classes used across multiple modules
- All exception classes
- All coordinator and entity classes
- Data classes (dataclasses, NamedTuples) used as public APIs
**When prefix can be omitted:**
- Private helper classes within a single module (prefix with `_` underscore)
- Type aliases and callbacks (e.g., `TimeServiceCallback`)
- Small internal NamedTuples for function returns
@ -71,6 +73,7 @@ class DataFetcher: # Should be TibberPricesDataFetcher
**Current Technical Debt:**
Many existing classes lack the `TibberPrices` prefix. Before refactoring:
1. Document the plan in `/planning/class-naming-refactoring.md`
2. Use `multi_replace_string_in_file` for bulk renames
3. Test thoroughly after each module

25
docs/developer/versioned_docs/version-v0.23.1/contributing.md
View file

@ -14,10 +14,10 @@ Welcome! This guide helps you contribute to the Tibber Prices integration.
1. Fork the repository on GitHub
2. Clone your fork:
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
```bash
git clone https://github.com/YOUR_USERNAME/hass.tibber_prices.git
cd hass.tibber_prices
```
3. Open in VS Code
4. Click "Reopen in Container" when prompted
@ -34,6 +34,7 @@ git checkout -b fix/issue-123-description
```
**Branch naming:**
- `feature/` - New features
- `fix/` - Bug fixes
- `docs/` - Documentation only
@ -45,6 +46,7 @@ git checkout -b fix/issue-123-description
Edit code, following [Coding Guidelines](coding-guidelines.md).
**Run checks frequently:**
```bash
./scripts/type-check # Pyright type checking
./scripts/lint # Ruff linting (auto-fix)
@ -78,6 +80,7 @@ async def test_your_feature(hass, coordinator):
```
Run your test:
```bash
./scripts/test tests/test_your_feature.py -v
```
@ -97,6 +100,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
```
**Commit types:**
- `feat:` - New feature
- `fix:` - Bug fix
- `docs:` - Documentation
@ -105,6 +109,7 @@ Impact: Users can predict when prices will stabilize or continue fluctuating."
- `chore:` - Maintenance
**Add scope when relevant:**
- `feat(sensors):` - Sensor platform
- `fix(coordinator):` - Data coordinator
- `docs(user):` - User documentation
@ -124,32 +129,40 @@ Then open Pull Request on GitHub.
Title: Short, descriptive (50 chars max)
Description should include:
```markdown
## What
Brief description of changes
## Why
Problem being solved or feature rationale
## How
Implementation approach
## Testing
- [ ] Manual testing in Home Assistant
- [ ] Unit tests added/updated
- [ ] Type checking passes
- [ ] Linting passes
## Breaking Changes
(If any - describe migration path)
## Related Issues
Closes #123
```
### PR Checklist
Before submitting:
- [ ] Code follows [Coding Guidelines](coding-guidelines.md)
- [ ] All tests pass (`./scripts/test`)
- [ ] Type checking passes (`./scripts/type-check`)
@ -170,6 +183,7 @@ Before submitting:
### What Reviewers Look For
✅ **Good:**
- Clear, self-explanatory code
- Appropriate comments for complex logic
- Tests covering edge cases
@ -177,6 +191,7 @@ Before submitting:
- Follows existing patterns
❌ **Avoid:**
- Large PRs (>500 lines) - split into smaller ones
- Mixing unrelated changes
- Missing tests for new features
@ -193,6 +208,7 @@ Before submitting:
## Finding Issues to Work On
Good first issues are labeled:
- `good first issue` - Beginner-friendly
- `help wanted` - Maintainers welcome contributions
- `documentation` - Docs improvements
@ -210,6 +226,7 @@ Be respectful, constructive, and patient. We're all volunteers! 🙏
---
💡 **Related:**
- [Setup Guide](setup.md) - DevContainer setup
- [Coding Guidelines](coding-guidelines.md) - Style guide
- [Testing](testing.md) - Writing tests

65
docs/developer/versioned_docs/version-v0.23.1/critical-patterns.md
View file

@ -12,6 +12,7 @@ comments: false
## 🎯 Why Are These Tests Critical?
Home Assistant integrations run **continuously** in the background. Resource leaks lead to:
- **Memory Leaks**: RAM usage grows over days/weeks until HA becomes unstable
- **Callback Leaks**: Listeners remain registered after entity removal → CPU load increases
- **Timer Leaks**: Timers continue running after unload → unnecessary background tasks
@ -26,6 +27,7 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.1 Listener Cleanup ✅
**What is tested:**
- Time-sensitive listeners are correctly removed (`async_add_time_sensitive_listener()`)
- Minute-update listeners are correctly removed (`async_add_minute_update_listener()`)
- Lifecycle callbacks are correctly unregistered (`register_lifecycle_callback()`)
@ -33,11 +35,13 @@ Home Assistant integrations run **continuously** in the background. Resource lea
- Binary sensor cleanup removes ALL registered listeners
**Why critical:**
- Each registered listener holds references to Entity + Coordinator
- Without cleanup: Entities are not freed by GC → Memory Leak
- With 80+ sensors × 3 listener types = 240+ callbacks that must be cleanly removed
**Code Locations:**
- `coordinator/listeners.py` → `async_add_time_sensitive_listener()`, `async_add_minute_update_listener()`
- `coordinator/core.py` → `register_lifecycle_callback()`
- `sensor/core.py` → `async_will_remove_from_hass()`
@ -46,32 +50,38 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 1.2 Timer Cleanup ✅
**What is tested:**
- Quarter-hour timer is cancelled and reference cleared
- Minute timer is cancelled and reference cleared
- Both timers are cancelled together
- Cleanup works even when timers are `None`
**Why critical:**
- Uncancelled timers continue running after integration unload
- HA's `async_track_utc_time_change()` creates persistent callbacks
- Without cleanup: Timers keep firing → CPU load + unnecessary coordinator updates
**Code Locations:**
- `coordinator/listeners.py` → `cancel_timers()`
- `coordinator/core.py` → `async_shutdown()`
#### 1.3 Config Entry Cleanup ✅
**What is tested:**
- Options update listener is registered via `async_on_unload()`
- Cleanup function is correctly passed to `async_on_unload()`
**Why critical:**
- `entry.add_update_listener()` registers permanent callback
- Without `async_on_unload()`: Listener remains active after reload → duplicate updates
- Pattern: `entry.async_on_unload(entry.add_update_listener(handler))`
**Code Locations:**
- `coordinator/core.py` → `__init__()` (listener registration)
- `__init__.py` → `async_unload_entry()`
@ -82,16 +92,19 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 2.1 Config Cache Invalidation
**What is tested:**
- DataTransformer config cache is invalidated on options change
- PeriodCalculator config + period cache is invalidated
- Trend calculator cache is cleared on coordinator update
**Why critical:**
- Stale config → Sensors use old user settings
- Stale period cache → Incorrect best/peak price periods
- Stale trend cache → Outdated trend analysis
**Code Locations:**
- `coordinator/data_transformation.py` → `invalidate_config_cache()`
- `coordinator/periods.py` → `invalidate_config_cache()`
- `sensor/calculators/trend.py` → `clear_trend_cache()`
@ -103,15 +116,18 @@ Home Assistant integrations run **continuously** in the background. Resource lea
#### 3.1 Persistent Storage Removal
**What is tested:**
- Storage file is deleted on config entry removal
- Cache is saved on shutdown (no data loss)
**Why critical:**
- Without storage removal: Old files remain after uninstallation
- Without cache save on shutdown: Data loss on HA restart
- Storage path: `.storage/tibber_prices.{entry_id}`
**Code Locations:**
- `__init__.py` → `async_remove_entry()`
- `coordinator/core.py` → `async_shutdown()`
@ -120,12 +136,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_timer_scheduling.py`
**What is tested:**
- Quarter-hour timer is registered with correct parameters
- Minute timer is registered with correct parameters
- Timers can be re-scheduled (override old timer)
- Midnight turnover detection works correctly
**Why critical:**
- Wrong timer parameters → Entities update at wrong times
- Without timer override on re-schedule → Multiple parallel timers → Performance problem
@ -134,12 +152,14 @@ Home Assistant integrations run **continuously** in the background. Resource lea
**File:** `tests/test_sensor_timer_assignment.py`
**What is tested:**
- All `TIME_SENSITIVE_ENTITY_KEYS` are valid entity keys
- All `MINUTE_UPDATE_ENTITY_KEYS` are valid entity keys
- Both lists are disjoint (no overlap)
- Sensor and binary sensor platforms are checked
**Why critical:**
- Wrong timer assignment → Sensors update at wrong times
- Overlap → Duplicate updates → Performance problem
@ -150,10 +170,12 @@ These patterns were analyzed and classified as **not critical**:
### 6. Async Task Management
**Current Status:** Fire-and-forget pattern for short tasks
- `sensor/core.py` → Chart data refresh (short-lived, max 1-2 seconds)
- `coordinator/core.py` → Cache storage (short-lived, max 100ms)
**Why no tests needed:**
- No long-running tasks (all < 2 seconds)
- HA's event loop handles short tasks automatically
- Task exceptions are already logged
@ -163,6 +185,7 @@ These patterns were analyzed and classified as **not critical**:
### 7. API Session Cleanup
**Current Status:** ✅ Correctly implemented
- `async_get_clientsession(hass)` is used (shared session)
- No new sessions are created
- HA manages session lifecycle automatically
@ -172,6 +195,7 @@ These patterns were analyzed and classified as **not critical**:
### 8. Translation Cache Memory
**Current Status:** ✅ Bounded cache
- Max ~5-10 languages × 5KB = 50KB total
- Module-level cache without re-loading
- Practically no memory issue
@ -181,11 +205,13 @@ These patterns were analyzed and classified as **not critical**:
### 9. Coordinator Data Structure Integrity
**Current Status:** Manually tested via `./scripts/develop`
- Midnight turnover works correctly (observed over several days)
- Missing keys are handled via `.get()` with defaults
- 80+ sensors access `coordinator.data` without errors
**Structure:**
```python
coordinator.data = {
"user_data": {...},
@ -197,6 +223,7 @@ coordinator.data = {
### 10. Service Response Memory
**Current Status:** HA's response lifecycle
- HA automatically frees service responses after return
- ApexCharts ~20KB response is one-time per call
- No response accumulation in integration code
@ -207,29 +234,30 @@ coordinator.data = {
### ✅ Implemented Tests (41 total)
| Category | Status | Tests | File | Coverage |
|----------|--------|-------|------|----------|
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
| Category | Status | Tests | File | Coverage |
| ----------------------- | ------ | ------ | --------------------------------- | ------------------- |
| Listener Cleanup | ✅ | 5 | `test_resource_cleanup.py` | 100% |
| Timer Cleanup | ✅ | 4 | `test_resource_cleanup.py` | 100% |
| Config Entry Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Cache Invalidation | ✅ | 3 | `test_resource_cleanup.py` | 100% |
| Storage Cleanup | ✅ | 1 | `test_resource_cleanup.py` | 100% |
| Storage Persistence | ✅ | 2 | `test_coordinator_shutdown.py` | 100% |
| Timer Scheduling | ✅ | 8 | `test_timer_scheduling.py` | 100% |
| Sensor-Timer Assignment | ✅ | 17 | `test_sensor_timer_assignment.py` | 100% |
| **TOTAL** | **✅** | **41** | | **100% (critical)** |
### 📋 Analyzed but Not Implemented (Nice-to-Have)
| Category | Status | Rationale |
|----------|--------|-----------|
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
| Category | Status | Rationale |
| ------------------------ | ------ | ---------------------------------------------------- |
| Async Task Management | 📋 | Fire-and-forget pattern used (no long-running tasks) |
| API Session Cleanup | ✅ | Pattern correct (`async_get_clientsession` used) |
| Translation Cache | ✅ | Cache size bounded (~50KB max for 10 languages) |
| Data Structure Integrity | 📋 | Would add test time without finding real issues |
| Service Response Memory | 📋 | HA automatically frees service responses |
**Legend:**
- ✅ = Fully tested or pattern verified correct
- 📋 = Analyzed, low priority for testing (no known issues)
@ -238,6 +266,7 @@ coordinator.data = {
### ✅ All Critical Patterns Tested
All essential memory leak prevention patterns are covered by 41 tests:
- ✅ Listeners are correctly removed (no callback leaks)
- ✅ Timers are cancelled (no background task leaks)
- ✅ Config entry cleanup works (no dangling listeners)

52
docs/developer/versioned_docs/version-v0.23.1/debugging.md
View file

@ -10,9 +10,9 @@ Add to `configuration.yaml`:
```yaml
logger:
default: info
logs:
custom_components.tibber_prices: debug
default: info
logs:
custom_components.tibber_prices: debug
```
Restart Home Assistant to apply.
@ -20,6 +20,7 @@ Restart Home Assistant to apply.
### Key Log Messages
**Coordinator Updates:**
```
[custom_components.tibber_prices.coordinator] Successfully fetched price data
[custom_components.tibber_prices.coordinator] Cache valid, using cached data
@ -27,6 +28,7 @@ Restart Home Assistant to apply.
```
**Period Calculation:**
```
[custom_components.tibber_prices.coordinator.periods] Calculating BEST PRICE periods: flex=15.0%
[custom_components.tibber_prices.coordinator.periods] Day 2024-12-06: Found 2 periods
@ -34,6 +36,7 @@ Restart Home Assistant to apply.
```
**API Errors:**
```
[custom_components.tibber_prices.api] API request failed: Unauthorized
[custom_components.tibber_prices.api] Retrying (attempt 2/3) after 2.0s
@ -47,26 +50,27 @@ Restart Home Assistant to apply.
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
"version": "0.2.0",
"configurations": [
{
"name": "Home Assistant",
"type": "debugpy",
"request": "launch",
"module": "homeassistant",
"args": ["-c", "config", "--debug"],
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}/.venv/lib/python3.13/site-packages"
}
}
]
}
```
### Set Breakpoints
**Coordinator update:**
```python
# coordinator/core.py
async def _async_update_data(self) -> dict:
@ -75,6 +79,7 @@ async def _async_update_data(self) -> dict:
```
**Period calculation:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(...) -> list[dict]:
@ -91,6 +96,7 @@ def calculate_periods(...) -> list[dict]:
```
**Flags:**
- `-v` - Verbose output
- `-s` - Show print statements
- `-k pattern` - Run tests matching pattern
@ -102,6 +108,7 @@ Set breakpoint in test file, use "Debug Test" CodeLens.
### Useful Test Patterns
**Print coordinator data:**
```python
def test_something(coordinator):
print(f"Coordinator data: {coordinator.data}")
@ -109,6 +116,7 @@ def test_something(coordinator):
```
**Inspect period attributes:**
```python
def test_periods(hass, coordinator):
periods = coordinator.data.get('best_price_periods', [])
@ -122,11 +130,13 @@ def test_periods(hass, coordinator):
### Integration Not Loading
**Check:**
```bash
grep "tibber_prices" config/home-assistant.log
```
**Common causes:**
- Syntax error in Python code → Check logs for traceback
- Missing dependency → Run `uv sync`
- Wrong file permissions → `chmod +x scripts/*`
@ -134,12 +144,14 @@ grep "tibber_prices" config/home-assistant.log
### Sensors Not Updating
**Check coordinator state:**
```python
# In Developer Tools > Template
{{ states.sensor.tibber_home_current_interval_price.last_updated }}
```
**Debug in code:**
```python
# Add logging in sensor/core.py
_LOGGER.debug("Updating sensor %s: old=%s new=%s",
@ -149,6 +161,7 @@ _LOGGER.debug("Updating sensor %s: old=%s new=%s",
### Period Calculation Wrong
**Enable detailed period logs:**
```python
# coordinator/period_handlers/period_building.py
_LOGGER.debug("Candidate intervals: %s",
@ -156,6 +169,7 @@ _LOGGER.debug("Candidate intervals: %s",
```
**Check filter statistics:**
```
[period_building] Flex filter blocked: 45 intervals
[period_building] Min distance blocked: 12 intervals
@ -200,6 +214,7 @@ python -m pstats profile.stats
### Remote Debugging with debugpy
Add to coordinator code:
```python
import debugpy
debugpy.listen(5678)
@ -212,11 +227,13 @@ Connect from VS Code with remote attach configuration.
### IPython REPL
Install in container:
```bash
uv pip install ipython
```
Add breakpoint:
```python
from IPython import embed
embed() # Drops into interactive shell
@ -225,6 +242,7 @@ embed() # Drops into interactive shell
---
💡 **Related:**
- [Testing Guide](testing.md) - Writing and running tests
- [Setup Guide](setup.md) - Development environment
- [Architecture](architecture.md) - Code structure

123
docs/developer/versioned_docs/version-v0.23.1/intro.md
View file

@ -8,25 +8,25 @@ This is an independent, community-maintained custom integration for Home Assista
## 📚 Developer Guides
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
- **[Setup](setup.md)** - DevContainer, environment setup, and dependencies
- **[Architecture](architecture.md)** - Code structure, patterns, and conventions
- **[Period Calculation Theory](period-calculation-theory.md)** - Mathematical foundations, Flex/Distance interaction, Relaxation strategy
- **[Timer Architecture](timer-architecture.md)** - Timer system, scheduling, coordination (3 independent timers)
- **[Caching Strategy](caching-strategy.md)** - Cache layers, invalidation, debugging
- **[Testing](testing.md)** - How to run tests and write new test cases
- **[Release Management](release-management.md)** - Release workflow and versioning process
- **[Coding Guidelines](coding-guidelines.md)** - Style guide, linting, and best practices
- **[Refactoring Guide](refactoring-guide.md)** - How to plan and execute major refactorings
## 🤖 AI Documentation
The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.1/AGENTS.md). This file serves as long-term memory for AI assistants and contains:
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
- Detailed architectural patterns
- Code quality rules and conventions
- Development workflow guidance
- Common pitfalls and anti-patterns
- Project-specific patterns and utilities
**Important:** When proposing changes to patterns or conventions, always update [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.1/AGENTS.md) to keep AI guidance consistent.
@ -34,32 +34,32 @@ The main AI/Copilot documentation is in [`AGENTS.md`](https://github.com/jpawlow
This integration is developed with extensive AI assistance (GitHub Copilot, Claude, and other AI tools). The AI handles:
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
- **Pattern Recognition**: Understanding and applying Home Assistant best practices
- **Code Generation**: Implementing features with proper type hints, error handling, and documentation
- **Refactoring**: Maintaining consistency across the codebase during structural changes
- **Translation Management**: Keeping 5 language files synchronized
- **Documentation**: Generating and maintaining comprehensive documentation
**Quality Assurance:**
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
- Automated linting with Ruff (120-char line length, max complexity 25)
- Home Assistant's type checking and validation
- Real-world testing in development environment
- Code review by maintainer before merging
**Benefits:**
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
- Rapid feature development while maintaining quality
- Consistent code patterns across all modules
- Comprehensive documentation maintained alongside code
- Quick bug fixes with proper understanding of context
**Limitations:**
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
- AI may occasionally miss edge cases or subtle bugs
- Some complex Home Assistant patterns may need human review
- Translation quality depends on AI's understanding of target language
- User feedback is crucial for discovering real-world issues
If you're working with AI tools on this project, the [`AGENTS.md`](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.1/AGENTS.md) file provides the context and patterns that ensure consistency.
@ -80,15 +80,15 @@ If you're working with AI tools on this project, the [`AGENTS.md`](https://githu
The project includes several helper scripts in `./scripts/`:
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
- `bootstrap` - Initial setup of dependencies
- `develop` - Start Home Assistant in debug mode (auto-cleans .egg-info)
- `clean` - Remove build artifacts and caches
- `lint` - Auto-fix code issues with ruff
- `lint-check` - Check code without modifications (CI mode)
- `hassfest` - Validate integration structure (JSON, Python syntax, required files)
- `setup` - Install development tools (git-cliff, @github/copilot)
- `prepare-release` - Prepare a new release (bump version, create tag)
- `generate-release-notes` - Generate release notes from commits
## 📦 Project Structure
@ -121,23 +121,23 @@ custom_components/tibber_prices/
**DataUpdateCoordinator Pattern:**
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
- Centralized data fetching and caching
- Automatic entity updates on data changes
- Persistent storage via `Store`
- Quarter-hour boundary refresh scheduling
**Price Data Enrichment:**
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
- Raw API data is enriched with statistical analysis
- Trailing/leading 24h averages calculated per interval
- Price differences and ratings added
- All via pure functions in `price_utils.py`
**Translation System:**
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
- Dual system: `/translations/` (HA schema) + `/custom_translations/` (extended)
- Both must stay in sync across all languages (de, en, nb, nl, sv)
- Async loading at integration setup
## 🧪 Testing
@ -159,18 +159,19 @@ pytest --cov=custom_components.tibber_prices tests/
Documentation is organized in two Docusaurus sites:
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
- **User docs** (`docs/user/`): Installation, configuration, usage guides
- Markdown files in `docs/user/docs/*.md`
- Navigation managed via `docs/user/sidebars.ts`
- **Developer docs** (`docs/developer/`): Architecture, patterns, contribution guides
- Markdown files in `docs/developer/docs/*.md`
- Navigation managed via `docs/developer/sidebars.ts`
- **AI guidance**: `AGENTS.md` (patterns, conventions, long-term memory)
**Best practices:**
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
- Use clear examples and code snippets
- Keep docs up-to-date with code changes
- Add new pages to appropriate `sidebars.ts` for navigation
## 🤝 Contributing

17
docs/developer/versioned_docs/version-v0.23.1/performance.md
View file

@ -5,6 +5,7 @@ Guidelines for maintaining and improving integration performance.
## Performance Goals
Target metrics:
- **Coordinator update**: &lt;500ms (typical: 200-300ms)
- **Sensor update**: &lt;10ms per sensor
- **Period calculation**: &lt;100ms (typical: 20-50ms)
@ -64,6 +65,7 @@ python -m aioprof homeassistant -c config
### Caching
**1. Persistent Cache** (API data):
```python
# Already implemented in coordinator/cache.py
store = Store(hass, STORAGE_VERSION, STORAGE_KEY)
@ -71,6 +73,7 @@ data = await store.async_load()
```
**2. Translation Cache** (in-memory):
```python
# Already implemented in const.py
_TRANSLATION_CACHE: dict[str, dict] = {}
@ -83,6 +86,7 @@ def get_translation(path: str, language: str) -> dict:
```
**3. Config Cache** (invalidated on options change):
```python
class DataTransformer:
def __init__(self):
@ -100,6 +104,7 @@ class DataTransformer:
### Lazy Loading
**Load data only when needed:**
```python
@property
def extra_state_attributes(self) -> dict | None:
@ -113,6 +118,7 @@ def extra_state_attributes(self) -> dict | None:
### Bulk Operations
**Process multiple items at once:**
```python
# ❌ Slow - loop with individual operations
for interval in intervals:
@ -126,6 +132,7 @@ results = enrich_intervals_bulk(intervals)
### Async Best Practices
**1. Concurrent API calls:**
```python
# ❌ Sequential (slow)
user_data = await fetch_user_data()
@ -139,6 +146,7 @@ user_data, price_data = await asyncio.gather(
```
**2. Don't block event loop:**
```python
# ❌ Blocking
result = heavy_computation() # Blocks for seconds
@ -152,6 +160,7 @@ result = await hass.async_add_executor_job(heavy_computation)
### Avoid Memory Leaks
**1. Clear references:**
```python
class Coordinator:
async def async_shutdown(self):
@ -162,6 +171,7 @@ class Coordinator:
```
**2. Use weak references for callbacks:**
```python
import weakref
@ -176,6 +186,7 @@ class Manager:
### Efficient Data Structures
**Use appropriate types:**
```python
# ❌ List for lookups (O(n))
if timestamp in timestamp_list:
@ -197,11 +208,13 @@ results = (x for x in items if condition(x))
### Minimize API Calls
**Already implemented:**
- Cache valid until midnight
- User data cached for 24h
- Only poll when tomorrow data expected
**Monitor API usage:**
```python
_LOGGER.debug("API call: %s (cache_age=%s)",
endpoint, cache_age)
@ -210,6 +223,7 @@ _LOGGER.debug("API call: %s (cache_age=%s)",
### Smart Updates
**Only update when needed:**
```python
async def _async_update_data(self) -> dict:
"""Fetch data from API."""
@ -226,6 +240,7 @@ async def _async_update_data(self) -> dict:
### State Class Selection
**Affects long-term statistics storage:**
```python
# ❌ MEASUREMENT for prices (stores every change)
state_class=SensorStateClass.MEASUREMENT # ~35K records/year
@ -240,6 +255,7 @@ state_class=SensorStateClass.TOTAL # For cumulative values
### Attribute Size
**Keep attributes minimal:**
```python
# ❌ Large nested structures (KB per update)
attributes = {
@ -317,6 +333,7 @@ _LOGGER.debug("Current memory usage: %.2f MB", memory_mb)
---
💡 **Related:**
- [Caching Strategy](caching-strategy.md) - Cache layers
- [Architecture](architecture.md) - System design
- [Debugging](debugging.md) - Profiling tools

314
docs/developer/versioned_docs/version-v0.23.1/period-calculation-theory.md
View file

@ -7,6 +7,7 @@ This document explains the mathematical foundations and design decisions behind
**Target Audience:** Developers maintaining or extending the period calculation logic.
**Related Files:**
- `coordinator/period_handlers/core.py` - Main calculation entry point
- `coordinator/period_handlers/level_filtering.py` - Flex and distance filtering
- `coordinator/period_handlers/relaxation.py` - Multi-phase relaxation strategy
@ -23,6 +24,7 @@ Period detection uses **three independent filters** (all must pass):
**Purpose:** Limit how far prices can deviate from the daily min/max.
**Logic:**
```python
# Best Price: Price must be within flex% ABOVE daily minimum
in_flex = price <= (daily_min + daily_min × flex)
@ -32,6 +34,7 @@ in_flex = price >= (daily_max - daily_max × flex)
```
**Example (Best Price):**
- Daily Min: 10 ct/kWh
- Flex: 15%
- Acceptance Range: 0 - 11.5 ct/kWh (10 + 10×0.15)
@ -41,6 +44,7 @@ in_flex = price >= (daily_max - daily_max × flex)
**Purpose:** Ensure periods are **significantly** cheaper/more expensive than average, not just marginally better.
**Logic:**
```python
# Best Price: Price must be at least min_distance% BELOW daily average
meets_distance = price <= (daily_avg × (1 - min_distance/100))
@ -50,6 +54,7 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
```
**Example (Best Price):**
- Daily Avg: 15 ct/kWh
- Min Distance: 5%
- Acceptance Range: 0 - 14.25 ct/kWh (15 × 0.95)
@ -65,17 +70,17 @@ meets_distance = price >= (daily_avg × (1 + min_distance/100))
The integration maintains **two independent sets** of volatility thresholds:
1. **Sensor Thresholds** (user-configurable via `CONF_VOLATILITY_*_THRESHOLD`)
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
- Purpose: Display classification in `sensor.tibber_home_volatility_*`
- Default: LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%
- User can adjust in config flow options
- Affects: Sensor state/attributes only
2. **Period Filter Thresholds** (internal, fixed)
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
- Purpose: Level filter criteria when using `level="volatility_low"` etc.
- Source: `PRICE_LEVEL_THRESHOLDS` in `const.py`
- Values: Same as sensor defaults (LOW < 10%, MEDIUM < 20%, HIGH ≥ 20%)
- User **cannot** adjust these
- Affects: Period candidate selection
**Rationale for Separation:**
@ -86,6 +91,7 @@ The integration maintains **two independent sets** of volatility thresholds:
- Period calculation has many interacting filters (Flex, Distance, Level) - exposing all internals would be error-prone
**Implementation:**
```python
# Sensor classification uses user config
user_low_threshold = config_entry.options.get(CONF_VOLATILITY_LOW_THRESHOLD, 10)
@ -107,36 +113,42 @@ period_low_threshold = PRICE_LEVEL_THRESHOLDS["volatility_low"] # Always 10%
#### Scenario: Best Price with Flex=50%, Min_Distance=5%
**Given:**
- Daily Min: 10 ct/kWh
- Daily Avg: 15 ct/kWh
- Daily Max: 20 ct/kWh
**Flex Filter (50%):**
```
Max accepted = 10 + (10 × 0.50) = 15 ct/kWh
```
**Min Distance Filter (5%):**
```
Max accepted = 15 × (1 - 0.05) = 14.25 ct/kWh
```
**Conflict:**
- Interval at 14.8 ct/kWh:
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
- ✅ Flex: 14.8 ≤ 15 (PASS)
- ❌ Distance: 14.8 > 14.25 (FAIL)
- **Result:** Rejected by Min_Distance even though Flex allows it!
**The Issue:** At high Flex values, Min_Distance becomes the dominant filter and blocks intervals that Flex would permit. This defeats the purpose of having high Flex.
### Mathematical Analysis
**Conflict condition for Best Price:**
```
daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
```
**Typical values:**
- Min = 10, Avg = 15, Min_Distance = 5%
- Conflict occurs when: `10 × (1 + flex) > 14.25`
- Simplify: `flex > 0.425` (42.5%)
@ -149,6 +161,7 @@ daily_min × (1 + flex) > daily_avg × (1 - min_distance/100)
**Approach:** Reduce Min_Distance proportionally as Flex increases.
**Formula:**
```python
if flex > 0.20: # 20% threshold
flex_excess = flex - 0.20
@ -158,15 +171,16 @@ if flex > 0.20: # 20% threshold
**Scaling Table (Original Min_Distance = 5%):**
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
|-------|--------------|----------------------|-----------|
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
| Flex | Scale Factor | Adjusted Min_Distance | Rationale |
| ---- | ------------ | --------------------- | --------------------------------- |
| ≤20% | 1.00 | 5.0% | Standard - both filters relevant |
| 25% | 0.88 | 4.4% | Slight reduction |
| 30% | 0.75 | 3.75% | Moderate reduction |
| 40% | 0.50 | 2.5% | Strong reduction - Flex dominates |
| 50% | 0.25 | 1.25% | Minimal distance - Flex decides |
**Why stop at 25% of original?**
- Min_Distance ensures periods are **significantly** different from average
- Even at 1.25%, prevents "flat days" (little price variation) from accepting every interval
- Maintains semantic meaning: "this is a meaningful best/peak price period"
@ -174,6 +188,7 @@ if flex > 0.20: # 20% threshold
**Implementation:** See `level_filtering.py` → `check_interval_criteria()`
**Code Extract:**
```python
# coordinator/period_handlers/level_filtering.py
@ -209,12 +224,14 @@ def check_interval_criteria(price, criteria):
```
**Why Linear Scaling?**
- Simple and predictable
- No abrupt behavior changes
- Easy to reason about for users and developers
- Alternative considered: Exponential scaling (rejected as too aggressive)
**Why 25% Minimum?**
- Below this, min_distance loses semantic meaning
- Even on flat days, some quality filter needed
- Prevents "every interval is a period" scenario
@ -227,12 +244,14 @@ def check_interval_criteria(price, criteria):
### Implementation Constants
**Defined in `coordinator/period_handlers/core.py`:**
```python
MAX_SAFE_FLEX = 0.50 # 50% - hard cap: above this, period detection becomes unreliable
MAX_OUTLIER_FLEX = 0.25 # 25% - cap for outlier filtering: above this, spike detection too permissive
```
**Defined in `const.py`:**
```python
DEFAULT_BEST_PRICE_FLEX = 15 # 15% base - optimal for relaxation mode (default enabled)
DEFAULT_PEAK_PRICE_FLEX = -20 # 20% base (negative for peak detection)
@ -255,16 +274,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Find practical time windows for running appliances
**Constraints:**
- Appliances need time to complete cycles (dishwasher: 2-3h, EV charging: 4-8h)
- Short periods are impractical (not worth automation overhead)
- User wants genuinely cheap times, not just "slightly below average"
**Defaults:**
- **60 min minimum** - Ensures period is long enough for meaningful use
- **15% flex** - Stricter selection, focuses on truly cheap times
- **Reasoning:** Better to find fewer, higher-quality periods than many mediocre ones
**User behavior:**
- Automations trigger actions (turn on devices)
- Wrong automation = wasted energy/money
- Preference: Conservative (miss some savings) over aggressive (false positives)
@ -274,16 +296,19 @@ The different defaults reflect fundamentally different use cases:
**Goal:** Alert users to expensive periods for consumption reduction
**Constraints:**
- Brief price spikes still matter (even 15-30 min is worth avoiding)
- Early warning more valuable than perfect accuracy
- User can manually decide whether to react
**Defaults:**
- **30 min minimum** - Catches shorter expensive spikes
- **20% flex** - More permissive, earlier detection
- **Reasoning:** Better to warn early (even if not peak) than miss expensive periods
**User behavior:**
- Notifications/alerts (informational)
- Wrong alert = minor inconvenience, not cost
- Preference: Sensitive (catch more) over specific (catch only extremes)
@ -293,17 +318,20 @@ The different defaults reflect fundamentally different use cases:
**Peak Price Volatility:**
Price curves tend to have:
- **Sharp spikes** during peak hours (morning/evening)
- **Shorter duration** at maximum (1-2 hours typical)
- **Higher variance** in peak times than cheap times
**Example day:**
```
Cheap period: 02:00-07:00 (5 hours at 10-12 ct) ← Gradual, stable
Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
```
**Implication:**
- Stricter flex on peak (15%) might miss real expensive periods (too brief)
- Longer min_length (60 min) might exclude legitimate spikes
- Solution: More flexible thresholds for peak detection
@ -311,16 +339,19 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
#### Design Alternatives Considered
**Option 1: Symmetric defaults (rejected)**
- Both 60 min, both 15% flex
- Problem: Misses short but expensive spikes
- User feedback: "Why didn't I get warned about the 30-min price spike?"
**Option 2: Same defaults, let users figure it out (rejected)**
- No guidance on best practices
- Users would need to experiment to find good values
- Most users stick with defaults, so defaults matter
**Option 3: Current approach (adopted)**
- **All values user-configurable** via config flow options
- **Different installation defaults** for Best Price vs. Peak Price
- Defaults reflect recommended practices for each use case
@ -336,12 +367,14 @@ Expensive period: 17:00-18:30 (1.5 hours at 35-40 ct) ← Sharp, brief
**Enforcement:** `core.py` caps `abs(flex)` at 0.50 (50%)
**Rationale:**
- Above 50%, period detection becomes unreliable
- Best Price: Almost entire day qualifies (Min + 50% typically covers 60-80% of intervals)
- Peak Price: Similar issue with Max - 50%
- **Result:** Either massive periods (entire day) or no periods (min_length not met)
**Warning Message:**
```
Flex XX% exceeds maximum safe value! Capping at 50%.
Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation.
@ -352,6 +385,7 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
**Enforcement:** `core.py` caps outlier filtering flex at 0.25 (25%)
**Rationale:**
- Outlier filtering uses Flex to determine "stable context" threshold
- At > 25% Flex, almost any price swing is considered "stable"
- **Result:** Legitimate price shifts aren't smoothed, breaking period formation
@ -363,23 +397,28 @@ Recommendation: Use 15-20% with relaxation enabled, or 25-35% without relaxation
#### With Relaxation Enabled (Recommended)
**Optimal:** 10-20%
- Relaxation increases Flex incrementally: 15% → 18% → 21% → ...
- Low baseline ensures relaxation has room to work
**Warning Threshold:** > 25%
- INFO log: "Base flex is on the high side"
**High Warning:** > 30%
- WARNING log: "Base flex is very high for relaxation mode!"
- Recommendation: Lower to 15-20%
#### Without Relaxation
**Optimal:** 20-35%
- No automatic adjustment, must be sufficient from start
- Higher baseline acceptable since no relaxation fallback
**Maximum Useful:** ~50%
- Above this, period detection degrades (see Hard Limits)
---
@ -395,6 +434,7 @@ Ensure **minimum periods per day** are found even when baseline filters are too
### Multi-Phase Approach
**Each day processed independently:**
1. Calculate baseline periods with user's config
2. If insufficient periods found, enter relaxation loop
3. Try progressively relaxed filter combinations
@ -418,6 +458,7 @@ for attempt in range(max_relaxation_attempts):
```
**Constants:**
```python
FLEX_WARNING_THRESHOLD_RELAXATION = 0.25 # 25% - INFO: suggest lowering to 15-20%
FLEX_HIGH_THRESHOLD_RELAXATION = 0.30 # 30% - WARNING: very high for relaxation mode
@ -427,26 +468,27 @@ MAX_FLEX_HARD_LIMIT = 0.50 # 50% - absolute maximum (enforced in core.py)
**Design Decisions:**
1. **Why 3% fixed increment?**
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
- Predictable escalation path (15% → 18% → 21% → ...)
- Independent of base flex (works consistently)
- 11 attempts covers full useful range (15% → 48%)
- Balance: Not too slow (2%), not too fast (5%)
2. **Why hard-coded, not configurable?**
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
- Prevents user misconfiguration
- Simplifies mental model (fewer knobs to turn)
- Reliable behavior across all configurations
- If needed, user adjusts `max_relaxation_attempts` (fewer/more steps)
3. **Why warn at 25% base flex?**
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
- At 25% base, first relaxation step reaches 28%
- Above 30%, entering diminishing returns territory
- User likely doesn't need relaxation with such high base flex
- Should either: (a) lower base flex, or (b) disable relaxation
**Historical Context (Pre-November 2025):**
The algorithm previously used percentage-based increments that scaled with base flex:
```python
increment = base_flex × (step_pct / 100) # REMOVED
```
@ -454,6 +496,7 @@ increment = base_flex × (step_pct / 100) # REMOVED
This caused exponential escalation with high base flex values (e.g., 40% → 50% → 60% → 70% in just 6 steps), making behavior unpredictable. The fixed 3% increment solves this by providing consistent, controlled escalation regardless of starting point.
**Warning Messages:**
```python
if base_flex >= FLEX_HIGH_THRESHOLD_RELAXATION: # 30%
_LOGGER.warning(
@ -472,12 +515,14 @@ elif base_flex >= FLEX_WARNING_THRESHOLD_RELAXATION: # 25%
### Filter Combination Strategy
**Per Flex level, try in order:**
1. Original Level filter
2. Level filter = "any" (disabled)
**Early Exit:** Stop immediately when target reached (don't try unnecessary combinations)
**Example Flow (target=2 periods/day):**
```
Day 2025-11-19:
1. Baseline flex=15%: Found 1 period (need 2)
@ -492,6 +537,7 @@ Day 2025-11-19:
### Key Files and Functions
**Period Calculation Entry Point:**
```python
# coordinator/period_handlers/core.py
def calculate_periods(
@ -502,6 +548,7 @@ def calculate_periods(
```
**Flex + Distance Filtering:**
```python
# coordinator/period_handlers/level_filtering.py
def check_interval_criteria(
@ -511,6 +558,7 @@ def check_interval_criteria(
```
**Relaxation Orchestration:**
```python
# coordinator/period_handlers/relaxation.py
def calculate_periods_with_relaxation(...) -> tuple[dict, dict]
@ -526,43 +574,45 @@ def relax_single_day(...) -> tuple[dict, dict]
**Algorithm Details:**
1. **Linear Regression Prediction:**
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
- Uses surrounding intervals to predict expected price
- Window size: 3+ intervals (MIN_CONTEXT_SIZE)
- Calculates trend slope and standard deviation
- Formula: `predicted = mean + slope × (position - center)`
2. **Confidence Intervals:**
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
- 95% confidence level (2 standard deviations)
- Tolerance = 2.0 × std_dev (CONFIDENCE_LEVEL constant)
- Outlier if: `|actual - predicted| > tolerance`
- Accounts for natural price volatility in context window
3. **Symmetry Check:**
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
- Rejects asymmetric outliers (threshold: 1.5 std dev)
- Preserves legitimate price shifts (morning/evening peaks)
- Algorithm:
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
```python
residual = abs(actual - predicted)
symmetry_threshold = 1.5 × std_dev
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
if residual > tolerance:
# Check if spike is symmetric in context
context_residuals = [abs(p - pred) for p, pred in context]
avg_context_residual = mean(context_residuals)
if residual > symmetry_threshold × avg_context_residual:
# Asymmetric spike → smooth it
else:
# Symmetric (part of trend) → keep it
```
4. **Enhanced Zigzag Detection:**
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
- Detects spike clusters via relative volatility
- Threshold: 2.0× local volatility (RELATIVE_VOLATILITY_THRESHOLD)
- Single-pass algorithm (no iteration needed)
- Catches patterns like: 18, 35, 19, 34, 18 (alternating spikes)
**Constants:**
```python
# coordinator/period_handlers/outlier_filtering.py
@ -573,18 +623,21 @@ MIN_CONTEXT_SIZE = 3 # Minimum intervals for regression
```
**Data Integrity:**
- Original prices stored in `_original_price` field
- All statistics (daily min/max/avg) use original prices
- Smoothing only affects period formation logic
- Smart counting: Only counts smoothing that changed period outcome
**Performance:**
- Single pass through price data
- O(n) complexity with small context window
- No iterative refinement needed
- Typical processing time: `<`1ms for 96 intervals
**Example Debug Output:**
```
DEBUG: [2025-11-11T14:30:00+01:00] Outlier detected: 35.2 ct
DEBUG: Context: 18.5, 19.1, 19.3, 19.8, 20.2 ct
@ -598,19 +651,19 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
**Why This Approach?**
1. **Linear regression over moving average:**
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
- Accounts for price trends (morning ramp-up, evening decline)
- Moving average can't predict direction, only level
- Better accuracy on non-stationary price curves
2. **Symmetry check over fixed threshold:**
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
- Prevents false positives on legitimate price shifts
- Adapts to local volatility patterns
- Preserves user expectation: "expensive during peak hours"
3. **Single-pass over iterative:**
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
- Predictable behavior (no convergence issues)
- Fast and deterministic
- Easier to debug and reason about
**Alternative Approaches Considered:**
@ -624,15 +677,17 @@ DEBUG: Asymmetry ratio: 3.2 (>1.5 threshold) → confirmed outlier
## Debugging Tips
**Enable DEBUG logging:**
```yaml
# configuration.yaml
logger:
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
default: info
logs:
custom_components.tibber_prices.coordinator.period_handlers: debug
```
**Key log messages to watch:**
1. `"Filter statistics: X intervals checked"` - Shows how many intervals filtered by each criterion
2. `"After build_periods: X raw periods found"` - Periods before min_length filtering
3. `"Day X: Success with flex=Y%"` - Relaxation succeeded
@ -645,52 +700,61 @@ logger:
### ❌ Anti-Pattern 1: High Flex with Relaxation
**Configuration:**
```yaml
best_price_flex: 40
enable_relaxation_best: true
```
**Problem:**
- Base Flex 40% already very permissive
- Relaxation increments further (43%, 46%, 49%, ...)
- Quickly approaches 50% cap with diminishing returns
**Solution:**
```yaml
best_price_flex: 15 # Let relaxation increase it
best_price_flex: 15 # Let relaxation increase it
enable_relaxation_best: true
```
### ❌ Anti-Pattern 2: Zero Min_Distance
**Configuration:**
```yaml
best_price_min_distance_from_avg: 0
```
**Problem:**
- "Flat days" (little price variation) accept all intervals
- Periods lose semantic meaning ("significantly cheap")
- May create periods during barely-below-average times
**Solution:**
```yaml
best_price_min_distance_from_avg: 5 # Use default 5%
best_price_min_distance_from_avg: 5 # Use default 5%
```
### ❌ Anti-Pattern 3: Conflicting Flex + Distance
**Configuration:**
```yaml
best_price_flex: 45
best_price_min_distance_from_avg: 10
```
**Problem:**
- Distance filter dominates, making Flex irrelevant
- Dynamic scaling helps but still suboptimal
**Solution:**
```yaml
best_price_flex: 20
best_price_min_distance_from_avg: 5
@ -706,11 +770,13 @@ best_price_min_distance_from_avg: 5
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: Should find 2-4 clear best price periods
- Flex 30%: Should find 4-8 periods (more lenient)
- Min_Distance 5%: Effective throughout range
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 12/96 (12.5%) ← Low percentage = good variation
@ -724,11 +790,13 @@ DEBUG: After build_periods: 3 raw periods found
**Average:** 15 ct/kWh
**Expected Behavior:**
- Flex 15%: May find 1-2 small periods (or zero if no clear winners)
- Min_Distance 5%: Critical here - ensures only truly cheaper intervals qualify
- Without Min_Distance: Would accept almost entire day as "best price"
**Debug Checks:**
```
DEBUG: Filter statistics: 96 intervals checked
DEBUG: Filtered by FLEX: 45/96 (46.9%) ← High percentage = poor variation
@ -743,11 +811,13 @@ DEBUG: Day 2025-11-11: Baseline insufficient (1 < 2), starting relaxation
**Average:** 18 ct/kWh
**Expected Behavior:**
- Flex 15%: Finds multiple very cheap periods (5-6 ct)
- Outlier filtering: May smooth isolated spikes (30-40 ct)
- Distance filter: Less impactful (clear separation between cheap/expensive)
**Debug Checks:**
```
DEBUG: Outlier detected: 38.5 ct (threshold: 4.2 ct)
DEBUG: Smoothed to: 20.1 ct (trend prediction)
@ -762,6 +832,7 @@ DEBUG: After build_periods: 4 raw periods found
**Initial State:** Baseline finds 1 period, target is 2
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 1 period (need 2)
@ -777,6 +848,7 @@ INFO: Day 2025-11-11: Success after 1 relaxation phase (2 periods)
**Initial State:** Strict filters, very flat day
**Expected Flow:**
```
INFO: Calculating BEST PRICE periods: relaxation=ON, target=2/day, flex=15.0%
DEBUG: Day 2025-11-11: Baseline found 0 periods (need 2)
@ -790,31 +862,31 @@ INFO: Period calculation completed: 1/2 days reached target
When debugging period calculation issues:
1. **Check Filter Statistics**
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
- Which filter blocks most intervals? (flex, distance, or level)
- High flex filtering (>30%) = Need more flexibility or relaxation
- High distance filtering (>50%) = Min_distance too strict or flat day
- High level filtering = Level filter too restrictive
2. **Check Relaxation Behavior**
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
- Did relaxation activate? Check for "Baseline insufficient" message
- Which phase succeeded? Early success (phase 1-3) = good config
- Late success (phase 8-11) = Consider adjusting base config
- Exhausted all phases = Unrealistic target for this day's price curve
3. **Check Flex Warnings**
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
- INFO at 25% base flex = On the high side
- WARNING at 30% base flex = Too high for relaxation
- If seeing these: Lower base flex to 15-20%
4. **Check Min_Distance Scaling**
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
- Debug messages show "High flex X% detected: Reducing min_distance Y% → Z%"
- If scale factor `<`0.8 (20% reduction): High flex is active
- If periods still not found: Filters conflict even with scaling
5. **Check Outlier Filtering**
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
- Look for "Outlier detected" messages
- Check `period_interval_smoothed_count` attribute
- If no smoothing but periods fragmented: Not isolated spikes, but legitimate price levels
---
@ -823,19 +895,19 @@ When debugging period calculation issues:
### Potential Improvements
1. **Adaptive Flex Calculation:**
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
- Auto-adjust Flex based on daily price variation
- High variation days: Lower Flex needed
- Low variation days: Higher Flex needed
2. **Machine Learning Approach:**
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn optimal Flex/Distance from user feedback
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
3. **Multi-Objective Optimization:**
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
- Balance period count vs. quality
- Consider period duration vs. price level
- Optimize for user's stated use case (EV charging vs. heat pump)
### Known Limitations
@ -854,6 +926,7 @@ When debugging period calculation issues:
**Concept:** Auto-adjust Flex based on daily price variation
**Algorithm:**
```python
# Pseudo-code for adaptive flex
variation = (daily_max - daily_min) / daily_avg
@ -867,11 +940,13 @@ else: # Normal day
```
**Benefits:**
- Eliminates need for relaxation on most days
- Self-adjusting to market conditions
- Better user experience (less configuration needed)
**Challenges:**
- Harder to predict behavior (less transparent)
- May conflict with user's mental model
- Needs extensive testing across different markets
@ -883,17 +958,20 @@ else: # Normal day
**Concept:** Learn optimal Flex/Distance from user feedback
**Approach:**
- Track which periods user actually uses (automation triggers)
- Classify days by pattern (normal/flat/volatile/bimodal)
- Apply pattern-specific defaults
- Learn per-user preferences over time
**Benefits:**
- Personalized to user's actual behavior
- Adapts to local market patterns
- Could discover non-obvious patterns
**Challenges:**
- Requires user feedback mechanism (not implemented)
- Privacy concerns (storing usage patterns)
- Complexity for users to understand "why this period?"
@ -906,22 +984,26 @@ else: # Normal day
**Concept:** Balance multiple goals simultaneously
**Goals:**
- Period count vs. quality (cheap vs. very cheap)
- Period duration vs. price level (long mediocre vs. short excellent)
- Temporal distribution (spread throughout day vs. clustered)
- User's stated use case (EV charging vs. heat pump vs. dishwasher)
**Algorithm:**
- Pareto optimization (find trade-off frontier)
- User chooses point on frontier via preferences
- Genetic algorithm or simulated annealing
**Benefits:**
- More sophisticated period selection
- Better match to user's actual needs
- Could handle complex appliance requirements
**Challenges:**
- Much more complex to implement
- Harder to explain to users
- Computational cost (may need caching)
@ -936,14 +1018,17 @@ else: # Normal day
**Current:** 3% cap may be too aggressive for very low base Flex
**Example:**
- Base flex 5% + 3% increment = 8% (60% increase!)
- Base flex 15% + 3% increment = 18% (20% increase)
**Possible Solution:**
- Percentage-based increment: `increment = max(base_flex × 0.20, 0.03)`
- This gives: 5% → 6% (20%), 15% → 18% (20%), 40% → 43% (7.5%)
**Why Not Implemented:**
- Very low base flex (`<`10%) unusual
- Users with strict requirements likely disable relaxation
- Simplicity preferred over edge case optimization
@ -953,6 +1038,7 @@ else: # Normal day
**Current:** Linear scaling may be too aggressive/conservative
**Alternative:** Non-linear curve
```python
# Example: Exponential scaling
scale_factor = 0.25 + 0.75 × exp(-5 × (flex - 0.20))
@ -962,6 +1048,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
```
**Why Not Implemented:**
- Linear is easier to reason about
- No evidence that non-linear is better
- Would need extensive testing
@ -971,15 +1058,18 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Issue:** May find all periods in one part of day
**Example:**
- All 3 "best price" periods between 02:00-08:00
- No periods in evening (when user might want to run appliances)
**Possible Solution:**
- Add "spread" parameter (prefer distributed periods)
- Weight periods by time-of-day preferences
- Consider user's typical usage patterns
**Why Not Implemented:**
- Adds complexity
- Users can work around with multiple automations
- Different users have different needs (no one-size-fits-all)
@ -991,6 +1081,7 @@ scale_factor = 0.25 + 0.75 / (1 + exp(10 × (flex - 0.35)))
**Design Principle:** Each interval is evaluated using its **own day's** reference prices (daily min/max/avg).
**Implementation:**
```python
# In period_building.py build_periods():
for price_data in all_prices:
@ -1042,6 +1133,7 @@ Period crossing midnight: 23:45 Day 1 → 00:15 Day 2
**Trade-off: Periods May Break at Midnight**
When days differ significantly, period can split:
```
Day 1: Min=10ct, Avg=20ct, 23:45=11ct → ✅ Cheap (relative to Day 1)
Day 2: Min=25ct, Avg=35ct, 00:00=21ct → ❌ Expensive (relative to Day 2)
@ -1053,6 +1145,7 @@ This is **mathematically correct** - 21ct is genuinely expensive on a day where
**Market Reality Explains Price Jumps:**
Day-ahead electricity markets (EPEX SPOT) set prices at 12:00 CET for all next-day hours:
- Late intervals (23:45): Priced ~36h before delivery → high forecast uncertainty → risk premium
- Early intervals (00:00): Priced ~12h before delivery → better forecasts → lower risk buffer
@ -1061,15 +1154,17 @@ This explains why absolute prices jump at midnight despite minimal demand change
**User-Facing Solution (Nov 2025):**
Added per-period day volatility attributes to detect when classification changes are meaningful:
- `day_volatility_%`: Percentage spread (span/avg × 100)
- `day_price_min`, `day_price_max`, `day_price_span`: Daily price range (ct/øre)
Automations can check volatility before acting:
```yaml
condition:
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
- condition: template
value_template: >
{{ state_attr('binary_sensor.tibber_home_best_price_period', 'day_volatility_%') | float(0) > 15 }}
```
Low volatility (< 15%) means classification changes are less economically significant.
@ -1077,24 +1172,25 @@ Low volatility (< 15%) means classification changes are less economically signif
**Alternative Approaches Rejected:**
1. **Use period start day for all intervals**
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
- Problem: Mathematically incorrect - lends cheap day's criteria to expensive day
- Rejected: Violates relative evaluation principle
2. **Adjust flex/distance at midnight**
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
- Problem: Complex, unpredictable, hides market reality
- Rejected: Users should understand price context, not have it hidden
3. **Split at midnight always**
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
- Problem: Artificially fragments natural periods
- Rejected: Worse user experience
4. **Use next day's reference after midnight**
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
- Problem: Period criteria inconsistent across duration
- Rejected: Confusing and unpredictable
**Status:** Per-day evaluation is intentional design prioritizing mathematical correctness.
**See Also:**
- User documentation: `docs/user/docs/period-calculation.md` → "Midnight Price Classification Changes"
- Implementation: `coordinator/period_handlers/period_building.py` (line ~126: `ref_date = date_key`)
- Attributes: `coordinator/period_handlers/period_statistics.py` (day volatility calculation)

72
docs/developer/versioned_docs/version-v0.23.1/recorder-optimization.md
View file

@ -29,6 +29,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
```
**Key Points:**
- Must be a **class attribute** (not instance attribute)
- Use `frozenset` for immutability and performance
- Applied automatically by Home Assistant's Recorder component
@ -40,6 +41,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `description`, `usage_tips`
**Reason:** Static, large text strings (100-500 chars each) that:
- Never change or change very rarely
- Don't provide analytical value in history
- Consume significant database space when recorded every state change
@ -50,6 +52,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
### 2. Large Nested Structures
**Attributes:**
- `periods` (binary_sensor) - Array of all period summaries
- `data` (chart_data_export) - Complete price data arrays
- `trend_attributes` - Detailed trend analysis
@ -58,6 +61,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
- `volatility_attributes` - Detailed volatility breakdown
**Reason:** Complex nested data structures that are:
- Serialized to JSON for storage (expensive)
- Create large database rows (2-20 KB each)
- Slow down history queries
@ -66,20 +70,21 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Impact:** ~10-30 KB saved per state change for affected sensors
**Example - periods array:**
```json
{
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8,
// ... 10+ more attributes × 10-20 periods
}
]
"periods": [
{
"start": "2025-12-07T06:00:00+01:00",
"end": "2025-12-07T08:00:00+01:00",
"duration_minutes": 120,
"price_mean": 18.5,
"price_median": 18.3,
"price_min": 17.2,
"price_max": 19.8
// ... 10+ more attributes × 10-20 periods
}
]
}
```
@ -88,6 +93,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `icon_color`, `cache_age`, `cache_validity`, `data_completeness`, `data_status`
**Reason:**
- Change every update cycle (every 15 minutes or more frequently)
- Don't provide long-term analytical value
- Create state changes even when core values haven't changed
@ -103,6 +109,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `tomorrow_expected_after`, `level_value`, `rating_value`, `level_id`, `rating_id`, `currency`, `resolution`, `yaxis_min`, `yaxis_max`
**Reason:**
- Configuration values that rarely change
- Wastes space when recorded repeatedly
- Can be derived from other attributes or from entity state
@ -114,6 +121,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `next_api_poll`, `next_midnight_turnover`, `last_api_fetch`, `last_cache_update`, `last_turnover`, `last_error`, `error`
**Reason:**
- Only relevant at moment of reading
- Won't be valid after some time
- Similar to `entity_picture` in HA core image entities
@ -128,6 +136,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `relaxation_level`, `relaxation_threshold_original_%`, `relaxation_threshold_applied_%`
**Reason:**
- Detailed technical information not needed for historical analysis
- Only useful for debugging during active development
- Boolean `relaxation_active` is kept for high-level analysis
@ -139,6 +148,7 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
**Attributes:** `price_spread`, `volatility`, `diff_%`, `rating_difference_%`, `period_price_diff_from_daily_min`, `period_price_diff_from_daily_min_%`, `periods_total`, `periods_remaining`
**Reason:**
- Can be calculated from other attributes
- Redundant information
- Doesn't add analytical value to history
@ -152,22 +162,27 @@ class TibberPricesSensor(TibberPricesEntity, SensorEntity):
These attributes **remain in history** because they provide essential analytical value:
### Time-Series Core
- `timestamp` - Critical for time-series analysis (ALWAYS FIRST)
- All price values - Core sensor states
### Diagnostics & Tracking
- `cache_age_minutes` - Numeric value for diagnostics tracking over time
- `updates_today` - Tracking API usage patterns
### Data Completeness
- `interval_count`, `intervals_available` - Data completeness metrics
- `yesterday_available`, `today_available`, `tomorrow_available` - Boolean status
### Period Data
- `start`, `end`, `duration_minutes` - Core period timing
- `price_mean`, `price_median`, `price_min`, `price_max` - Core price statistics
### High-Level Status
- `relaxation_active` - Whether relaxation was used (boolean, useful for analyzing when periods needed relaxation)
## Expected Database Impact
@ -175,6 +190,7 @@ These attributes **remain in history** because they provide essential analytical
### Space Savings
**Per state change:**
- Before: ~3-8 KB average
- After: ~0.5-1.5 KB average
- **Reduction: 60-85%**
@ -196,6 +212,7 @@ These attributes **remain in history** because they provide essential analytical
### Real-World Impact
For a typical installation with:
- 80+ sensors
- Updates every 15 minutes
- ~10 sensors updating every minute
@ -207,14 +224,14 @@ For a typical installation with:
## Implementation Files
- **Sensor Platform**: `custom_components/tibber_prices/sensor/core.py`
- Class: `TibberPricesSensor`
- 47 attributes excluded
- Class: `TibberPricesSensor`
- 47 attributes excluded
- **Binary Sensor Platform**: `custom_components/tibber_prices/binary_sensor/core.py`
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
- Class: `TibberPricesBinarySensor`
- 30 attributes excluded
## When to Update _unrecorded_attributes
## When to Update \_unrecorded_attributes
### Add to Exclusion List When:
@ -236,24 +253,24 @@ For a typical installation with:
When adding a new attribute, ask:
1. **Will this be useful in history queries 1 week from now?**
- No → Exclude
- Yes → Keep
- No → Exclude
- Yes → Keep
2. **Can this be calculated from other recorded attributes?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
3. **Is this primarily for current UI display?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
4. **Does this change frequently without indicating state change?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
5. **Is this larger than 100 bytes and not essential for analysis?**
- Yes → Exclude
- No → Keep
- Yes → Exclude
- No → Keep
## Testing
@ -265,6 +282,7 @@ After modifying `_unrecorded_attributes`:
4. **Confirm excluded attributes** don't appear in new state writes
**SQL Query to check attribute presence:**
```sql
SELECT
state_id,

200
docs/developer/versioned_docs/version-v0.23.1/refactoring-guide.md
View file

@ -8,22 +8,22 @@ Not every code change needs a detailed plan. Create a refactoring plan when:
🔴 **Major changes requiring planning:**
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
- Splitting modules into packages (>5 files affected, >500 lines moved)
- Architectural changes (new packages, module restructuring)
- Breaking changes (API changes, config format migrations)
🟡 **Medium changes that might benefit from planning:**
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
- Complex features with multiple moving parts
- Changes affecting many files (>3 files, unclear best approach)
- Refactorings with unclear scope
🟢 **Small changes - no planning needed:**
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
- Bug fixes (straightforward, `<`100 lines)
- Small features (`<`3 files, clear approach)
- Documentation updates
- Cosmetic changes (formatting, renaming)
## The Planning Process
@ -51,34 +51,34 @@ Every planning document should include:
## Problem Statement
- What's the issue?
- Why does it need fixing?
- Current pain points
- What's the issue?
- Why does it need fixing?
- Current pain points
## Proposed Solution
- High-level approach
- File structure (before/after)
- Module responsibilities
- High-level approach
- File structure (before/after)
- Module responsibilities
## Migration Strategy
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
- Phase-by-phase breakdown
- File lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Dependencies between phases
- Testing checkpoints
## Risks & Mitigation
- What could go wrong?
- How to prevent it?
- Rollback strategy
- What could go wrong?
- How to prevent it?
- Rollback strategy
## Success Criteria
- Measurable improvements
- Testing requirements
- Verification steps
- Measurable improvements
- Testing requirements
- Verification steps
```
See `planning/README.md` for detailed template explanation.
@ -87,19 +87,19 @@ See `planning/README.md` for detailed template explanation.
Since `planning/` is git-ignored:
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
- Draft multiple versions
- Get AI assistance without commit pressure
- Refine until the plan is solid
- No need to clean up intermediate versions
### 4. Implementation Phase
Once plan is approved:
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
- Follow the phases defined in the plan
- Test after each phase (don't skip!)
- Update plan if issues discovered
- Track progress through phase status
### 5. After Completion
@ -134,13 +134,13 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Before:**
- `sensor.py` - 2,574 lines, hard to navigate
- `sensor.py` - 2,574 lines, hard to navigate
**After:**
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
- `sensor/` package with 5 focused modules
- Each module `<`800 lines
- Clear separation of concerns
**Process:**
@ -153,10 +153,10 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
**Key learnings:**
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
- Temporary `_impl.py` files avoid Python package conflicts
- Test after EVERY phase (don't accumulate changes)
- Clear file lifecycle (CREATE/MODIFY/DELETE/RENAME)
- Phase-by-phase approach enables safe rollback
**Note:** The complete module splitting plan was documented during implementation but has been superseded by the actual code structure.
@ -166,11 +166,11 @@ The **sensor/ package refactoring** (Nov 2025) is a successful example:
Breaking refactorings into phases:
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
- ✅ Enables testing after each change (catch bugs early)
- ✅ Allows rollback to last good state
- ✅ Makes progress visible
- ✅ Reduces cognitive load (focus on one thing)
- ❌ Takes more time (but worth it!)
### Phase Structure
@ -191,8 +191,8 @@ Each phase should:
**File Lifecycle**:
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
- ✨ CREATE `sensor/helpers.py` (utility functions)
- ✏️ MODIFY `sensor/core.py` (import from helpers.py)
**Steps**:
@ -205,10 +205,10 @@ Each phase should:
**Success criteria**:
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
- ✅ All pure functions moved
- ✅ `./scripts/lint-check` passes
- ✅ HA starts successfully
- ✅ All entities work correctly
```
## Testing Strategy
@ -238,13 +238,13 @@ Minimum testing checklist:
After completing all phases:
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
- Test all entities (sensors, binary sensors)
- Test configuration flow (add/modify/remove)
- Test options flow (change settings)
- Test services (custom service calls)
- Test error handling (disconnect API, invalid data)
- Test caching (restart HA, verify cache loads)
- Test time-based updates (quarter-hour refresh)
## Common Pitfalls
@ -286,21 +286,21 @@ This project uses AI heavily (GitHub Copilot, Claude). The planning process supp
**AI reads from:**
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
- `AGENTS.md` - Long-term memory, patterns, conventions (AI-focused)
- `docs/development/` - Human-readable guides (human-focused)
- `planning/` - Active refactoring plans (shared context)
**AI updates:**
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
- `AGENTS.md` - When patterns change
- `planning/*.md` - During refactoring implementation
- `docs/development/` - After successful completion
**Why separate AGENTS.md and docs/development/?**
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
- `AGENTS.md`: Technical, comprehensive, AI-optimized
- `docs/development/`: Practical, focused, human-optimized
- Both stay in sync but serve different audiences
See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.1/AGENTS.md) section "Planning Major Refactorings" for AI-specific guidance.
@ -308,16 +308,16 @@ See [AGENTS.md](https://github.com/jpawlowski/hass.tibber_prices/blob/v0.23.1/AG
### Planning Directory
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
- `planning/` - Git-ignored workspace for drafts
- `planning/README.md` - Detailed planning documentation
- `planning/*.md` - Active refactoring plans
### Example Plans
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
- `docs/development/module-splitting-plan.md` - ✅ Completed, archived
- `planning/config-flow-refactoring-plan.md` - 🔄 Planned (1013 lines → 4 modules)
- `planning/binary-sensor-refactoring-plan.md` - 🔄 Planned (644 lines → 4 modules)
- `planning/coordinator-refactoring-plan.md` - 🔄 Planned (1446 lines, high complexity)
### Helper Scripts
@ -341,21 +341,21 @@ Simple rule: If you can't describe the entire change in 3 sentences, create a pl
Good plan level:
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
- Lists all files affected (CREATE/MODIFY/DELETE)
- Defines phases with clear boundaries
- Includes testing strategy
- Estimates time per phase
Too detailed:
- Exact code snippets for every change
- Line-by-line instructions
- Exact code snippets for every change
- Line-by-line instructions
Too vague:
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
- "Refactor sensor.py to be better"
- No phase breakdown
- No testing strategy
### Q: What if the plan changes during implementation?
@ -363,9 +363,9 @@ Too vague:
If you discover:
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
- Better approach → Update "Proposed Solution"
- More phases needed → Add to "Migration Strategy"
- New risks → Update "Risks & Mitigation"
Document WHY the plan changed (helps future refactorings).
@ -373,9 +373,9 @@ Document WHY the plan changed (helps future refactorings).
**A:** No! Use judgment:
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
- **Small changes (`<`100 lines, clear approach)**: Just do it, no plan needed
- **Medium changes (unclear scope)**: Write rough outline, refine if needed
- **Large changes (>500 lines, >5 files)**: Full planning process
### Q: How do I know when a refactoring is successful?
@ -383,12 +383,12 @@ Document WHY the plan changed (helps future refactorings).
Typical criteria:
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
- ✅ All linting checks pass
- ✅ HA starts without errors
- ✅ All entities functional
- ✅ No regressions (existing features work)
- ✅ Code easier to understand/modify
- ✅ Documentation updated
If you can't tick all boxes, the refactoring isn't done.
@ -409,6 +409,6 @@ If you can't tick all boxes, the refactoring isn't done.
**Next steps:**
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans
- Read `planning/README.md` for detailed template
- Check `docs/development/module-splitting-plan.md` for real example
- Browse `planning/` for active refactoring plans

Some files were not shown because too many files have changed in this diff Show more