Cache validity now checks _last_coordinator_update (within 30min)
instead of _api_calls_today counter. Fixes false "stale" status
when coordinator runs every 15min but cache validation was only
checking API call counter.
Bug #1: Cache validity shows "stale" at 05:57 AM
Bug #2: Cache age calculation incorrect after midnight turnover
Bug #3: get_cache_validity inconsistent with cache_age sensor
Changes:
- Coordinator: Use _last_coordinator_update for cache validation
- Lifecycle: Extract cache validation to dedicated helper function
- Tests: 7 new tests covering midnight scenarios and edge cases
Impact: Cache validity sensor now accurately reflects coordinator
activity, not just explicit API calls. Correctly handles midnight
turnover without false "stale" status.
Set up pytest with Home Assistant support and created 6 tests for
midnight-crossing period logic (5 unit tests + 1 integration test).
Added pytest configuration, test dependencies, test runner script
(./scripts/test), and comprehensive tests for group_periods_by_day()
and midnight turnover consistency.
All tests pass in 0.12s.
Impact: Provides regression testing for midnight-crossing period bugs.
Tests validate periods remain visible across midnight turnover.
Periods can now naturally cross midnight boundaries, and new diagnostic
attributes help users understand price classification changes at midnight.
**New Features:**
1. Midnight-Crossing Period Support (relaxation.py):
- group_periods_by_day() assigns periods to ALL spanned days
- Periods crossing midnight appear in both yesterday and today
- Enables period formation across calendar day boundaries
- Ensures min_periods checking works correctly at midnight
2. Extended Price Data Window (relaxation.py):
- Period calculation now uses full 3-day data (yesterday+today+tomorrow)
- Enables natural period formation without artificial midnight cutoff
- Removed date filter that excluded yesterday's prices
3. Day Volatility Diagnostic Attributes (period_statistics.py, core.py):
- day_volatility_%: Daily price spread as percentage (span/avg × 100)
- day_price_min/max/span: Daily price range in minor currency (ct/øre)
- Helps detect when midnight classification changes are economically significant
- Uses period start day's reference prices for consistency
**Documentation:**
4. Design Principles (period-calculation-theory.md):
- Clarified per-day evaluation principle (always was the design)
- Added comprehensive section on midnight boundary handling
- Documented volatility threshold separation (sensor vs period filters)
- Explained market context for midnight price jumps (EPEX SPOT timing)
5. User Guides (period-calculation.md, automation-examples.md):
- Added \"Midnight Price Classification Changes\" troubleshooting section
- Provided automation examples using volatility attributes
- Explained why Best→Peak classification can change at midnight
- Documented level filter volatility threshold behavior
**Architecture:**
- Per-day evaluation: Each interval evaluated against its OWN day's min/max/avg
(not period start day) ensures mathematical correctness across midnight
- Period boundaries: Periods can naturally cross midnight but may split when
consecutive days differ significantly (intentional, mathematically correct)
- Volatility thresholds: Sensor thresholds (user-configurable) remain separate
from period filter thresholds (fixed internal) to prevent unexpected behavior
Impact: Periods crossing midnight are now consistently visible before and
after midnight turnover. Users can understand and handle edge cases where
price classification changes at midnight on low-volatility days.
Synchronized coordinator._cached_price_data after API calls to ensure tomorrow data is available for sensor value calculations and lifecycle state detection.
Impact: Tomorrow sensors now display values correctly after afternoon data fetch. Lifecycle sensor status remains stable without flickering between "searching_tomorrow" and other states.
Restored mark_periods_with_relaxation() function and added call in
relax_all_prices() to properly mark periods found through relaxation.
Problem: Periods found via relaxation were missing metadata attributes:
- relaxation_active
- relaxation_level
- relaxation_threshold_original_%
- relaxation_threshold_applied_%
These attributes are expected by:
- period_overlap.py: For merging periods with correct relaxation info
- binary_sensor/attributes.py: For displaying relaxation info to users
Implementation:
- Added reverse_sort parameter to preserve sign semantics
- For Best Price: Store positive thresholds (e.g., +15%, +18%)
- For Peak Price: Store negative thresholds (e.g., -20%, -23%)
- Mark periods immediately after calculate_periods() and before
resolve_period_overlaps() so metadata is preserved during merging
Impact: Users can now see which periods were found through relaxation
and at what flex threshold. Peak Price periods show negative thresholds
matching the user's configuration semantics (negative = below maximum).
Changed description attribute behavior from "add separate long_description
attribute" to "switch description content" when CONF_EXTENDED_DESCRIPTIONS
is enabled.
OLD: description always shown, long_description added as separate attribute
NEW: description content switches between short and long based on config
Implementation:
- Check extended_descriptions flag BEFORE loading translation
- Load "long_description" key if enabled, fallback to "description" if missing
- Assign loaded content to "description" attribute (same key always)
- usage_tips remains separate attribute (only when extended=true)
- Updated both sync (entities) and async (services) versions
Added PLR0912 noqa: Branch complexity justified by feature requirements
(extended check + fallback logic + position handling).
Impact: Users see more detailed descriptions when extended mode enabled,
without attribute clutter. Fallback ensures robustness if long_description
missing in translations.
Fixed uninitialized self.time attribute causing AttributeError during
config entry creation. Added explicit initialization to None with
Optional type annotation and guard in _get_price_info_for_specific_homes().
Impact: Config flow no longer crashes when creating initial config entry.
Users can complete setup without errors.
- Added new validation functions for various parameters including flexibility percentage, distance percentage, minimum periods, gap count, relaxation attempts, price rating thresholds, volatility threshold, and price trend thresholds.
- Updated constants in `const.py` to define maximum and minimum limits for the new validation criteria.
- Improved error messages in translations for invalid parameters to provide clearer guidance to users.
- Adjusted existing validation functions to ensure they align with the new constants and validation logic.
Add comprehensive data_lifecycle_status sensor showing real-time cache
vs fresh API data status with 6 states and 13+ detailed attributes.
Key features:
- 6 lifecycle states: cached, fresh, refreshing, searching_tomorrow,
turnover_pending, error
- Push-update system for instant state changes (refreshing→fresh→error)
- Quarter-hour polling for turnover_pending detection at 23:45
- Accurate next_api_poll prediction using Timer #1 offset tracking
- Tomorrow prediction with actual timer schedule (not fixed 13:00)
- 13+ formatted attributes: cache_age, data_completeness, api_calls_today,
next_api_poll, etc.
Implementation:
- sensor/calculators/lifecycle.py: New calculator with state logic
- sensor/attributes/lifecycle.py: Attribute builders with formatting
- coordinator/core.py: Lifecycle tracking + callback system (+16 lines)
- sensor/core.py: Push callback registration (+3 lines)
- coordinator/constants.py: Added to TIME_SENSITIVE_ENTITY_KEYS
- Translations: All 5 languages (de, en, nb, nl, sv)
Timing optimization:
- Extended turnover warning: 5min → 15min (catches 23:45 quarter boundary)
- No minute-timer needed: quarter-hour updates + push = optimal
- Push-updates: <1sec latency for refreshing/fresh/error states
- Timer offset tracking: Accurate tomorrow predictions
Removed obsolete sensors:
- data_timestamp (replaced by lifecycle attributes)
- price_forecast (never implemented, removed from definitions)
Impact: Users can monitor data freshness, API call patterns, cache age,
and understand integration behavior. Perfect for troubleshooting and
visibility into when data updates occur.
Moved Chart Data Export sensor configuration from config flow textarea
to configuration.yaml for better maintainability and consistency with
Home Assistant standards.
Changes:
- __init__.py: Added async_setup() with CONFIG_SCHEMA for tibber_prices.chart_export
- const.py: Added DATA_CHART_CONFIG constant for hass.data storage
- options_flow.py: Simplified chart_data_export step to info-only page
- schemas.py: get_chart_data_export_schema() returns empty schema (no input fields)
- sensor/chart_data.py: Reads config from hass.data instead of config_entry.options
- All 5 translation files: Updated chart_data_export description with:
- Clear heading: "📊 Chart Data Export Sensor"
- Intro line explaining sensor purpose
- Legacy warning (⚠️) recommending service use
- Two valid use cases (✅): attribute-only tools, auto-updating data
- One discouraged use case (❌): automations should use service directly
- 3-step activation instructions
- YAML configuration example with all parameters
- Correct default behavior: today+tomorrow, 15-minute intervals, prices only
Impact: Users configure chart export in configuration.yaml instead of UI.
Sensor remains disabled by default (diagnostic sensor). Config flow shows
prominent info page guiding users toward service usage while keeping
sensor available for legacy dashboard tools that only read attributes.
Standardized config flow translations (nb, nl, sv) to match German/English
format with minimal field labels and comprehensive data_descriptions.
Changes across Norwegian, Dutch, and Swedish translations:
- Updated step_progress format: **{step_progress}** → _{step_progress}_
- Made all step descriptions bold with **text** formatting
- Simplified field labels (removed verbose explanations)
- Added data_description for price_rating (low/high thresholds)
- Added data_description for price_trend (rising/falling thresholds)
- Added data_description for volatility (moderate/high/very high thresholds)
- Ensured all steps have: emojis, italic step_progress, separator (---)
- Added missing emoji to Swedish price_rating step (📊)
Impact: All 5 languages now have consistent UX with minimal, scannable
field labels and detailed optional descriptions accessible via ⓘ icon.
Users get cleaner config flow with better clarity.
When adding a new integration (no existing cache), metadata sensors
(grid_company, estimated_annual_consumption, etc.) were marked as
unavailable because coordinator._cached_user_data remained None even
after successful API call.
Root cause: update_user_data_if_needed() stored user data in
_data_fetcher.cached_user_data, but the sync back to coordinator
only happened during _load_cache() (before the API call).
Solution: Added explicit sync of cached_user_data after
handle_main_entry_update() completes, ensuring metadata is available
when sensors first access get_user_homes().
Changes:
- coordinator/core.py: Sync _cached_user_data after main entry update
- __init__.py: Kept preload cache call (helps with HA restarts)
Impact: Metadata sensors now show values immediately on fresh integration
setup, without requiring a second update cycle or manual sensor activation.
BREAKING CHANGE: Period overlap resolution now merges adjacent/overlapping periods
instead of marking them as extensions. This simplifies automation logic and provides
clearer period boundaries for users.
Previous Behavior:
- Adjacent periods created by relaxation were marked with is_extension=true
- Multiple short periods instead of one continuous period
- Complex logic needed to determine actual period length in automations
New Behavior:
- Adjacent/overlapping periods are merged into single continuous periods
- Newer period's relaxation attributes override older period's
- Simpler automation: one period = one continuous time window
Changes:
- Period Overlap Resolution (new file: period_overlap.py):
* Added merge_adjacent_periods() to combine periods and preserve attributes
* Rewrote resolve_period_overlaps() with simplified merge logic
* Removed split_period_by_overlaps() (no longer needed)
* Removed is_extension marking logic
* Removed unused parameters: min_period_length, baseline_periods
- Relaxation Strategy (relaxation.py):
* Removed all is_extension filtering from period counting
* Simplified standalone counting to just len(periods)
* Changed from period_merging import to period_overlap import
* Added MAX_FLEX_HARD_LIMIT constant (0.50)
* Improved debug logging for merged periods
- Code Quality:
* Fixed all remaining linter errors (N806, PLR2004, PLR0912)
* Extracted magic values to module-level constants:
- FLEX_SCALING_THRESHOLD = 0.20
- SCALE_FACTOR_WARNING_THRESHOLD = 0.8
- MAX_FLEX_HARD_LIMIT = 0.50
* Added appropriate noqa comments for unavoidable patterns
- Configuration (from previous work in this session):
* Removed CONF_RELAXATION_STEP_BEST, CONF_RELAXATION_STEP_PEAK
* Hard-coded 3% relaxation increment for reliability
* Optimized defaults: RELAXATION_ATTEMPTS 8→11, ENABLE_MIN_PERIODS False→True,
MIN_PERIODS undefined→2
* Removed relaxation_step UI fields from config flow
* Updated all 5 translation files
- Documentation:
* Updated period_handlers/__init__.py: period_merging → period_overlap
* No user-facing docs changes needed (already described continuous periods)
Rationale - Period Merging:
User experience was complicated by fragmented periods:
- Automations had to check multiple adjacent periods
- Binary sensors showed ON/OFF transitions within same cheap time
- No clear way to determine actual continuous period length
With merging:
- One continuous cheap time = one period
- Binary sensor clearly ON during entire period
- Attributes show merge history via merged_from dict
- Relaxation info preserved from newest/highest flex period
Rationale - Hard-Coded Relaxation Increment:
The configurable relaxation_step parameter proved problematic:
- High base flex + high step → rapid explosion (40% base + 10% step → 100% in 6 steps)
- Users don't understand the multiplicative nature
- 3% increment provides optimal balance: 11 attempts to reach 50% hard cap
Impact:
- Existing installations: Periods may appear longer (merged instead of split)
- Automations benefit from simpler logic (no is_extension checks needed)
- Custom relaxation_step values will use new 3% increment
- Users may need to adjust relaxation_attempts if they relied on high step sizes
Introduce TimeService as single source of truth for all datetime operations,
replacing direct dt_util calls throughout the codebase. This establishes
consistent time context across update cycles and enables future time-travel
testing capability.
Core changes:
- NEW: coordinator/time_service.py with timezone-aware datetime API
- Coordinator now creates TimeService per update cycle, passes to calculators
- Timer callbacks (#2, #3) inject TimeService into entity update flow
- All sensor calculators receive TimeService via coordinator reference
- Attribute builders accept time parameter for timestamp calculations
Key patterns replaced:
- dt_util.now() → time.now() (single reference time per cycle)
- dt_util.parse_datetime() + as_local() → time.get_interval_time()
- Manual interval arithmetic → time.get_interval_offset_time()
- Manual day boundaries → time.get_day_boundaries()
- round_to_nearest_quarter_hour() → time.round_to_nearest_quarter()
Import cleanup:
- Removed dt_util imports from ~30 files (calculators, attributes, utils)
- Restricted dt_util to 3 modules: time_service.py (operations), api/client.py
(rate limiting), entity_utils/icons.py (cosmetic updates)
- datetime/timedelta only for TYPE_CHECKING (type hints) or duration arithmetic
Interval resolution abstraction:
- Removed hardcoded MINUTES_PER_INTERVAL constant from 10+ files
- New methods: time.minutes_to_intervals(), time.get_interval_duration()
- Supports future 60-minute resolution (legacy data) via TimeService config
Timezone correctness:
- API timestamps (startsAt) already localized by data transformation
- TimeService operations preserve HA user timezone throughout
- DST transitions handled via get_expected_intervals_for_day() (future use)
Timestamp ordering preserved:
- Attribute builders generate default timestamp (rounded quarter)
- Sensors override when needed (next interval, daily midnight, etc.)
- Platform ensures timestamp stays FIRST in attribute dict
Timer integration:
- Timer #2 (quarter-hour): Creates TimeService, calls _handle_time_sensitive_update(time)
- Timer #3 (30-second): Creates TimeService, calls _handle_minute_update(time)
- Consistent time reference for all entities in same update batch
Time-travel readiness:
- TimeService.with_reference_time() enables time injection (not yet used)
- All calculations use time.now() → easy to simulate past/future states
- Foundation for debugging period calculations with historical data
Impact: Eliminates timestamp drift within update cycles (previously 60+ independent
dt_util.now() calls could differ by milliseconds). Establishes architecture for
time-based testing and debugging features.
Created entity_utils/helpers.py with platform-agnostic utility functions:
- get_price_value(): Price unit conversion (major/minor currency)
- translate_level(): Price level translation
- translate_rating_level(): Rating level translation
- find_rolling_hour_center_index(): Rolling hour window calculations
These functions moved from sensor/helpers.py as they are used by both
sensor and binary_sensor platforms. Remaining sensor/helpers.py now
contains only sensor-specific helpers (aggregate_price_data, etc.).
Updated imports:
- sensor/core.py: Import from entity_utils instead of sensor.helpers
- entity_utils/icons.py: Fixed find_rolling_hour_center_index import
- binary_sensor platforms: Can now use shared helpers
Added clear docstrings explaining:
- entity_utils/helpers.py: Platform-agnostic utilities
- sensor/helpers.py: Sensor-specific aggregation functions
Impact: Better code reuse, clearer responsibility boundaries between
platform-specific and shared utilities.
This commit completes multiple refactoring efforts and documentation improvements:
Code Structure Changes:
- Move round_to_nearest_quarter_hour() from sensor/helpers.py to average_utils.py
- Resolve circular import between price_utils.py and sensor/helpers.py
- Split api.py into api/ package (client.py, queries.py, exceptions.py, helpers.py)
- Split coordinator.py into coordinator/ package (core.py, cache.py, listeners.py, etc.)
- Move period_utils/ to coordinator/period_handlers/ for better organization
- All lint checks passing (no PLC0415 local import warnings)
Documentation Additions:
- Add docs/development/architecture.md with Mermaid diagrams (end-to-end flow, cache coordination)
- Add docs/development/timer-architecture.md (comprehensive 3-timer system documentation)
- Add docs/development/caching-strategy.md (4-layer cache system with invalidation logic)
- Update docs/development/README.md with cross-references
- Update AGENTS.md with new module structure and patterns
Smart Boundary Tolerance:
- Implement ±2 second tolerance for quarter-hour rounding
- Prevents premature interval switching during HA restarts (14:59:30 stays at 14:45)
- Enables boundary snapping for timer jitter (14:59:58 → 15:00)
Atomic Midnight Coordination:
- Add _check_midnight_turnover_needed() for race-free midnight handling
- Coordinate Timer #1 (HA DataUpdateCoordinator) with Timer #2 (quarter-hour refresh)
- Whoever runs first performs turnover, other skips gracefully
Timer Optimization:
- Change timer scheduling from second=1 to second=0 (absolute-time scheduling)
- Document load distribution rationale (unsynchronized API polling prevents thundering herd)
- Comprehensive explanation of 3 independent timers and their coordination
Impact: Cleaner code structure with resolved circular dependencies, comprehensive
documentation of timer and caching systems, and improved reliability during
boundary conditions and midnight turnovers. All changes are developer-facing
improvements with no user-visible behavior changes.
Migrated chart_data_export from binary_sensor to sensor to enable
compatibility with dashboard integrations (ApexCharts, etc.) that
require sensor entities for data selection.
Changes:
- Moved chart_data_export from binary_sensor/ to sensor/ platform
- Changed from boolean state (ON/OFF) to ENUM states ("pending", "ready", "error")
- Maintained all functionality: service call, attribute structure, caching
- Updated translations in all 5 languages (de, en, nb, nl, sv)
- Updated user documentation (sensors.md, services.md)
- Removed all chart_data_export code from binary_sensor platform
Technical details:
- State: "pending" (before first call), "ready" (data available), "error" (service failed)
- Attributes: timestamp + error (metadata) → descriptions → service response data
- Cache (_chart_data_response) bridges async service call and sync property access
- Service call: Triggered on async_added_to_hass() and async_update()
Impact: Dashboard integrations can now select chart_data_export sensor
in their entity pickers. No breaking changes for existing users - entity ID
changes from binary_sensor.* to sensor.*, but functionality identical.
Added optional diagnostic binary sensor that exposes get_chartdata
service results as entity attributes for legacy dashboard tools.
Key features:
- Entity: binary_sensor.tibber_home_NAME_chart_data_export
- Configurable via Options Flow Step 7 (YAML parameters)
- Calls get_chartdata service with user configuration
- Exposes response as attributes for chart cards
- Disabled by default (opt-in)
- Auto-refreshes on coordinator updates
- Manual refresh via homeassistant.update_entity
Implementation details:
- Added chart_data_export entity description to definitions.py
- Implemented state/attribute logic in binary_sensor/core.py
- Added YAML configuration schema in schemas.py
- Added validation in options_flow.py (Step 7)
- Service call validation with detailed error messages
- Attribute ordering: metadata first, descriptions next, service data last
- Dynamic icon mapping (database-export/database-alert)
Translations:
- Added chart_data_export_config to all 5 languages
- Added Step 7 descriptions with legacy warning
- Added invalid_yaml_syntax/invalid_yaml_structure error messages
- Added custom_translations for sensor descriptions
Documentation:
- Added Chart Data Export section to sensors.md
- Added comprehensive service guide to services.md
- Migration path from sensor to service
- Configuration instructions via Options Flow
Impact: Provides backward compatibility for dashboard tools that can
only read entity attributes (e.g., older ApexCharts versions). New
integrations should use tibber_prices.get_chartdata service directly.
