Fixes bug where lifecycle sensor attributes (data_completeness, tomorrow_available)
didn't update after tomorrow data was successfully fetched from API.
Root cause: DataTransformer had cached transformation data but no mechanism to detect
when source API data changed (only checked config and midnight turnover).
Changes:
- coordinator/data_transformation.py: Track source_data_timestamp and invalidate cache
when timestamp changes (detects new API data arrival)
- coordinator/data_transformation.py: Integrate period calculation into DataTransformer
(calculate_periods_fn parameter) for complete single-layer caching
- coordinator/core.py: Remove duplicate transformation cache (_cached_transformed_data,
_last_transformation_config), simplify _transform_data_for_*() to direct delegation
- tests/test_tomorrow_data_refresh.py: Add 3 regression tests for cache invalidation
(new timestamp, config change behavior, cache preservation)
Impact: Lifecycle sensor attributes now update correctly when new API data arrives.
Reduced code by ~40 lines in coordinator, consolidated caching to single layer.
All 350 tests passing.
Cleaned up AGENTS.md to focus on patterns and conventions:
Removed:
- "Current State (as of Nov 2025)" section
- "Classes that need renaming" outdated list
- "Action Required" checklist
- Temporal statements about current project state
Added:
- TypedDict exemption in "When prefix can be omitted" list
- Clear rationale: documentation-only, never instantiated
Rationale:
AGENTS.md documents patterns and conventions that help AI understand
the codebase structure. Status tracking belongs in git history or
planning documents. The file should be timeless guidance, not a
snapshot of work in progress.
Impact: Documentation is now focused on "how to write code correctly"
rather than "what state is the code in now".
Removed tests for the lifecycle callback system that was removed in
commit 48d6e25.
Also fixed commit f373c01 which incorrectly added test_lifecycle_tomorrow_update.py
instead of deleting it - this commit properly removes it.
Changes:
- tests/test_chart_data_push_updates.py: Deleted (235 lines)
- tests/test_lifecycle_tomorrow_update.py: Deleted (174 lines)
- tests/test_resource_cleanup.py: Removed lifecycle callback test method
Impact: Test suite now has 343 tests (down from 349). All tests pass.
No functionality affected - only test cleanup.
Simplified _has_future_periods() to check for ANY future periods instead
of limiting to 6-hour window. This ensures icons show 'waiting' state
whenever periods are scheduled, not just within artificial time limit.
Also added pragmatic fallback in timing calculator _find_next_period():
when skip_current=True but only one future period exists, return it
anyway instead of showing 'unknown'. This prevents timing sensors from
showing unknown during active periods.
Changes:
- binary_sensor/definitions.py: Removed PERIOD_LOOKAHEAD_HOURS constant
- binary_sensor/core.py: Simplified _has_future_periods() logic
- sensor/calculators/timing.py: Added pragmatic fallback for single period
Impact: Better user experience - icons always show future periods, timing
sensors show values even during edge cases.
Deleted test_lifecycle_tomorrow_update.py (2 tests) which validated the
now-removed lifecycle callback system.
These tests were rendered obsolete by the removal of the custom lifecycle
callback mechanism in favor of Home Assistant's standard coordinator pattern.
Impact: Test suite reduced from 355 to 349 tests, all passing.
Removed custom lifecycle callback push-update mechanism after confirming
it was redundant with Home Assistant's built-in DataUpdateCoordinator
pattern.
Root cause analysis showed HA's async_update_listeners() is called
synchronously (no await) immediately after _async_update_data() returns,
making separate lifecycle callbacks unnecessary.
Changes:
- coordinator/core.py: Removed lifecycle callback methods and notifications
- sensor/core.py: Removed lifecycle callback registration and cleanup
- sensor/attributes/lifecycle.py: Removed next_tomorrow_check attribute
- sensor/calculators/lifecycle.py: Removed get_next_tomorrow_check_time()
Impact: Simplified coordinator pattern, no user-visible changes. Standard
HA coordinator mechanism provides same immediate update guarantee without
custom callback complexity.
Added documentation file explaining why period calculation functions
are tested via integration tests rather than unit tests.
Rationale:
- Period building requires full coordinator context (TimeService, price_context)
- Complex enriched price data with multiple calculated fields
- Helper functions (split_intervals_by_day, calculate_reference_prices)
are simple transformations that can't fail independently
- Integration tests provide better coverage than mocked unit tests
Testing strategy:
- test_midnight_periods.py: Period calculation across day boundaries
- test_midnight_turnover.py: Cache invalidation and recalculation
- docs/development/period-calculation-theory.md: Algorithm documentation
Impact: Clarifies testing approach for future contributors. Prevents
wasted effort on low-value unit tests for complex integrated functions.
Fixed state inconsistencies during auth errors:
Bug #4: tomorrow_data_available incorrectly returns True during auth failure
- Now returns None (unknown) when coordinator.last_exception is ConfigEntryAuthFailed
- Prevents misleading "data available" when API connection lost
Bug #5: Sensor states inconsistent during error conditions
- connection: False during auth error (even with cached data)
- tomorrow_data_available: None during auth error (cannot verify)
- lifecycle_status: "error" during auth error
Changes:
- binary_sensor/core.py: Check last_exception before evaluating tomorrow data
- tests: 25 integration tests covering all error scenarios
Impact: All three sensors show consistent states during auth errors,
API timeouts, and normal operation. No misleading "available" status
when connection is lost.
Introduced TibberPricesMidnightHandler to prevent duplicate midnight
turnover when multiple timers fire simultaneously.
Problem: Timer #1 (API poll) and Timer #2 (quarter-hour refresh) both
wake at midnight, each detecting day change and triggering cache clear.
Race condition caused duplicate turnover operations.
Solution:
- Atomic flag coordination: First timer sets flag, subsequent timers skip
- Persistent state survives HA restart (cache stores last_turnover_time)
- Day-boundary detection: Compares current.date() vs last_check.date()
- 13 comprehensive tests covering race conditions and HA restart scenarios
Architecture:
- coordinator/midnight_handler.py: 165 lines, atomic coordination logic
- coordinator/core.py: Integrated handler in coordinator initialization
- coordinator/listeners.py: Delegate midnight check to handler
Impact: Eliminates duplicate cache clears at midnight. Single atomic
turnover operation regardless of how many timers fire simultaneously.
Fixed multiple calculation issues with negative prices (Norway/Germany
renewable surplus scenarios):
Bug #6: Rating threshold validation with dead code
- Added threshold validation (low >= high) with warning
- Returns NORMAL as fallback for misconfigured thresholds
Bug #7: Min/Max functions returning 0.0 instead of None
- Changed default from 0.0 to None when window is empty
- Prevents misinterpretation (0.0 looks like price with negatives)
Bug #9: Period price diff percentage wrong sign with negative reference
- Use abs(ref_price) in percentage calculation
- Correct percentage direction for negative prices
Bug #10: Trend diff percentage wrong sign with negative current price
- Use abs(current_interval_price) in percentage calculation
- Correct trend direction when prices cross zero
Bug #11: later_half_diff calculation failed for negative prices
- Changed condition from `if current_interval_price > 0` to `!= 0`
- Use abs(current_interval_price) for percentage
Changes:
- utils/price.py: Add threshold validation, use abs() in percentages
- utils/average.py: Return None instead of 0.0 for empty windows
- period_statistics.py: Use abs() for reference prices
- trend.py: Use abs() for current prices, fix zero-check condition
- tests: 95+ new tests covering negative/zero/mixed price scenarios
Impact: All calculations work correctly with negative electricity prices.
Percentages show correct direction regardless of sign.
Best Price min_distance now uses negative values (-50 to 0) to match
semantic meaning "below average". Peak Price continues using positive
values (0 to 50) for "above average".
Uniform formula: avg * (1 + distance/100) works for both period types.
Sign indicates direction: negative = toward MIN (cheap), positive = toward MAX (expensive).
Changes:
- const.py: DEFAULT_BEST_PRICE_MIN_DISTANCE_FROM_AVG = -5 (was 5)
- schemas.py: Best Price range -50 to 0, Peak Price range 0 to 50
- validators.py: Separate validate_best_price_distance_percentage()
- level_filtering.py: Simplified to uniform formula (removed conditionals)
- translations: Separate error messages for Best/Peak distance validation
- tests: 37 comprehensive validator tests (100% coverage)
Impact: Configuration UI now visually represents direction relative to average.
Users see intuitive negative values for "below average" pricing.
chart_data_export now registers lifecycle callback for immediate
updates when coordinator data changes ("fresh" lifecycle state).
Previously only updated via polling intervals.
Changes:
- Register callback in sensor constructor (when entity_key matches)
- Callback triggers async_write_ha_state() on "fresh" lifecycle
- 5 new tests covering callback registration and triggering
Impact: Chart data export updates immediately on API data arrival,
enabling real-time dashboard updates without polling delay.
Cache validity now checks _last_coordinator_update (within 30min)
instead of _api_calls_today counter. Fixes false "stale" status
when coordinator runs every 15min but cache validation was only
checking API call counter.
Bug #1: Cache validity shows "stale" at 05:57 AM
Bug #2: Cache age calculation incorrect after midnight turnover
Bug #3: get_cache_validity inconsistent with cache_age sensor
Changes:
- Coordinator: Use _last_coordinator_update for cache validation
- Lifecycle: Extract cache validation to dedicated helper function
- Tests: 7 new tests covering midnight scenarios and edge cases
Impact: Cache validity sensor now accurately reflects coordinator
activity, not just explicit API calls. Correctly handles midnight
turnover without false "stale" status.
Set up pytest with Home Assistant support and created 6 tests for
midnight-crossing period logic (5 unit tests + 1 integration test).
Added pytest configuration, test dependencies, test runner script
(./scripts/test), and comprehensive tests for group_periods_by_day()
and midnight turnover consistency.
All tests pass in 0.12s.
Impact: Provides regression testing for midnight-crossing period bugs.
Tests validate periods remain visible across midnight turnover.
Periods can now naturally cross midnight boundaries, and new diagnostic
attributes help users understand price classification changes at midnight.
**New Features:**
1. Midnight-Crossing Period Support (relaxation.py):
- group_periods_by_day() assigns periods to ALL spanned days
- Periods crossing midnight appear in both yesterday and today
- Enables period formation across calendar day boundaries
- Ensures min_periods checking works correctly at midnight
2. Extended Price Data Window (relaxation.py):
- Period calculation now uses full 3-day data (yesterday+today+tomorrow)
- Enables natural period formation without artificial midnight cutoff
- Removed date filter that excluded yesterday's prices
3. Day Volatility Diagnostic Attributes (period_statistics.py, core.py):
- day_volatility_%: Daily price spread as percentage (span/avg × 100)
- day_price_min/max/span: Daily price range in minor currency (ct/øre)
- Helps detect when midnight classification changes are economically significant
- Uses period start day's reference prices for consistency
**Documentation:**
4. Design Principles (period-calculation-theory.md):
- Clarified per-day evaluation principle (always was the design)
- Added comprehensive section on midnight boundary handling
- Documented volatility threshold separation (sensor vs period filters)
- Explained market context for midnight price jumps (EPEX SPOT timing)
5. User Guides (period-calculation.md, automation-examples.md):
- Added \"Midnight Price Classification Changes\" troubleshooting section
- Provided automation examples using volatility attributes
- Explained why Best→Peak classification can change at midnight
- Documented level filter volatility threshold behavior
**Architecture:**
- Per-day evaluation: Each interval evaluated against its OWN day's min/max/avg
(not period start day) ensures mathematical correctness across midnight
- Period boundaries: Periods can naturally cross midnight but may split when
consecutive days differ significantly (intentional, mathematically correct)
- Volatility thresholds: Sensor thresholds (user-configurable) remain separate
from period filter thresholds (fixed internal) to prevent unexpected behavior
Impact: Periods crossing midnight are now consistently visible before and
after midnight turnover. Users can understand and handle edge cases where
price classification changes at midnight on low-volatility days.
Synchronized coordinator._cached_price_data after API calls to ensure tomorrow data is available for sensor value calculations and lifecycle state detection.
Impact: Tomorrow sensors now display values correctly after afternoon data fetch. Lifecycle sensor status remains stable without flickering between "searching_tomorrow" and other states.
Restored mark_periods_with_relaxation() function and added call in
relax_all_prices() to properly mark periods found through relaxation.
Problem: Periods found via relaxation were missing metadata attributes:
- relaxation_active
- relaxation_level
- relaxation_threshold_original_%
- relaxation_threshold_applied_%
These attributes are expected by:
- period_overlap.py: For merging periods with correct relaxation info
- binary_sensor/attributes.py: For displaying relaxation info to users
Implementation:
- Added reverse_sort parameter to preserve sign semantics
- For Best Price: Store positive thresholds (e.g., +15%, +18%)
- For Peak Price: Store negative thresholds (e.g., -20%, -23%)
- Mark periods immediately after calculate_periods() and before
resolve_period_overlaps() so metadata is preserved during merging
Impact: Users can now see which periods were found through relaxation
and at what flex threshold. Peak Price periods show negative thresholds
matching the user's configuration semantics (negative = below maximum).
Changed description attribute behavior from "add separate long_description
attribute" to "switch description content" when CONF_EXTENDED_DESCRIPTIONS
is enabled.
OLD: description always shown, long_description added as separate attribute
NEW: description content switches between short and long based on config
Implementation:
- Check extended_descriptions flag BEFORE loading translation
- Load "long_description" key if enabled, fallback to "description" if missing
- Assign loaded content to "description" attribute (same key always)
- usage_tips remains separate attribute (only when extended=true)
- Updated both sync (entities) and async (services) versions
Added PLR0912 noqa: Branch complexity justified by feature requirements
(extended check + fallback logic + position handling).
Impact: Users see more detailed descriptions when extended mode enabled,
without attribute clutter. Fallback ensures robustness if long_description
missing in translations.
Fixed uninitialized self.time attribute causing AttributeError during
config entry creation. Added explicit initialization to None with
Optional type annotation and guard in _get_price_info_for_specific_homes().
Impact: Config flow no longer crashes when creating initial config entry.
Users can complete setup without errors.
- Added new validation functions for various parameters including flexibility percentage, distance percentage, minimum periods, gap count, relaxation attempts, price rating thresholds, volatility threshold, and price trend thresholds.
- Updated constants in `const.py` to define maximum and minimum limits for the new validation criteria.
- Improved error messages in translations for invalid parameters to provide clearer guidance to users.
- Adjusted existing validation functions to ensure they align with the new constants and validation logic.
Add comprehensive data_lifecycle_status sensor showing real-time cache
vs fresh API data status with 6 states and 13+ detailed attributes.
Key features:
- 6 lifecycle states: cached, fresh, refreshing, searching_tomorrow,
turnover_pending, error
- Push-update system for instant state changes (refreshing→fresh→error)
- Quarter-hour polling for turnover_pending detection at 23:45
- Accurate next_api_poll prediction using Timer #1 offset tracking
- Tomorrow prediction with actual timer schedule (not fixed 13:00)
- 13+ formatted attributes: cache_age, data_completeness, api_calls_today,
next_api_poll, etc.
Implementation:
- sensor/calculators/lifecycle.py: New calculator with state logic
- sensor/attributes/lifecycle.py: Attribute builders with formatting
- coordinator/core.py: Lifecycle tracking + callback system (+16 lines)
- sensor/core.py: Push callback registration (+3 lines)
- coordinator/constants.py: Added to TIME_SENSITIVE_ENTITY_KEYS
- Translations: All 5 languages (de, en, nb, nl, sv)
Timing optimization:
- Extended turnover warning: 5min → 15min (catches 23:45 quarter boundary)
- No minute-timer needed: quarter-hour updates + push = optimal
- Push-updates: <1sec latency for refreshing/fresh/error states
- Timer offset tracking: Accurate tomorrow predictions
Removed obsolete sensors:
- data_timestamp (replaced by lifecycle attributes)
- price_forecast (never implemented, removed from definitions)
Impact: Users can monitor data freshness, API call patterns, cache age,
and understand integration behavior. Perfect for troubleshooting and
visibility into when data updates occur.
Moved Chart Data Export sensor configuration from config flow textarea
to configuration.yaml for better maintainability and consistency with
Home Assistant standards.
Changes:
- __init__.py: Added async_setup() with CONFIG_SCHEMA for tibber_prices.chart_export
- const.py: Added DATA_CHART_CONFIG constant for hass.data storage
- options_flow.py: Simplified chart_data_export step to info-only page
- schemas.py: get_chart_data_export_schema() returns empty schema (no input fields)
- sensor/chart_data.py: Reads config from hass.data instead of config_entry.options
- All 5 translation files: Updated chart_data_export description with:
- Clear heading: "📊 Chart Data Export Sensor"
- Intro line explaining sensor purpose
- Legacy warning (⚠️) recommending service use
- Two valid use cases (✅): attribute-only tools, auto-updating data
- One discouraged use case (❌): automations should use service directly
- 3-step activation instructions
- YAML configuration example with all parameters
- Correct default behavior: today+tomorrow, 15-minute intervals, prices only
Impact: Users configure chart export in configuration.yaml instead of UI.
Sensor remains disabled by default (diagnostic sensor). Config flow shows
prominent info page guiding users toward service usage while keeping
sensor available for legacy dashboard tools that only read attributes.
Standardized config flow translations (nb, nl, sv) to match German/English
format with minimal field labels and comprehensive data_descriptions.
Changes across Norwegian, Dutch, and Swedish translations:
- Updated step_progress format: **{step_progress}** → _{step_progress}_
- Made all step descriptions bold with **text** formatting
- Simplified field labels (removed verbose explanations)
- Added data_description for price_rating (low/high thresholds)
- Added data_description for price_trend (rising/falling thresholds)
- Added data_description for volatility (moderate/high/very high thresholds)
- Ensured all steps have: emojis, italic step_progress, separator (---)
- Added missing emoji to Swedish price_rating step (📊)
Impact: All 5 languages now have consistent UX with minimal, scannable
field labels and detailed optional descriptions accessible via ⓘ icon.
Users get cleaner config flow with better clarity.
