Periods can now naturally cross midnight boundaries, and new diagnostic
attributes help users understand price classification changes at midnight.
**New Features:**
1. Midnight-Crossing Period Support (relaxation.py):
- group_periods_by_day() assigns periods to ALL spanned days
- Periods crossing midnight appear in both yesterday and today
- Enables period formation across calendar day boundaries
- Ensures min_periods checking works correctly at midnight
2. Extended Price Data Window (relaxation.py):
- Period calculation now uses full 3-day data (yesterday+today+tomorrow)
- Enables natural period formation without artificial midnight cutoff
- Removed date filter that excluded yesterday's prices
3. Day Volatility Diagnostic Attributes (period_statistics.py, core.py):
- day_volatility_%: Daily price spread as percentage (span/avg × 100)
- day_price_min/max/span: Daily price range in minor currency (ct/øre)
- Helps detect when midnight classification changes are economically significant
- Uses period start day's reference prices for consistency
**Documentation:**
4. Design Principles (period-calculation-theory.md):
- Clarified per-day evaluation principle (always was the design)
- Added comprehensive section on midnight boundary handling
- Documented volatility threshold separation (sensor vs period filters)
- Explained market context for midnight price jumps (EPEX SPOT timing)
5. User Guides (period-calculation.md, automation-examples.md):
- Added \"Midnight Price Classification Changes\" troubleshooting section
- Provided automation examples using volatility attributes
- Explained why Best→Peak classification can change at midnight
- Documented level filter volatility threshold behavior
**Architecture:**
- Per-day evaluation: Each interval evaluated against its OWN day's min/max/avg
(not period start day) ensures mathematical correctness across midnight
- Period boundaries: Periods can naturally cross midnight but may split when
consecutive days differ significantly (intentional, mathematically correct)
- Volatility thresholds: Sensor thresholds (user-configurable) remain separate
from period filter thresholds (fixed internal) to prevent unexpected behavior
Impact: Periods crossing midnight are now consistently visible before and
after midnight turnover. Users can understand and handle edge cases where
price classification changes at midnight on low-volatility days.
BREAKING CHANGE: Period overlap resolution now merges adjacent/overlapping periods
instead of marking them as extensions. This simplifies automation logic and provides
clearer period boundaries for users.
Previous Behavior:
- Adjacent periods created by relaxation were marked with is_extension=true
- Multiple short periods instead of one continuous period
- Complex logic needed to determine actual period length in automations
New Behavior:
- Adjacent/overlapping periods are merged into single continuous periods
- Newer period's relaxation attributes override older period's
- Simpler automation: one period = one continuous time window
Changes:
- Period Overlap Resolution (new file: period_overlap.py):
* Added merge_adjacent_periods() to combine periods and preserve attributes
* Rewrote resolve_period_overlaps() with simplified merge logic
* Removed split_period_by_overlaps() (no longer needed)
* Removed is_extension marking logic
* Removed unused parameters: min_period_length, baseline_periods
- Relaxation Strategy (relaxation.py):
* Removed all is_extension filtering from period counting
* Simplified standalone counting to just len(periods)
* Changed from period_merging import to period_overlap import
* Added MAX_FLEX_HARD_LIMIT constant (0.50)
* Improved debug logging for merged periods
- Code Quality:
* Fixed all remaining linter errors (N806, PLR2004, PLR0912)
* Extracted magic values to module-level constants:
- FLEX_SCALING_THRESHOLD = 0.20
- SCALE_FACTOR_WARNING_THRESHOLD = 0.8
- MAX_FLEX_HARD_LIMIT = 0.50
* Added appropriate noqa comments for unavoidable patterns
- Configuration (from previous work in this session):
* Removed CONF_RELAXATION_STEP_BEST, CONF_RELAXATION_STEP_PEAK
* Hard-coded 3% relaxation increment for reliability
* Optimized defaults: RELAXATION_ATTEMPTS 8→11, ENABLE_MIN_PERIODS False→True,
MIN_PERIODS undefined→2
* Removed relaxation_step UI fields from config flow
* Updated all 5 translation files
- Documentation:
* Updated period_handlers/__init__.py: period_merging → period_overlap
* No user-facing docs changes needed (already described continuous periods)
Rationale - Period Merging:
User experience was complicated by fragmented periods:
- Automations had to check multiple adjacent periods
- Binary sensors showed ON/OFF transitions within same cheap time
- No clear way to determine actual continuous period length
With merging:
- One continuous cheap time = one period
- Binary sensor clearly ON during entire period
- Attributes show merge history via merged_from dict
- Relaxation info preserved from newest/highest flex period
Rationale - Hard-Coded Relaxation Increment:
The configurable relaxation_step parameter proved problematic:
- High base flex + high step → rapid explosion (40% base + 10% step → 100% in 6 steps)
- Users don't understand the multiplicative nature
- 3% increment provides optimal balance: 11 attempts to reach 50% hard cap
Impact:
- Existing installations: Periods may appear longer (merged instead of split)
- Automations benefit from simpler logic (no is_extension checks needed)
- Custom relaxation_step values will use new 3% increment
- Users may need to adjust relaxation_attempts if they relied on high step sizes
Replaced absolute volatility thresholds (ct/øre) with relative coefficient
of variation (CV = std_dev / mean * 100%) for scale-independent volatility
measurement that works across all price levels.
Changes to volatility calculation:
- price_utils.py: Rewrote calculate_volatility_level() to accept price list
instead of spread value, using statistics.mean() and statistics.stdev()
- sensor.py: Updated volatility sensors to pass price lists (not spread)
- services.py: Modified _get_price_stats() to calculate CV from prices
- period_statistics.py: Extract prices for CV calculation in period summaries
- const.py: Updated default thresholds to 15%/30%/50% (was 5/15/30 ct)
with comprehensive documentation explaining CV-based approach
Dead code removal:
- period_utils/core.py: Removed filter_periods_by_volatility() function
(86 lines of code that was never actually called)
- period_utils/__init__.py: Removed dead function export
- period_utils/relaxation.py: Simplified callback signature from
Callable[[str|None, str|None], bool] to Callable[[str|None], bool]
- coordinator.py: Updated lambda callbacks to match new signature
- const.py: Replaced RELAXATION_VOLATILITY_ANY with RELAXATION_LEVEL_ANY
Bug fix:
- relaxation.py: Added int() conversion for max_relaxation_attempts
(line 435: attempts = max(1, int(max_relaxation_attempts)))
Fixes TypeError when config value arrives as float
Configuration UI:
- config_flow.py: Changed volatility threshold unit display from "ct" to "%"
Translations (all 5 languages):
- Updated volatility descriptions to explain coefficient of variation
- Changed threshold labels from "spread ≥ value" to "CV ≥ percentage"
- Languages: de, en, nb, nl, sv
Documentation:
- period-calculation.md: Removed volatility filter section (dead feature)
Impact: Breaking change for users with custom volatility thresholds.
Old absolute values (e.g., 5 ct) will be interpreted as percentages (5%).
However, new defaults (15%/30%/50%) are more conservative and work
universally across all currencies and price levels. No data migration
needed - existing configs continue to work with new interpretation.
- Skip asymmetry/zigzag rejection near the data tail and refactor spike
validation so legitimate end-of-day spikes stop breaking periods.
- Expose relaxation attempt sliders for both Best/Peak flows, wire the values
through the coordinator, and extend the relaxation engine to honor the new
max-attempt cap with richer logging & metadata.
- Raise the default attempt count to eight flex levels so the 25% increment
pattern can stretch much further before stopping, keeping translations and
docs (including the matrix explanation) in sync across all locales.
Impact: Tail spikes no longer get thrown out incorrectly, users can tune how
aggressively the period search relaxes, and the defaults now find more viable
periods on volatile days.
* feat(period-calc): adaptive defaults + remove volatility filter
Major improvements to period calculation with smarter defaults and
simplified configuration:
**Adaptive Defaults:**
- ENABLE_MIN_PERIODS: true (was false) - Always try to find periods
- MIN_PERIODS target: 2 periods/day (ensures coverage)
- BEST_PRICE_MAX_LEVEL: "cheap" (was "any") - Prefer genuinely cheap
- PEAK_PRICE_MIN_LEVEL: "expensive" (was "any") - Prefer genuinely expensive
- GAP_TOLERANCE: 1 (was 0) - Allow 1-level deviations in sequences
- MIN_DISTANCE_FROM_AVG: 5% (was 2%) - Ensure significance
- PEAK_PRICE_MIN_PERIOD_LENGTH: 30min (was 60min) - More responsive
- PEAK_PRICE_FLEX: -20% (was -15%) - Better peak detection
**Volatility Filter Removal:**
- Removed CONF_BEST_PRICE_MIN_VOLATILITY from const.py
- Removed CONF_PEAK_PRICE_MIN_VOLATILITY from const.py
- Removed volatility filter UI controls from config_flow.py
- Removed filter_periods_by_volatility() calls from coordinator.py
- Updated all 5 translations (de, en, nb, nl, sv)
**Period Calculation Logic:**
- Level filter now integrated into _build_periods() (applied during
interval qualification, not as post-filter)
- Gap tolerance implemented via _check_level_with_gap_tolerance()
- Short periods (<1.5h) use strict filtering (no gap tolerance)
- Relaxation now passes level_filter + gap_count directly to
PeriodConfig
- show_periods check skipped when relaxation enabled (relaxation
tries "any" as fallback)
**Documentation:**
- Complete rewrite of docs/user/period-calculation.md:
* Visual examples with timelines
* Step-by-step explanation of 4-step process
* Configuration scenarios (5 common use cases)
* Troubleshooting section with specific fixes
* Advanced topics (per-day independence, early stop, etc.)
- Updated README.md: "volatility" → "distance from average"
Impact: Periods now reliably appear on most days with meaningful
quality filters. Users get warned about expensive periods and notified
about cheap opportunities without manual tuning. Relaxation ensures
coverage while keeping filters as strict as possible.
Breaking change: Volatility filter removed (was never a critical
feature, often confused users). Existing configs continue to work
(removed keys are simply ignored).
* feat(periods): modularize period_utils and add statistical outlier filtering
Refactored monolithic period_utils.py (1800 lines) into focused modules
for better maintainability and added advanced outlier filtering with
smart impact tracking.
Modular structure:
- types.py: Type definitions and constants (89 lines)
- level_filtering.py: Level filtering with gap tolerance (121 lines)
- period_building.py: Period construction from intervals (238 lines)
- period_statistics.py: Statistics and summaries (318 lines)
- period_merging.py: Overlap resolution (382 lines)
- relaxation.py: Per-day relaxation strategy (547 lines)
- core.py: Main API orchestration (251 lines)
- outlier_filtering.py: Statistical spike detection (294 lines)
- __init__.py: Public API exports (62 lines)
New statistical outlier filtering:
- Linear regression for trend-based spike detection
- 2 standard deviation confidence intervals (95%)
- Symmetry checking to preserve legitimate price shifts
- Enhanced zigzag detection with relative volatility (catches clusters)
- Replaces simple average smoothing with trend-based predictions
Smart impact tracking:
- Tests if original price would have passed criteria
- Only counts smoothed intervals that actually changed period formation
- Tracks level gap tolerance usage separately
- Both attributes only appear when > 0 (clean UI)
New period attributes:
- period_interval_smoothed_count: Intervals kept via outlier smoothing
- period_interval_level_gap_count: Intervals kept via gap tolerance
Impact: Statistical outlier filtering prevents isolated price spikes from
breaking continuous periods while preserving data integrity. All statistics
use original prices. Smart tracking shows only meaningful interventions,
making it clear when tolerance mechanisms actually influenced results.
Backwards compatible: All public APIs re-exported from period_utils package.
* Update docs/user/period-calculation.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update custom_components/tibber_prices/const.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update custom_components/tibber_prices/coordinator.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update custom_components/tibber_prices/const.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* docs(periods): fix corrupted period-calculation.md and add outlier filtering documentation
Completely rewrote period-calculation.md after severe corruption (massive text
duplication throughout the file made it 2489 lines).
Changes:
- Fixed formatting: Removed all duplicate text and headers
- Reduced file size: 2594 lines down to 516 lines (clean, readable structure)
- Added section 5: "Statistical Outlier Filtering (NEW)" explaining:
- Linear regression-based spike detection (95% confidence intervals)
- Symmetry checking to preserve legitimate price shifts
- Enhanced zigzag detection with relative volatility
- Data integrity guarantees (original prices always used)
- New period attributes: period_interval_smoothed_count
- Added troubleshooting: "Price spikes breaking periods" section
- Added technical details: Algorithm constants and implementation notes
Impact: Users can now understand how outlier filtering prevents isolated
price spikes from breaking continuous periods. Documentation is readable
again with no duplicate content.
* fix(const): improve clarity in comments regarding period lengths for price alerts
* docs(periods): improve formatting and clarity in period-calculation.md
* Initial plan
* refactor: convert flexibility_pct to ratio once at function entry
Co-authored-by: jpawlowski <75446+jpawlowski@users.noreply.github.com>
* Update custom_components/tibber_prices/const.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update custom_components/tibber_prices/period_utils/period_building.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update custom_components/tibber_prices/period_utils/relaxation.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Julian Pawlowski <jpawlowski@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Completely rewrote period-calculation.md based on user feedback and
live development understanding.
Changes:
- Replaced outdated 3-phase relaxation description with correct 4×4
matrix approach (4 flex levels × 4 filter combinations)
- Added per-day independence explanation (each day relaxes independently)
- Documented replacement logic (larger periods replace smaller ones)
- Added extension logic (baseline periods get expanded, not replaced)
- Updated metadata format examples (price_diff_27.3%+level_any)
- Restructured for clarity: Quick Start → How It Works → Config →
Relaxation → Scenarios → Troubleshooting
- Added 4 real-world scenarios with automation examples (dishwasher,
heat pump, EV charging, peak avoidance)
- Added visual timeline examples
- Reduced technical complexity, focused on user understanding
- Added practical troubleshooting with specific solutions
Impact: Users can now understand how period calculation actually works,
with correct information matching the implemented 4×4 relaxation
strategy. Documentation evolved from cold code reading to live
development insights with user feedback.
