akkudoktoreos.core.databaseabc.DatabaseRecordProtocolMixin

class akkudoktoreos.core.databaseabc.DatabaseRecordProtocolMixin

Bases: ConfigMixin, DatabaseMixin, Generic[T_Record]

Database Record Protocol Mixin.

Completely manages in memory records and database storage.

Expects records with date_time (DatabaseTimestamp) property and the a record list in self.records of the derived class.

DatabaseRecordProtocolMixin expects the derived classes to be singletons.

__init__()

Methods

__init__()
db_autosave()
db_compact([compact_tiers])	Apply tiered compaction policy to all records in this namespace.
db_compact_tiers()	Compaction tiers as (age_threshold, target_interval) pairs.
db_count_records()	Return total logical number of records.
db_delete_records([start_timestamp, ...])
db_generate_timestamps(start_timestamp, ...)	Generate database timestamps using fixed absolute time stepping.
db_get_record(target_timestamp, *[, time_window])	Get the record at or nearest to the specified timestamp.
db_get_stats()	Get comprehensive statistics about database storage.
db_initial_time_window()	Return the initial time window used for database loading.
db_insert_record(record, *[, mark_dirty])
db_iterate_records([start_timestamp, ...])	Iterate records in requested range.
db_keep_duration()	Duration for which database records should be retained.
db_load_records([start_timestamp, end_timestamp])	Load records from database into memory.
db_mark_dirty_record(record)
db_namespace()	Namespace of database.
db_next_timestamp(timestamp)	Find the smallest timestamp > given timestamp.
db_previous_timestamp(timestamp)	Find the largest timestamp < given timestamp.
db_save_records()
db_timestamp_range()	Get the timestamp range of records in database.
db_vacuum([keep_hours, keep_timestamp])	Remove old records from database to free space.
model_post_init(...)	Initialize DB state attributes immediately after Pydantic construction.

Attributes

config
database
db_enabled

records: list[T_Record]

db_initial_time_window() → Duration | None

Return the initial time window used for database loading.

This window defines the initial symmetric time span around a target datetime that should be loaded from the database when no explicit search time window is specified. It serves as a loading hint and may be expanded by the caller if no records are found within the initial range.

Subclasses may override this method to provide a domain-specific default.

Returns:

The initial loading time window as a Duration, or None to indicate that no initial window constraint should be applied.

model_post_init(_DatabaseRecordProtocolMixin__context: Any) → None

Initialize DB state attributes immediately after Pydantic construction.

db_previous_timestamp(timestamp: DatabaseTimestamp) → DatabaseTimestamp | None

Find the largest timestamp < given timestamp.

Search memory-first, then fallback to database if necessary.

db_next_timestamp(timestamp: DatabaseTimestamp) → DatabaseTimestamp | None

Find the smallest timestamp > given timestamp.

Search memory-first, then fallback to database if necessary.

db_keep_duration() → Duration | None

Duration for which database records should be retained.

Used when removing old records from database to free space.

Defaults to general database configuration.

May be provided by derived class.

Returns:

Duration or None (forever).

db_namespace() → str

Namespace of database.

To be implemented by derived class.

property db_enabled: bool

db_timestamp_range() → tuple[DatabaseTimestamp | None, DatabaseTimestamp | None]

Get the timestamp range of records in database.

Regards records in storage plus extra records in memory.

db_generate_timestamps(start_timestamp: DatabaseTimestamp, values_count: int, interval: Duration | None = None) → Iterator[DatabaseTimestamp]

Generate database timestamps using fixed absolute time stepping.

The iterator advances strictly in UTC, guaranteeing constant spacing in seconds across daylight saving transitions.

Returned database timestamps are in UTC. This avoids ambiguity during fall-back transitions and prevents accidental overwriting when inserting into UTC-normalized storage backends.

Parameters:

• start_timestamp (DatabaseTimestamp) – Starting database timestamp.

• values_count (int) – Number of timestamps to generate.

• interval (Optional[Duration]) – Fixed duration between timestamps. Defaults to 1 hour if not provided.

Yields:

DatabaseTimestamp – UTC-based database timestamps.

Raises:

ValueError – If values_count is negative.

db_get_record(target_timestamp: DatabaseTimestamp, *, time_window: Duration | None | _DatabaseTimeWindowUnbound = None) → T_Record | None

Get the record at or nearest to the specified timestamp.

The search strategies are:

• None - exact match only.

• UNBOUND_WINDOW - nearest record across all stored records.

• Duration - nearest record within a symmetric window of this total width around target_timestamp.

Parameters:

• target_timestamp – The timestamp to search for.

• time_window – Controls the search strategy (None, UNBOUND_WINDOW, Duration).

Returns:

Exact match, nearest record within the window, or None.

db_insert_record(record: T_Record, *, mark_dirty: bool = True) → None

db_load_records(start_timestamp: DatabaseTimestamp | _DatabaseTimestampUnbound | None = None, end_timestamp: DatabaseTimestamp | _DatabaseTimestampUnbound | None = None) → int

Load records from database into memory.

Merges database records into in-memory records while preserving: - Memory-only records - Sorted order - No duplicates (DB overwrites memory)

This requested load range is extended to include the first record < start_timestamp and the first record >= end_timestamp, so nearest-neighbor searches do not require additional DB lookups.

The _db_loaded_range is updated to reflect the total timestamp span currently present in memory after this method completes.

Parameters:

• start_timestamp – Load records from this timestamp (inclusive)

• end_timestamp – Load records until this timestamp (exclusive)

Returns:

Number of records loaded from database

Note

record.date_time shall be DateTime or None

db_delete_records(start_timestamp: DatabaseTimestamp | _DatabaseTimestampUnbound | None = None, end_timestamp: DatabaseTimestamp | _DatabaseTimestampUnbound | None = None) → int

db_iterate_records(start_timestamp: DatabaseTimestamp | _DatabaseTimestampUnbound | None = None, end_timestamp: DatabaseTimestamp | _DatabaseTimestampUnbound | None = None) → Iterator[T_Record]

Iterate records in requested range.

Ensures storage is loaded into memory first, then iterates over in-memory records only.

db_mark_dirty_record(record: T_Record) → None

db_save_records() → int

db_autosave() → int

db_vacuum(keep_hours: int | None = None, keep_timestamp: DatabaseTimestamp | _DatabaseTimestampUnbound | None = None) → int

Remove old records from database to free space.

Semantics:

• keep_hours is relative to the DB’s max timestamp: cutoff = db_max - keep_hours, and records

  with timestamp < cutoff are deleted.

• keep_timestamp is an absolute cutoff; records with timestamp < cutoff are deleted (exclusive).

Uses self.keep_duration() if both of keep_hours and keep_timestamp are None.

Parameters:

• keep_hours – Keep only records from the last N hours (relative to the data’s max timestamp)

• keep_timestamp – Keep only records from this timestamp on (absolute cutoff)

Returns:

Number of records deleted

db_count_records() → int

Return total logical number of records.

Memory is authoritative. If DB is enabled but not fully loaded, we conservatively include storage-only records.

db_get_stats() → dict

Get comprehensive statistics about database storage.

Returns:

Dictionary with statistics

db_compact_tiers() → list[tuple[Duration, Duration]]

Compaction tiers as (age_threshold, target_interval) pairs.

Records older than age_threshold are downsampled to target_interval. Tiers must be ordered from shortest to longest age threshold.

Default policy:

• older than 2 hours → 15 min resolution

• older than 14 days → 1 hour resolution

Return empty list to disable compaction entirely. Override in derived classes for domain-specific behaviour.

Example override to disable:

Example override for price data (already at 15 min, skip first tier):

config = ConfigEOS(general=GeneralSettings(home_assistant_addon=False, version='0.2.0.dev58204789', data_folder_path=Path('/home/docs/.local/share/net.akkudoktor.eos'), data_output_subpath='output', latitude=52.52, longitude=13.405, timezone='Europe/Berlin', data_output_path=Path('/home/docs/.local/share/net.akkudoktor.eos/output'), config_folder_path=Path('/home/docs/checkouts/readthedocs.org/user_builds/akkudoktor-eos/checkouts/latest/docs'), config_file_path=Path('/home/docs/checkouts/readthedocs.org/user_builds/akkudoktor-eos/checkouts/latest/docs/EOS.config.json')), cache=CacheCommonSettings(subpath='cache', cleanup_interval=300.0), database=DatabaseCommonSettings(provider=None, compression_level=9, initial_load_window_h=None, keep_duration_h=None, autosave_interval_sec=10, compaction_interval_sec=604800, batch_size=100, providers=['LMDB', 'SQLite']), ems=EnergyManagementCommonSettings(startup_delay=5, interval=300.0, mode=None), logging=LoggingCommonSettings(console_level='INFO', file_level=None, file_path=Path('/home/docs/.local/share/net.akkudoktor.eos/output/eos.log')), devices=DevicesCommonSettings(batteries=None, max_batteries=None, electric_vehicles=None, max_electric_vehicles=None, inverters=None, max_inverters=None, home_appliances=None, max_home_appliances=None, measurement_keys=[]), measurement=MeasurementCommonSettings(historic_hours=17520, load_emr_keys=None, grid_export_emr_keys=None, grid_import_emr_keys=None, pv_production_emr_keys=None, keys=[]), optimization=OptimizationCommonSettings(horizon_hours=24, interval=3600, algorithm='GENETIC', genetic=GeneticCommonSettings(individuals=300, generations=400, seed=None, penalties=None), keys=[]), prediction=PredictionCommonSettings(hours=48, historic_hours=48), elecprice=ElecPriceCommonSettings(provider=None, charges_kwh=None, vat_rate=1.19, elecpriceimport=ElecPriceImportCommonSettings(import_file_path=None, import_json=None), energycharts=ElecPriceEnergyChartsCommonSettings(bidding_zone=<EnergyChartsBiddingZones.DE_LU: 'DE-LU'>), providers=['ElecPriceAkkudoktor', 'ElecPriceEnergyCharts', 'ElecPriceImport']), feedintariff=FeedInTariffCommonSettings(provider=None, provider_settings=FeedInTariffCommonProviderSettings(FeedInTariffFixed=None, FeedInTariffImport=None), providers=['FeedInTariffFixed', 'FeedInTariffImport']), load=LoadCommonSettings(provider=None, provider_settings=LoadCommonProviderSettings(LoadAkkudoktor=None, LoadVrm=None, LoadImport=None), providers=['LoadAkkudoktor', 'LoadAkkudoktorAdjusted', 'LoadVrm', 'LoadImport']), pvforecast=PVForecastCommonSettings(provider=None, provider_settings=PVForecastCommonProviderSettings(PVForecastImport=None, PVForecastVrm=None), planes=None, max_planes=0, providers=['PVForecastAkkudoktor', 'PVForecastVrm', 'PVForecastImport'], planes_peakpower=[], planes_azimuth=[], planes_tilt=[], planes_userhorizon=[], planes_inverter_paco=[]), weather=WeatherCommonSettings(provider=None, provider_settings=WeatherCommonProviderSettings(WeatherImport=None), providers=['BrightSky', 'ClearOutside', 'WeatherImport']), server=ServerCommonSettings(host='127.0.0.1', port=8503, verbose=False, startup_eosdash=True, eosdash_host=None, eosdash_port=None), utils=UtilsCommonSettings(), adapter=AdapterCommonSettings(provider=None, homeassistant=HomeAssistantAdapterCommonSettings(config_entity_ids=None, load_emr_entity_ids=None, grid_export_emr_entity_ids=None, grid_import_emr_entity_ids=None, pv_production_emr_entity_ids=None, device_measurement_entity_ids=None, device_instruction_entity_ids=None, solution_entity_ids=None, homeassistant_entity_ids=[], eos_solution_entity_ids=[], eos_device_instruction_entity_ids=[]), nodered=NodeREDAdapterCommonSettings(host='127.0.0.1', port=1880), providers=['HomeAssistant', 'NodeRED']))

database = <akkudoktoreos.core.database.Database object>

db_compact(compact_tiers: list[tuple[Duration, Duration]] | None = None) → int

Apply tiered compaction policy to all records in this namespace.

Tiers are processed coarsest-first (longest age threshold first) to avoid compacting fine-grained data that an inner tier would immediately re-compact anyway.

Parameters:

compact_tiers – Override tiers for this call. If None, uses db_compact_tiers(). Each entry is (age_threshold, target_interval), ordered shortest to longest age threshold.

Returns:

Total number of original records deleted across all tiers.