Persistency subsystem update - Generics and history records (#22114)

# Summary

This PR is the next update to the persistency subsystem. The goal of
this PR is to provide more framework like functions to allow more types
of content to be made persistent.

Currently only volatile game objects created during a round can be (*in
a [clean](https://www.youtube.com/watch?v=rZ3ETK7-ZM8) way*) saved and
made persistent. This update attempts to provide methods to make
*everything*¹ persistent.

This introduces persistent generics and history.

## Database

The following things are going to be changed and added in the database  
(open in new tab for better visibility, PNG file includes the drawIO
code):

<img width="1692" height="1041" alt="aurora_persistency_db drawio"
src="https://github.com/user-attachments/assets/ea53f419-f9aa-4592-af8f-3a8d5edf3177"
/>

**Deviations on database implementation from diagram:**  
- Removed unique constraint on history table - Prevented adding multiple
records per round per attribute.

## Framework surface changes

- MC/VV: Moved global object track register to subsystem var space.
- MC/VV: Point of interest: Added history_cache and generic_cache to
subsystem var space.
- MC/VV: Updated subsystem stat entry message, now providing information
on cache sizes of new types.
- Added `singleton/persistent_type` defines (Type, clean-up rules,
finalization hook) and macros allowing new definitions of said types.
- Added cache structures that are also used for returns on public procs
in generics and history persistent types.
- Major new framework features: Persistent history (example: Mining
yield records) and persistent generics (example: Persistent Horizon
overmap position). See documentation for more information.

DrawIO diagram for documentation, includes source in it (open in new
tab):

<img width="200" height="200" alt="Persistence-subsystem-flowchart
drawio"
src="https://github.com/user-attachments/assets/51f28331-f999-49a2-a7cc-58278f7ae416"
/>

## Tasks

(These lists are not comprehensive.)

**General**
- [x] Update DB - Write SQL scripts.
- [x] Add subsystem modular files for generics and history.
- [x] Add type definition logic, macros.
- [x] Add type-DB init logic.
- [x] Logging.
- [x] A lot of testing. *A lot.*
- [x] Changelog.
- [x] Self-Review.
- [x] Update documentation on the persistence subsystem.

**"Persistent history"**
- [x] Add init logic.
- [x] Add finalize logic.
- [x] Add framework surface procs.
  - [x] Get last record.
  - [x] Get last X records.
  - [x] Add record.
  - [x] Add character ID related validation.
- [x] Add initial example mechanic.

**"Persistent generics"**
- [x] Add init logic.
- [x] Add finalize logic.
- [x] Add framework surface procs.
  - [x] Save.
  - [x] Load.
- [x] Add initial example mechanic.

## Changes

Too many changes to be listed here - Check changelog and actual changes.

## Warning

There are certain use/test cases that *cannot* be tested locally due to
missing preexisting data in the database. This should only affect new
data structures (new persistent types), not existing data.

¹ _Large scale persistent mapping is excluded for this version._

---------

Co-authored-by: Copilot <copilot@github.com>
This commit is contained in:
FabianK3
2026-07-19 20:56:17 +02:00
committed by GitHub
co-authored by Copilot
parent 4eccb61c31
commit 802d3fe01f
39 changed files with 3534 additions and 2112 deletions
@@ -4,7 +4,7 @@
*/
/datum/controller/subsystem/persistence/proc/objectsInitialize()
PRIVATE_PROC(TRUE)
GLOB.persistence_object_track_register = list()
object_track_register = list()
if(SSatlas.current_map.path != "sccv_horizon") // The persistence system only supports objects from the main map levels for multiple reasons, e.g. Z level value, mapping support
log_subsystem_persistence_info("Persistent objects: Current map did not match SCCV Horizon, skipping persistent object initialization.")
@@ -42,8 +42,8 @@
if(SSatlas.current_map.path != "sccv_horizon") // The persistence system only supports objects from the main map levels for multiple reasons, e.g. Z level value, mapping support
log_subsystem_persistence_info("Persistent objects: Current map did not match SCCV Horizon, skipping persistent object finalization.")
if(length(GLOB.persistence_object_track_register) > 0)
log_subsystem_persistence_warning("Persistent objects: There are [length(GLOB.persistence_object_track_register)] tracked objects at finalization, while the map is not supported! These track will not be saved! Verify that SSatlas.current_map.path has not changed during the round!")
if(length(object_track_register) > 0)
log_subsystem_persistence_warning("Persistent objects: There are [length(object_track_register)] tracked objects at finalization, while the map is not supported! These track will not be saved! Verify that SSatlas.current_map.path has not changed during the round!")
return
// Subsystem shutdown:
@@ -52,7 +52,7 @@
// Delete persistent records that no longer exist in the registry (removed during the round)
// Run checks on each track that might prevent further persistence
for (var/obj/track as anything in GLOB.persistence_object_track_register)
for (var/obj/track in object_track_register)
CHECK_TICK
var/turf/T = get_turf(track)
if(!T || !is_station_level(T.z)) // The persistence system only supports objects from the main map levels for multiple reasons, e.g. Z level value, mapping support
@@ -65,7 +65,7 @@
// Get already stored data before saving new tracks so we can compare what has been updated or removed during the round.
var/list/existing_data = objectsDatabaseGetActiveEntries()
for (var/obj/track as anything in GLOB.persistence_object_track_register)
for (var/obj/track in object_track_register)
CHECK_TICK
if (track.persistent_objects_track_id == 0)
// Tracked object has no ID meaning it is new, create a new persistent record for it
@@ -76,7 +76,7 @@
// If we find the track, we need to check if it requires an update instead
for (var/record in existing_data)
var/found = FALSE
for (var/obj/track as anything in GLOB.persistence_object_track_register)
for (var/obj/track in object_track_register)
CHECK_TICK
if (record["id"] == track.persistent_objects_track_id)
// A track with the same ID has been found in the register, it still exists, check if we need to update it instead
@@ -114,7 +114,7 @@
if(length(content))
result = json_encode(content)
catch(var/exception/e)
log_subsystem_persistence_error("Error during json serialization for persistent object. Failed to get/encode track content: [e]")
log_subsystem_persistence_error("Error during json serialization or retrieval of content for persistent object. Type: [track.type]", e)
return result
/**
@@ -129,4 +129,4 @@
try
track.persistent_objects_apply_content(json_decode(json), x, y, z)
catch(var/exception/e)
log_subsystem_persistence_error("Error during json deserialization for persistent object. Failed to apply/decode track content: [e]")
log_subsystem_persistence_error("Error during json deserialization or applying content for persistent object. Type: [track.type]", e)