Persistency subsystem update - Generics and history records (#22114)

# Summary

This PR is the next update to the persistency subsystem. The goal of
this PR is to provide more framework like functions to allow more types
of content to be made persistent.

Currently only volatile game objects created during a round can be (*in
a [clean](https://www.youtube.com/watch?v=rZ3ETK7-ZM8) way*) saved and
made persistent. This update attempts to provide methods to make
*everything*¹ persistent.

This introduces persistent generics and history.

## Database

The following things are going to be changed and added in the database  
(open in new tab for better visibility, PNG file includes the drawIO
code):

<img width="1692" height="1041" alt="aurora_persistency_db drawio"
src="https://github.com/user-attachments/assets/ea53f419-f9aa-4592-af8f-3a8d5edf3177"
/>

**Deviations on database implementation from diagram:**  
- Removed unique constraint on history table - Prevented adding multiple
records per round per attribute.

## Framework surface changes

- MC/VV: Moved global object track register to subsystem var space.
- MC/VV: Point of interest: Added history_cache and generic_cache to
subsystem var space.
- MC/VV: Updated subsystem stat entry message, now providing information
on cache sizes of new types.
- Added `singleton/persistent_type` defines (Type, clean-up rules,
finalization hook) and macros allowing new definitions of said types.
- Added cache structures that are also used for returns on public procs
in generics and history persistent types.
- Major new framework features: Persistent history (example: Mining
yield records) and persistent generics (example: Persistent Horizon
overmap position). See documentation for more information.

DrawIO diagram for documentation, includes source in it (open in new
tab):

<img width="200" height="200" alt="Persistence-subsystem-flowchart
drawio"
src="https://github.com/user-attachments/assets/51f28331-f999-49a2-a7cc-58278f7ae416"
/>

## Tasks

(These lists are not comprehensive.)

**General**
- [x] Update DB - Write SQL scripts.
- [x] Add subsystem modular files for generics and history.
- [x] Add type definition logic, macros.
- [x] Add type-DB init logic.
- [x] Logging.
- [x] A lot of testing. *A lot.*
- [x] Changelog.
- [x] Self-Review.
- [x] Update documentation on the persistence subsystem.

**"Persistent history"**
- [x] Add init logic.
- [x] Add finalize logic.
- [x] Add framework surface procs.
  - [x] Get last record.
  - [x] Get last X records.
  - [x] Add record.
  - [x] Add character ID related validation.
- [x] Add initial example mechanic.

**"Persistent generics"**
- [x] Add init logic.
- [x] Add finalize logic.
- [x] Add framework surface procs.
  - [x] Save.
  - [x] Load.
- [x] Add initial example mechanic.

## Changes

Too many changes to be listed here - Check changelog and actual changes.

## Warning

There are certain use/test cases that *cannot* be tested locally due to
missing preexisting data in the database. This should only affect new
data structures (new persistent types), not existing data.

¹ _Large scale persistent mapping is excluded for this version._

---------

Co-authored-by: Copilot <copilot@github.com>

code/controllers/subsystems/persistence/persistence.dm

@@ -2,24 +2,43 @@
  * Persistence subsystem
  * Subsytem for managing any form of persistent content across rounds.
  *
- * This subsystem consists of multiple partial files, following the structure:
- * - persistence.dm						- Subsystem definition and generic code.
- * - persistence_objects.dm				- Persistent objects related code.
- * - persistence_objects_sql.dm			- Persistent objects database code.
- * - persistence_objects_public.dm		- Persistent objects public procs.
+ * This subsystem consists of multiple partial files, split into different responsibilities:
+ *   persistence.dm - Subsystem define and related code
+ *   Objects and types (with Generics and History respectively), each containing:
+ *     Base file (no suffix), public procs (_public.dm suffix), SQL code (_sql.dm suffix)
  */
 
 SUBSYSTEM_DEF(persistence)
 	name = "Persistence"
 	init_order = INIT_ORDER_PERSISTENCE // The order is tied with the init and maploading subsystem.
 	flags = SS_NO_FIRE // This subsystem has no continues workload, it's init and shutdown only.
-	var/prevent_saving = FALSE // Toggle to prevent saving at round end, changed by toggle_persistence proc, used for admin purposes.
+	/// Sanity check to confirm init was a success before finalizing.
+	var/init_success = FALSE
+	/// Global toggle to prevent saving at round end, changed by toggle_persistence proc, used for admin purposes.
+	var/prevent_saving = FALSE
+	/// In-memory register of all persistent objects that were loaded or created during the round, used for tracking and finalization purposes.
+	var/object_track_register = list()
+	/// Dictionary<"[type](+[attribute])" cache of persistent history records.
+	var/history_cache = alist()
+	/// Manual record counter of cache containers.
+	var/history_cache_count = 0
+	/// ID of last found history record.
+	/// Higher found IDs mean the record is not yet found in the database, lower or equal found ID means the are record that are already in the database.
+	/// Used during history_virtual_id init and read-through cache hits.
+	var/history_last_database_id = 0
+	/// ID used for instanciating new history records during the round, used for cache tracking.
+	/// Their database ID will be set during insert/finalization.
+	var/history_virtual_id = 0
+	/// Dictionary<char_id, charname> cache of Character name by ID for history/character helper.
+	var/char_cache = alist()
+	/// Dictionary<"[type](+[attribute])", container> cache of persistent generics.
+	var/generic_cache = alist()
 
 /**
  * Subsystem info stub message generation.
  */
 /datum/controller/subsystem/persistence/stat_entry(msg)
-	msg = ("Register: [length(GLOB.persistence_object_track_register)] | Prevent saving: [SSpersistence.prevent_saving ? "TRUE" : "FALSE"]")
+	msg = ("[init_success ? "" : "INIT FAILED!!!|"][prevent_saving ? "SAVING DISABLED!|" : ""]Objects:[length(object_track_register)]|Containers:[length(history_cache)];Records:[history_cache_count]|Generics:[length(generic_cache)]")
 	return msg
 
 /**
@@ -77,7 +96,7 @@ SUBSYSTEM_DEF(persistence)
 	else
 		return
 
-	feedback_add_details("admin_verb","TP") //If you are copy-pasting this, ensure the 2nd parameter is unique to the new proc!
+	feedback_add_details("admin_verb","TPS") //If you are copy-pasting this, ensure the 2nd parameter is unique to the new proc!
 
 /**
  * Initialization of the persistence subsystem.
@@ -90,14 +109,22 @@ SUBSYSTEM_DEF(persistence)
 		return SS_INIT_SUCCESS
 
 	if(!databaseCheckConnection("subsystem init"))
+		log_subsystem_persistence_error("SQL connection unavailable. Init not possible.")
 		return SS_INIT_FAILURE
 
 	try
 		objectsInitialize()
-	catch(var/exception/e)
-		log_subsystem_persistence_panic("Unhandled exception during persistent objects initialization: [e]")
+	catch(var/exception/e_objects)
+		log_subsystem_persistence_panic("Unhandled exception during persistent objects initialization!", e_objects)
 		return SS_INIT_FAILURE
 
+	try
+		typesInitialize()
+	catch(var/exception/e_types)
+		log_subsystem_persistence_panic("Unhandled exception during persistent type initialization!", e_types)
+		return SS_INIT_FAILURE
+
+	init_success = TRUE
 	return SS_INIT_SUCCESS
 
 /**
@@ -105,6 +132,10 @@ SUBSYSTEM_DEF(persistence)
  * The shutdown consists of finalization steps for each persistent data type.
  */
 /datum/controller/subsystem/persistence/Shutdown()
+	if(!init_success)
+		log_subsystem_persistence_panic("Init success flag is FALSE. Something went wrong during subsystem init! Aborting finalization to prevent corrupt data!")
+		return
+
 	if(prevent_saving)
 		log_subsystem_persistence_warning("Persistence subsystem was toggled to not save. Skipping subsystem finalization.")
 		return
@@ -115,6 +146,10 @@ SUBSYSTEM_DEF(persistence)
 
 	try
 		objectsFinalize()
-	catch(var/exception/e)
-		log_subsystem_persistence_panic("Unhandled exception during persistent objects finalization: [e]")
-		return
+	catch(var/exception/e_objects)
+		log_subsystem_persistence_panic("Unhandled exception during persistent objects finalization!", e_objects)
+
+	try
+		typesFinalize()
+	catch(var/exception/e_types)
+		log_subsystem_persistence_panic("Unhandled exception during persistent types finalization!", e_types)