Persistency subsystem update - Generics and history records (#22114)

# Summary This PR is the next update to the persistency subsystem. The goal of this PR is to provide more framework like functions to allow more types of content to be made persistent. Currently only volatile game objects created during a round can be (*in a [clean](https://www.youtube.com/watch?v=rZ3ETK7-ZM8) way*) saved and made persistent. This update attempts to provide methods to make *everything*¹ persistent. This introduces persistent generics and history. ## Database The following things are going to be changed and added in the database (open in new tab for better visibility, PNG file includes the drawIO code): <img width="1692" height="1041" alt="aurora_persistency_db drawio" src="https://github.com/user-attachments/assets/ea53f419-f9aa-4592-af8f-3a8d5edf3177" /> **Deviations on database implementation from diagram:** - Removed unique constraint on history table - Prevented adding multiple records per round per attribute. ## Framework surface changes - MC/VV: Moved global object track register to subsystem var space. - MC/VV: Point of interest: Added history_cache and generic_cache to subsystem var space. - MC/VV: Updated subsystem stat entry message, now providing information on cache sizes of new types. - Added `singleton/persistent_type` defines (Type, clean-up rules, finalization hook) and macros allowing new definitions of said types. - Added cache structures that are also used for returns on public procs in generics and history persistent types. - Major new framework features: Persistent history (example: Mining yield records) and persistent generics (example: Persistent Horizon overmap position). See documentation for more information. DrawIO diagram for documentation, includes source in it (open in new tab): <img width="200" height="200" alt="Persistence-subsystem-flowchart drawio" src="https://github.com/user-attachments/assets/51f28331-f999-49a2-a7cc-58278f7ae416" /> ## Tasks (These lists are not comprehensive.) **General** - [x] Update DB - Write SQL scripts. - [x] Add subsystem modular files for generics and history. - [x] Add type definition logic, macros. - [x] Add type-DB init logic. - [x] Logging. - [x] A lot of testing. *A lot.* - [x] Changelog. - [x] Self-Review. - [x] Update documentation on the persistence subsystem. **"Persistent history"** - [x] Add init logic. - [x] Add finalize logic. - [x] Add framework surface procs. - [x] Get last record. - [x] Get last X records. - [x] Add record. - [x] Add character ID related validation. - [x] Add initial example mechanic. **"Persistent generics"** - [x] Add init logic. - [x] Add finalize logic. - [x] Add framework surface procs. - [x] Save. - [x] Load. - [x] Add initial example mechanic. ## Changes Too many changes to be listed here - Check changelog and actual changes. ## Warning There are certain use/test cases that *cannot* be tested locally due to missing preexisting data in the database. This should only affect new data structures (new persistent types), not existing data. ¹ _Large scale persistent mapping is excluded for this version._ --------- Co-authored-by: Copilot <copilot@github.com>
2026-08-22 04:22:39 +01:00 · 2026-07-19 20:56:17 +02:00
parent 4eccb61c31
commit 802d3fe01f
39 changed files with 3534 additions and 2112 deletions
@@ -90,7 +90,7 @@

 # Persistence, Fabian's area
 **/persistence @Arrow768 @NonQueueingMatt @FabianK3
-/code/__DEFINES/persistence.dm @Arrow768 @NonQueueingMatt @FabianK3
+/code/__DEFINES/persistence/ @Arrow768 @NonQueueingMatt @FabianK3
 /code/__HELPERS/logging/subsystems/persistence.dm @Arrow768 @NonQueueingMatt @FabianK3
 /code/controllers/subsystems/persistence/ @Arrow768 @NonQueueingMatt @FabianK3

@@ -343,6 +343,45 @@ jobs:
          tools/bootstrap/python -m mapmerge2.dmm_test
          tools/bootstrap/python -m tools.maplint.source

+  ##################################################
+  ############### Forbidden map types ##############
+  ##################################################
+  lint-forbidden-map-types:
+    name: Lint Forbidden map types
+    runs-on: ubuntu-24.04
+    needs: validate-structure
+    if: needs.validate-structure.outputs.skipci != 'true'
+
+    concurrency:
+      group: lint-forbidden-map-types-${{ github.head_ref || github.run_id }}
+      cancel-in-progress: true
+
+    steps:
+      #Checkout the repository
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      #Initialize the environment variables
+      - name: Set ENV variables
+        run: bash dependencies.sh
+
+      #Restores python cache
+      - name: Restore Python Cache
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+          cache: "pip"
+
+      #Install python packages and tools
+      - name: Install Python Packages
+        run: |
+          pip install -r tools/requirements.txt
+          pip3 install setuptools
+
+      - name: Check forbidden map types
+        run: |
+          tools/bootstrap/python tools/findForbiddenMapTypes.py
+
  ###########################################
  ############## GENERIC TESTS ##############
  ###########################################
@@ -0,0 +1,41 @@
+-- PR: https://github.com/Aurorastation/Aurora.3/pull/22114
+
+-- Updates to existing objects table
+ALTER TABLE `ss13_persistent_objects` MODIFY COLUMN `x` INT NOT NULL;
+ALTER TABLE `ss13_persistent_objects` MODIFY COLUMN `y` INT NOT NULL;
+ALTER TABLE `ss13_persistent_objects` MODIFY COLUMN `z` INT NOT NULL;
+ALTER TABLE `ss13_persistent_objects` MODIFY COLUMN `content` MEDIUMTEXT NULL;
+
+-- New persistent content types - Generics and records
+CREATE TABLE `ss13_persistent_type_definitions` (
+    `id` INT AUTO_INCREMENT PRIMARY KEY,
+    `type` VARCHAR(128) NOT NULL UNIQUE,
+    `title` VARCHAR(128) NOT NULL,
+    `description` VARCHAR(256) NOT NULL,
+    `definition_type` INT NOT NULL COMMENT '1 = GENERIC, 2 = HISTORY'
+);
+
+CREATE TABLE `ss13_persistent_generics` (
+    `id` INT AUTO_INCREMENT PRIMARY KEY,
+    `type` INT NOT NULL,
+    `attribute` VARCHAR(64) NULL,
+    `created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    `expires_at` DATETIME NOT NULL,
+    `content` MEDIUMTEXT NOT NULL,
+    CONSTRAINT `fk_generics_type_definition` FOREIGN KEY (`type`) REFERENCES `ss13_persistent_type_definitions` (`id`),
+    CONSTRAINT `unique_generics_type_attribute` UNIQUE (`type`, `attribute`),
+    INDEX `idx_generics_attribute` (`attribute`),
+    INDEX `idx_generics_expires_at` (`expires_at`)
+);
+
+CREATE TABLE `ss13_persistent_history` (
+    `id` INT AUTO_INCREMENT PRIMARY KEY,
+    `type` INT NOT NULL,
+    `created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    `attribute` VARCHAR(64) NULL,
+    `value` VARCHAR(64) NOT NULL,
+    `game_id` VARCHAR(30) NOT NULL,
+    CONSTRAINT `fk_history_type_definition` FOREIGN KEY (`type`) REFERENCES `ss13_persistent_type_definitions` (`id`),
+    INDEX `idx_history_created_at` (`created_at`),
+    INDEX `idx_history_attribute` (`attribute`)
+);
@@ -117,7 +117,6 @@
 #include "code\__DEFINES\outfit.dm"
 #include "code\__DEFINES\overmap.dm"
 #include "code\__DEFINES\path.dm"
-#include "code\__DEFINES\persistence.dm"
 #include "code\__DEFINES\pipes.dm"
 #include "code\__DEFINES\prefs.dm"
 #include "code\__DEFINES\procpath.dm"
@@ -201,6 +200,10 @@
 #include "code\__DEFINES\dcs\signals\signals_mob\signals_mob_living.dm"
 #include "code\__DEFINES\dcs\signals\signals_mob\signals_mob_main.dm"
 #include "code\__DEFINES\dcs\signals\signals_object\signals_object.dm"
+#include "code\__DEFINES\persistence\cleanup.dm"
+#include "code\__DEFINES\persistence\models.dm"
+#include "code\__DEFINES\persistence\type_defines.dm"
+#include "code\__DEFINES\persistence\types.dm"
 #include "code\__HELPERS\_global_objects.dm"
 #include "code\__HELPERS\_lists.dm"
 #include "code\__HELPERS\area_movement.dm"
@@ -444,6 +447,12 @@
 #include "code\controllers\subsystems\persistence\persistence_objects.dm"
 #include "code\controllers\subsystems\persistence\persistence_objects_public.dm"
 #include "code\controllers\subsystems\persistence\persistence_objects_sql.dm"
+#include "code\controllers\subsystems\persistence\persistence_types.dm"
+#include "code\controllers\subsystems\persistence\persistence_types_generic_public.dm"
+#include "code\controllers\subsystems\persistence\persistence_types_generic_sql.dm"
+#include "code\controllers\subsystems\persistence\persistence_types_history_public.dm"
+#include "code\controllers\subsystems\persistence\persistence_types_history_sql.dm"
+#include "code\controllers\subsystems\persistence\persistence_types_sql.dm"
 #include "code\controllers\subsystems\processing\airflow.dm"
 #include "code\controllers\subsystems\processing\calamity.dm"
 #include "code\controllers\subsystems\processing\disease.dm"
@@ -109,10 +109,6 @@ GLOBAL_VAR(custom_event_msg)
 GLOBAL_DATUM(dbcon, /DBConnection)
 GLOBAL_PROTECT(dbcon)

-// Persistence subsystem object track register - List of all persistent objects tracked by the subsystem.
-GLOBAL_LIST_EMPTY(persistence_object_track_register)
-GLOBAL_PROTECT(persistence_object_track_register)
-
 // Added for Xenoarchaeology, might be useful for other stuff.
 GLOBAL_LIST_INIT(alphabet_uppercase, list("A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z"))

@@ -1,6 +0,0 @@
-/*#############################################
-	Constants for the persistence subsystem
-#############################################*/
-
-#define PERSISTENT_DEFAULT_EXPIRATION_DAYS 30 // Default expire timespan for newly created persistent objects
-#define PERSISTENT_EXPIRATION_CLEANUP_DELAY_DAYS 30 // Grace period for expired database entries before they get cleaned up.
@@ -0,0 +1,58 @@
+/*####################################################
+	Defines for cleanups and expirations
+####################################################*/
+
+#define PERSISTENT_DEFAULT_EXPIRATION_DAYS 30 // Default expire timespan for newly created persistent content
+#define PERSISTENT_EXPIRATION_CLEANUP_DELAY_DAYS 30 // Grace period for expired database entries before they get cleaned up, objects only
+
+// ##### Persistent type "history" expiration rules
+// Rules are applied on the combination type+attribute
+// See type definition macros on their usage
+// Abstract marked types are used for type catching in code and not to be used in type definitions
+
+ABSTRACT_TYPE(/singleton/persistent_type_history_expiration_rule)
+
+// Keep last X rows - This rule removes all records exceeding the newest X records by count
+ABSTRACT_TYPE(/singleton/persistent_type_history_expiration_rule/row_count)
+	var/max_row_count = 0
+
+/singleton/persistent_type_history_expiration_rule/row_count/ten
+	max_row_count = 10
+
+/singleton/persistent_type_history_expiration_rule/row_count/hundred
+	max_row_count = 100
+
+/singleton/persistent_type_history_expiration_rule/row_count/thousand
+	max_row_count = 1000
+
+// Keep records for X rounds - This rule removes all records that don't belong to the last X finished rounds
+ABSTRACT_TYPE(/singleton/persistent_type_history_expiration_rule/round_count)
+	var/max_round_count = 0
+
+/singleton/persistent_type_history_expiration_rule/round_count/ten
+	max_round_count = 10
+
+/singleton/persistent_type_history_expiration_rule/round_count/fifty
+	max_round_count = 50
+
+/singleton/persistent_type_history_expiration_rule/round_count/hundred
+	max_round_count = 100
+
+// Keep records for X days - This rule removes all records that are older then X days since their creation
+ABSTRACT_TYPE(/singleton/persistent_type_history_expiration_rule/age)
+	var/max_age_days = 0
+
+/singleton/persistent_type_history_expiration_rule/age/default
+	max_age_days = PERSISTENT_DEFAULT_EXPIRATION_DAYS
+
+/singleton/persistent_type_history_expiration_rule/age/week
+	max_age_days = 7
+
+/singleton/persistent_type_history_expiration_rule/age/quarter_year
+	max_age_days = 90
+
+/singleton/persistent_type_history_expiration_rule/age/half_year
+	max_age_days = 180
+
+/singleton/persistent_type_history_expiration_rule/age/year
+	max_age_days = 365
@@ -0,0 +1,22 @@
+/*###################################################
+	Subsystem cache structures
+###################################################*/
+
+// Data transfer objects - Used by the subsystem for aggregation and result returns, these should be treated as read-only when handed by the subsystem
+
+/datum/persistent_record_container // Container for combining records of type(+attribute)
+	var/singleton/persistent_type/history/type_define = null // Definition type
+	var/attribute = null // Attribute for aggregation records into type+attribute groups
+	var/list/datum/persistent_record/records = list() // Container contents
+
+/datum/persistent_record // Single persistent record
+	var/id = 0 // Database ID - Might be a non-existend (virtual) ID if the record hasn't been saved yet
+	var/created_at = "" // Timestamp when the record was saved
+	var/game_id = "" // Game Id when the record was saved
+	var/value = null // Treat this as string value
+
+/datum/persistent_generic // Persistent generic data holder
+	var/singleton/persistent_type/history/type_define = null // Definition type
+	var/attribute = null // Attribute for aggregation into type+attribute
+	var/content = null // Treat this as a string - Open for implementing caller (e.g. raw string or json)
+	var/expires_in_days = PERSISTENT_EXPIRATION_CLEANUP_DELAY_DAYS // Expiration timespan used when generic is saved
@@ -0,0 +1,78 @@
+/*###################################################
+	Base types and macros for type definitions
+###################################################*/
+
+// ##### Base type definitions
+
+// Persistent type definition found in database
+ABSTRACT_TYPE(/singleton/persistent_type)
+	var/database_id = 0 // Set during subsystem init - DO NOT MODIFY
+	var/definition_type_value = 0 // DO NOT MODIFY - DATABASE CONSTANT
+	var/title = ""
+	var/description = ""
+	var/requires_attribute = FALSE // Whether or not this type requires/has an attribute, relevant for subsystem when saving or pulling type data - Should not be changed after release for the given type
+
+// Hard coded in "ss13_persistent_type_definitions.definition_type", DO NOT MODIFY - DATABASE CONSTANTS
+#define PERSISTENCE_INTERNAL_TYPE_DEFINE_TYPE_VALUE_GENERIC 1
+#define PERSISTENCE_INTERNAL_TYPE_DEFINE_TYPE_VALUE_HISTORY 2
+
+/**
+ *	Hook proc that is called by the subsystem starting finalization on each persistent type definition.
+ *  Hooks are used for implementing finalization logic for mechanics that either
+ *  don't have a single trigger to save or where repetitive saving would be too costly.
+ *  Should return nothing, returned values are discarded.
+ *  Implementation can be put where applicable, e.g. next to loading logic of the type.
+ */
+/singleton/persistent_type/proc/finalization_hook()
+	SHOULD_CALL_PARENT(FALSE)
+	return
+
+ABSTRACT_TYPE(/singleton/persistent_type/generic) // Base type for "persistent generics"
+	definition_type_value = PERSISTENCE_INTERNAL_TYPE_DEFINE_TYPE_VALUE_GENERIC // DO NOT MODIFY - DATABASE CONSTANT
+
+ABSTRACT_TYPE(/singleton/persistent_type/history) // Base type for "persistent history"
+	definition_type_value = PERSISTENCE_INTERNAL_TYPE_DEFINE_TYPE_VALUE_HISTORY // DO NOT MODIFY - DATABASE CONSTANT
+	var/singleton/persistent_type_history_expiration_rule/expiration_rule = null
+
+ABSTRACT_TYPE(/singleton/persistent_type/history/character) // Base type with extended validation for persistent history in relation to characters
+	// Empty stub
+
+// ##### Macros for new custom type definitions
+// - TYPE_NAME = Name of the type definition, used to upsert into database, cannot be updated - Changes result in a new type definition in the DB
+// - TITLE = Title of the type definition, used for display purposes
+// - DESCRIPTION = Description of the type definition, used for display purposes
+// - REQUIRES_ATTRIBUTE = Boolean, whether this type definition requires an attribute to be specified
+// - EXPIRATION_RULE = For history type definitions, the expiration rule to apply to records of this type.
+//					   See /singleton/persistent_type_history_expiration_rule and subtypes for available rules.
+
+// Persistent generic
+// CREATE_PERSISTENT_TYPE_GENERIC(my_type_name, "My custom type", "This type is a test and has no purpose", TRUE)
+#define CREATE_PERSISTENT_TYPE_GENERIC(TYPE_NAME, TITLE, DESCRIPTION, REQUIRES_ATTRIBUTE) \
+	/singleton/persistent_type/generic/##TYPE_NAME \
+	{ \
+		title = #TITLE; \
+		description = #DESCRIPTION; \
+		requires_attribute = ##REQUIRES_ATTRIBUTE; \
+	}
+
+// Persistent history
+// CREATE_PERSISTENT_TYPE_HISTORY(my_type_name, "My custom type", "This type is a test and has no purpose", TRUE, /singleton/persistent_type_history_expiration_rule/age/default)
+#define CREATE_PERSISTENT_TYPE_HISTORY(TYPE_NAME, TITLE, DESCRIPTION, REQUIRES_ATTRIBUTE, EXPIRATION_RULE) \
+	/singleton/persistent_type/history/##TYPE_NAME \
+	{ \
+		title = #TITLE; \
+		description = #DESCRIPTION; \
+		requires_attribute = ##REQUIRES_ATTRIBUTE; \
+		expiration_rule = ##EXPIRATION_RULE; \
+	}
+
+// Persistent history with extended validation on character relation - Attribute automatically required compared to parent persistent history type definition
+// CREATE_PERSISTENT_TYPE_HISTORY_CHARACTER(my_type_name, "My custom type", "This type is a test and has no purpose", /singleton/persistent_type_history_expiration_rule/age/default)
+#define CREATE_PERSISTENT_TYPE_HISTORY_CHARACTER(TYPE_NAME, TITLE, DESCRIPTION, EXPIRATION_RULE) \
+	/singleton/persistent_type/history/character/##TYPE_NAME \
+	{ \
+		title = #TITLE; \
+		description = #DESCRIPTION; \
+		requires_attribute = TRUE; \
+		expiration_rule = ##EXPIRATION_RULE; \
+	}
@@ -0,0 +1,27 @@
+/*###################################################
+	Type definitions
+###################################################*/
+
+// Type definitions using macros - This file contains all types getting made accessible in code and later in database during first subsystem run
+// See macros for detailed information on parameters and the underlaying types
+// Generally, after being released (as in, run against the database once), these should NOT be modified if possible, explicit warnings:
+
+//		+------------------------------------------------------------------------------------------------------------------+
+//		| MODIFYING THE TYPE NAME OF EXISTING DEFINES WILL CREATE A NEW TYPE DEFINITION IN THE DATABASE                    |
+//		+------------------------------------------------------------------------------------------------------------------+
+//		| MODIFYING THE ATTRIBUTE FLAG OF EXISTING DEFINES CAN HAVE BREAKING CONSEQUENCES, TESTING REQUIRED BEFORE RELEASE |
+//		+------------------------------------------------------------------------------------------------------------------+
+
+// Below are the defines for generics, history and history with character validation
+
+// ##### Persistent generics
+
+CREATE_PERSISTENT_TYPE_GENERIC(horizon_overmap_position, "SCCV Horizon sector position", "Position of the SCCV Horizon on the overmap.", FALSE)
+
+// ##### Persistent history
+
+
+
+// ##### Persistent history with character validation
+
+CREATE_PERSISTENT_TYPE_HISTORY_CHARACTER(mining_points, "Mining yield history", "History of mining points yield of individual miners.", /singleton/persistent_type_history_expiration_rule/age/week)
@@ -426,18 +426,24 @@
 /**
 * List of lists, sorts by element[key] - for things like crew monitoring computer sorting records by name.
 */
-/proc/sortByKey(var/list/L, var/key)
+/proc/sortByKeyText(var/list/L, var/key)
 	if(L.len < 2)
 		return L
 	var/middle = L.len / 2 + 1
-	return mergeKeyedLists(sortByKey(L.Copy(0, middle), key), sortByKey(L.Copy(middle), key), key)
+	return mergeKeyedLists(sortByKeyText(L.Copy(0, middle), key), sortByKeyText(L.Copy(middle), key), key)

-/proc/mergeKeyedLists(var/list/L, var/list/R, var/key)
+/proc/sortByKeyNumber(var/list/L, var/key)
+	if(L.len < 2)
+		return L
+	var/middle = L.len / 2 + 1
+	return mergeKeyedLists(sortByKeyText(L.Copy(0, middle), key), sortByKeyText(L.Copy(middle), key), key, value_is_number = TRUE)
+
+/proc/mergeKeyedLists(var/list/L, var/list/R, var/key, var/value_is_number = FALSE)
 	var/Li=1
 	var/Ri=1
 	var/list/result = new()
 	while(Li <= L.len && Ri <= R.len)
-		if(sorttext(L[Li][key], R[Ri][key]) < 1)
+		if((value_is_number && L[Li][key] < R[Ri][key]) || (!value_is_number && sorttext(L[Li][key], R[Ri][key]) < 1))
 			// Works around list += list2 merging lists; it's not pretty but it works.
 			result += "temp item"
 			result[result.len] = R[Ri++]
@@ -12,8 +12,18 @@
 /proc/log_subsystem_persistence_warning(text)
 	log_subsystem_persistence("WARNING: [text]")

-/proc/log_subsystem_persistence_error(text)
-	log_subsystem_persistence("ERROR: [text]")
+/proc/log_subsystem_persistence_error(text, exception/e = null)
+	if(e)
+		log_subsystem_persistence("ERROR: [text] - [e]")
+		if(SSsentry)
+			SSsentry.capture_exception(e)
+	else
+		log_subsystem_persistence("ERROR: [text]")

-/proc/log_subsystem_persistence_panic(text)
-	log_subsystem_persistence("PANIC: [text]")
+/proc/log_subsystem_persistence_panic(text, exception/e = null)
+	if(e)
+		log_subsystem_persistence("PANIC: [text] - [e]")
+		if(SSsentry)
+			SSsentry.capture_exception(e)
+	else
+		log_subsystem_persistence("PANIC: [text]")
@@ -2,24 +2,43 @@
 * Persistence subsystem
 * Subsytem for managing any form of persistent content across rounds.
 *
- * This subsystem consists of multiple partial files, following the structure:
- * - persistence.dm						- Subsystem definition and generic code.
- * - persistence_objects.dm				- Persistent objects related code.
- * - persistence_objects_sql.dm			- Persistent objects database code.
- * - persistence_objects_public.dm		- Persistent objects public procs.
+ * This subsystem consists of multiple partial files, split into different responsibilities:
+ *   persistence.dm - Subsystem define and related code
+ *   Objects and types (with Generics and History respectively), each containing:
+ *     Base file (no suffix), public procs (_public.dm suffix), SQL code (_sql.dm suffix)
 */

 SUBSYSTEM_DEF(persistence)
 	name = "Persistence"
 	init_order = INIT_ORDER_PERSISTENCE // The order is tied with the init and maploading subsystem.
 	flags = SS_NO_FIRE // This subsystem has no continues workload, it's init and shutdown only.
-	var/prevent_saving = FALSE // Toggle to prevent saving at round end, changed by toggle_persistence proc, used for admin purposes.
+	/// Sanity check to confirm init was a success before finalizing.
+	var/init_success = FALSE
+	/// Global toggle to prevent saving at round end, changed by toggle_persistence proc, used for admin purposes.
+	var/prevent_saving = FALSE
+	/// In-memory register of all persistent objects that were loaded or created during the round, used for tracking and finalization purposes.
+	var/object_track_register = list()
+	/// Dictionary<"[type](+[attribute])" cache of persistent history records.
+	var/history_cache = alist()
+	/// Manual record counter of cache containers.
+	var/history_cache_count = 0
+	/// ID of last found history record.
+	/// Higher found IDs mean the record is not yet found in the database, lower or equal found ID means the are record that are already in the database.
+	/// Used during history_virtual_id init and read-through cache hits.
+	var/history_last_database_id = 0
+	/// ID used for instanciating new history records during the round, used for cache tracking.
+	/// Their database ID will be set during insert/finalization.
+	var/history_virtual_id = 0
+	/// Dictionary<char_id, charname> cache of Character name by ID for history/character helper.
+	var/char_cache = alist()
+	/// Dictionary<"[type](+[attribute])", container> cache of persistent generics.
+	var/generic_cache = alist()

 /**
 * Subsystem info stub message generation.
 */
 /datum/controller/subsystem/persistence/stat_entry(msg)
-	msg = ("Register: [length(GLOB.persistence_object_track_register)] | Prevent saving: [SSpersistence.prevent_saving ? "TRUE" : "FALSE"]")
+	msg = ("[init_success ? "" : "INIT FAILED!!!|"][prevent_saving ? "SAVING DISABLED!|" : ""]Objects:[length(object_track_register)]|Containers:[length(history_cache)];Records:[history_cache_count]|Generics:[length(generic_cache)]")
 	return msg

 /**
@@ -77,7 +96,7 @@ SUBSYSTEM_DEF(persistence)
 	else
 		return

-	feedback_add_details("admin_verb","TP") //If you are copy-pasting this, ensure the 2nd parameter is unique to the new proc!
+	feedback_add_details("admin_verb","TPS") //If you are copy-pasting this, ensure the 2nd parameter is unique to the new proc!

 /**
 * Initialization of the persistence subsystem.
@@ -90,14 +109,22 @@ SUBSYSTEM_DEF(persistence)
 		return SS_INIT_SUCCESS

 	if(!databaseCheckConnection("subsystem init"))
+		log_subsystem_persistence_error("SQL connection unavailable. Init not possible.")
 		return SS_INIT_FAILURE

 	try
 		objectsInitialize()
-	catch(var/exception/e)
-		log_subsystem_persistence_panic("Unhandled exception during persistent objects initialization: [e]")
+	catch(var/exception/e_objects)
+		log_subsystem_persistence_panic("Unhandled exception during persistent objects initialization!", e_objects)
 		return SS_INIT_FAILURE

+	try
+		typesInitialize()
+	catch(var/exception/e_types)
+		log_subsystem_persistence_panic("Unhandled exception during persistent type initialization!", e_types)
+		return SS_INIT_FAILURE
+
+	init_success = TRUE
 	return SS_INIT_SUCCESS

 /**
@@ -105,6 +132,10 @@ SUBSYSTEM_DEF(persistence)
 * The shutdown consists of finalization steps for each persistent data type.
 */
 /datum/controller/subsystem/persistence/Shutdown()
+	if(!init_success)
+		log_subsystem_persistence_panic("Init success flag is FALSE. Something went wrong during subsystem init! Aborting finalization to prevent corrupt data!")
+		return
+
 	if(prevent_saving)
 		log_subsystem_persistence_warning("Persistence subsystem was toggled to not save. Skipping subsystem finalization.")
 		return
@@ -115,6 +146,10 @@ SUBSYSTEM_DEF(persistence)

 	try
 		objectsFinalize()
-	catch(var/exception/e)
-		log_subsystem_persistence_panic("Unhandled exception during persistent objects finalization: [e]")
-		return
+	catch(var/exception/e_objects)
+		log_subsystem_persistence_panic("Unhandled exception during persistent objects finalization!", e_objects)
+
+	try
+		typesFinalize()
+	catch(var/exception/e_types)
+		log_subsystem_persistence_panic("Unhandled exception during persistent types finalization!", e_types)
@@ -4,7 +4,7 @@
 */
 /datum/controller/subsystem/persistence/proc/objectsInitialize()
 	PRIVATE_PROC(TRUE)
-	GLOB.persistence_object_track_register = list()
+	object_track_register = list()

 	if(SSatlas.current_map.path != "sccv_horizon") // The persistence system only supports objects from the main map levels for multiple reasons, e.g. Z level value, mapping support
 		log_subsystem_persistence_info("Persistent objects: Current map did not match SCCV Horizon, skipping persistent object initialization.")
@@ -42,8 +42,8 @@

 	if(SSatlas.current_map.path != "sccv_horizon") // The persistence system only supports objects from the main map levels for multiple reasons, e.g. Z level value, mapping support
 		log_subsystem_persistence_info("Persistent objects: Current map did not match SCCV Horizon, skipping persistent object finalization.")
-		if(length(GLOB.persistence_object_track_register) > 0)
-			log_subsystem_persistence_warning("Persistent objects: There are [length(GLOB.persistence_object_track_register)] tracked objects at finalization, while the map is not supported! These track will not be saved! Verify that SSatlas.current_map.path has not changed during the round!")
+		if(length(object_track_register) > 0)
+			log_subsystem_persistence_warning("Persistent objects: There are [length(object_track_register)] tracked objects at finalization, while the map is not supported! These track will not be saved! Verify that SSatlas.current_map.path has not changed during the round!")
 		return

 	// Subsystem shutdown:
@@ -52,7 +52,7 @@
 	// Delete persistent records that no longer exist in the registry (removed during the round)

 	// Run checks on each track that might prevent further persistence
-	for (var/obj/track as anything in GLOB.persistence_object_track_register)
+	for (var/obj/track in object_track_register)
 		CHECK_TICK
 		var/turf/T = get_turf(track)
 		if(!T || !is_station_level(T.z)) // The persistence system only supports objects from the main map levels for multiple reasons, e.g. Z level value, mapping support
@@ -65,7 +65,7 @@
 	// Get already stored data before saving new tracks so we can compare what has been updated or removed during the round.
 	var/list/existing_data = objectsDatabaseGetActiveEntries()

-	for (var/obj/track as anything in GLOB.persistence_object_track_register)
+	for (var/obj/track in object_track_register)
 		CHECK_TICK
 		if (track.persistent_objects_track_id == 0)
 			// Tracked object has no ID meaning it is new, create a new persistent record for it
@@ -76,7 +76,7 @@
 	// If we find the track, we need to check if it requires an update instead
 	for (var/record in existing_data)
 		var/found = FALSE
-		for (var/obj/track as anything in GLOB.persistence_object_track_register)
+		for (var/obj/track in object_track_register)
 			CHECK_TICK
 			if (record["id"] == track.persistent_objects_track_id)
 				// A track with the same ID has been found in the register, it still exists, check if we need to update it instead
@@ -114,7 +114,7 @@
 		if(length(content))
 			result = json_encode(content)
 	catch(var/exception/e)
-		log_subsystem_persistence_error("Error during json serialization for persistent object. Failed to get/encode track content: [e]")
+		log_subsystem_persistence_error("Error during json serialization or retrieval of content for persistent object. Type: [track.type]", e)
 	return result

 /**
@@ -129,4 +129,4 @@
 	try
 		track.persistent_objects_apply_content(json_decode(json), x, y, z)
 	catch(var/exception/e)
-		log_subsystem_persistence_error("Error during json deserialization for persistent object. Failed to apply/decode track content: [e]")
+		log_subsystem_persistence_error("Error during json deserialization or applying content for persistent object. Type: [track.type]", e)
@@ -11,7 +11,7 @@

 	new_track.persistent_objects_track_active = TRUE
 	new_track.persistent_objects_author_ckey = ckey
-	GLOB.persistence_object_track_register += new_track
+	object_track_register += new_track

 /**
 * Removes the given object from the list of tracked objects. At shutdown the tracked object will be remove from the database.
@@ -21,4 +21,4 @@
 		return

 	old_track.persistent_objects_track_active = FALSE
-	GLOB.persistence_object_track_register -= old_track
+	object_track_register -= old_track
@@ -71,7 +71,7 @@
 		list(
 			"author_ckey" = track.persistent_objects_author_ckey,
 			"type" = "[track.type]",
-			"expire_in_days" = track.persistant_objects_expiration_time_days,
+			"expire_in_days" = track.persistent_objects_expiration_time_days,
 			"content" = objectsGetTrackContent(track),
 			"x" = T.x,
 			"y" = T.y,
@@ -99,7 +99,7 @@
 		"UPDATE ss13_persistent_objects SET author_ckey=:author_ckey, expires_at=DATE_ADD(NOW(), INTERVAL :expire_in_days DAY), content=:content, x=:x, y=:y, z=:z WHERE id = :id",
 		list(
 			"author_ckey" = track.persistent_objects_author_ckey,
-			"expire_in_days" = track.persistant_objects_expiration_time_days,
+			"expire_in_days" = track.persistent_objects_expiration_time_days,
 			"content" = objectsGetTrackContent(track),
 			"x" = T.x,
 			"y" = T.y,
@@ -0,0 +1,170 @@
+/**
+ * Called during subsystem init to upssert persistent type definitions into the database.
+ */
+/datum/controller/subsystem/persistence/proc/typesInitialize()
+	PRIVATE_PROC(TRUE)
+	// Types init:
+	// Upsert all types found in code into database
+	// Get type ID of each type found in code by name lookup
+	// -- Records
+	// Init history cache
+	// Run cleanup on history records
+	// -- Generics
+	// Init generic cache
+	// Run cleanup on generics
+
+	// Types upsert
+	// Base types to exclude
+	var/base_types = list(/singleton/persistent_type, /singleton/persistent_type/generic, /singleton/persistent_type/history, /singleton/persistent_type/history/character)
+	var/custom_types = typesof(/singleton/persistent_type) - base_types // These are the types we are actually dealing with
+
+	// Upsert all persistent type definitions found in code
+	// Whether or not it's new, get its database ID
+	for (var/C in custom_types)
+		CHECK_TICK
+		var/singleton/persistent_type/T = GET_SINGLETON(C)
+		typesDatabaseUpsertType("[T]", T.title, T.description, T.definition_type_value)
+		T.database_id = typesDatabaseGetTypeIdByName("[T]")
+
+	// ### Records
+
+	// Init internal history cache
+	history_last_database_id = historyDatabaseGetLastID()
+	if(history_last_database_id == 0)
+		log_subsystem_persistence_warning("Failed to get last ID of persistent type history records from the database during initialization. Either the table is empty or something went wrong.")
+	history_virtual_id = history_last_database_id
+	history_cache = list()
+
+	// Clean history records
+	for(var/type_combination in historyDatabaseGetTypeAttributeCombinations()) // Iterate through each distinct type+attribute combination
+		CHECK_TICK
+		var/type_id = type_combination["type_id"]
+		var/attribute = type_combination["attribute"]
+		var/singleton/persistent_type/history/found_type
+		for (var/C in custom_types)
+			var/singleton/persistent_type/T = GET_SINGLETON(C)
+			if(istype(T, /singleton/persistent_type/history) && T.database_id == type_id)
+				found_type = T
+		if(!found_type)
+			continue // The type found in the database is no longer available in the codebase
+
+		// Clean by the individual cleanup rule
+		if(ispath(found_type.expiration_rule, /singleton/persistent_type_history_expiration_rule/row_count)) // row_count
+			var/singleton/persistent_type_history_expiration_rule/row_count/rule = GET_SINGLETON(found_type.expiration_rule)
+			historyDatabaseCleanByRowCount(found_type.database_id, attribute, rule.max_row_count)
+
+		if(ispath(found_type.expiration_rule, /singleton/persistent_type_history_expiration_rule/round_count)) // round_count
+			var/singleton/persistent_type_history_expiration_rule/round_count/rule = GET_SINGLETON(found_type.expiration_rule)
+			historyDatabaseCleanByRoundCount(found_type.database_id, attribute, rule.max_round_count)
+
+		if(ispath(found_type.expiration_rule, /singleton/persistent_type_history_expiration_rule/age)) // age
+			var/singleton/persistent_type_history_expiration_rule/age/rule = GET_SINGLETON(found_type.expiration_rule)
+			historyDatabaseCleanByMaxAgeDays(found_type.database_id, attribute, rule.max_age_days)
+
+	// ### Generics
+
+	// Init internal generic cache
+	generic_cache = alist()
+	// Cleanup
+	genericDatabaseCleanup()
+
+	// ### Char lookup
+	char_cache = alist()
+
+/**
+ * Finalize persistent types.
+ * Adds new persistent generics and history.
+ */
+/datum/controller/subsystem/persistence/proc/typesFinalize()
+	PRIVATE_PROC(TRUE)
+
+	// Subsystem shutdown:
+	// Call finalization hook for each known persistent_type
+	// Save all history records in cache to database which have an ID higher then last known database ID - Records created during the round.
+	// Save all generics in cache to database which have an ID higher then last known database ID - Generics created during the round.
+
+	// ##### Hooks
+	var/base_types = list(/singleton/persistent_type, /singleton/persistent_type/generic, /singleton/persistent_type/history, /singleton/persistent_type/history/character)
+	var/custom_types = typesof(/singleton/persistent_type) - base_types // These are the types we are actually dealing with
+	for (var/C in custom_types)
+		CHECK_TICK
+		var/singleton/persistent_type/type_instance = GET_SINGLETON(C)
+		try
+			type_instance.finalization_hook()
+		catch(var/exception/e)
+			log_subsystem_persistence_error("Unhandled exception during finalization_hook of [type_instance]", e)
+
+	// ##### Saving history
+	var/total_saved_count = 0
+	for(var/key in history_cache)
+		CHECK_TICK
+		var/datum/persistent_record_container/container = history_cache[key] // Dictionary<"[type](+[attribute])", container>
+		if(!length(container.records))
+			continue // Container was queried, got no hits and nothing was added.
+
+		var/list/datum/persistent_record/new_records = list()
+		for(var/datum/persistent_record/record in container.records)
+			if(record.id > history_last_database_id) // ID assigned by virtual ID is larger then last known database ID, record is new and needs to be saved.
+				new_records += record
+
+		for(var/datum/persistent_record/record in new_records)
+			var/singleton/persistent_type/type_instance = GET_SINGLETON(container.type_define)
+			historyDatabaseInsertRecord(type_instance.database_id, container.attribute, record.value)
+			total_saved_count++
+
+	log_subsystem_persistence_info("Saved new [length(total_saved_count)] persistent history records.")
+
+	// ##### Saving generics
+	for(var/key in generic_cache)
+		CHECK_TICK
+		var/datum/persistent_generic/container = generic_cache[key] // Dictionary<"[type](+[attribute])", container>
+		var/singleton/persistent_type/type_instance = GET_SINGLETON(container.type_define)
+		genericDatabaseSave(type_instance.database_id, container.attribute, container.expires_in_days, container.content)
+
+	log_subsystem_persistence_info("Saved [length(generic_cache)] persistent generics.")
+
+/**
+ * Internal proc for assigning new IDs to history records, these are used for internal cache tracking and will be discard by database IDs at finalization.
+ */
+/datum/controller/subsystem/persistence/proc/typesGetVirtualRecordID()
+	PRIVATE_PROC(TRUE)
+	history_virtual_id += 1
+	return history_virtual_id
+
+/**
+ * Internal proc for finding the top K records (by ID) in a record container.
+ * Top K insertion sort selection (Manual leaderboard sort).
+ * PARAMS:
+ * 	k = Number of records to return.
+ * 	Container = Container to search in.
+ * RETURN:
+ * 	List of top K records by ID, sorted from highest to lowest.
+ */
+/datum/controller/subsystem/persistence/proc/typesHistoryCacheSelectTopK(k, datum/persistent_record_container/container)
+	PRIVATE_PROC(TRUE)
+	if(length(container.records))
+		return list()
+
+	var/list/datum/persistent_record/top = list()
+
+	for(var/datum/persistent_record/r in container.records)
+		var/insert_pos = 1
+
+		// Find position for insert when top isn't full yet or when a value in top is smaller then current record to be replaced
+		while(insert_pos <= top.len && top[insert_pos].id > r.id)
+			insert_pos++
+
+		if(top.len < k) // Top isn't full yet, insert without cutting
+			top.Insert(insert_pos, r)
+		else if(r.id > top[top.len].id) // Top is full, replace next lowest pos with current record and cut list back to size k
+			top.Insert(insert_pos, r)
+			top.Cut(k+1)
+
+	return top
+
+/datum/controller/subsystem/persistence/proc/typesGetCacheName(var/singleton/persistent_type/target_type, var/attribute)
+	PRIVATE_PROC(TRUE)
+	if(attribute && length(attribute) > 0)
+		return "[target_type]+[attribute]"
+	else
+		return "[target_type]"
@@ -0,0 +1,72 @@
+/**
+ * Saves or overrides generic content for a type(+attribute)
+ * PARAMS:
+ * 	target_type =		Singleton persistent type definition. See /singleton/persistent_type/generic and subtypes.
+ *  content = 			List of associative values to be saved. ("id" = 123, "value" = "lorem ipsum")
+ *  attribute =			Custom attribute of the generic, can be null if the type definition doesn't require it. Defaults to null.
+ *	expires_in_days =	Days until the content is deemed expired. Defaults to PERSISTENT_DEFAULT_EXPIRATION_DAYS.
+ */
+/datum/controller/subsystem/persistence/proc/genericSave(var/singleton/persistent_type/generic/target_type, content, attribute = null, expires_in_days = PERSISTENT_DEFAULT_EXPIRATION_DAYS)
+	if(!content || !length(content))
+		return
+
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to add generic with null target type.")
+		return
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	if(type_instance.requires_attribute && !length(attribute))
+		log_subsystem_persistence_warning("Attempted to add generic of type [target_type] without required attribute.")
+		return
+
+	if(!expires_in_days || expires_in_days <= 0)
+		expires_in_days = PERSISTENT_DEFAULT_EXPIRATION_DAYS
+
+	attribute = length("[attribute]") > 0 ? attribute : null
+	var/datum/persistent_generic/generic = generic_cache[typesGetCacheName(target_type, attribute)]
+	if(generic)
+		generic.content = json_encode(content)
+		generic.expires_in_days = expires_in_days
+		return
+
+	var/datum/persistent_generic/new_generic = new /datum/persistent_generic/
+	new_generic.type_define = target_type
+	new_generic.attribute = attribute
+	new_generic.content = json_encode(content)
+	new_generic.expires_in_days = expires_in_days
+	generic_cache[typesGetCacheName(target_type, attribute)] = new_generic
+
+/**
+ * Retrieve/Loads generic content of a type(+attribute)
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type/generic and subtypes.
+ *  attribute =		Custom attribute of the generic, can be null if the type definition doesn't require it. Defaults to null.
+ * RETURN:
+ *	/persistent_generic or null if not available.
+ */
+/datum/controller/subsystem/persistence/proc/genericLoad(var/singleton/persistent_type/generic/target_type, attribute = null)
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to load generic with null target type.")
+		return
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	if(type_instance.requires_attribute && !length(attribute))
+		log_subsystem_persistence_warning("Attempted to load generic of type [target_type] without required attribute.")
+		return
+
+	attribute = length("[attribute]") > 0 ? attribute : null
+	var/datum/persistent_generic/generic = generic_cache[typesGetCacheName(target_type, attribute)]
+	if(generic)
+		return generic
+
+	var/result = genericDatabaseLoad(type_instance.database_id, attribute)
+	if(!result)
+		return null
+
+	var/datum/persistent_generic/new_generic = new /datum/persistent_generic/
+	new_generic.type_define = target_type
+	new_generic.attribute = attribute
+	new_generic.content = json_decode(result["content"])
+	new_generic.expires_in_days = 0
+	generic_cache[typesGetCacheName(target_type, attribute)] = new_generic
+	return new_generic
@@ -0,0 +1,137 @@
+/**
+ * Get the last ID in the generics table.
+ * RETURN:
+ *  Last ID or zero.
+ */
+/datum/controller/subsystem/persistence/proc/genericDatabaseGetLastID()
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("genericDatabaseGetLastID"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id FROM ss13_persistent_generics ORDER BY id DESC LIMIT 1"
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "genericDatabaseGetLastID"))
+		qdel(query)
+		return 0
+
+	var/last_id = 0
+	if(query.NextRow())
+		last_id = query.item[1]
+	qdel(query)
+	return last_id
+
+/**
+ * Runs a cleanup query on generics that have expired.
+ */
+/datum/controller/subsystem/persistence/proc/genericDatabaseCleanup()
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("genericDatabaseCleanup"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"DELETE FROM ss13_persistent_generics WHERE expires_at < NOW()"
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "genericDatabaseCleanup")
+	qdel(query)
+
+/**
+ * Save a generic persistent type(+attribute).
+ * PARAMS:
+ * 	type_id =			Type of ID.
+ *  attribute =			Custom attribute of the record, can be null.
+ *	expires_in_days =	Number of days until the content expires.
+ *  content =			JSON content to be saved.
+ */
+/datum/controller/subsystem/persistence/proc/genericDatabaseSave(type_id, attribute, expires_in_days, content)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("genericDatabaseSave"))
+		return 0
+
+	// Because MariaDB doesn't consider NULL in the attribute to be a violation of the UNIQUE constraint,
+	// we have to verify if the generic already exists, if the attribute is null.
+	// If the attribute is null and the generic exists, manually update the row instead of INSERT + ON DUPLICATE KEY.
+	// Otherwise, with valid unique constraint (not null attributes), we can continue with the regular INSERT + ON DUPLICATE KEY directly.
+	if(!attribute)
+		var/datum/db_query/null_attribute_query = SSdbcore.NewQuery(
+			"SELECT id FROM ss13_persistent_generics WHERE type = :type_id AND attribute IS NULL",
+			list(
+				"type_id" = type_id
+			)
+		)
+		null_attribute_query.Execute()
+
+		if(!databaseCheckQueryResult(null_attribute_query, "genericDatabaseSaveNullAttributeCheck"))
+			qdel(null_attribute_query)
+			return 0
+
+		var/id = 0
+		if(null_attribute_query.NextRow())
+			id = null_attribute_query.item[1]
+		qdel(null_attribute_query)
+		if(id > 0) // Attribute null and row found - Invalid unique contraint for MariaDB - Update manually.
+			var/datum/db_query/update_query = SSdbcore.NewQuery(
+				"UPDATE ss13_persistent_generics SET created_at = NOW(), expires_at = DATE_ADD(NOW(), INTERVAL :expires_in_days DAY), content = :content WHERE id = :id",
+				list(
+					"expires_in_days" = expires_in_days,
+					"content" = content,
+					"id" = id
+				)
+			)
+			update_query.Execute()
+
+			databaseCheckQueryResult(update_query, "genericDatabaseSaveNullAttributeUpdate")
+			qdel(update_query)
+			return // Skip regular upcoming query due to the reasons above
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"INSERT INTO ss13_persistent_generics (type, attribute, created_at, expires_at, content) VALUES (:type_id, :attribute, NOW(), DATE_ADD(NOW(), INTERVAL :expires_in_days DAY), :content) \
+		ON DUPLICATE KEY UPDATE created_at = NOW(), expires_at = DATE_ADD(NOW(), INTERVAL :expires_in_days DAY), content = :content",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"expires_in_days" = expires_in_days,
+			"content" = content
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "genericDatabaseSave")
+	qdel(query)
+
+/**
+ * Load a generic persistent type(+attribute).
+ * PARAMS:
+ * 	type_id =			Type of ID.
+ *  attribute =			Custom attribute of the record, can be null.
+ * RETURN:
+ *  Associative list of keys "id", "content" (JSON).
+ */
+/datum/controller/subsystem/persistence/proc/genericDatabaseLoad(type_id, attribute)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("genericDatabaseLoad"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id, content FROM ss13_persistent_generics \
+		WHERE type = :type_id AND attribute <=> :attribute",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute
+		)
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "genericDatabaseLoad"))
+		qdel(query)
+		return null
+
+	var/result = null
+	while(query.NextRow())
+		result = list("id" = query.item[1], "content" = query.item[2])
+	qdel(query)
+	return result
@@ -0,0 +1,263 @@
+#define PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT 1000 // Max row count of records allowed to be drawn from database for performance reasons
+
+/**
+ * Helper proc for history/character persistent types to retrieve a character name based of it's character ID.
+ * PARAMS:
+ *  char_id = ID of character.
+ * RETURN:
+ *  Character name or null if not found.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetCharnameByID(char_id)
+	if(!char_id)
+		return null
+
+	var/char_id_num = text2num(char_id)
+	if(char_id_num <= 0)
+		return null
+
+	var/cache_hit = char_cache["[char_id_num]"]
+	if(cache_hit)
+		return cache_hit
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT name FROM ss13_characters WHERE id = :char_id",
+		list("char_id" = char_id_num)
+	)
+	query.Execute()
+
+	var/char_name
+	while(query.NextRow())
+		char_name += query.item[1]
+	qdel(query)
+	if(!char_name)
+		return null
+	else
+		char_cache["[char_id_num]"] = char_name
+		return char_name
+
+/**
+ * Add a new record to the history for the given type/attribute.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type/history and subtypes.
+ *  attribute =		Custom attribute of the record, can be null if the type definition doesn't require it.
+ *	value =			Value of the record, cannot be null or empty.
+ */
+/datum/controller/subsystem/persistence/proc/historyAddRecord(var/singleton/persistent_type/history/target_type, attribute, value)
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to add history record with null target type.")
+		return
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	if(type_instance.requires_attribute && length(attribute) > 0)
+		log_subsystem_persistence_warning("Attempted to add history record of type [target_type] without required attribute.")
+		return
+
+	if(!value)
+		log_subsystem_persistence_warning("Attempted to add history record of type [target_type] with empty value.")
+		return
+
+	// Sanity check if a character record is added using this proc directly instead of the overload historyAddCharacterRecord.
+	if(istype(type_instance, /singleton/persistent_type/history/character) && (length(attribute) > 0 || !isnum(attribute)))
+		log_subsystem_persistence_warning("Attempted to add character history record of target type [target_type], but the attribute was either empty or failed the isnum check.")
+		return
+
+	// Add record to cache for DB insert at finalization and quick access
+	// Check if record container exists, if not, create it
+	var/datum/persistent_record_container/container = null
+	attribute = length("[attribute]") > 0 ? attribute : null
+	container = history_cache[typesGetCacheName(target_type, attribute)]
+
+	if(!container)
+		container = new /datum/persistent_record_container
+		container.type_define = target_type.type
+		container.attribute = attribute
+		container.records = list()
+		history_cache[typesGetCacheName(target_type, attribute)] = container
+
+	// Create record and add to container
+	var/datum/persistent_record/r = new /datum/persistent_record
+	r.id = typesGetVirtualRecordID()
+	r.created_at = "[worlddate2text()] [worldtime2text()]"
+	r.game_id = GLOB.round_id
+	r.value = value
+	container.records += r
+	history_cache_count++
+
+/**
+ * Add a new record that belongs to a specific character to the history for the given type.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type/history/character and subtypes.
+ *  char_id =		Character ID that the record should belong to.
+ *	value =			Value of the record, cannot be null or empty.
+ */
+/datum/controller/subsystem/persistence/proc/historyAddCharacterRecord(var/singleton/persistent_type/history/character/target_type, char_id, value)
+	if(!ispath(target_type, /singleton/persistent_type/history/character))
+		log_subsystem_persistence_warning("Attempted to add character history record, but the provided target type didn't match a character persistent type, provided was [target_type]")
+		return
+	if(!isnum(char_id))
+		log_subsystem_persistence_warning("Attempted to add character history record of type [target_type] but char_id failed the isnum check.")
+		return
+	return historyAddRecord(target_type, char_id, value)
+
+/**
+ * Queries the last record of a specified type/attribute.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *					If the type definition is a character record type, the attribute must be a valid character ID or the record will be rejected.
+ *  attribute =		Custom attribute of the record, can be null if the type definition doesn't require it.
+ * RETURN:
+ * 	Single /persistent_record or null.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetLastRecord(var/singleton/persistent_type/history/target_type, attribute)
+	var/result = historyGetLastRecords(target_type, attribute, 1)
+	if(length(result) == 0)
+		return null
+	else
+		return result[1]
+
+/**
+ * Queries the last X records of a specified type/attribute.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *					If the type definition is a character record type, the attribute must be a valid character ID or the record will be rejected.
+ *  attribute =		Custom attribute of the record, can be null if the type definition doesn't require it.
+ *  limit =			Number of records to retrieve.
+ *  skip_caching =	If set to TRUE, the results won't be added to the types cache, defaults to FALSE.
+ * RETURN:
+ * 	List of /persistent_record or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetLastRecords(var/singleton/persistent_type/history/target_type, attribute, limit, skip_caching = FALSE)
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to get history records with null target type.")
+		return list()
+
+	if(limit > PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT)
+		limit = PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT
+		log_subsystem_persistence_warning("Attempted to draw more records then allowed for target type [target_type].")
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	if(type_instance.requires_attribute && !attribute)
+		log_subsystem_persistence_warning("Attempted to get history records of type [target_type] without required attribute.")
+		return list()
+
+	// Query order
+	// 1 - Check if record container exists, if so, check if last X records are in there, aggregate found records, step to DB (2) for missing remainders.
+	// 2 - Query database for last X records of type and add it to record container as new cache
+
+	var/datum/persistent_record_container/container = null
+	attribute = length("[attribute]") > 0 ? attribute : null
+	container = history_cache[typesGetCacheName(target_type, attribute)]
+
+	var/list/datum/persistent_record/top = list()
+
+	// Query order - 1
+	if(container)
+		top = typesHistoryCacheSelectTopK(limit, container)
+		if(length(top) == limit) // All X records got hit in cache, return
+			return top
+	else if (!skip_caching)
+		container = new /datum/persistent_record_container
+		container.type_define = target_type.type
+		container.attribute = attribute
+		container.records = list()
+		history_cache[typesGetCacheName(target_type, attribute)] = container
+
+	// Query order - 2
+	var/list/db_records = historyDatabaseGetRecords(type_instance.database_id, attribute, limit - length(top)) // Draw remaining missing records from DB
+	var/len = length(db_records)
+	if(!len)
+		return list()
+
+	if(!skip_caching)
+		history_cache_count += len
+
+	for(var/alist/record in db_records)
+		var/datum/persistent_record/r = new /datum/persistent_record
+		r.id = record["id"]
+		r.created_at = record["created_at"]
+		r.game_id = record["game_id"]
+		r.value = record["value"]
+		if(!skip_caching)
+			container.records += r // Add to cache
+		top += r // Records in top are either newly created or read from DB already, append newly queries records.
+
+	return top
+
+/**
+ * Queries all records of a specified type/attribute.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *					If the type definition is a character record type, the attribute must be a valid character ID or the record will be rejected.
+ *  attribute =		Custom attribute of the record, can be null if the type definition doesn't require it.
+ *  skip_caching =	If set to TRUE, the results won't be added to the types cache, defaults to TRUE.
+ * RETURN:
+ * 	List of /persistent_record or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetAllRecords(var/singleton/persistent_type/history/target_type, attribute, skip_caching = TRUE)
+	var/result = historyGetLastRecords(target_type, attribute, PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT, skip_caching)
+	if(length(result) == 0)
+		return null
+	else
+		return result
+
+/**
+ * Queries the last record of the specified type for all attributes.
+ * PARAMS:
+ * 	target_type = Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ * RETURN:
+ * 	Associative list with "attribute" and "records" of type list(/persistent_record) or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetLastRecordForAllAttributes(var/singleton/persistent_type/history/target_type)
+	var/result = historyGetLastRecordsForAllAttributes(target_type, 1)
+	if(length(result) == 0)
+		return list()
+	else
+		return result
+
+/**
+ * Queries the last X records of a specified type for all attributes.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *  limit =			Number of records to retrieve.
+ *  skip_caching =	If set to TRUE, the results won't be added to the types cache, defaults to TRUE.
+ * RETURN:
+ * 	List of associative list with "attribute" and "records" of type list(/persistent_record) or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetLastRecordsForAllAttributes(var/singleton/persistent_type/history/target_type, limit, skip_caching = TRUE)
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to get history records with null target type.")
+		return list()
+
+	if(limit > PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT)
+		limit = PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT
+		log_subsystem_persistence_warning("Attempted to draw more records then allowed for target type [target_type].")
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	var/list/result = list()
+
+	var/attributes = historyDatabaseGetAllAttributes(type_instance.database_id)
+	if(attributes && length(attributes) > 0)
+		for(var/attribute in attributes)
+			result += list(alist("attribute" = attribute, "records" = historyGetAllRecords(target_type, attribute, skip_caching)))
+	else
+		var/no_attribute_records = historyGetAllRecords(target_type, null, skip_caching)
+		if(no_attribute_records && length(no_attribute_records) > 0)
+			result = list(alist("attribute" = null, "records" = no_attribute_records))
+	return result
+
+/**
+ * Queries all records of a specified type for all attributes.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *  skip_caching =	If set to TRUE, the results won't be added to the types cache, defaults to TRUE.
+ * RETURN:
+ * 	List of associative list with "attribute" and "records" of type list(/persistent_record) or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetAllRecordsForAllAttributes(var/singleton/persistent_type/history/target_type, skip_caching = TRUE)
+	var/result = historyGetLastRecordsForAllAttributes(target_type, PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT, skip_caching)
+	if(length(result) == 0)
+		return null
+	else
+		return result
+
+#undef PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT
@@ -0,0 +1,235 @@
+/**
+ * Get the last ID in the history table.
+ * RETURN:
+ *  Last ID or zero.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseGetLastID()
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseGetLastID"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id FROM ss13_persistent_history ORDER BY id DESC LIMIT 1"
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "historyDatabaseGetLastID"))
+		qdel(query)
+		return 0
+
+	var/last_id = 0
+	if(query.NextRow())
+		last_id = query.item[1]
+	qdel(query)
+	return last_id
+
+/**
+ * Returns all combinations of types+attributes from persistent history.
+ * RETURN:
+ * 	Distinct list of list with keys "type_id" and "attribute" (possibly null).
+ *  Example: (("type_id" = 1, "attribute" = null), ("type_id" = 1, "attribute" = "lorem ipsum"), ("type_id" = 2, "attribute" = "dolor sit amet"))
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseGetTypeAttributeCombinations()
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseGetTypeAttributeCombinations"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT DISTINCT type, attribute FROM ss13_persistent_history"
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "historyDatabaseGetTypeAttributeCombinations"))
+		qdel(query)
+		return null
+
+	var/result = list()
+	while(query.NextRow())
+		result += list(alist("type_id" = query.item[1], "attribute" = query.item[2]))
+	qdel(query)
+	return result
+
+/**
+ * Returns all attributes from persistent history for a specified type.
+ * RETURN:
+ * 	Distinct list of attributes or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseGetAllAttributes(type_id)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseGetAllAttributes"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT DISTINCT attribute FROM ss13_persistent_history WHERE type = :type_id",
+		list("type_id" = type_id)
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "historyDatabaseGetAllAttributes"))
+		qdel(query)
+		return null
+
+	var/result = list()
+	while(query.NextRow())
+		result += query.item[1]
+	qdel(query)
+	return result
+
+/**
+ * Clean up history records of type+attribute by specified row count
+ * PARAMS:
+ * 	type_id =	ID of type.
+ *  attribute =	Custom attribute of the record, can be null.
+ *	row_count =	Count of rows to keep for the specified grouping.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseCleanByRowCount(type_id, attribute, row_count)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseCleanByRowCount"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"\
+			DELETE FROM ss13_persistent_history \
+			WHERE type = :type_id AND attribute <=> :attribute \
+			AND id NOT IN ( \
+				SELECT id FROM ss13_persistent_history \
+				WHERE type = :type_id AND attribute <=> :attribute \
+				ORDER BY created_at DESC, id DESC \
+				LIMIT :row_count \
+			)",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"row_count" = row_count
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "historyDatabaseCleanByRowCount")
+	qdel(query)
+
+/**
+ * Clean up history records of type+attribute by specified round count.
+ * PARAMS:
+ * 	type_id =		Type of ID.
+ *  attribute =		Custom attribute of the record, can be null.
+ *	round_count =	Number of rounds to keep for specified grouping.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseCleanByRoundCount(type_id, attribute, round_count)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseCleanByRoundCount"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"\
+			DELETE FROM ss13_persistent_history \
+			WHERE type = :type_id AND attribute <=> :attribute \
+			AND game_id NOT IN ( \
+				SELECT game_id FROM ( \
+					SELECT game_id FROM ss13_persistent_history \
+					WHERE type = :type_id AND attribute <=> :attribute \
+					GROUP BY game_id \
+					ORDER BY MAX(created_at) DESC, game_id DESC \
+					LIMIT :round_count \
+				) AS recent_games \
+			)",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"round_count" = round_count
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "historyDatabaseCleanByRoundCount")
+	qdel(query)
+
+/**
+ * Clean up history records of type+attribute by max age of record in days
+ * PARAMS:
+ * 	type_id =		ID of type.
+ *  attribute =		Custom attribute of the record, can be null.
+ *	max_age_days =	Max age of records in days.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseCleanByMaxAgeDays(type_id, attribute, max_age_days)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseCleanByMaxAgeDays"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"\
+			DELETE FROM ss13_persistent_history \
+			WHERE type = :type_id AND attribute <=> :attribute \
+			AND created_at < (NOW() - INTERVAL :max_age_days DAY)",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"max_age_days" = max_age_days
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "historyDatabaseCleanByMaxAgeDays")
+	qdel(query)
+
+/**
+ * Insert a new history record into the history table.
+ * PARAMS:
+ * 	type_id =	ID of type.
+ *  attribute =	Custom attribute of the record, can be null.
+ *	value =		Value of the record, cannot be null or empty.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseInsertRecord(type_id, attribute, value)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseInsertRecord"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"INSERT INTO ss13_persistent_history (type, created_at, attribute, value, game_id) VALUES (:type_id, NOW(), :attribute, :value, :game_id)",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"value" = "[value]",
+			"game_id" = "[GLOB.round_id]"
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "historyDatabaseInsertRecord")
+	qdel(query)
+
+/**
+ * Get the last X history records for a type+attribute.
+ * PARAMS:
+ * 	type_id =	ID of type
+ *  attribute =	Custom attribute of the record, can be null.
+ *	count =		Number of records to be returned.
+ * RETURN:
+ *  List of records, each as a list consisting of keys "id", "created_at" and "value".
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseGetRecords(type_id, attribute, count)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseGetRecords"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id, created_at, value, game_id FROM ss13_persistent_history \
+		WHERE type = :type_id AND attribute <=> :attribute \
+		ORDER BY id DESC LIMIT :count",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"count" = count
+		)
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "historyDatabaseGetRecords"))
+		qdel(query)
+		return null
+
+	var/records = list()
+	while(query.NextRow())
+		records += list(alist("id" = query.item[1], "created_at" = query.item[2], "value" = query.item[3]))
+	qdel(query)
+	return records
@@ -0,0 +1,55 @@
+/**
+ * Insert or update a type definition in the database. If a type with the same name already exists, it will be updated with the new title and description.
+ * PARAMS:
+ * 	type =				Name of type to be upserted.
+ *  title =				Custom display title of the type.
+ *  description =		Custom display description of the type.
+ *  definition_type =	Enum value of the definition type. See /singleton/persistent_type.
+ */
+/datum/controller/subsystem/persistence/proc/typesDatabaseUpsertType(type, title, description, definition_type)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("typesDatabaseUpsertType"))
+		return
+
+	var/datum/db_query/upsert_query = SSdbcore.NewQuery(
+		"INSERT INTO ss13_persistent_type_definitions (type, title, description, definition_type) VALUES (:type, :title, :description, :definition_type) \
+		ON DUPLICATE KEY UPDATE title = VALUES(title), description = VALUES(description)",
+		list(
+			"type" = type,
+			"title" = title,
+			"description" = description,
+			"definition_type" = definition_type
+		)
+	)
+	upsert_query.Execute()
+
+	databaseCheckQueryResult(upsert_query, "typesDatabaseUpsertType")
+	qdel(upsert_query)
+
+/**
+ * Get ID of type definition.
+ * PARAMS:
+ *	type_name = Type name of singleton definition.
+ * RETURN:
+ *	Database ID of type.
+ */
+/datum/controller/subsystem/persistence/proc/typesDatabaseGetTypeIdByName(type_name)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("typesDatabaseGetTypeIdByName"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id FROM ss13_persistent_type_definitions WHERE type = :type_name",
+		list("type_name" = type_name)
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "typesDatabaseGetTypeIdByName"))
+		qdel(query)
+		return 0
+
+	var/database_id = null
+	if(query.NextRow())
+		database_id = query.item[1]
+	qdel(query)
+	return database_id
@@ -95,7 +95,7 @@ GLOBAL_DATUM_INIT(crew_repository, /datum/repository/crew, new())

 				crewmembers += list(crewmemberData)

-	crewmembers = sortByKey(crewmembers, "name")
+	crewmembers = sortByKeyText(crewmembers, "name")
 	cache_entry.timestamp = world.time + 5 SECONDS
 	cache_entry.data = crewmembers

@@ -28,8 +28,6 @@ GLOBAL_LIST_EMPTY(additional_antag_types)

 	var/station_was_nuked = 0                // See nuclearbomb.dm and malfunction.dm.
 	var/explosion_in_progress = 0            // Sit back and relax
-	var/waittime_l = 60 SECONDS                     // Lower bound on time before intercept arrives (in tenths of seconds)
-	var/waittime_h = 180 SECONDS                    // Upper bound on time before intercept arrives (in tenths of seconds)

 	var/event_delay_mod_moderate             // Modifies the timing of random events.
 	var/event_delay_mod_major                // As above.
@@ -281,8 +279,7 @@ GLOBAL_LIST_EMPTY(additional_antag_types)

 	addtimer(CALLBACK(GLOBAL_PROC, GLOBAL_PROC_REF(display_logout_report)), ROUNDSTART_LOGOUT_REPORT_TIME)

-	var/welcome_delay = rand(waittime_l, waittime_h)
-	addtimer(CALLBACK(SSatlas.current_map, TYPE_PROC_REF(/datum/map, send_welcome)), welcome_delay)
+	SSatlas.current_map.post_gamemode_setup()

 	addtimer(CALLBACK(SSatlas.current_map, TYPE_PROC_REF(/datum/map, load_holodeck_programs)), 5 MINUTES)

@@ -531,7 +531,7 @@
 	if(T)
 		var/area/A = get_area(T)
 		if(A && !(A.area_flags & AREA_FLAG_PREVENT_PERSISTENT_TRASH))
-			persistant_objects_expiration_time_days = 3 // Ensure expiration date is set to prevent long term trash
+			persistent_objects_expiration_time_days = 3 // Ensure expiration date is set to prevent long term trash
 			SSpersistence.objectsRegisterTrack(src, usr == null ? null : ckey(usr.key))
 			return

@@ -6,7 +6,7 @@
 	item_flags = ITEM_FLAG_NO_BLUDGEON
 	w_class = WEIGHT_CLASS_TINY
 	vis_flags = VIS_INHERIT_LAYER | VIS_INHERIT_DIR
-	persistant_objects_expiration_time_days = 2
+	persistent_objects_expiration_time_days = 2

 	var/datum/weakref/attached
 	var/list/rand_icons
@@ -11,7 +11,7 @@

 /obj/item/material/ashtray/Initialize(newloc, material_key)
 	. = ..()
-	persistant_objects_expiration_time_days = rand(7, 180) // Imagine they get stolen, lost or break...
+	persistent_objects_expiration_time_days = rand(7, 180) // Imagine they get stolen, lost or break...
 	max_butts = round(material.hardness/10) //This is arbitrary but whatever.
 	randpixel_xy()
 	update_icon()
@@ -105,7 +105,7 @@
 	// This should be considered for any moderation purpose
 	var/persistent_objects_author_ckey = null
 	// Expiration time used when saving/updating a persistent type, this can be changed depending on the use case by assigning a new value
-	var/persistant_objects_expiration_time_days = PERSISTENT_DEFAULT_EXPIRATION_DAYS
+	var/persistent_objects_expiration_time_days = PERSISTENT_DEFAULT_EXPIRATION_DAYS
 	/* END PERSISTENCE VARS */

 	/// for easy reference of talking atoms
@@ -158,7 +158,7 @@ ABSTRACT_TYPE(/obj/item/package)
 		Probably more expensive then it should be."
 	icon_state = "supply_package"
 	item_state = "supply_package"
-	persistant_objects_expiration_time_days = 360
+	persistent_objects_expiration_time_days = 360

 /obj/item/package/persistent_supply/Initialize()
 	. = ..()
@@ -325,7 +325,7 @@
 	desc = "A specialized charge card that holds a certain amount of money. This type of charge card is in use for special purposes and not generally available."
 	icon_state = "efundcard_special"
 	var/initial_worth = 0 // Used for calculating how much cash was spend, needs to be set using VV after spawning it.
-	persistant_objects_expiration_time_days = 360
+	persistent_objects_expiration_time_days = 360

 /obj/item/spacecash/ewallet/persistent_charge_card/Initialize()
 	. = ..()
@@ -192,6 +192,9 @@
 					ID.mining_points += points
 					if(points != 0)
 						ping("<b>\The [src]</b> pings, \"Point transfer complete! Transaction total: [points] points!\"")
+						var/character_id = astype(ID.mob_id.resolve(), /mob/living/carbon/human/)?.character_id
+						if(character_id)
+							SSpersistence.historyAddCharacterRecord(/singleton/persistent_type/history/character/mining_points, character_id, points)
 					points = 0
 				else
 					ping("<b>\The [src]</b> pings, \"Transaction failed due to a negative point value. No transaction can be done until this value has returned to a positive one.\"")
@@ -60,8 +60,8 @@

 			robots += list(robotData)

-	data["mechs"] = sortByKey(mechs, "pilot")
-	data["robots"] = sortByKey(robots, "pilot")
+	data["mechs"] = sortByKeyText(mechs, "pilot")
+	data["robots"] = sortByKeyText(robots, "pilot")

 	data["current_cam_loc"] = current_camera ? "[REF(current_camera.loc)]" : null

@@ -66,7 +66,7 @@
 	var/can_change_icon_state = TRUE
 	var/set_unsafe_on_init = FALSE

-	persistant_objects_expiration_time_days = 180
+	persistent_objects_expiration_time_days = 180

 /obj/item/paper/Destroy()
 	info = null
@@ -0,0 +1,23 @@
+author: FabianK3
+
+delete-after: True
+
+changes:
+  - server: "Database: Added new tables for persistent generics and history, refactored persistent objects."
+  - bugfix: "Fix duplicate code-behind ID for admin persistence toggle verb."
+  - admin: "MC/VV: Moved Persistent object register from GLOB to subsystem var space."
+  - admin: "MC/VV: Cache list for persistent generics (generic_cache) and persistent history (history_cache) added to subsystem var space."
+  - admin: "MC/VV: Update stat entry of persistence subsystem. It now displays master-save-toggle, persistent object register, generics cache, history cache with entry count and possible panic-mode warning."
+  - admin: "Logging: Added sentry integration to persistence subsystem error/panic logging."
+  - code_imp: "Renamed map send_welcome proc to post_gamemode_setup and moved SCCV Horizon round start fax logic into new proc. This proc can be used for map specific init logic for any map respectively."
+  - rscadd: "CI/CD: Added new CI job to find object types on maps that should be excluded. Config file under maps/forbiddenMapTypes.json and currently includes persistent ash trays on the SCCV Horizon and the operations warehouse submap."
+  - experiment: "Updated the persistence subsystem/framework and introduced persistent generics and persistent history. Technical documentation for maintainers and contributors will be available on the repository wiki. Update includes a near-complete overhaul of the existing subsystem and added a lot of newly available methods to make a lot of things persistent that couldn't be made possible with the previous object-only persistence."
+  - rscadd: "Added macros to easily create new persistent type definitions, the basis for persistent generics and history."
+  - rscadd: "Added cleanup-rule definitions for persistent history definitions, used to specify how different persistent history mechanics should be cleaned up on a database level."
+  - rscadd: "Added dedicated file for persistent type definitions."
+  - code_imp: "Misc. changes to file structures, defines and the persistence subsystem related to the persistence framework update."
+  - rscadd: "New persistent mechanics: Persistent Horizon position (persistent generic example mechanic) - The SCCV Horizon will now retain its position on the overmap from round to round. Where did we park again?"
+  - rscadd: "New persistent mechanics: Persistent records of mining yields (persistent history example mechanic) - When miners claim their ore points using the ore console, the points will be saved. Every shift a report will be generated based of the points collected by each individual miner over the last 7 days. Who met their quota last week?"
+  - bugfix: "Added an init flag to the persistence subsystem to verify init success. If any issues arise during the subsystems init, saving at round end will be skipped (panic mode)."
+  - rscadd: "Implemented numeric version of list insert sort, based of existing test list insert sort."
+  - rscadd: "Implemented top K insertion sort selection for persistence subsystem cache."
@@ -283,7 +283,7 @@

 	return list(spawn_cost, player_cost, ship_cost)

-/datum/map/proc/send_welcome()
+/datum/map/proc/post_gamemode_setup()
 	return

 /datum/map/proc/load_holodeck_programs()
@@ -0,0 +1,9 @@
+{
+	"forbidden_type_mappings": [
+		{
+			"title": "Objects excluded due to persistent mechanics",
+			"maps": ["sccv_horizon.dmm", "ops_warehouse_small_storage.dmm"],
+			"forbidden_types": ["/obj/item/material/ashtray"]
+		}
+	]
+}
@@ -163,22 +163,49 @@
 	)
 	shuttle_missions = list("Exploration", "Research", "Prospecting", "Salvaging", "Transport", "Combat", "Rescue", "Training", "Humanitarian", "Expedition", "Recreation", "Other")

-/datum/map/sccv_horizon/send_welcome()
+/singleton/persistent_type/generic/horizon_overmap_position/finalization_hook()
+	if(SSatlas.current_map.use_overmap)
+		var/obj/effect/overmap/visitable/ship/sccv_horizon/ship = locate(/obj/effect/overmap/visitable/ship/sccv_horizon) in GLOB.map_overmap
+		if(ship)
+			SSpersistence.genericSave(/singleton/persistent_type/generic/horizon_overmap_position, list("x" = ship.x, "y" = ship.y), 1)
+
+/datum/map/sccv_horizon/post_gamemode_setup()
+	// ##### Set persistent Horizon position on the overmap
+	var/datum/persistent_generic/horizon_location_generic = SSpersistence.genericLoad(/singleton/persistent_type/generic/horizon_overmap_position)
+	if(horizon_location_generic)
+		var/area/overmap/map = GLOB.map_overmap
+		// Set Horizon location
+		var/obj/effect/overmap/visitable/ship/sccv_horizon/ship = locate(/obj/effect/overmap/visitable/ship/sccv_horizon) in map
+		ship.x = horizon_location_generic.content["x"]
+		ship.y = horizon_location_generic.content["y"]
+		// Make safe space for the Horizon
+		for(var/obj/effect/overmap/hazard in map)
+			if(hazard.x == ship.x && hazard.y == ship.y && istype(hazard, /obj/effect/overmap/event/)) // Ions, dust, carps, meteors, etc.
+				qdel(hazard)
+
+	// ##### Send different faxes after slight delay
+	var/faxes_send_delay = rand(60 SECONDS , 180 SECONDS)
+	addtimer(CALLBACK(SSatlas.current_map, TYPE_PROC_REF(/datum/map/sccv_horizon, send_roundstart_faxes)), faxes_send_delay)
+
+/datum/map/sccv_horizon/proc/send_roundstart_faxes()
+
+	// #### Send welcome fax with overmap information
+
 	var/obj/effect/overmap/visitable/ship/horizon = SSshuttle.ship_by_type(overmap_visitable_type)

-	var/welcome_text = "<center><img src = scclogo.png><br />[FONT_LARGE("<b>SCCV Horizon</b> Ultra-Range Sensor Readings:")]<br>"
+	var/welcome_text = "<center><img src = scclogo.png><br />[FONT_LARGE("<b>SCCV Horizon</b> Ultra-Range Sensor Readings:")]<br />"
 	welcome_text += "Report generated on [worlddate2text()] at [worldtime2text()]</center><br /><br />"
-	welcome_text += "<hr>Current sector:<br /><b>[SSatlas.current_sector.name]</b><br /><br>"
+	welcome_text += "<hr>Current sector:<br /><b>[SSatlas.current_sector.name]</b><br /><br />"

 	if (horizon) //If the overmap is disabled, it's possible for there to be no Horizon.
 		var/list/space_things = list()
-		welcome_text += "Current Coordinates:<br /><b>[horizon.x]:[horizon.y]</b><br /><br>"
-		welcome_text += "Available Ports of Call: <b>[english_list(SSatlas.current_sector.ports_of_call, "none")]</b><br>"
+		welcome_text += "Current Coordinates:<br /><b>[horizon.x]:[horizon.y]</b><br /><br />"
+		welcome_text += "Available Ports of Call: <b>[english_list(SSatlas.current_sector.ports_of_call, "none")]</b><br />"
 		if(SSatlas.current_sector.next_port_visit)
-			welcome_text += "Next Port Visit: <b>in [SSatlas.current_sector.next_port_visit] days</b><br>"
+			welcome_text += "Next Port Visit: <b>in [SSatlas.current_sector.next_port_visit] days</b><br />"
 		else
-			welcome_text += "<b>There is no port visit scheduled.</b><br><br>"
-		welcome_text += "<b>It is advised to inform crew of the available ports of call and the date of the next port visit.</b><br><br>"
+			welcome_text += "<b>There is no port visit scheduled.</b><br /><br />"
+		welcome_text += "<b>It is advised to inform crew of the available ports of call and the date of the next port visit.</b><br /><br />"
 		welcome_text += "Scan results show the following points of interest:<br />"

 		for(var/zlevel in GLOB.map_sectors)
@@ -206,6 +233,56 @@
 	var/report = "The long-range sensor readings have been printed out at all communication consoles."
 	priority_announcement.Announce(message = report)

+	// #### Mining yield report for operations
+
+	var/list/name_points_kvps = SSpersistence.historyGetAllRecordsForAllAttributes(/singleton/persistent_type/history/character/mining_points)
+	if(name_points_kvps && length(name_points_kvps) > 0)
+		var/list/leaderboard_kvps = list()
+		for(var/kvp in name_points_kvps) // Get name, validate, add to leaderboard
+			var/char_name = SSpersistence.historyGetCharnameByID(kvp["attribute"])
+			if(!char_name)
+				continue
+			var/point_sum = 0
+			for(var/datum/persistent_record/r in kvp["records"])
+				point_sum += text2num(r.value)
+			if(point_sum <= 0)
+				continue
+			leaderboard_kvps += list(list("score" = point_sum, "name" = char_name, "claims" = length(kvp["records"])))
+
+		if(length(leaderboard_kvps))
+			var/report_text = ""
+			report_text += "<center><H3>7-day mining yield report</H3>"
+			report_text += "<table border=1 cellspacing=0 cellpadding=3 style='border: 1px solid black;'></td><tr>"
+			report_text += "<td><img src = scclogo_small.png>"
+			report_text += "<td><font size = \"1\">7-day history of worker mining yields.<br />Manifest version: [worlddate2text()] [worldtime2text()]</font>"
+			report_text += "</td></tr></table><br />"
+
+			leaderboard_kvps = sortByKeyNumber(leaderboard_kvps, "score")
+
+			report_text += "<font size=\"4\"><b>[leaderboard_kvps[1]["name"]]</b> is leading the 7-day<br />yield report with <b>[leaderboard_kvps[1]["score"]]</b> points.</font><br /><br />"
+
+			report_text += "<table border=1 cellspacing=0 cellpadding=3 style='border: 1px solid black;'>"
+			for(var/leaderboard_kvp in leaderboard_kvps)
+				report_text += "</td><tr><td><font size=\"4\"><B>[leaderboard_kvp["name"]]</font></B><br />Points: <B>[leaderboard_kvp["score"]]</B><br />Claims: [leaderboard_kvp["claims"]]<br />"
+			report_text += "</td></tr></table><br />"
+
+			report_text += "<font size = \"1\"><table border=1 cellspacing=0 cellpadding=3 style='border: 1px solid black;'>"
+			report_text += "</td><tr><td>Powered by Orion Express logistics software.<br /><center><I>Faster than light.</I></center><td><img src = orionlogo_small.png>"
+			report_text += "</td></tr></table></font>"
+
+			report_text += "<br /><I>This report has been generated based of all mining yield claims in the past 7 days. Points are aggregated, claim count does not reflect on shift or trip count. Point claims are authenticated by identification cards.</I>"
+			report_text += "</font><br /><font size = \"1\"></center>"
+			report_text += "Generated by OE.SCC.MiningOps 1.7<br />Manifest form version 5.87, Hash:<br />576520646F206C6F7665206F7572207061706572776F726B21</font>"
+
+			for(var/obj/structure/machinery/requests_console/console in GLOB.allConsoles)
+				var/area/console_area = get_area(console)
+				if(console_area.type in typesof(/area/horizon/command/heads/om, /area/horizon/operations/office, /area/horizon/operations/mining_main/refinery, /area/horizon/command/bridge/controlroom))
+					if(console.paperstock <= 0)
+						continue
+					new /obj/item/paper(get_turf(console), report_text, "7-day mining yield report")
+					console.audible_message("<b>The Requests Console</b> beeps, \"Fax received.\"")
+					console.paperstock -= 1
+
 /datum/map/sccv_horizon/load_holodeck_programs()
 	// loads only if at least two engineers are present
 	// so as to not drain power on deadpop
@@ -0,0 +1,67 @@
+import json
+import os
+import sys
+from pathlib import Path
+
+def check_forbidden_types():
+    # Read the forbidden type mappings
+    json_path = Path("maps/forbiddenMapTypes.json")
+
+    with open(json_path, 'r') as f:
+        data = json.load(f)
+
+    failed = False
+    missing_maps = []
+    illegal_types = []
+
+    # Loop through each forbidden type mapping
+    for mapping in data["forbidden_type_mappings"]:
+        title = mapping["title"]
+        maps = mapping["maps"]
+        forbidden_types = mapping["forbidden_types"]
+        print(f"Linting step - {title}")
+        print(f"  - Maps: {', '.join(maps)}")
+        print(f"  - Forbidden types: {', '.join(forbidden_types)}")
+
+        # Loop through each map in this mapping
+        for map_name in maps:
+            # Recursively search for the map file in maps/ directory
+            map_path = None
+            for root, dirs, files in os.walk(Path("maps")):
+                if map_name in files:
+                    map_path = Path(root) / map_name
+                    break
+
+            # Check if map file was found
+            if map_path is None:
+                missing_maps.append(f"- {map_name} (step: {title})")
+                failed = True
+                break
+
+            # Read the map file
+            with open(map_path, 'r') as f:
+                map_content = f.read()
+
+            # Search for each forbidden type
+            for forbidden_type in forbidden_types:
+                if forbidden_type in map_content:
+                    illegal_types.append(f"- {map_name} - {forbidden_type} (step: {title})")
+                    failed = True
+
+    # Print results
+    if failed:
+        print(" ===== LINT FAILED =====")
+        if missing_maps:
+            print("Maps not found:")
+            for item in missing_maps:
+                print(item)
+        if illegal_types:
+            print("Forbidden types found:")
+            for item in illegal_types:
+                print(item)
+        sys.exit(1)
+    else:
+        print(" ===== LINT PASSED =====")
+
+if __name__ == "__main__":
+    check_forbidden_types()