Persistency subsystem update - Generics and history records (#22114)

# Summary

This PR is the next update to the persistency subsystem. The goal of
this PR is to provide more framework like functions to allow more types
of content to be made persistent.

Currently only volatile game objects created during a round can be (*in
a [clean](https://www.youtube.com/watch?v=rZ3ETK7-ZM8) way*) saved and
made persistent. This update attempts to provide methods to make
*everything*¹ persistent.

This introduces persistent generics and history.

## Database

The following things are going to be changed and added in the database  
(open in new tab for better visibility, PNG file includes the drawIO
code):

<img width="1692" height="1041" alt="aurora_persistency_db drawio"
src="https://github.com/user-attachments/assets/ea53f419-f9aa-4592-af8f-3a8d5edf3177"
/>

**Deviations on database implementation from diagram:**  
- Removed unique constraint on history table - Prevented adding multiple
records per round per attribute.

## Framework surface changes

- MC/VV: Moved global object track register to subsystem var space.
- MC/VV: Point of interest: Added history_cache and generic_cache to
subsystem var space.
- MC/VV: Updated subsystem stat entry message, now providing information
on cache sizes of new types.
- Added `singleton/persistent_type` defines (Type, clean-up rules,
finalization hook) and macros allowing new definitions of said types.
- Added cache structures that are also used for returns on public procs
in generics and history persistent types.
- Major new framework features: Persistent history (example: Mining
yield records) and persistent generics (example: Persistent Horizon
overmap position). See documentation for more information.

DrawIO diagram for documentation, includes source in it (open in new
tab):

<img width="200" height="200" alt="Persistence-subsystem-flowchart
drawio"
src="https://github.com/user-attachments/assets/51f28331-f999-49a2-a7cc-58278f7ae416"
/>

## Tasks

(These lists are not comprehensive.)

**General**
- [x] Update DB - Write SQL scripts.
- [x] Add subsystem modular files for generics and history.
- [x] Add type definition logic, macros.
- [x] Add type-DB init logic.
- [x] Logging.
- [x] A lot of testing. *A lot.*
- [x] Changelog.
- [x] Self-Review.
- [x] Update documentation on the persistence subsystem.

**"Persistent history"**
- [x] Add init logic.
- [x] Add finalize logic.
- [x] Add framework surface procs.
  - [x] Get last record.
  - [x] Get last X records.
  - [x] Add record.
  - [x] Add character ID related validation.
- [x] Add initial example mechanic.

**"Persistent generics"**
- [x] Add init logic.
- [x] Add finalize logic.
- [x] Add framework surface procs.
  - [x] Save.
  - [x] Load.
- [x] Add initial example mechanic.

## Changes

Too many changes to be listed here - Check changelog and actual changes.

## Warning

There are certain use/test cases that *cannot* be tested locally due to
missing preexisting data in the database. This should only affect new
data structures (new persistent types), not existing data.

¹ _Large scale persistent mapping is excluded for this version._

---------

Co-authored-by: Copilot <copilot@github.com>

code/controllers/subsystems/persistence/persistence.dm

@@ -2,24 +2,43 @@
  * Persistence subsystem
  * Subsytem for managing any form of persistent content across rounds.
  *
- * This subsystem consists of multiple partial files, following the structure:
- * - persistence.dm						- Subsystem definition and generic code.
- * - persistence_objects.dm				- Persistent objects related code.
- * - persistence_objects_sql.dm			- Persistent objects database code.
- * - persistence_objects_public.dm		- Persistent objects public procs.
+ * This subsystem consists of multiple partial files, split into different responsibilities:
+ *   persistence.dm - Subsystem define and related code
+ *   Objects and types (with Generics and History respectively), each containing:
+ *     Base file (no suffix), public procs (_public.dm suffix), SQL code (_sql.dm suffix)
  */
 
 SUBSYSTEM_DEF(persistence)
 	name = "Persistence"
 	init_order = INIT_ORDER_PERSISTENCE // The order is tied with the init and maploading subsystem.
 	flags = SS_NO_FIRE // This subsystem has no continues workload, it's init and shutdown only.
-	var/prevent_saving = FALSE // Toggle to prevent saving at round end, changed by toggle_persistence proc, used for admin purposes.
+	/// Sanity check to confirm init was a success before finalizing.
+	var/init_success = FALSE
+	/// Global toggle to prevent saving at round end, changed by toggle_persistence proc, used for admin purposes.
+	var/prevent_saving = FALSE
+	/// In-memory register of all persistent objects that were loaded or created during the round, used for tracking and finalization purposes.
+	var/object_track_register = list()
+	/// Dictionary<"[type](+[attribute])" cache of persistent history records.
+	var/history_cache = alist()
+	/// Manual record counter of cache containers.
+	var/history_cache_count = 0
+	/// ID of last found history record.
+	/// Higher found IDs mean the record is not yet found in the database, lower or equal found ID means the are record that are already in the database.
+	/// Used during history_virtual_id init and read-through cache hits.
+	var/history_last_database_id = 0
+	/// ID used for instanciating new history records during the round, used for cache tracking.
+	/// Their database ID will be set during insert/finalization.
+	var/history_virtual_id = 0
+	/// Dictionary<char_id, charname> cache of Character name by ID for history/character helper.
+	var/char_cache = alist()
+	/// Dictionary<"[type](+[attribute])", container> cache of persistent generics.
+	var/generic_cache = alist()
 
 /**
  * Subsystem info stub message generation.
  */
 /datum/controller/subsystem/persistence/stat_entry(msg)
-	msg = ("Register: [length(GLOB.persistence_object_track_register)] | Prevent saving: [SSpersistence.prevent_saving ? "TRUE" : "FALSE"]")
+	msg = ("[init_success ? "" : "INIT FAILED!!!|"][prevent_saving ? "SAVING DISABLED!|" : ""]Objects:[length(object_track_register)]|Containers:[length(history_cache)];Records:[history_cache_count]|Generics:[length(generic_cache)]")
 	return msg
 
 /**
@@ -77,7 +96,7 @@ SUBSYSTEM_DEF(persistence)
 	else
 		return
 
-	feedback_add_details("admin_verb","TP") //If you are copy-pasting this, ensure the 2nd parameter is unique to the new proc!
+	feedback_add_details("admin_verb","TPS") //If you are copy-pasting this, ensure the 2nd parameter is unique to the new proc!
 
 /**
  * Initialization of the persistence subsystem.
@@ -90,14 +109,22 @@ SUBSYSTEM_DEF(persistence)
 		return SS_INIT_SUCCESS
 
 	if(!databaseCheckConnection("subsystem init"))
+		log_subsystem_persistence_error("SQL connection unavailable. Init not possible.")
 		return SS_INIT_FAILURE
 
 	try
 		objectsInitialize()
-	catch(var/exception/e)
-		log_subsystem_persistence_panic("Unhandled exception during persistent objects initialization: [e]")
+	catch(var/exception/e_objects)
+		log_subsystem_persistence_panic("Unhandled exception during persistent objects initialization!", e_objects)
 		return SS_INIT_FAILURE
 
+	try
+		typesInitialize()
+	catch(var/exception/e_types)
+		log_subsystem_persistence_panic("Unhandled exception during persistent type initialization!", e_types)
+		return SS_INIT_FAILURE
+
+	init_success = TRUE
 	return SS_INIT_SUCCESS
 
 /**
@@ -105,6 +132,10 @@ SUBSYSTEM_DEF(persistence)
  * The shutdown consists of finalization steps for each persistent data type.
  */
 /datum/controller/subsystem/persistence/Shutdown()
+	if(!init_success)
+		log_subsystem_persistence_panic("Init success flag is FALSE. Something went wrong during subsystem init! Aborting finalization to prevent corrupt data!")
+		return
+
 	if(prevent_saving)
 		log_subsystem_persistence_warning("Persistence subsystem was toggled to not save. Skipping subsystem finalization.")
 		return
@@ -115,6 +146,10 @@ SUBSYSTEM_DEF(persistence)
 
 	try
 		objectsFinalize()
-	catch(var/exception/e)
-		log_subsystem_persistence_panic("Unhandled exception during persistent objects finalization: [e]")
-		return
+	catch(var/exception/e_objects)
+		log_subsystem_persistence_panic("Unhandled exception during persistent objects finalization!", e_objects)
+
+	try
+		typesFinalize()
+	catch(var/exception/e_types)
+		log_subsystem_persistence_panic("Unhandled exception during persistent types finalization!", e_types)

code/controllers/subsystems/persistence/persistence_objects.dm

@@ -4,7 +4,7 @@
  */
 /datum/controller/subsystem/persistence/proc/objectsInitialize()
 	PRIVATE_PROC(TRUE)
-	GLOB.persistence_object_track_register = list()
+	object_track_register = list()
 
 	if(SSatlas.current_map.path != "sccv_horizon") // The persistence system only supports objects from the main map levels for multiple reasons, e.g. Z level value, mapping support
 		log_subsystem_persistence_info("Persistent objects: Current map did not match SCCV Horizon, skipping persistent object initialization.")
@@ -42,8 +42,8 @@
 
 	if(SSatlas.current_map.path != "sccv_horizon") // The persistence system only supports objects from the main map levels for multiple reasons, e.g. Z level value, mapping support
 		log_subsystem_persistence_info("Persistent objects: Current map did not match SCCV Horizon, skipping persistent object finalization.")
-		if(length(GLOB.persistence_object_track_register) > 0)
-			log_subsystem_persistence_warning("Persistent objects: There are [length(GLOB.persistence_object_track_register)] tracked objects at finalization, while the map is not supported! These track will not be saved! Verify that SSatlas.current_map.path has not changed during the round!")
+		if(length(object_track_register) > 0)
+			log_subsystem_persistence_warning("Persistent objects: There are [length(object_track_register)] tracked objects at finalization, while the map is not supported! These track will not be saved! Verify that SSatlas.current_map.path has not changed during the round!")
 		return
 
 	// Subsystem shutdown:
@@ -52,7 +52,7 @@
 	// Delete persistent records that no longer exist in the registry (removed during the round)
 
 	// Run checks on each track that might prevent further persistence
-	for (var/obj/track as anything in GLOB.persistence_object_track_register)
+	for (var/obj/track in object_track_register)
 		CHECK_TICK
 		var/turf/T = get_turf(track)
 		if(!T || !is_station_level(T.z)) // The persistence system only supports objects from the main map levels for multiple reasons, e.g. Z level value, mapping support
@@ -65,7 +65,7 @@
 	// Get already stored data before saving new tracks so we can compare what has been updated or removed during the round.
 	var/list/existing_data = objectsDatabaseGetActiveEntries()
 
-	for (var/obj/track as anything in GLOB.persistence_object_track_register)
+	for (var/obj/track in object_track_register)
 		CHECK_TICK
 		if (track.persistent_objects_track_id == 0)
 			// Tracked object has no ID meaning it is new, create a new persistent record for it
@@ -76,7 +76,7 @@
 	// If we find the track, we need to check if it requires an update instead
 	for (var/record in existing_data)
 		var/found = FALSE
-		for (var/obj/track as anything in GLOB.persistence_object_track_register)
+		for (var/obj/track in object_track_register)
 			CHECK_TICK
 			if (record["id"] == track.persistent_objects_track_id)
 				// A track with the same ID has been found in the register, it still exists, check if we need to update it instead
@@ -114,7 +114,7 @@
 		if(length(content))
 			result = json_encode(content)
 	catch(var/exception/e)
-		log_subsystem_persistence_error("Error during json serialization for persistent object. Failed to get/encode track content: [e]")
+		log_subsystem_persistence_error("Error during json serialization or retrieval of content for persistent object. Type: [track.type]", e)
 	return result
 
 /**
@@ -129,4 +129,4 @@
 	try
 		track.persistent_objects_apply_content(json_decode(json), x, y, z)
 	catch(var/exception/e)
-		log_subsystem_persistence_error("Error during json deserialization for persistent object. Failed to apply/decode track content: [e]")
+		log_subsystem_persistence_error("Error during json deserialization or applying content for persistent object. Type: [track.type]", e)

code/controllers/subsystems/persistence/persistence_objects_public.dm

@@ -11,7 +11,7 @@
 
 	new_track.persistent_objects_track_active = TRUE
 	new_track.persistent_objects_author_ckey = ckey
-	GLOB.persistence_object_track_register += new_track
+	object_track_register += new_track
 
 /**
  * Removes the given object from the list of tracked objects. At shutdown the tracked object will be remove from the database.
@@ -21,4 +21,4 @@
 		return
 
 	old_track.persistent_objects_track_active = FALSE
-	GLOB.persistence_object_track_register -= old_track
+	object_track_register -= old_track

code/controllers/subsystems/persistence/persistence_objects_sql.dm

@@ -71,7 +71,7 @@
 		list(
 			"author_ckey" = track.persistent_objects_author_ckey,
 			"type" = "[track.type]",
-			"expire_in_days" = track.persistant_objects_expiration_time_days,
+			"expire_in_days" = track.persistent_objects_expiration_time_days,
 			"content" = objectsGetTrackContent(track),
 			"x" = T.x,
 			"y" = T.y,
@@ -99,7 +99,7 @@
 		"UPDATE ss13_persistent_objects SET author_ckey=:author_ckey, expires_at=DATE_ADD(NOW(), INTERVAL :expire_in_days DAY), content=:content, x=:x, y=:y, z=:z WHERE id = :id",
 		list(
 			"author_ckey" = track.persistent_objects_author_ckey,
-			"expire_in_days" = track.persistant_objects_expiration_time_days,
+			"expire_in_days" = track.persistent_objects_expiration_time_days,
 			"content" = objectsGetTrackContent(track),
 			"x" = T.x,
 			"y" = T.y,

code/controllers/subsystems/persistence/persistence_types.dm

@@ -0,0 +1,170 @@
+/**
+ * Called during subsystem init to upssert persistent type definitions into the database.
+ */
+/datum/controller/subsystem/persistence/proc/typesInitialize()
+	PRIVATE_PROC(TRUE)
+	// Types init:
+	// Upsert all types found in code into database
+	// Get type ID of each type found in code by name lookup
+	// -- Records
+	// Init history cache
+	// Run cleanup on history records
+	// -- Generics
+	// Init generic cache
+	// Run cleanup on generics
+
+	// Types upsert
+	// Base types to exclude
+	var/base_types = list(/singleton/persistent_type, /singleton/persistent_type/generic, /singleton/persistent_type/history, /singleton/persistent_type/history/character)
+	var/custom_types = typesof(/singleton/persistent_type) - base_types // These are the types we are actually dealing with
+
+	// Upsert all persistent type definitions found in code
+	// Whether or not it's new, get its database ID
+	for (var/C in custom_types)
+		CHECK_TICK
+		var/singleton/persistent_type/T = GET_SINGLETON(C)
+		typesDatabaseUpsertType("[T]", T.title, T.description, T.definition_type_value)
+		T.database_id = typesDatabaseGetTypeIdByName("[T]")
+
+	// ### Records
+
+	// Init internal history cache
+	history_last_database_id = historyDatabaseGetLastID()
+	if(history_last_database_id == 0)
+		log_subsystem_persistence_warning("Failed to get last ID of persistent type history records from the database during initialization. Either the table is empty or something went wrong.")
+	history_virtual_id = history_last_database_id
+	history_cache = list()
+
+	// Clean history records
+	for(var/type_combination in historyDatabaseGetTypeAttributeCombinations()) // Iterate through each distinct type+attribute combination
+		CHECK_TICK
+		var/type_id = type_combination["type_id"]
+		var/attribute = type_combination["attribute"]
+		var/singleton/persistent_type/history/found_type
+		for (var/C in custom_types)
+			var/singleton/persistent_type/T = GET_SINGLETON(C)
+			if(istype(T, /singleton/persistent_type/history) && T.database_id == type_id)
+				found_type = T
+		if(!found_type)
+			continue // The type found in the database is no longer available in the codebase
+
+		// Clean by the individual cleanup rule
+		if(ispath(found_type.expiration_rule, /singleton/persistent_type_history_expiration_rule/row_count)) // row_count
+			var/singleton/persistent_type_history_expiration_rule/row_count/rule = GET_SINGLETON(found_type.expiration_rule)
+			historyDatabaseCleanByRowCount(found_type.database_id, attribute, rule.max_row_count)
+
+		if(ispath(found_type.expiration_rule, /singleton/persistent_type_history_expiration_rule/round_count)) // round_count
+			var/singleton/persistent_type_history_expiration_rule/round_count/rule = GET_SINGLETON(found_type.expiration_rule)
+			historyDatabaseCleanByRoundCount(found_type.database_id, attribute, rule.max_round_count)
+
+		if(ispath(found_type.expiration_rule, /singleton/persistent_type_history_expiration_rule/age)) // age
+			var/singleton/persistent_type_history_expiration_rule/age/rule = GET_SINGLETON(found_type.expiration_rule)
+			historyDatabaseCleanByMaxAgeDays(found_type.database_id, attribute, rule.max_age_days)
+
+	// ### Generics
+
+	// Init internal generic cache
+	generic_cache = alist()
+	// Cleanup
+	genericDatabaseCleanup()
+
+	// ### Char lookup
+	char_cache = alist()
+
+/**
+ * Finalize persistent types.
+ * Adds new persistent generics and history.
+ */
+/datum/controller/subsystem/persistence/proc/typesFinalize()
+	PRIVATE_PROC(TRUE)
+
+	// Subsystem shutdown:
+	// Call finalization hook for each known persistent_type
+	// Save all history records in cache to database which have an ID higher then last known database ID - Records created during the round.
+	// Save all generics in cache to database which have an ID higher then last known database ID - Generics created during the round.
+
+	// ##### Hooks
+	var/base_types = list(/singleton/persistent_type, /singleton/persistent_type/generic, /singleton/persistent_type/history, /singleton/persistent_type/history/character)
+	var/custom_types = typesof(/singleton/persistent_type) - base_types // These are the types we are actually dealing with
+	for (var/C in custom_types)
+		CHECK_TICK
+		var/singleton/persistent_type/type_instance = GET_SINGLETON(C)
+		try
+			type_instance.finalization_hook()
+		catch(var/exception/e)
+			log_subsystem_persistence_error("Unhandled exception during finalization_hook of [type_instance]", e)
+
+	// ##### Saving history
+	var/total_saved_count = 0
+	for(var/key in history_cache)
+		CHECK_TICK
+		var/datum/persistent_record_container/container = history_cache[key] // Dictionary<"[type](+[attribute])", container>
+		if(!length(container.records))
+			continue // Container was queried, got no hits and nothing was added.
+
+		var/list/datum/persistent_record/new_records = list()
+		for(var/datum/persistent_record/record in container.records)
+			if(record.id > history_last_database_id) // ID assigned by virtual ID is larger then last known database ID, record is new and needs to be saved.
+				new_records += record
+
+		for(var/datum/persistent_record/record in new_records)
+			var/singleton/persistent_type/type_instance = GET_SINGLETON(container.type_define)
+			historyDatabaseInsertRecord(type_instance.database_id, container.attribute, record.value)
+			total_saved_count++
+
+	log_subsystem_persistence_info("Saved new [length(total_saved_count)] persistent history records.")
+
+	// ##### Saving generics
+	for(var/key in generic_cache)
+		CHECK_TICK
+		var/datum/persistent_generic/container = generic_cache[key] // Dictionary<"[type](+[attribute])", container>
+		var/singleton/persistent_type/type_instance = GET_SINGLETON(container.type_define)
+		genericDatabaseSave(type_instance.database_id, container.attribute, container.expires_in_days, container.content)
+
+	log_subsystem_persistence_info("Saved [length(generic_cache)] persistent generics.")
+
+/**
+ * Internal proc for assigning new IDs to history records, these are used for internal cache tracking and will be discard by database IDs at finalization.
+ */
+/datum/controller/subsystem/persistence/proc/typesGetVirtualRecordID()
+	PRIVATE_PROC(TRUE)
+	history_virtual_id += 1
+	return history_virtual_id
+
+/**
+ * Internal proc for finding the top K records (by ID) in a record container.
+ * Top K insertion sort selection (Manual leaderboard sort).
+ * PARAMS:
+ * 	k = Number of records to return.
+ * 	Container = Container to search in.
+ * RETURN:
+ * 	List of top K records by ID, sorted from highest to lowest.
+ */
+/datum/controller/subsystem/persistence/proc/typesHistoryCacheSelectTopK(k, datum/persistent_record_container/container)
+	PRIVATE_PROC(TRUE)
+	if(length(container.records))
+		return list()
+
+	var/list/datum/persistent_record/top = list()
+
+	for(var/datum/persistent_record/r in container.records)
+		var/insert_pos = 1
+
+		// Find position for insert when top isn't full yet or when a value in top is smaller then current record to be replaced
+		while(insert_pos <= top.len && top[insert_pos].id > r.id)
+			insert_pos++
+
+		if(top.len < k) // Top isn't full yet, insert without cutting
+			top.Insert(insert_pos, r)
+		else if(r.id > top[top.len].id) // Top is full, replace next lowest pos with current record and cut list back to size k
+			top.Insert(insert_pos, r)
+			top.Cut(k+1)
+
+	return top
+
+/datum/controller/subsystem/persistence/proc/typesGetCacheName(var/singleton/persistent_type/target_type, var/attribute)
+	PRIVATE_PROC(TRUE)
+	if(attribute && length(attribute) > 0)
+		return "[target_type]+[attribute]"
+	else
+		return "[target_type]"

code/controllers/subsystems/persistence/persistence_types_generic_public.dm

@@ -0,0 +1,72 @@
+/**
+ * Saves or overrides generic content for a type(+attribute)
+ * PARAMS:
+ * 	target_type =		Singleton persistent type definition. See /singleton/persistent_type/generic and subtypes.
+ *  content = 			List of associative values to be saved. ("id" = 123, "value" = "lorem ipsum")
+ *  attribute =			Custom attribute of the generic, can be null if the type definition doesn't require it. Defaults to null.
+ *	expires_in_days =	Days until the content is deemed expired. Defaults to PERSISTENT_DEFAULT_EXPIRATION_DAYS.
+ */
+/datum/controller/subsystem/persistence/proc/genericSave(var/singleton/persistent_type/generic/target_type, content, attribute = null, expires_in_days = PERSISTENT_DEFAULT_EXPIRATION_DAYS)
+	if(!content || !length(content))
+		return
+
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to add generic with null target type.")
+		return
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	if(type_instance.requires_attribute && !length(attribute))
+		log_subsystem_persistence_warning("Attempted to add generic of type [target_type] without required attribute.")
+		return
+
+	if(!expires_in_days || expires_in_days <= 0)
+		expires_in_days = PERSISTENT_DEFAULT_EXPIRATION_DAYS
+
+	attribute = length("[attribute]") > 0 ? attribute : null
+	var/datum/persistent_generic/generic = generic_cache[typesGetCacheName(target_type, attribute)]
+	if(generic)
+		generic.content = json_encode(content)
+		generic.expires_in_days = expires_in_days
+		return
+
+	var/datum/persistent_generic/new_generic = new /datum/persistent_generic/
+	new_generic.type_define = target_type
+	new_generic.attribute = attribute
+	new_generic.content = json_encode(content)
+	new_generic.expires_in_days = expires_in_days
+	generic_cache[typesGetCacheName(target_type, attribute)] = new_generic
+
+/**
+ * Retrieve/Loads generic content of a type(+attribute)
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type/generic and subtypes.
+ *  attribute =		Custom attribute of the generic, can be null if the type definition doesn't require it. Defaults to null.
+ * RETURN:
+ *	/persistent_generic or null if not available.
+ */
+/datum/controller/subsystem/persistence/proc/genericLoad(var/singleton/persistent_type/generic/target_type, attribute = null)
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to load generic with null target type.")
+		return
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	if(type_instance.requires_attribute && !length(attribute))
+		log_subsystem_persistence_warning("Attempted to load generic of type [target_type] without required attribute.")
+		return
+
+	attribute = length("[attribute]") > 0 ? attribute : null
+	var/datum/persistent_generic/generic = generic_cache[typesGetCacheName(target_type, attribute)]
+	if(generic)
+		return generic
+
+	var/result = genericDatabaseLoad(type_instance.database_id, attribute)
+	if(!result)
+		return null
+
+	var/datum/persistent_generic/new_generic = new /datum/persistent_generic/
+	new_generic.type_define = target_type
+	new_generic.attribute = attribute
+	new_generic.content = json_decode(result["content"])
+	new_generic.expires_in_days = 0
+	generic_cache[typesGetCacheName(target_type, attribute)] = new_generic
+	return new_generic

code/controllers/subsystems/persistence/persistence_types_generic_sql.dm

@@ -0,0 +1,137 @@
+/**
+ * Get the last ID in the generics table.
+ * RETURN:
+ *  Last ID or zero.
+ */
+/datum/controller/subsystem/persistence/proc/genericDatabaseGetLastID()
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("genericDatabaseGetLastID"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id FROM ss13_persistent_generics ORDER BY id DESC LIMIT 1"
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "genericDatabaseGetLastID"))
+		qdel(query)
+		return 0
+
+	var/last_id = 0
+	if(query.NextRow())
+		last_id = query.item[1]
+	qdel(query)
+	return last_id
+
+/**
+ * Runs a cleanup query on generics that have expired.
+ */
+/datum/controller/subsystem/persistence/proc/genericDatabaseCleanup()
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("genericDatabaseCleanup"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"DELETE FROM ss13_persistent_generics WHERE expires_at < NOW()"
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "genericDatabaseCleanup")
+	qdel(query)
+
+/**
+ * Save a generic persistent type(+attribute).
+ * PARAMS:
+ * 	type_id =			Type of ID.
+ *  attribute =			Custom attribute of the record, can be null.
+ *	expires_in_days =	Number of days until the content expires.
+ *  content =			JSON content to be saved.
+ */
+/datum/controller/subsystem/persistence/proc/genericDatabaseSave(type_id, attribute, expires_in_days, content)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("genericDatabaseSave"))
+		return 0
+
+	// Because MariaDB doesn't consider NULL in the attribute to be a violation of the UNIQUE constraint,
+	// we have to verify if the generic already exists, if the attribute is null.
+	// If the attribute is null and the generic exists, manually update the row instead of INSERT + ON DUPLICATE KEY.
+	// Otherwise, with valid unique constraint (not null attributes), we can continue with the regular INSERT + ON DUPLICATE KEY directly.
+	if(!attribute)
+		var/datum/db_query/null_attribute_query = SSdbcore.NewQuery(
+			"SELECT id FROM ss13_persistent_generics WHERE type = :type_id AND attribute IS NULL",
+			list(
+				"type_id" = type_id
+			)
+		)
+		null_attribute_query.Execute()
+
+		if(!databaseCheckQueryResult(null_attribute_query, "genericDatabaseSaveNullAttributeCheck"))
+			qdel(null_attribute_query)
+			return 0
+
+		var/id = 0
+		if(null_attribute_query.NextRow())
+			id = null_attribute_query.item[1]
+		qdel(null_attribute_query)
+		if(id > 0) // Attribute null and row found - Invalid unique contraint for MariaDB - Update manually.
+			var/datum/db_query/update_query = SSdbcore.NewQuery(
+				"UPDATE ss13_persistent_generics SET created_at = NOW(), expires_at = DATE_ADD(NOW(), INTERVAL :expires_in_days DAY), content = :content WHERE id = :id",
+				list(
+					"expires_in_days" = expires_in_days,
+					"content" = content,
+					"id" = id
+				)
+			)
+			update_query.Execute()
+
+			databaseCheckQueryResult(update_query, "genericDatabaseSaveNullAttributeUpdate")
+			qdel(update_query)
+			return // Skip regular upcoming query due to the reasons above
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"INSERT INTO ss13_persistent_generics (type, attribute, created_at, expires_at, content) VALUES (:type_id, :attribute, NOW(), DATE_ADD(NOW(), INTERVAL :expires_in_days DAY), :content) \
+		ON DUPLICATE KEY UPDATE created_at = NOW(), expires_at = DATE_ADD(NOW(), INTERVAL :expires_in_days DAY), content = :content",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"expires_in_days" = expires_in_days,
+			"content" = content
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "genericDatabaseSave")
+	qdel(query)
+
+/**
+ * Load a generic persistent type(+attribute).
+ * PARAMS:
+ * 	type_id =			Type of ID.
+ *  attribute =			Custom attribute of the record, can be null.
+ * RETURN:
+ *  Associative list of keys "id", "content" (JSON).
+ */
+/datum/controller/subsystem/persistence/proc/genericDatabaseLoad(type_id, attribute)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("genericDatabaseLoad"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id, content FROM ss13_persistent_generics \
+		WHERE type = :type_id AND attribute <=> :attribute",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute
+		)
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "genericDatabaseLoad"))
+		qdel(query)
+		return null
+
+	var/result = null
+	while(query.NextRow())
+		result = list("id" = query.item[1], "content" = query.item[2])
+	qdel(query)
+	return result

code/controllers/subsystems/persistence/persistence_types_history_public.dm

@@ -0,0 +1,263 @@
+#define PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT 1000 // Max row count of records allowed to be drawn from database for performance reasons
+
+/**
+ * Helper proc for history/character persistent types to retrieve a character name based of it's character ID.
+ * PARAMS:
+ *  char_id = ID of character.
+ * RETURN:
+ *  Character name or null if not found.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetCharnameByID(char_id)
+	if(!char_id)
+		return null
+
+	var/char_id_num = text2num(char_id)
+	if(char_id_num <= 0)
+		return null
+
+	var/cache_hit = char_cache["[char_id_num]"]
+	if(cache_hit)
+		return cache_hit
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT name FROM ss13_characters WHERE id = :char_id",
+		list("char_id" = char_id_num)
+	)
+	query.Execute()
+
+	var/char_name
+	while(query.NextRow())
+		char_name += query.item[1]
+	qdel(query)
+	if(!char_name)
+		return null
+	else
+		char_cache["[char_id_num]"] = char_name
+		return char_name
+
+/**
+ * Add a new record to the history for the given type/attribute.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type/history and subtypes.
+ *  attribute =		Custom attribute of the record, can be null if the type definition doesn't require it.
+ *	value =			Value of the record, cannot be null or empty.
+ */
+/datum/controller/subsystem/persistence/proc/historyAddRecord(var/singleton/persistent_type/history/target_type, attribute, value)
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to add history record with null target type.")
+		return
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	if(type_instance.requires_attribute && length(attribute) > 0)
+		log_subsystem_persistence_warning("Attempted to add history record of type [target_type] without required attribute.")
+		return
+
+	if(!value)
+		log_subsystem_persistence_warning("Attempted to add history record of type [target_type] with empty value.")
+		return
+
+	// Sanity check if a character record is added using this proc directly instead of the overload historyAddCharacterRecord.
+	if(istype(type_instance, /singleton/persistent_type/history/character) && (length(attribute) > 0 || !isnum(attribute)))
+		log_subsystem_persistence_warning("Attempted to add character history record of target type [target_type], but the attribute was either empty or failed the isnum check.")
+		return
+
+	// Add record to cache for DB insert at finalization and quick access
+	// Check if record container exists, if not, create it
+	var/datum/persistent_record_container/container = null
+	attribute = length("[attribute]") > 0 ? attribute : null
+	container = history_cache[typesGetCacheName(target_type, attribute)]
+
+	if(!container)
+		container = new /datum/persistent_record_container
+		container.type_define = target_type.type
+		container.attribute = attribute
+		container.records = list()
+		history_cache[typesGetCacheName(target_type, attribute)] = container
+
+	// Create record and add to container
+	var/datum/persistent_record/r = new /datum/persistent_record
+	r.id = typesGetVirtualRecordID()
+	r.created_at = "[worlddate2text()] [worldtime2text()]"
+	r.game_id = GLOB.round_id
+	r.value = value
+	container.records += r
+	history_cache_count++
+
+/**
+ * Add a new record that belongs to a specific character to the history for the given type.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type/history/character and subtypes.
+ *  char_id =		Character ID that the record should belong to.
+ *	value =			Value of the record, cannot be null or empty.
+ */
+/datum/controller/subsystem/persistence/proc/historyAddCharacterRecord(var/singleton/persistent_type/history/character/target_type, char_id, value)
+	if(!ispath(target_type, /singleton/persistent_type/history/character))
+		log_subsystem_persistence_warning("Attempted to add character history record, but the provided target type didn't match a character persistent type, provided was [target_type]")
+		return
+	if(!isnum(char_id))
+		log_subsystem_persistence_warning("Attempted to add character history record of type [target_type] but char_id failed the isnum check.")
+		return
+	return historyAddRecord(target_type, char_id, value)
+
+/**
+ * Queries the last record of a specified type/attribute.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *					If the type definition is a character record type, the attribute must be a valid character ID or the record will be rejected.
+ *  attribute =		Custom attribute of the record, can be null if the type definition doesn't require it.
+ * RETURN:
+ * 	Single /persistent_record or null.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetLastRecord(var/singleton/persistent_type/history/target_type, attribute)
+	var/result = historyGetLastRecords(target_type, attribute, 1)
+	if(length(result) == 0)
+		return null
+	else
+		return result[1]
+
+/**
+ * Queries the last X records of a specified type/attribute.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *					If the type definition is a character record type, the attribute must be a valid character ID or the record will be rejected.
+ *  attribute =		Custom attribute of the record, can be null if the type definition doesn't require it.
+ *  limit =			Number of records to retrieve.
+ *  skip_caching =	If set to TRUE, the results won't be added to the types cache, defaults to FALSE.
+ * RETURN:
+ * 	List of /persistent_record or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetLastRecords(var/singleton/persistent_type/history/target_type, attribute, limit, skip_caching = FALSE)
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to get history records with null target type.")
+		return list()
+
+	if(limit > PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT)
+		limit = PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT
+		log_subsystem_persistence_warning("Attempted to draw more records then allowed for target type [target_type].")
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	if(type_instance.requires_attribute && !attribute)
+		log_subsystem_persistence_warning("Attempted to get history records of type [target_type] without required attribute.")
+		return list()
+
+	// Query order
+	// 1 - Check if record container exists, if so, check if last X records are in there, aggregate found records, step to DB (2) for missing remainders.
+	// 2 - Query database for last X records of type and add it to record container as new cache
+
+	var/datum/persistent_record_container/container = null
+	attribute = length("[attribute]") > 0 ? attribute : null
+	container = history_cache[typesGetCacheName(target_type, attribute)]
+
+	var/list/datum/persistent_record/top = list()
+
+	// Query order - 1
+	if(container)
+		top = typesHistoryCacheSelectTopK(limit, container)
+		if(length(top) == limit) // All X records got hit in cache, return
+			return top
+	else if (!skip_caching)
+		container = new /datum/persistent_record_container
+		container.type_define = target_type.type
+		container.attribute = attribute
+		container.records = list()
+		history_cache[typesGetCacheName(target_type, attribute)] = container
+
+	// Query order - 2
+	var/list/db_records = historyDatabaseGetRecords(type_instance.database_id, attribute, limit - length(top)) // Draw remaining missing records from DB
+	var/len = length(db_records)
+	if(!len)
+		return list()
+
+	if(!skip_caching)
+		history_cache_count += len
+
+	for(var/alist/record in db_records)
+		var/datum/persistent_record/r = new /datum/persistent_record
+		r.id = record["id"]
+		r.created_at = record["created_at"]
+		r.game_id = record["game_id"]
+		r.value = record["value"]
+		if(!skip_caching)
+			container.records += r // Add to cache
+		top += r // Records in top are either newly created or read from DB already, append newly queries records.
+
+	return top
+
+/**
+ * Queries all records of a specified type/attribute.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *					If the type definition is a character record type, the attribute must be a valid character ID or the record will be rejected.
+ *  attribute =		Custom attribute of the record, can be null if the type definition doesn't require it.
+ *  skip_caching =	If set to TRUE, the results won't be added to the types cache, defaults to TRUE.
+ * RETURN:
+ * 	List of /persistent_record or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetAllRecords(var/singleton/persistent_type/history/target_type, attribute, skip_caching = TRUE)
+	var/result = historyGetLastRecords(target_type, attribute, PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT, skip_caching)
+	if(length(result) == 0)
+		return null
+	else
+		return result
+
+/**
+ * Queries the last record of the specified type for all attributes.
+ * PARAMS:
+ * 	target_type = Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ * RETURN:
+ * 	Associative list with "attribute" and "records" of type list(/persistent_record) or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetLastRecordForAllAttributes(var/singleton/persistent_type/history/target_type)
+	var/result = historyGetLastRecordsForAllAttributes(target_type, 1)
+	if(length(result) == 0)
+		return list()
+	else
+		return result
+
+/**
+ * Queries the last X records of a specified type for all attributes.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *  limit =			Number of records to retrieve.
+ *  skip_caching =	If set to TRUE, the results won't be added to the types cache, defaults to TRUE.
+ * RETURN:
+ * 	List of associative list with "attribute" and "records" of type list(/persistent_record) or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetLastRecordsForAllAttributes(var/singleton/persistent_type/history/target_type, limit, skip_caching = TRUE)
+	if(!target_type)
+		log_subsystem_persistence_warning("Attempted to get history records with null target type.")
+		return list()
+
+	if(limit > PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT)
+		limit = PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT
+		log_subsystem_persistence_warning("Attempted to draw more records then allowed for target type [target_type].")
+
+	var/singleton/persistent_type/type_instance = GET_SINGLETON(target_type)
+	var/list/result = list()
+
+	var/attributes = historyDatabaseGetAllAttributes(type_instance.database_id)
+	if(attributes && length(attributes) > 0)
+		for(var/attribute in attributes)
+			result += list(alist("attribute" = attribute, "records" = historyGetAllRecords(target_type, attribute, skip_caching)))
+	else
+		var/no_attribute_records = historyGetAllRecords(target_type, null, skip_caching)
+		if(no_attribute_records && length(no_attribute_records) > 0)
+			result = list(alist("attribute" = null, "records" = no_attribute_records))
+	return result
+
+/**
+ * Queries all records of a specified type for all attributes.
+ * PARAMS:
+ * 	target_type =	Singleton persistent type definition. See /singleton/persistent_type and subtypes.
+ *  skip_caching =	If set to TRUE, the results won't be added to the types cache, defaults to TRUE.
+ * RETURN:
+ * 	List of associative list with "attribute" and "records" of type list(/persistent_record) or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyGetAllRecordsForAllAttributes(var/singleton/persistent_type/history/target_type, skip_caching = TRUE)
+	var/result = historyGetLastRecordsForAllAttributes(target_type, PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT, skip_caching)
+	if(length(result) == 0)
+		return null
+	else
+		return result
+
+#undef PERSISTENCE_INTERNAL_MAX_RECORD_QUERY_COUNT

code/controllers/subsystems/persistence/persistence_types_history_sql.dm

@@ -0,0 +1,235 @@
+/**
+ * Get the last ID in the history table.
+ * RETURN:
+ *  Last ID or zero.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseGetLastID()
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseGetLastID"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id FROM ss13_persistent_history ORDER BY id DESC LIMIT 1"
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "historyDatabaseGetLastID"))
+		qdel(query)
+		return 0
+
+	var/last_id = 0
+	if(query.NextRow())
+		last_id = query.item[1]
+	qdel(query)
+	return last_id
+
+/**
+ * Returns all combinations of types+attributes from persistent history.
+ * RETURN:
+ * 	Distinct list of list with keys "type_id" and "attribute" (possibly null).
+ *  Example: (("type_id" = 1, "attribute" = null), ("type_id" = 1, "attribute" = "lorem ipsum"), ("type_id" = 2, "attribute" = "dolor sit amet"))
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseGetTypeAttributeCombinations()
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseGetTypeAttributeCombinations"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT DISTINCT type, attribute FROM ss13_persistent_history"
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "historyDatabaseGetTypeAttributeCombinations"))
+		qdel(query)
+		return null
+
+	var/result = list()
+	while(query.NextRow())
+		result += list(alist("type_id" = query.item[1], "attribute" = query.item[2]))
+	qdel(query)
+	return result
+
+/**
+ * Returns all attributes from persistent history for a specified type.
+ * RETURN:
+ * 	Distinct list of attributes or empty list.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseGetAllAttributes(type_id)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseGetAllAttributes"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT DISTINCT attribute FROM ss13_persistent_history WHERE type = :type_id",
+		list("type_id" = type_id)
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "historyDatabaseGetAllAttributes"))
+		qdel(query)
+		return null
+
+	var/result = list()
+	while(query.NextRow())
+		result += query.item[1]
+	qdel(query)
+	return result
+
+/**
+ * Clean up history records of type+attribute by specified row count
+ * PARAMS:
+ * 	type_id =	ID of type.
+ *  attribute =	Custom attribute of the record, can be null.
+ *	row_count =	Count of rows to keep for the specified grouping.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseCleanByRowCount(type_id, attribute, row_count)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseCleanByRowCount"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"\
+			DELETE FROM ss13_persistent_history \
+			WHERE type = :type_id AND attribute <=> :attribute \
+			AND id NOT IN ( \
+				SELECT id FROM ss13_persistent_history \
+				WHERE type = :type_id AND attribute <=> :attribute \
+				ORDER BY created_at DESC, id DESC \
+				LIMIT :row_count \
+			)",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"row_count" = row_count
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "historyDatabaseCleanByRowCount")
+	qdel(query)
+
+/**
+ * Clean up history records of type+attribute by specified round count.
+ * PARAMS:
+ * 	type_id =		Type of ID.
+ *  attribute =		Custom attribute of the record, can be null.
+ *	round_count =	Number of rounds to keep for specified grouping.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseCleanByRoundCount(type_id, attribute, round_count)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseCleanByRoundCount"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"\
+			DELETE FROM ss13_persistent_history \
+			WHERE type = :type_id AND attribute <=> :attribute \
+			AND game_id NOT IN ( \
+				SELECT game_id FROM ( \
+					SELECT game_id FROM ss13_persistent_history \
+					WHERE type = :type_id AND attribute <=> :attribute \
+					GROUP BY game_id \
+					ORDER BY MAX(created_at) DESC, game_id DESC \
+					LIMIT :round_count \
+				) AS recent_games \
+			)",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"round_count" = round_count
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "historyDatabaseCleanByRoundCount")
+	qdel(query)
+
+/**
+ * Clean up history records of type+attribute by max age of record in days
+ * PARAMS:
+ * 	type_id =		ID of type.
+ *  attribute =		Custom attribute of the record, can be null.
+ *	max_age_days =	Max age of records in days.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseCleanByMaxAgeDays(type_id, attribute, max_age_days)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseCleanByMaxAgeDays"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"\
+			DELETE FROM ss13_persistent_history \
+			WHERE type = :type_id AND attribute <=> :attribute \
+			AND created_at < (NOW() - INTERVAL :max_age_days DAY)",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"max_age_days" = max_age_days
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "historyDatabaseCleanByMaxAgeDays")
+	qdel(query)
+
+/**
+ * Insert a new history record into the history table.
+ * PARAMS:
+ * 	type_id =	ID of type.
+ *  attribute =	Custom attribute of the record, can be null.
+ *	value =		Value of the record, cannot be null or empty.
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseInsertRecord(type_id, attribute, value)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseInsertRecord"))
+		return
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"INSERT INTO ss13_persistent_history (type, created_at, attribute, value, game_id) VALUES (:type_id, NOW(), :attribute, :value, :game_id)",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"value" = "[value]",
+			"game_id" = "[GLOB.round_id]"
+		)
+	)
+	query.Execute()
+
+	databaseCheckQueryResult(query, "historyDatabaseInsertRecord")
+	qdel(query)
+
+/**
+ * Get the last X history records for a type+attribute.
+ * PARAMS:
+ * 	type_id =	ID of type
+ *  attribute =	Custom attribute of the record, can be null.
+ *	count =		Number of records to be returned.
+ * RETURN:
+ *  List of records, each as a list consisting of keys "id", "created_at" and "value".
+ */
+/datum/controller/subsystem/persistence/proc/historyDatabaseGetRecords(type_id, attribute, count)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("historyDatabaseGetRecords"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id, created_at, value, game_id FROM ss13_persistent_history \
+		WHERE type = :type_id AND attribute <=> :attribute \
+		ORDER BY id DESC LIMIT :count",
+		list(
+			"type_id" = type_id,
+			"attribute" = attribute,
+			"count" = count
+		)
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "historyDatabaseGetRecords"))
+		qdel(query)
+		return null
+
+	var/records = list()
+	while(query.NextRow())
+		records += list(alist("id" = query.item[1], "created_at" = query.item[2], "value" = query.item[3]))
+	qdel(query)
+	return records

code/controllers/subsystems/persistence/persistence_types_sql.dm

@@ -0,0 +1,55 @@
+/**
+ * Insert or update a type definition in the database. If a type with the same name already exists, it will be updated with the new title and description.
+ * PARAMS:
+ * 	type =				Name of type to be upserted.
+ *  title =				Custom display title of the type.
+ *  description =		Custom display description of the type.
+ *  definition_type =	Enum value of the definition type. See /singleton/persistent_type.
+ */
+/datum/controller/subsystem/persistence/proc/typesDatabaseUpsertType(type, title, description, definition_type)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("typesDatabaseUpsertType"))
+		return
+
+	var/datum/db_query/upsert_query = SSdbcore.NewQuery(
+		"INSERT INTO ss13_persistent_type_definitions (type, title, description, definition_type) VALUES (:type, :title, :description, :definition_type) \
+		ON DUPLICATE KEY UPDATE title = VALUES(title), description = VALUES(description)",
+		list(
+			"type" = type,
+			"title" = title,
+			"description" = description,
+			"definition_type" = definition_type
+		)
+	)
+	upsert_query.Execute()
+
+	databaseCheckQueryResult(upsert_query, "typesDatabaseUpsertType")
+	qdel(upsert_query)
+
+/**
+ * Get ID of type definition.
+ * PARAMS:
+ *	type_name = Type name of singleton definition.
+ * RETURN:
+ *	Database ID of type.
+ */
+/datum/controller/subsystem/persistence/proc/typesDatabaseGetTypeIdByName(type_name)
+	PRIVATE_PROC(TRUE)
+	if(!databaseCheckConnection("typesDatabaseGetTypeIdByName"))
+		return 0
+
+	var/datum/db_query/query = SSdbcore.NewQuery(
+		"SELECT id FROM ss13_persistent_type_definitions WHERE type = :type_name",
+		list("type_name" = type_name)
+	)
+	query.Execute()
+
+	if(!databaseCheckQueryResult(query, "typesDatabaseGetTypeIdByName"))
+		qdel(query)
+		return 0
+
+	var/database_id = null
+	if(query.NextRow())
+		database_id = query.item[1]
+	qdel(query)
+	return database_id