mirror of
https://github.com/Aurorastation/Aurora.3.git
synced 2026-08-27 06:54:18 +01:00
Persistency subsystem update - Generics and history records (#22114)
# Summary This PR is the next update to the persistency subsystem. The goal of this PR is to provide more framework like functions to allow more types of content to be made persistent. Currently only volatile game objects created during a round can be (*in a [clean](https://www.youtube.com/watch?v=rZ3ETK7-ZM8) way*) saved and made persistent. This update attempts to provide methods to make *everything*¹ persistent. This introduces persistent generics and history. ## Database The following things are going to be changed and added in the database (open in new tab for better visibility, PNG file includes the drawIO code): <img width="1692" height="1041" alt="aurora_persistency_db drawio" src="https://github.com/user-attachments/assets/ea53f419-f9aa-4592-af8f-3a8d5edf3177" /> **Deviations on database implementation from diagram:** - Removed unique constraint on history table - Prevented adding multiple records per round per attribute. ## Framework surface changes - MC/VV: Moved global object track register to subsystem var space. - MC/VV: Point of interest: Added history_cache and generic_cache to subsystem var space. - MC/VV: Updated subsystem stat entry message, now providing information on cache sizes of new types. - Added `singleton/persistent_type` defines (Type, clean-up rules, finalization hook) and macros allowing new definitions of said types. - Added cache structures that are also used for returns on public procs in generics and history persistent types. - Major new framework features: Persistent history (example: Mining yield records) and persistent generics (example: Persistent Horizon overmap position). See documentation for more information. DrawIO diagram for documentation, includes source in it (open in new tab): <img width="200" height="200" alt="Persistence-subsystem-flowchart drawio" src="https://github.com/user-attachments/assets/51f28331-f999-49a2-a7cc-58278f7ae416" /> ## Tasks (These lists are not comprehensive.) **General** - [x] Update DB - Write SQL scripts. - [x] Add subsystem modular files for generics and history. - [x] Add type definition logic, macros. - [x] Add type-DB init logic. - [x] Logging. - [x] A lot of testing. *A lot.* - [x] Changelog. - [x] Self-Review. - [x] Update documentation on the persistence subsystem. **"Persistent history"** - [x] Add init logic. - [x] Add finalize logic. - [x] Add framework surface procs. - [x] Get last record. - [x] Get last X records. - [x] Add record. - [x] Add character ID related validation. - [x] Add initial example mechanic. **"Persistent generics"** - [x] Add init logic. - [x] Add finalize logic. - [x] Add framework surface procs. - [x] Save. - [x] Load. - [x] Add initial example mechanic. ## Changes Too many changes to be listed here - Check changelog and actual changes. ## Warning There are certain use/test cases that *cannot* be tested locally due to missing preexisting data in the database. This should only affect new data structures (new persistent types), not existing data. ¹ _Large scale persistent mapping is excluded for this version._ --------- Co-authored-by: Copilot <copilot@github.com>
This commit is contained in:
FabianK3
@@ -109,10 +109,6 @@ GLOBAL_VAR(custom_event_msg)
|
||||
GLOBAL_DATUM(dbcon, /DBConnection)
|
||||
GLOBAL_PROTECT(dbcon)
|
||||
|
||||
// Persistence subsystem object track register - List of all persistent objects tracked by the subsystem.
|
||||
GLOBAL_LIST_EMPTY(persistence_object_track_register)
|
||||
GLOBAL_PROTECT(persistence_object_track_register)
|
||||
|
||||
// Added for Xenoarchaeology, might be useful for other stuff.
|
||||
GLOBAL_LIST_INIT(alphabet_uppercase, list("A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z"))
|
||||
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
/*#############################################
|
||||
Constants for the persistence subsystem
|
||||
#############################################*/
|
||||
|
||||
#define PERSISTENT_DEFAULT_EXPIRATION_DAYS 30 // Default expire timespan for newly created persistent objects
|
||||
#define PERSISTENT_EXPIRATION_CLEANUP_DELAY_DAYS 30 // Grace period for expired database entries before they get cleaned up.
|
||||
@@ -0,0 +1,58 @@
|
||||
/*####################################################
|
||||
Defines for cleanups and expirations
|
||||
####################################################*/
|
||||
|
||||
#define PERSISTENT_DEFAULT_EXPIRATION_DAYS 30 // Default expire timespan for newly created persistent content
|
||||
#define PERSISTENT_EXPIRATION_CLEANUP_DELAY_DAYS 30 // Grace period for expired database entries before they get cleaned up, objects only
|
||||
|
||||
// ##### Persistent type "history" expiration rules
|
||||
// Rules are applied on the combination type+attribute
|
||||
// See type definition macros on their usage
|
||||
// Abstract marked types are used for type catching in code and not to be used in type definitions
|
||||
|
||||
ABSTRACT_TYPE(/singleton/persistent_type_history_expiration_rule)
|
||||
|
||||
// Keep last X rows - This rule removes all records exceeding the newest X records by count
|
||||
ABSTRACT_TYPE(/singleton/persistent_type_history_expiration_rule/row_count)
|
||||
var/max_row_count = 0
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/row_count/ten
|
||||
max_row_count = 10
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/row_count/hundred
|
||||
max_row_count = 100
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/row_count/thousand
|
||||
max_row_count = 1000
|
||||
|
||||
// Keep records for X rounds - This rule removes all records that don't belong to the last X finished rounds
|
||||
ABSTRACT_TYPE(/singleton/persistent_type_history_expiration_rule/round_count)
|
||||
var/max_round_count = 0
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/round_count/ten
|
||||
max_round_count = 10
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/round_count/fifty
|
||||
max_round_count = 50
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/round_count/hundred
|
||||
max_round_count = 100
|
||||
|
||||
// Keep records for X days - This rule removes all records that are older then X days since their creation
|
||||
ABSTRACT_TYPE(/singleton/persistent_type_history_expiration_rule/age)
|
||||
var/max_age_days = 0
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/age/default
|
||||
max_age_days = PERSISTENT_DEFAULT_EXPIRATION_DAYS
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/age/week
|
||||
max_age_days = 7
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/age/quarter_year
|
||||
max_age_days = 90
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/age/half_year
|
||||
max_age_days = 180
|
||||
|
||||
/singleton/persistent_type_history_expiration_rule/age/year
|
||||
max_age_days = 365
|
||||
@@ -0,0 +1,22 @@
|
||||
/*###################################################
|
||||
Subsystem cache structures
|
||||
###################################################*/
|
||||
|
||||
// Data transfer objects - Used by the subsystem for aggregation and result returns, these should be treated as read-only when handed by the subsystem
|
||||
|
||||
/datum/persistent_record_container // Container for combining records of type(+attribute)
|
||||
var/singleton/persistent_type/history/type_define = null // Definition type
|
||||
var/attribute = null // Attribute for aggregation records into type+attribute groups
|
||||
var/list/datum/persistent_record/records = list() // Container contents
|
||||
|
||||
/datum/persistent_record // Single persistent record
|
||||
var/id = 0 // Database ID - Might be a non-existend (virtual) ID if the record hasn't been saved yet
|
||||
var/created_at = "" // Timestamp when the record was saved
|
||||
var/game_id = "" // Game Id when the record was saved
|
||||
var/value = null // Treat this as string value
|
||||
|
||||
/datum/persistent_generic // Persistent generic data holder
|
||||
var/singleton/persistent_type/history/type_define = null // Definition type
|
||||
var/attribute = null // Attribute for aggregation into type+attribute
|
||||
var/content = null // Treat this as a string - Open for implementing caller (e.g. raw string or json)
|
||||
var/expires_in_days = PERSISTENT_EXPIRATION_CLEANUP_DELAY_DAYS // Expiration timespan used when generic is saved
|
||||
@@ -0,0 +1,78 @@
|
||||
/*###################################################
|
||||
Base types and macros for type definitions
|
||||
###################################################*/
|
||||
|
||||
// ##### Base type definitions
|
||||
|
||||
// Persistent type definition found in database
|
||||
ABSTRACT_TYPE(/singleton/persistent_type)
|
||||
var/database_id = 0 // Set during subsystem init - DO NOT MODIFY
|
||||
var/definition_type_value = 0 // DO NOT MODIFY - DATABASE CONSTANT
|
||||
var/title = ""
|
||||
var/description = ""
|
||||
var/requires_attribute = FALSE // Whether or not this type requires/has an attribute, relevant for subsystem when saving or pulling type data - Should not be changed after release for the given type
|
||||
|
||||
// Hard coded in "ss13_persistent_type_definitions.definition_type", DO NOT MODIFY - DATABASE CONSTANTS
|
||||
#define PERSISTENCE_INTERNAL_TYPE_DEFINE_TYPE_VALUE_GENERIC 1
|
||||
#define PERSISTENCE_INTERNAL_TYPE_DEFINE_TYPE_VALUE_HISTORY 2
|
||||
|
||||
/**
|
||||
* Hook proc that is called by the subsystem starting finalization on each persistent type definition.
|
||||
* Hooks are used for implementing finalization logic for mechanics that either
|
||||
* don't have a single trigger to save or where repetitive saving would be too costly.
|
||||
* Should return nothing, returned values are discarded.
|
||||
* Implementation can be put where applicable, e.g. next to loading logic of the type.
|
||||
*/
|
||||
/singleton/persistent_type/proc/finalization_hook()
|
||||
SHOULD_CALL_PARENT(FALSE)
|
||||
return
|
||||
|
||||
ABSTRACT_TYPE(/singleton/persistent_type/generic) // Base type for "persistent generics"
|
||||
definition_type_value = PERSISTENCE_INTERNAL_TYPE_DEFINE_TYPE_VALUE_GENERIC // DO NOT MODIFY - DATABASE CONSTANT
|
||||
|
||||
ABSTRACT_TYPE(/singleton/persistent_type/history) // Base type for "persistent history"
|
||||
definition_type_value = PERSISTENCE_INTERNAL_TYPE_DEFINE_TYPE_VALUE_HISTORY // DO NOT MODIFY - DATABASE CONSTANT
|
||||
var/singleton/persistent_type_history_expiration_rule/expiration_rule = null
|
||||
|
||||
ABSTRACT_TYPE(/singleton/persistent_type/history/character) // Base type with extended validation for persistent history in relation to characters
|
||||
// Empty stub
|
||||
|
||||
// ##### Macros for new custom type definitions
|
||||
// - TYPE_NAME = Name of the type definition, used to upsert into database, cannot be updated - Changes result in a new type definition in the DB
|
||||
// - TITLE = Title of the type definition, used for display purposes
|
||||
// - DESCRIPTION = Description of the type definition, used for display purposes
|
||||
// - REQUIRES_ATTRIBUTE = Boolean, whether this type definition requires an attribute to be specified
|
||||
// - EXPIRATION_RULE = For history type definitions, the expiration rule to apply to records of this type.
|
||||
// See /singleton/persistent_type_history_expiration_rule and subtypes for available rules.
|
||||
|
||||
// Persistent generic
|
||||
// CREATE_PERSISTENT_TYPE_GENERIC(my_type_name, "My custom type", "This type is a test and has no purpose", TRUE)
|
||||
#define CREATE_PERSISTENT_TYPE_GENERIC(TYPE_NAME, TITLE, DESCRIPTION, REQUIRES_ATTRIBUTE) \
|
||||
/singleton/persistent_type/generic/##TYPE_NAME \
|
||||
{ \
|
||||
title = #TITLE; \
|
||||
description = #DESCRIPTION; \
|
||||
requires_attribute = ##REQUIRES_ATTRIBUTE; \
|
||||
}
|
||||
|
||||
// Persistent history
|
||||
// CREATE_PERSISTENT_TYPE_HISTORY(my_type_name, "My custom type", "This type is a test and has no purpose", TRUE, /singleton/persistent_type_history_expiration_rule/age/default)
|
||||
#define CREATE_PERSISTENT_TYPE_HISTORY(TYPE_NAME, TITLE, DESCRIPTION, REQUIRES_ATTRIBUTE, EXPIRATION_RULE) \
|
||||
/singleton/persistent_type/history/##TYPE_NAME \
|
||||
{ \
|
||||
title = #TITLE; \
|
||||
description = #DESCRIPTION; \
|
||||
requires_attribute = ##REQUIRES_ATTRIBUTE; \
|
||||
expiration_rule = ##EXPIRATION_RULE; \
|
||||
}
|
||||
|
||||
// Persistent history with extended validation on character relation - Attribute automatically required compared to parent persistent history type definition
|
||||
// CREATE_PERSISTENT_TYPE_HISTORY_CHARACTER(my_type_name, "My custom type", "This type is a test and has no purpose", /singleton/persistent_type_history_expiration_rule/age/default)
|
||||
#define CREATE_PERSISTENT_TYPE_HISTORY_CHARACTER(TYPE_NAME, TITLE, DESCRIPTION, EXPIRATION_RULE) \
|
||||
/singleton/persistent_type/history/character/##TYPE_NAME \
|
||||
{ \
|
||||
title = #TITLE; \
|
||||
description = #DESCRIPTION; \
|
||||
requires_attribute = TRUE; \
|
||||
expiration_rule = ##EXPIRATION_RULE; \
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
/*###################################################
|
||||
Type definitions
|
||||
###################################################*/
|
||||
|
||||
// Type definitions using macros - This file contains all types getting made accessible in code and later in database during first subsystem run
|
||||
// See macros for detailed information on parameters and the underlaying types
|
||||
// Generally, after being released (as in, run against the database once), these should NOT be modified if possible, explicit warnings:
|
||||
|
||||
// +------------------------------------------------------------------------------------------------------------------+
|
||||
// | MODIFYING THE TYPE NAME OF EXISTING DEFINES WILL CREATE A NEW TYPE DEFINITION IN THE DATABASE |
|
||||
// +------------------------------------------------------------------------------------------------------------------+
|
||||
// | MODIFYING THE ATTRIBUTE FLAG OF EXISTING DEFINES CAN HAVE BREAKING CONSEQUENCES, TESTING REQUIRED BEFORE RELEASE |
|
||||
// +------------------------------------------------------------------------------------------------------------------+
|
||||
|
||||
// Below are the defines for generics, history and history with character validation
|
||||
|
||||
// ##### Persistent generics
|
||||
|
||||
CREATE_PERSISTENT_TYPE_GENERIC(horizon_overmap_position, "SCCV Horizon sector position", "Position of the SCCV Horizon on the overmap.", FALSE)
|
||||
|
||||
// ##### Persistent history
|
||||
|
||||
|
||||
|
||||
// ##### Persistent history with character validation
|
||||
|
||||
CREATE_PERSISTENT_TYPE_HISTORY_CHARACTER(mining_points, "Mining yield history", "History of mining points yield of individual miners.", /singleton/persistent_type_history_expiration_rule/age/week)
|
||||
Reference in New Issue
Block a user