Aurora.3

cyberpresidentvanellope/Aurora.3

Fork 0

mirror of https://github.com/Aurorastation/Aurora.3.git synced 2026-08-23 13:05:44 +01:00

Commit Graph

Author	SHA1	Message	Date
FabianK3 GitHub Copilot	802d3fe01f	Persistency subsystem update - Generics and history records (#22114 ) # Summary This PR is the next update to the persistency subsystem. The goal of this PR is to provide more framework like functions to allow more types of content to be made persistent. Currently only volatile game objects created during a round can be (in a [clean](https://www.youtube.com/watch?v=rZ3ETK7-ZM8) way) saved and made persistent. This update attempts to provide methods to make everything¹ persistent. This introduces persistent generics and history. ## Database The following things are going to be changed and added in the database (open in new tab for better visibility, PNG file includes the drawIO code): <img width="1692" height="1041" alt="aurora_persistency_db drawio" src="https://github.com/user-attachments/assets/ea53f419-f9aa-4592-af8f-3a8d5edf3177" /> Deviations on database implementation from diagram: - Removed unique constraint on history table - Prevented adding multiple records per round per attribute. ## Framework surface changes - MC/VV: Moved global object track register to subsystem var space. - MC/VV: Point of interest: Added history_cache and generic_cache to subsystem var space. - MC/VV: Updated subsystem stat entry message, now providing information on cache sizes of new types. - Added `singleton/persistent_type` defines (Type, clean-up rules, finalization hook) and macros allowing new definitions of said types. - Added cache structures that are also used for returns on public procs in generics and history persistent types. - Major new framework features: Persistent history (example: Mining yield records) and persistent generics (example: Persistent Horizon overmap position). See documentation for more information. DrawIO diagram for documentation, includes source in it (open in new tab): <img width="200" height="200" alt="Persistence-subsystem-flowchart drawio" src="https://github.com/user-attachments/assets/51f28331-f999-49a2-a7cc-58278f7ae416" /> ## Tasks (These lists are not comprehensive.) General - [x] Update DB - Write SQL scripts. - [x] Add subsystem modular files for generics and history. - [x] Add type definition logic, macros. - [x] Add type-DB init logic. - [x] Logging. - [x] A lot of testing. A lot. - [x] Changelog. - [x] Self-Review. - [x] Update documentation on the persistence subsystem. "Persistent history" - [x] Add init logic. - [x] Add finalize logic. - [x] Add framework surface procs. - [x] Get last record. - [x] Get last X records. - [x] Add record. - [x] Add character ID related validation. - [x] Add initial example mechanic. "Persistent generics" - [x] Add init logic. - [x] Add finalize logic. - [x] Add framework surface procs. - [x] Save. - [x] Load. - [x] Add initial example mechanic. ## Changes Too many changes to be listed here - Check changelog and actual changes. ## Warning There are certain use/test cases that cannot be tested locally due to missing preexisting data in the database. This should only affect new data structures (new persistent types), not existing data. ¹ _Large scale persistent mapping is excluded for this version._ --------- Co-authored-by: Copilot <copilot@github.com>	2026-07-19 20:56:17 +02:00
FabianK3andGitHub	36d267d83e	Persistent data framework (#21055 ) # Summary This PR has the goal of adding a base framework for tracking, saving and otherwise managing persistent data between rounds, written from scratch. Persistent data can be anything - Most expectedly noticeboards, dirt, papers, etc. _Technical details about the PR below are subject to change._ ## Estimated scope The first iteration of the persistence framework should include the functioning subsystem and the first implementation of a persistent type using said subsystem. - [x] Prepare database for persistent data - [x] Add new features to `/obj/` - [x] Logic to get and load persistent data into objects for the subsystem - [x] Internal logic for tracking and other - [x] Implement subsystem - [x] Basics, files, integration of subsystem - [x] Database queries for different operations - [x] Methods for registering and de-registering object tracking for `/obj` - [x] Round start logic - [x] Round end logic - [x] Implement first persistent type: Papers on notice boards - [ ] Code review - [x] Scope validation - [x] Testing - [x] Full changelog - Merge - [ ] Setup logging config for new subsystem - [x] Documentation for developers ([How to add new persistent types](https://github.com/Aurorastation/Aurora.3/wiki/Persistence)) ## Inner workings The new persistence subsystem has the following concept: At round start the subsystem reads existing data from the database and creates objects for them. During the round objects can be added to the tracking using a register function (or removed by a de-register function). At round end the subsystem goes over all it's actively tracked objects: Create new objects (that don't have a persistent ID from the database), update objects that changed during the round and expire objects that are no longer tracked. The objects in questions can register themselves, de-register themselves and decide themselves what content should be stored and handle how to set themselves up during the subsystems init phase. ## Information on the new table `id` - database generated ID for tracking persistent objects over multiple rounds, objects without an ID are considered new and created during the round. `author_ckey` - With the first implementation unused, but expected to be required in the future to allow any staff/moderation on persistent data. Nullable as persistent data also can be things like decals without an actual owner. `type` - Type of persistent data used for creating objects at round start. `created_at` - Statistical values. `expire_at` - Used for cleaning the database and setting limits for persistent data types. It's optional and allows permanent persistent data (command notice boards?). `content` - JSON formatted data containing the actual properties of an individual tracked object (e.g. what's written on a paper, what's its title). `x`, `y`, `z` - Coordinates taken and given from/to the object during save/load. Those can be null for future purposes. ## The goal on how to add new persistent data types `/obj` received two new methods which need to be overriden in the to-be added new persistent type: `persistence_get_content()` and `persistence_apply_content(content, x, y, z)`. The first method has the responsibility to provide all custom content of a new type to the subsystem in the form of an associated list (`["title" = "Hello, World!", "value" = 1337]. What does that mean? In the example of a piece of paper, the method needs to put the papers title and text content into a an associated list, that list will be saved by the persistence subsystem. The second method has the responsibility to apply the previously saved content when persistent data is initialized at the start of the round: The associated list needs to be read back and applied to the paper, additionally coordinates are provided for the object that have been taken during saving. Note that the coordinates might be null and could be ignored during setup. During the round, when a new object is created, like a paper, you need to call the subsystem and register a track: `SSpersistence.register_track(your_object_to_track, ckey (optional)`. The subsystem will be using the methods above when the round ends and saves the object - and loads it at the start of the round using the second method. The ckey needs to be given for all persistent that contains user generated content, e.g. papers. In case a persistent object gets removed, you need to call `SSpersistence.deregister_track(your_tracked_object)`. At the end of the round the object will be removed from the storage. `/obj` contains the variable (along some other technical vars) `persistance_initial_expiration_time_days`, which has a default value of 30 days, but can be safely overriden on a per-type basis. This value (in days) will be used for the persistent data entry expiration date when a new object of said type is stored in the database. ## PR description changelog - Added The goal on how to add new persistent data types. - Updates Estimated scope format. - Updates Inner workings. - Final iteration. (Grammer and formatting not tracked manually)	2025-08-16 22:00:12 +00:00

Author

SHA1

Message

Date

802d3fe01f

Persistency subsystem update - Generics and history records (#22114 )

# Summary

This PR is the next update to the persistency subsystem. The goal of
this PR is to provide more framework like functions to allow more types
of content to be made persistent.

Currently only volatile game objects created during a round can be (*in
a [clean](https://www.youtube.com/watch?v=rZ3ETK7-ZM8) way*) saved and
made persistent. This update attempts to provide methods to make
*everything*¹ persistent.

This introduces persistent generics and history.

## Database

The following things are going to be changed and added in the database  
(open in new tab for better visibility, PNG file includes the drawIO
code):

<img width="1692" height="1041" alt="aurora_persistency_db drawio"
src="https://github.com/user-attachments/assets/ea53f419-f9aa-4592-af8f-3a8d5edf3177"
/>

**Deviations on database implementation from diagram:**  
- Removed unique constraint on history table - Prevented adding multiple
records per round per attribute.

## Framework surface changes

- MC/VV: Moved global object track register to subsystem var space.
- MC/VV: Point of interest: Added history_cache and generic_cache to
subsystem var space.
- MC/VV: Updated subsystem stat entry message, now providing information
on cache sizes of new types.
- Added `singleton/persistent_type` defines (Type, clean-up rules,
finalization hook) and macros allowing new definitions of said types.
- Added cache structures that are also used for returns on public procs
in generics and history persistent types.
- Major new framework features: Persistent history (example: Mining
yield records) and persistent generics (example: Persistent Horizon
overmap position). See documentation for more information.

DrawIO diagram for documentation, includes source in it (open in new
tab):

<img width="200" height="200" alt="Persistence-subsystem-flowchart
drawio"
src="https://github.com/user-attachments/assets/51f28331-f999-49a2-a7cc-58278f7ae416"
/>

## Tasks

(These lists are not comprehensive.)

**General**
- [x] Update DB - Write SQL scripts.
- [x] Add subsystem modular files for generics and history.
- [x] Add type definition logic, macros.
- [x] Add type-DB init logic.
- [x] Logging.
- [x] A lot of testing. *A lot.*
- [x] Changelog.
- [x] Self-Review.
- [x] Update documentation on the persistence subsystem.

**"Persistent history"**
- [x] Add init logic.
- [x] Add finalize logic.
- [x] Add framework surface procs.
  - [x] Get last record.
  - [x] Get last X records.
  - [x] Add record.
  - [x] Add character ID related validation.
- [x] Add initial example mechanic.

**"Persistent generics"**
- [x] Add init logic.
- [x] Add finalize logic.
- [x] Add framework surface procs.
  - [x] Save.
  - [x] Load.
- [x] Add initial example mechanic.

## Changes

Too many changes to be listed here - Check changelog and actual changes.

## Warning

There are certain use/test cases that *cannot* be tested locally due to
missing preexisting data in the database. This should only affect new
data structures (new persistent types), not existing data.

¹ _Large scale persistent mapping is excluded for this version._

---------

Co-authored-by: Copilot <copilot@github.com>

2026-07-19 20:56:17 +02:00

FabianK3andGitHub

36d267d83e

Persistent data framework (#21055 )

# Summary

This PR has the goal of adding a base framework for tracking, saving and
otherwise managing persistent data between rounds, written from scratch.

Persistent data can be anything - Most expectedly noticeboards, dirt,
papers, etc.

_Technical details about the PR below are subject to change._

## Estimated scope

The first iteration of the persistence framework should include the
functioning subsystem and the first implementation of a persistent type
using said subsystem.

- [x] Prepare database for persistent data
- [x] Add new features to `/obj/`
- [x] Logic to get and load persistent data into objects for the
subsystem
  - [x] Internal logic for tracking and other
- [x] Implement subsystem
  - [x] Basics, files, integration of subsystem
  - [x] Database queries for different operations
- [x] Methods for registering and de-registering object tracking for
`/obj`
  - [x] Round start logic
  - [x] Round end logic
- [x] Implement first persistent type: Papers on notice boards
- [ ] Code review
  - [x] Scope validation
  - [x] Testing
  - [x] Full changelog
- Merge
- [ ] Setup logging config for new subsystem
- [x] Documentation for developers ([How to add new persistent
types](https://github.com/Aurorastation/Aurora.3/wiki/Persistence))

## Inner workings

The new persistence subsystem has the following concept:
At round start the subsystem reads existing data from the database and
creates objects for them.
During the round objects can be added to the tracking using a register
function (or removed by a de-register function).
At round end the subsystem goes over all it's actively tracked objects:
Create new objects (that don't have a persistent ID from the database),
update objects that changed during the round and expire objects that are
no longer tracked.

The objects in questions can register themselves, de-register themselves
and decide themselves what content should be stored and handle how to
set themselves up during the subsystems init phase.

## Information on the new table

`id` - database generated ID for tracking persistent objects over
multiple rounds, objects without an ID are considered new and created
during the round.
`author_ckey` - With the first implementation unused, but expected to be
required in the future to allow any staff/moderation on persistent data.
Nullable as persistent data also can be things like decals without an
actual owner.
`type` - Type of persistent data used for creating objects at round
start.
`created_at` - Statistical values.
`expire_at` - Used for cleaning the database and setting limits for
persistent data types. It's optional and allows permanent persistent
data (command notice boards?).
`content` - JSON formatted data containing the actual properties of an
individual tracked object (e.g. what's written on a paper, what's its
title).
`x`, `y`, `z` - Coordinates taken and given from/to the object during
save/load. Those can be null for future purposes.

## The goal on how to add new persistent data types

`/obj` received two new methods which need to be overriden in the to-be
added new persistent type:
`persistence_get_content()` and `persistence_apply_content(content, x,
y, z)`.

The first method has the responsibility to provide all custom content of
a new type to the subsystem in the form of an associated list (`["title"
= "Hello, World!", "value" = 1337].

What does that mean?  
In the example of a piece of paper, the method needs to put the papers
title and text content into a an associated list, that list will be
saved by the persistence subsystem.

The second method has the responsibility to apply the previously saved
content when persistent data is initialized at the start of the round:
The associated list needs to be read back and applied to the paper,
additionally coordinates are provided for the object that have been
taken during saving. Note that the coordinates might be null and could
be ignored during setup.

During the round, when a new object is created, like a paper, you need
to call the subsystem and register a track:
`SSpersistence.register_track(your_object_to_track, ckey (optional)`.
The subsystem will be using the methods above when the round ends and
saves the object - and loads it at the start of the round using the
second method. The ckey needs to be given for all persistent that
contains user generated content, e.g. papers.

In case a persistent object gets removed, you need to call
`SSpersistence.deregister_track(your_tracked_object)`. At the end of the
round the object will be removed from the storage.

`/obj` contains the variable (along some other technical vars)
`persistance_initial_expiration_time_days`, which has a default value of
30 days, but can be safely overriden on a per-type basis. This value (in
days) will be used for the persistent data entry expiration date when a
new object of said type is stored in the database.

## PR description changelog

- Added *The goal on how to add new persistent data types*.
- Updates *Estimated scope* format.
- Updates *Inner workings*.
- Final iteration.

*(Grammer and formatting not tracked manually)*

2025-08-16 22:00:12 +00:00