Commit Graph
102 Commits
Author SHA1 Message Date
+37 21b4095dfd [MDB IGNORE] [IDB IGNORE] Upstream Sync - 04/17/2026 (#5453)
Upstream 04/17/2026

fixes https://github.com/Bubberstation/Bubberstation/issues/5549

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: tgstation-ci[bot] <179393467+tgstation-ci[bot]@users.noreply.github.com>
Co-authored-by: ArcaneMusic <41715314+ArcaneMusic@users.noreply.github.com>
Co-authored-by: MrMelbert <51863163+MrMelbert@users.noreply.github.com>
Co-authored-by: Rhials <28870487+Rhials@users.noreply.github.com>
Co-authored-by: rageguy505 <54517726+rageguy505@users.noreply.github.com>
Co-authored-by: LemonInTheDark <58055496+LemonInTheDark@users.noreply.github.com>
Co-authored-by: John Willard <53777086+JohnFulpWillard@users.noreply.github.com>
Co-authored-by: Aliceee2ch <160794176+Aliceee2ch@users.noreply.github.com>
Co-authored-by: Time-Green <7501474+Time-Green@users.noreply.github.com>
Co-authored-by: Tsar-Salat <62388554+Tsar-Salat@users.noreply.github.com>
Co-authored-by: SmArtKar <44720187+SmArtKar@users.noreply.github.com>
Co-authored-by: Maxipat <108554989+Maxipat112@users.noreply.github.com>
Co-authored-by: SyncIt21 <110812394+SyncIt21@users.noreply.github.com>
Co-authored-by: deltanedas <39013340+deltanedas@users.noreply.github.com>
Co-authored-by: SimplyLogan <47579821+loganuk@users.noreply.github.com>
Co-authored-by: loganuk <fakeemail123@aol.com>
Co-authored-by: Leland Kemble <70413276+lelandkemble@users.noreply.github.com>
Co-authored-by: FalloutFalcon <86381784+FalloutFalcon@users.noreply.github.com>
Co-authored-by: Roxy <75404941+TealSeer@users.noreply.github.com>
Co-authored-by: Lucy <lucy@absolucy.moe>
Co-authored-by: siliconOpossum <138069572+siliconOpossum@users.noreply.github.com>
Co-authored-by: Isratosh <Isratosh@hotmail.com>
Co-authored-by: TheRyeGuyWhoWillNowDie <70169560+TheRyeGuyWhoWillNowDie@users.noreply.github.com>
Co-authored-by: Neocloudy <88008002+Neocloudy@users.noreply.github.com>
Co-authored-by: Alexander V. <volas@ya.ru>
Co-authored-by: ElGitificador <168473461+ElGitificador@users.noreply.github.com>
Co-authored-by: Twaticus <46540570+Twaticus@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bloop <13398309+vinylspiders@users.noreply.github.com>
Co-authored-by: Cameron Lennox <killer65311@gmail.com>
Co-authored-by: Tim <timothymtorres@gmail.com>
Co-authored-by: Iamgoofball <iamgoofball@gmail.com>
Co-authored-by: Layzu666 <121319428+Layzu666@users.noreply.github.com>
Co-authored-by: Arturlang <24881678+Arturlang@users.noreply.github.com>
Co-authored-by: _0Steven <42909981+00-Steven@users.noreply.github.com>
Co-authored-by: mrmanlikesbt <99309552+mrmanlikesbt@users.noreply.github.com>
Co-authored-by: Ben10Omintrix <138636438+Ben10Omintrix@users.noreply.github.com>
Co-authored-by: John F. Kennedy <54908920+MacaroniCritter@users.noreply.github.com>
Co-authored-by: Cursor <102828457+theselfish@users.noreply.github.com>
Co-authored-by: Josh <josh.adam.powell@gmail.com>
Co-authored-by: Josh Powell <josh.powell@softwire.com>
Co-authored-by: Yobrocharlie <Charliemiller5617@gmail.com>
Co-authored-by: Hardly3D <66234359+Hardly3D@users.noreply.github.com>
Co-authored-by: shayoki <96078776+shayoki@users.noreply.github.com>
Co-authored-by: LT3 <83487515+lessthnthree@users.noreply.github.com>
2026-05-16 00:56:00 +02:00
nevimer 6e9f2ccfc0 Merge remote-tracking branch 'tgstation/master' into upstream-12-15
# Conflicts:
#	.github/workflows/compile_all_maps.yml
#	.github/workflows/run_integration_tests.yml
#	_maps/map_files/CatwalkStation/CatwalkStation_2023.dmm
#	code/_onclick/hud/credits.dm
#	code/controllers/subsystem/networks/id_access.dm
#	code/datums/diseases/advance/advance.dm
#	code/datums/diseases/advance/symptoms/heal.dm
#	code/game/machinery/doors/door.dm
#	code/game/objects/structures/crates_lockers/closets/secure/medical.dm
#	code/game/objects/structures/crates_lockers/closets/secure/security.dm
#	code/modules/antagonists/malf_ai/malf_ai_modules.dm
#	code/modules/jobs/job_types/_job.dm
#	code/modules/loadout/categories/accessories.dm
#	code/modules/loadout/loadout_helpers.dm
#	code/modules/loadout/loadout_items.dm
#	code/modules/loadout/loadout_preference.dm
#	code/modules/mob/living/silicon/robot/robot_defense.dm
#	code/modules/mod/mod_theme.dm
#	code/modules/projectiles/ammunition/energy/laser.dm
#	code/modules/reagents/reagent_containers/cups/drinks.dm
#	code/modules/shuttle/mobile_port/variants/supply.dm
#	code/modules/surgery/organs/internal/eyes/_eyes.dm
#	code/modules/unit_tests/screenshots/screenshot_antag_icons_heretic.png
#	icons/hud/screen_full.dmi
2025-12-15 18:12:29 -05:00
san7890andGitHub d25647c250 log_world() the Round ID (#94429)
## About The Pull Request

Better correspondence in higher-level logging files (e.g. TGS's
DreamDaemonLog Diagnostic), also helpful in instances where Round ID is
not known (like if someone gives you a random log and the round ID has
been forgotten), pretty good to have it logged if we know it in all the
important stuff like dd.log.
## Why It's Good For The Game

Higher standard of information tracking without having to do infinite
amounts of UTC Timezone maths and what-not as it's becoming more common
for me to see stuff and then have to pull all sorts of other data in
order to make sure what I'm looking at is accurate to what I want (and
still having that degree of uncertainty is unideal).
## Changelog
🆑
server: Round ID is now output to `log_world()`, so it'll show up stuff
like world.log, dd.log, etc.
/🆑
2025-12-13 20:32:43 -05:00
san7890andGitHub a4a1b7f3b4 Cleaner End-of-Round DB Shutdown Message (#94229)
## About The Pull Request

The end of round DB shutdown message has been posting everywhere for a
while now, but it was just put down as a quick debugging thing for
something on production. If we're going to keep it around, it should at
least look nicer instead of having weird maligned spacing rules and what
not.
2025-11-30 19:28:24 -05:00
LemonInTheDarkandnevimer eb7d505bef Makes the MC stat panel tab usable, and also nicer (#92405)
(cherry picked from commit 0909348cea)
2025-08-08 15:33:44 -04:00
LemonInTheDarkandGitHub 0909348cea Makes the MC stat panel tab usable, and also nicer (#92405) 2025-08-08 19:32:46 +02:00
Watermelon914andShadow-Quill 79b00baad2 Refactors subsystems to use dependency-ordering to determine init order. Subsystems can now declare their own dependencies. (#90268)
## About The Pull Request
As the title says.
`init_order` is no more, subsystems ordering now depends on their
declared dependencies.
Subsystems can now declare which other subsystems need to init before
them using a list and the subsystem's typepath
I.e.
```dm
dependencies = list(
    /datum/controller/subsystem/atoms,
    /datum/controller/subsystem/mapping
)
```
The reverse can also be done, if a subsystem must initialize after your
own:
```dm
dependents = list(
    /datum/controller/subsystem/atoms
)
```
Cyclical dependencies are not allowed and will throw an error on
initialization if one is found.
There's also a debug tool to visualize the dependency graph, although
it's a bit basic:

![image](https://github.com/user-attachments/assets/80c854d9-c2a5-4f2f-92db-a031e9a8e257)

Subsystem load ordering can still be controlled using `init_stage`, some
subsystems use this in cases where they must initialize first or last
regardless of dependencies. An error will be thrown if a subsystem has
an `init_stage` before one of their dependencies.

## Why It's Good For The Game
Makes dealing with subsystem dependencies easier, and reduces the chance
of making a dependency error when needing to shift around subsystem
inits.

## Changelog
🆑
refactor: Refactored subsystem initialization
/🆑
2025-04-29 17:11:39 -06:00
Watermelon914andGitHub 6c017cf1e1 Refactors subsystems to use dependency-ordering to determine init order. Subsystems can now declare their own dependencies. (#90268)
## About The Pull Request
As the title says.
`init_order` is no more, subsystems ordering now depends on their
declared dependencies.
Subsystems can now declare which other subsystems need to init before
them using a list and the subsystem's typepath
I.e.
```dm
dependencies = list(
    /datum/controller/subsystem/atoms,
    /datum/controller/subsystem/mapping
)
```
The reverse can also be done, if a subsystem must initialize after your
own:
```dm
dependents = list(
    /datum/controller/subsystem/atoms
)
```
Cyclical dependencies are not allowed and will throw an error on
initialization if one is found.
There's also a debug tool to visualize the dependency graph, although
it's a bit basic:

![image](https://github.com/user-attachments/assets/80c854d9-c2a5-4f2f-92db-a031e9a8e257)

Subsystem load ordering can still be controlled using `init_stage`, some
subsystems use this in cases where they must initialize first or last
regardless of dependencies. An error will be thrown if a subsystem has
an `init_stage` before one of their dependencies.

## Why It's Good For The Game
Makes dealing with subsystem dependencies easier, and reduces the chance
of making a dependency error when needing to shift around subsystem
inits.

## Changelog
🆑
refactor: Refactored subsystem initialization
/🆑
2025-04-03 17:04:30 -04:00
Majkl-J b6b8306fda Merge branch 'master' of https://github.com/tgstation/tgstation into upstream-25-02a 2025-02-20 00:00:19 -08:00
John WillardandGitHub 2e4d70afe5 Updates href uses for 516 (#88699)
## About The Pull Request

Was just scrolling through the Paradise github since they seem to have
more work done for 516 to see if there's anything I can port over, found
this and thought why not.

Ports parts of https://github.com/ParadiseSS13/Paradise/pull/25105
Specifically, updaing all hrefs to use the internal ``byond://``, and
adding it to grep.

## Why It's Good For The Game

More work towards 516.

## Changelog

Nothing player-facing.
2024-12-24 11:42:20 -08:00
LucyandGitHub 6046120e5f Updating player last seen info and connection log no longer blocks (#87906)
## About The Pull Request

Was investigating some stuff downstream related to undeleted SQL
queries, and these two queries were related.

thought it'd be best to make them not block
`/client/proc/set_client_age_from_db`, as the results are never checked
anyways, not even for errors.

I've added a new proc to SSdbcore, `FireAndForget` - takes the same
parameters as NewQuery, but instead of returning the query, it just
asynchronously executes and then deletes it.

## Why It's Good For The Game

less undeleted sql queries, less blocking. yay?

## Changelog

does this count as user-facing? unsure how i'd changelog this tbh
2024-11-15 23:49:13 +01:00
GhomandGitHub e6253c7812 Adds a score for all species of fish that you've caught. (#86049)
## About The Pull Request
I'm adding a score that tracks which types of fish you've caught across
multiple rounds. To do so, I had to add a new score subtype that manages
the score value not being a number. Thankfully the achievement code is
fairly flexible so not a whole lot had to be done, although I've to add
a new column to the achievements table in the DB, because the 'value' is
for integers, while we need one for text strings ~~(the contents of the
list are converted to text with a delimiter before being saved cuz I'm
not sure if and how our DM slash SQL integration handles using lists
directly and I don't want to waste time finding it out)~~.

EDIT: It's mostly done beside the reviews that are going to point out
things that need to be changed. The UI changes are done. It's time for
reviews.

Here are screenshots of the UI with all fish still uncatched beside one
(I've since then the typo on its name and removed an extra zero from the
index number, as well as a nit with the spacing between cells):

![immagine](https://github.com/user-attachments/assets/a1dcfeb6-6d26-461e-aaa1-97c619f5cbfa)

![immagine](https://github.com/user-attachments/assets/768f6621-c992-4932-9bca-979dd1e43d6f)


## Why It's Good For The Game
We have about dozens over dozens of different fish in the game now, many
of which are just fluff anyway. It's getting to the point it's perhaps
doable to add a score or something to be a braggard about.

## Changelog
🆑
add: Added a new score that keeps track of all different fish that
you've caught between shifts.
server: Added a new schema table to store the aforementioned entries and
the ckeys associated to them, with an additional timestamp column.
/🆑
2024-11-04 13:48:25 -08:00
9def6a3113 [MIRROR] Fix shutdown hanging if the db went away. gives shutdowns a time out. improves db shutdown logging (#26754)
Fix shutdown hanging if the db went away. gives shutdowns a time out. improves db shutdown logging (#81813)

terry right now:
```
[20:11:31] Runtime in code/controllers/subsystem/dbcore.dm,353: Attempting to create a new db query during the world shutdown
...
[21:25:29] Runtime in code/controllers/subsystem/dbcore.dm,353: Attempting to create a new db query during the world shutdown
```

Co-authored-by: Kyle Spier-Swenson <kyleshome@gmail.com>
2024-03-15 13:31:23 -04:00
7ae5117833 [MIRROR] refactor db connection timeouts (#26780)
* refactor db connection timeouts (#81816)

Give it exponential back off and smerter logic.

* refactor db connection timeouts

---------

Co-authored-by: Kyle Spier-Swenson <kyleshome@gmail.com>
2024-03-07 08:55:25 -06:00
Kyle Spier-SwensonandGitHub 3086ed4f8c refactor db connection timeouts (#81816)
Give it exponential back off and smerter logic.
2024-03-07 12:25:01 +13:00
Kyle Spier-SwensonandGitHub a88013783a Fix shutdown hanging if the db went away. gives shutdowns a time out. improves db shutdown logging (#81813)
terry right now:
```
[20:11:31] Runtime in code/controllers/subsystem/dbcore.dm,353: Attempting to create a new db query during the world shutdown
...
[21:25:29] Runtime in code/controllers/subsystem/dbcore.dm,353: Attempting to create a new db query during the world shutdown
```
2024-03-04 19:00:42 -07:00
339dc45593 [MIRROR] Remove delayed in MassInsert [MDB IGNORE] (#25554)
* Remove delayed in MassInsert (#80196)

Necessary to update MariaDB to a newer version. Was necessary until we
had async queries, apparently.

* Remove delayed in MassInsert

* Delayed arg is no more

---------

Co-authored-by: Mothblocks <35135081+Mothblocks@users.noreply.github.com>
Co-authored-by: Giz <13398309+vinylspiders@users.noreply.github.com>
2023-12-11 05:27:28 -05:00
MothblocksandGitHub 588da7a4d4 Remove delayed in MassInsert (#80196)
Necessary to update MariaDB to a newer version. Was necessary until we
had async queries, apparently.
2023-12-11 08:28:31 +00:00
f19eaf891c [MIRROR] Try waiting for db queries to finish during db shutdown [MDB IGNORE] (#22892)
* Wait for db queries to finish during db shutdown (#77311)

* Try waiting for db queries to finish during db shutdown

---------

Co-authored-by: oranges <email@oranges.net.nz>
2023-08-05 18:38:37 -04:00
orangesandGitHub 3611eec535 Wait for db queries to finish during db shutdown (#77311) 2023-08-05 16:14:00 -06:00
02bb6a0588 [MIRROR] Stop stack tracing if ezdb daemon is already running [MDB IGNORE] (#22284)
* Stop stack tracing if ezdb daemon is already running (#76555)

## About The Pull Request

This is an annoyance for me because I often kill the game but not the
EZDB daemon (leaving the cmd window open), and it just triggers this
stack trace while I have a debugger running. It's still able to connect
to the daemon in-game and functions perfectly fine, just hate dealing
with a completely needless stack_trace() every single time.

![image](https://github.com/tgstation/tgstation/assets/34697715/f8ae3cec-1cf6-4e70-b9bc-c7088121b85c)

![image](https://github.com/tgstation/tgstation/assets/34697715/a13774b8-addf-43fd-82f0-3d0e644d2dc1)
## Why It's Good For The Game

LET ME WORK!!! PLEASE!!!
## Changelog
Nothing that players should care about.

* Stop stack tracing if ezdb daemon is already running

---------

Co-authored-by: san7890 <the@san7890.com>
2023-07-12 05:50:27 +00:00
san7890andGitHub ed59cdd458 Stop stack tracing if ezdb daemon is already running (#76555)
## About The Pull Request

This is an annoyance for me because I often kill the game but not the
EZDB daemon (leaving the cmd window open), and it just triggers this
stack trace while I have a debugger running. It's still able to connect
to the daemon in-game and functions perfectly fine, just hate dealing
with a completely needless stack_trace() every single time.


![image](https://github.com/tgstation/tgstation/assets/34697715/f8ae3cec-1cf6-4e70-b9bc-c7088121b85c)


![image](https://github.com/tgstation/tgstation/assets/34697715/a13774b8-addf-43fd-82f0-3d0e644d2dc1)
## Why It's Good For The Game

LET ME WORK!!! PLEASE!!!
## Changelog
Nothing that players should care about.
2023-07-07 06:30:47 -04:00
d91555b79e ezdb - A one click script to quickly setting up a development database (#75053) (#21458)
* ezdb - A one click script to quickly setting up a development database (#75053)

https://user-images.githubusercontent.com/35135081/235344815-8e825ba9-52cf-44e8-b8e2-a2aeb5d47276.mp4

- Downloads a portable MariaDB (doesn't pollute your main system)
- Sets up a database with a random password on port 1338 (configurable)
- Installs the initial schema
- Every time after, will run updates

Major versions right now explicitly escape hatch, because those
historically come with something like a Python script, and I do not want
it to pretend to work.

---------

Co-authored-by: san7890 <the@san7890.com>

* touchups

* oh well

---------

Co-authored-by: Mothblocks <35135081+Mothblocks@users.noreply.github.com>
Co-authored-by: san7890 <the@san7890.com>
Co-authored-by: Useroth <37159550+Useroth@users.noreply.github.com>
2023-06-03 23:15:41 +02:00
649c250702 [MIRROR] JSON Logging Take Two [MDB IGNORE] (#21329)
* JSON Logging Take Two

* Fixes the conflicts and makes our logs all comply with the new logging framework, hopefully.

---------

Co-authored-by: Zephyr <12817816+ZephyrTFA@users.noreply.github.com>
Co-authored-by: GoldenAlpharex <jerego1234@hotmail.com>
2023-05-28 18:10:07 -04:00
fbec9c14e9 JSON Logging Take Two (#73604)
## About The Pull Request

Converts all logging, excluding perf and investigate, to json.
I focused on making the system as easy to use and as easy to add new
categories as possible.

Due to issues related to logging to world at global creation logger is
now a byond real, which is created directly before Master

Log categories support versioning, secret flagging, and sub-category
filtering. Although all of this is entirely optional for coders.
If you ever want to add a new category and use it, all you need to do is
make the barebones category datum and the define.
I've kept existing procs such as log_game, and simply turned them into a
wrapper for Logger.Log(xxx, ...)
## Why It's Good For The Game

Makes processing and filtering logs much easier in the future, while
only minimally downgrading log crawling experience.
I am also working on a log viewer frontend for admin usage however that
will take a little bit longer to finish up.
Also makes special logging and data tracking much easier thanks to a
data list processing implementation and handling
## Changelog
🆑
server: All logs are now formatted in json, excluding perf and
investigations
/🆑

---------

Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: tattle <66640614+dragomagol@users.noreply.github.com>
Co-authored-by: Kyle Spier-Swenson <kyleshome@gmail.com>
Co-authored-by: GoldenAlpharex <58045821+GoldenAlpharex@users.noreply.github.com>
2023-05-22 14:51:00 +12:00
980f3adc70 ezdb - A one click script to quickly setting up a development database (#75053)
https://user-images.githubusercontent.com/35135081/235344815-8e825ba9-52cf-44e8-b8e2-a2aeb5d47276.mp4

- Downloads a portable MariaDB (doesn't pollute your main system)
- Sets up a database with a random password on port 1338 (configurable)
- Installs the initial schema
- Every time after, will run updates

Major versions right now explicitly escape hatch, because those
historically come with something like a Python script, and I do not want
it to pretend to work.

---------

Co-authored-by: san7890 <the@san7890.com>
2023-05-15 12:49:29 -06:00
e3a638f179 [MIRROR] RUNLEVEL_INIT does not exist [MDB IGNORE] (#20857)
* RUNLEVEL_INIT does not exist (#75023)

This define shouldn't have been added (by me when I created runlevels)
people get it confused with RUNLEVEL_LOBBY and misuse it, like in this
verb subsystem I have no knowledge of.

- Removes RUNLEVEL_INIT.
- Fixed SSverb_manager not running during the lobby phase.
- Fixed Master.SetRunLevel having the potential to accept a scuffed
runlevel.
- Other standardizing cleanups to runlevels usage.

* RUNLEVEL_INIT does not exist

---------

Co-authored-by: Jordan Dominion <Cyberboss@users.noreply.github.com>
2023-05-02 15:49:14 -04:00
Jordan DominionandGitHub a20baac88e RUNLEVEL_INIT does not exist (#75023)
This define shouldn't have been added (by me when I created runlevels)
people get it confused with RUNLEVEL_LOBBY and misuse it, like in this
verb subsystem I have no knowledge of.

- Removes RUNLEVEL_INIT.
- Fixed SSverb_manager not running during the lobby phase.
- Fixed Master.SetRunLevel having the potential to accept a scuffed
runlevel.
- Other standardizing cleanups to runlevels usage.
2023-04-30 14:54:40 -07:00
abf148d62b [MIRROR] World Initialization Refactor [MDB IGNORE] (#20755)
* World Initialization Refactor

* Update .github/CODEOWNERS

* Update code/__HELPERS/global_lists.dm

* Add logging for manually changing your targeted zone (#72814)

See title.
Surgery hud is exempt from this.

Requested by @Mothblocks

Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: Mothblocks <35135081+Mothblocks@users.noreply.github.com>
Co-authored-by: dragomagol <66640614+dragomagol@users.noreply.github.com>

* eee

* Update tgstation.dme

---------

Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: Jordan Dominion <Cyberboss@users.noreply.github.com>
Co-authored-by: Tom <8881105+tf-4@users.noreply.github.com>
Co-authored-by: Zephyr <12817816+ZephyrTFA@users.noreply.github.com>
Co-authored-by: Mothblocks <35135081+Mothblocks@users.noreply.github.com>
Co-authored-by: dragomagol <66640614+dragomagol@users.noreply.github.com>
Co-authored-by: Gandalf <9026500+Gandalf2k15@users.noreply.github.com>
2023-04-26 22:46:55 +01:00
3ceee2aab4 World Initialization Refactor (#74808)
- Removes unnecessary real global vars.
- Adds comments pointing to the init order defined in
/code/game/world.dm.
- Prevent people using `GLOBAL_REAL_VAR` and `GLOBAL_REAL` to circumvent
init order.
- Properly type `PROFILE_STORE` real global.
- Refactored `make_datum_references_lists()` and moved the call to it
into `GLOB` init with duct tape.
- Renamed `GLOB.admin_log` to `GLOB.admin_activities` as it wasn't
actually a log file.
- Whitelist loading happens in config.
- Renamed `SSdbcore`'s `SetRoundID()` to `InitializeRound()`. Now
handles calling `CheckSchemaVersion()`.
- Created macro for setting up log `GLOB`s.
- Removed log line for `GLOB` count.
- Moved call to `make_datum_reference_lists()` to
`/datum/controller/global_vars/Initialize()`. I slimmed it down where
possible too.
- Updated comments about world init order.
- Move `load_admins()` call to after log setup.
- Removes unused function `gib_stack_trace()`.
- Removes a bunch of unused log `GLOB`s.
- Unlocks the secrets of the universe by finally making the first
executed line of code deterministic.

No functional changes. Closes #74792
Testmerge thoroughly.

---------

Co-authored-by: Mothblocks <35135081+Mothblocks@users.noreply.github.com>
2023-04-24 18:38:38 -06:00
244558afdc [MIRROR] Refactors and defuckulates dbcore. Adds support for min_threads rustg setting, Reduce query delay, Make unit tests faster [MDB IGNORE] (#20668)
* Refactors and defuckulates dbcore. Adds support for min_threads rustg setting, Reduce query delay, Make unit tests faster (#74852)

dbcore was very fuckulated.

It had 3 lists of queries, but they all had their own current_run style
list to support mc_tick_check (as it was already being done before with
the undeleted query check, so i can understand why they ~~cargo culted~~
mirrored the behavior) This was silly and confusing and unneeded given
two of those loops can only process at most 25 items at a time on
default config, plus these were cheap operations (ask rustg to start
thread, ask rustg to check on thread).

Because of the confusingness of the 6 lists for 3 query states, The code
to run pending/queued queries immediately during world shutdown was
instead looking at the current_run list for active queries, **meaning
those queries got ran twice.**

The queued query system only checked the current active query count in
fire(), meaning even when there was nothing going on in this subsystem
new queries had to wait for the next fire() to run (10 ticks, so 500ms
on default config)

Those have all been fixed.

the config `BSQL_THREAD_LIMIT` has been renamed to
`POOLING_MAX_SQL_CONNECTIONS` and its default was lowered to match
MAX_CONCURRENT_QUERIES .

added a new config `POOLING_MIN_SQL_CONNECTIONS`, allowing you to
pre-allocate a reserve of sql threads.

The queue processing part of SSdbcore's fire() has been made to not obey
mc_tick_check for clarity and to make the following change easier to do:

If there is less than `MAX_CONCURRENT_QUERIES` in the active queue, new
queries activate immediately.

(its ok that there are two configs that kinda do the same thing,
POOLING_MAX_SQL_CONNECTIONS maps to max-threads in the mysql crate, and
it seems to only be a suggestion, meanwhile MAX_CONCURRENT_QUERIES can't
do anything during init, which is when the highest amount of concurrent
queries tend to happen.)

🆑
config: database configs have been updated for better control over the
connection pool
server: BSQL_THREAD_LIMIT has been renamed to
POOLING_MAX_SQL_CONNECTIONS, old configs will whine but still work.
fix: fixed rare race condition that could lead to a sql query being ran
twice during world shutdown.
/🆑

I have not tested this pr.

* Refactors and defuckulates dbcore. Adds support for min_threads rustg setting, Reduce query delay, Make unit tests faster

---------

Co-authored-by: Kyle Spier-Swenson <kyleshome@gmail.com>
2023-04-21 02:03:25 +01:00
Kyle Spier-SwensonandGitHub 200b739c0a Refactors and defuckulates dbcore. Adds support for min_threads rustg setting, Reduce query delay, Make unit tests faster (#74852)
dbcore was very fuckulated.

It had 3 lists of queries, but they all had their own current_run style
list to support mc_tick_check (as it was already being done before with
the undeleted query check, so i can understand why they ~~cargo culted~~
mirrored the behavior) This was silly and confusing and unneeded given
two of those loops can only process at most 25 items at a time on
default config, plus these were cheap operations (ask rustg to start
thread, ask rustg to check on thread).

Because of the confusingness of the 6 lists for 3 query states, The code
to run pending/queued queries immediately during world shutdown was
instead looking at the current_run list for active queries, **meaning
those queries got ran twice.**

The queued query system only checked the current active query count in
fire(), meaning even when there was nothing going on in this subsystem
new queries had to wait for the next fire() to run (10 ticks, so 500ms
on default config)

Those have all been fixed.

the config `BSQL_THREAD_LIMIT` has been renamed to
`POOLING_MAX_SQL_CONNECTIONS` and its default was lowered to match
MAX_CONCURRENT_QUERIES .

added a new config `POOLING_MIN_SQL_CONNECTIONS`, allowing you to
pre-allocate a reserve of sql threads.

The queue processing part of SSdbcore's fire() has been made to not obey
mc_tick_check for clarity and to make the following change easier to do:

If there is less than `MAX_CONCURRENT_QUERIES` in the active queue, new
queries activate immediately.

(its ok that there are two configs that kinda do the same thing,
POOLING_MAX_SQL_CONNECTIONS maps to max-threads in the mysql crate, and
it seems to only be a suggestion, meanwhile MAX_CONCURRENT_QUERIES can't
do anything during init, which is when the highest amount of concurrent
queries tend to happen.)

🆑
config: database configs have been updated for better control over the
connection pool
server: BSQL_THREAD_LIMIT has been renamed to
POOLING_MAX_SQL_CONNECTIONS, old configs will whine but still work.
fix: fixed rare race condition that could lead to a sql query being ran
twice during world shutdown.
/🆑

I have not tested this pr.
2023-04-20 17:09:32 -06:00
9a594755f3 [MIRROR] Renames delta time to be a more obvious name [MDB IGNORE] (#20507)
* Renames delta time to be a more obvious name

* updates to our code

---------

Co-authored-by: oranges <email@oranges.net.nz>
Co-authored-by: Gandalf <9026500+Gandalf2k15@users.noreply.github.com>
2023-04-12 20:45:43 +01:00
orangesandGitHub 4c48966ff8 Renames delta time to be a more obvious name (#74654)
This tracks the seconds per tick of a subsystem, however note that it is
not completely accurate, as subsystems can be delayed, however it's
useful to have this number as a multiplier or ratio, so that if in
future someone changes the subsystem wait time code correctly adjusts
how fast it applies effects

regexes used

git grep --files-with-matches --name-only 'DT_PROB' | xargs -l sed -i
's/DT_PROB/SPT_PROB/g'
git grep --files-with-matches --name-only 'delta_time' | xargs -l sed -i
's/delta_time/seconds_per_tick/g'
2023-04-11 21:31:07 -07:00
f7c26bbf25 515 Compat (#17465)
* ONLY SKYRAT CHANGES

* ACTUALLY SKYRAT CHANGES

* yolo, revert later

* Update alternate_byond_versions.txt

Co-authored-by: AnturK <AnturK@users.noreply.github.com>
2022-11-15 06:59:06 +00:00
4d6a8bc537 515 Compatibility (#71161)
Makes the code compatible with 515.1594+

Few simple changes and one very painful one.
Let's start with the easy:
* puts call behind `LIBCALL` define, so call_ext is properly used in 515
* Adds `NAMEOF_STATIC(_,X)` macro for nameof in static definitions since
src is now invalid there.
* Fixes tgui and devserver. From 515 onward the tmp3333{procid} cache
directory is not appened to base path in browser controls so we don't
check for it in base js and put the dev server dummy window file in
actual directory not the byond root.
* Renames the few things that had /final/ in typepath to ultimate since
final is a new keyword

And the very painful change:
`.proc/whatever` format is no longer valid, so we're replacing it with
new nameof() function. All this wrapped in three new macros.
`PROC_REF(X)`,`TYPE_PROC_REF(TYPE,X)`,`GLOBAL_PROC_REF(X)`. Global is
not actually necessary but if we get nameof that does not allow globals
it would be nice validation.
This is pretty unwieldy but there's no real alternative.
If you notice anything weird in the commits let me know because majority
was done with regex replace.

@tgstation/commit-access Since the .proc/stuff is pretty big change.

Co-authored-by: san7890 <the@san7890.com>
Co-authored-by: Mothblocks <35135081+Mothblocks@users.noreply.github.com>
2022-11-15 03:50:11 +00:00
3b7abd584a [MIRROR] Change admin message for undeleted queries to stack trace. [MDB IGNORE] (#17137)
* Change admin message for undeleted queries to stack trace. (#70589)

Change admin message to stack trace

The admins don't know what to do with this error and I swear to god if I get another ping without a round id or server name I am going to literally explode

* Change admin message for undeleted queries to stack trace.

Co-authored-by: oranges <email@oranges.net.nz>
2022-10-24 22:31:39 -04:00
orangesandGitHub 41db5e3a13 Change admin message for undeleted queries to stack trace. (#70589)
Change admin message to stack trace

The admins don't know what to do with this error and I swear to god if I get another ping without a round id or server name I am going to literally explode
2022-10-24 18:08:30 -07:00
a00cad0164 [MIRROR] Clean up subsystem Initialize(), require an explicit result returned, give a formal way to fail (for SSlua) [MDB IGNORE] (#16248)
* Clean up subsystem Initialize(), require an explicit result returned, give a formal way to fail (for SSlua)

* [PR for MIRROR PR] Changes for 16248 (#16277)

* Merge skyrat changes, update SR SS's, and remove lobby_eye

* Apply suggestions from code review

Co-authored-by: GoldenAlpharex <58045821+GoldenAlpharex@users.noreply.github.com>

* Update modular_skyrat/modules/autotransfer/code/autotransfer.dm

Co-authored-by: GoldenAlpharex <58045821+GoldenAlpharex@users.noreply.github.com>

* restore lobby_cam for now

Co-authored-by: GoldenAlpharex <58045821+GoldenAlpharex@users.noreply.github.com>

Co-authored-by: Tastyfish <crazychris32@gmail.com>
Co-authored-by: GoldenAlpharex <58045821+GoldenAlpharex@users.noreply.github.com>
2022-09-24 17:32:40 -04:00
TastyfishandGitHub 4733643f39 Clean up subsystem Initialize(), require an explicit result returned, give a formal way to fail (for SSlua) (#69775)
* cleanup SS API, give SSlua a proper way to error out

* New SS_INIT_ system
2022-09-14 23:52:10 -04:00
4adcaed888 [MIRROR] Minor QuerySelect code improvements, QuerySelect documented. [MDB IGNORE] (#15940)
* Minor QuerySelect code improvements, QuerySelect documented. (#69383)

* Minor QuerySelect improvements coded from the porcelain throne.

We don't handle bad values being given in the query list well enough. This normally won't matter, runtimes are runtimes, but if mixed with real queries, it can lead to inconsistent state where we have query datums that have been ran, and query datums that have not been ran.

pre-checking is also an option, so that it can just refuse to run any of them if one is bad by checking before, but my main goal was to prevent runtimes from bad inputs leading to undeleted queries spam while being more clear about where the bug is, not try to perfectly handle the side effects of bad code.

(To be clear, i started intending to just add codedoc but now it uses as anything and typechecks because that was just eating at me while typing up the codedoc.)

* Update code/controllers/subsystem/dbcore.dm

Co-authored-by: Mothblocks <35135081+Mothblocks@ users.noreply.github.com>

* Minor QuerySelect code improvements, QuerySelect documented.

Co-authored-by: Kyle Spier-Swenson <kyleshome@gmail.com>
Co-authored-by: Mothblocks <35135081+Mothblocks@ users.noreply.github.com>
2022-08-31 09:33:04 -07:00
bbac075bbe Minor QuerySelect code improvements, QuerySelect documented. (#69383)
* Minor QuerySelect improvements coded from the porcelain throne.

We don't handle bad values being given in the query list well enough. This normally won't matter, runtimes are runtimes, but if mixed with real queries, it can lead to inconsistent state where we have query datums that have been ran, and query datums that have not been ran.

pre-checking is also an option, so that it can just refuse to run any of them if one is bad by checking before, but my main goal was to prevent runtimes from bad inputs leading to undeleted queries spam while being more clear about where the bug is, not try to perfectly handle the side effects of bad code.

(To be clear, i started intending to just add codedoc but now it uses as anything and typechecks because that was just eating at me while typing up the codedoc.)

* Update code/controllers/subsystem/dbcore.dm

Co-authored-by: Mothblocks <35135081+Mothblocks@users.noreply.github.com>
2022-08-30 19:14:01 -07:00
GoldenAlpharexandGitHub bbbe0682d8 Optimizes SQL logging and adds missing related default config entries (#15330)
* Adds a new config entry for SQL game log bundle query size

Also adds documentation to existing config entries that don't currently appear in any default entry files on the repo.

* Adds a new config file for database-related configs for Skyrat

* Adds a `add_log_to_mass_insert_queue` proc to handle bundling insert queries into bigger, less frequent queries

* Modifies log_message() to utilise the new add_log_to_mass_insert_queue() proc

* Actually makes the MassInsert() call asynchronous, to completely remove the freezes

* Adds code to handle sending all unsent logs to the database before the subsystem is shutdown, to reduce the loss of logs at the end of a round
2022-08-02 18:04:18 -04:00
b87c7279b0 [MIRROR] Adds MC initialization stages. Earlier stages can fire while later ones init. Fixes tgui chat reconnection banner showing during init. [MDB IGNORE] (#13089)
* Adds MC initialization stages. Earlier stages can fire while later ones init. Fixes tgui chat reconnection banner showing during init. (#66473)

* Adds MC initialization stages. Earlier stages can fire while later ones init.
Removes TICK_LIMIT_MC_INIT config for barely doing anything to speed up init and being inconvenient to work with if fires and inits can happen at the same time.

* Adds MC initialization stages. Earlier stages can fire while later ones init. Fixes tgui chat reconnection banner showing during init.

Co-authored-by: Kyle Spier-Swenson <kyleshome@gmail.com>
2022-04-25 14:25:23 +01:00
Kyle Spier-SwensonandGitHub 565319095f Adds MC initialization stages. Earlier stages can fire while later ones init. Fixes tgui chat reconnection banner showing during init. (#66473)
* Adds MC initialization stages. Earlier stages can fire while later ones init.
Removes TICK_LIMIT_MC_INIT config for barely doing anything to speed up init and being inconvenient to work with if fires and inits can happen at the same time.
2022-04-24 21:29:06 -07:00
eb384bd2d7 Telemetry 'n shit (#10810)
* Refactors dbcore and limits the maximum amount of concurrent async queries to a variable amount (#59676)

Refactors dbcore to work off a subsystem if executed async and limits the maximum amount of concurrent async queries to 25.

This has been tested locally on a mysql docker image and there were no crashes (as long as you didn't run it with debug extools) + data was getting recorded fine.
Why It's Good For The Game

May or may not resolve terry crashes, however, each query creates a new thread which takes up 2mb, preventing the game from using that 2mb. This can lead to ooms if they stack up, e.g. due to poor connectivity. This solves that issue.

maintainer note: this did not actually resolve the crashes, but has value anyway. Crashes were sidestepped fixed by finding out Large Address Awareness works

cl
refactor: Refactors dbcore.dm to possibly resolve the crashes that happen on Terry.
/cl

* Fixes an oversight in database code and cleans up telemetry (#64177)

As it is right now, we never actually clear the temporary list processing_queries
So if the subsystem is for some reason unable to complete a run, we will just whip right back around to it again
If it's been long enough, this could even cause horrific log spam. There was just now a manuel round with roughly 30k undeleted query errors. not good.

But what was actually not deleting you may ask?
Well

When you create a db request, a 5 minute timer starts. after those 5 minutes are up, the request is qdeleted by the db subsystem
This is to prevent the creation of unused requests, and to handle requests that are never cleaned up

Telemetry code was creating all of its db requests inside a for loop that could check tick, and then later
attempting to call them in series

Since requests by default sleep, this almost always lead to undeleted queries, which harddel'd given long enough periods

I've fixed this by moving the data gathering away from the query creation
Why is it good for the game

I was working on atmos code, happy, safe in my delusion, when suddenly I got a ping from tattle freaking out over 200 undeleted queries a second
This resolves that issue, so I can once again live in peace
Changelog

cl
admin: Telemetry code will spam you with undeleted query logs much less often now!
server: Improved how the db subsystem handles undeleted queries, should never have an incident like that again
/cl

* Fixes an error in telemetry queries (#64205)

* Hardsynced time_track.dm with upstream

Co-authored-by: Watermelon914 <37270891+Watermelon914@users.noreply.github.com>
Co-authored-by: LemonInTheDark <58055496+LemonInTheDark@users.noreply.github.com>
2022-01-19 00:21:07 +00:00
LemonInTheDarkandGitHub 6478dc5fce Fixes an oversight in database code and cleans up telemetry (#64177)
As it is right now, we never actually clear the temporary list processing_queries
So if the subsystem is for some reason unable to complete a run, we will just whip right back around to it again
If it's been long enough, this could even cause horrific log spam. There was just now a manuel round with roughly 30k undeleted query errors. not good.

But what was actually not deleting you may ask?
Well

When you create a db request, a 5 minute timer starts. after those 5 minutes are up, the request is qdeleted by the db subsystem
This is to prevent the creation of unused requests, and to handle requests that are never cleaned up

Telemetry code was creating all of its db requests inside a for loop that could check tick, and then later
attempting to call them in series

Since requests by default sleep, this almost always lead to undeleted queries, which harddel'd given long enough periods

I've fixed this by moving the data gathering away from the query creation
Why is it good for the game

I was working on atmos code, happy, safe in my delusion, when suddenly I got a ping from tattle freaking out over 200 undeleted queries a second
This resolves that issue, so I can once again live in peace
Changelog

cl
admin: Telemetry code will spam you with undeleted query logs much less often now!
server: Improved how the db subsystem handles undeleted queries, should never have an incident like that again
/cl
2022-01-18 10:17:40 +13:00
ba64528c4c [DNM] Revert "[MIRROR] Refactors dbcore and limits the maximum amount of concurrent async queries to a variable amount" (#6478)
* Revert "[MIRROR] Refactors dbcore and limits the maximum amount of concurrent async queries to a variable amount (#6436)"

This reverts commit e2519525f9.

* Update time_track.dm

Co-authored-by: Gandalf <jzo123@hotmail.com>
2021-08-06 23:56:35 +01:00
e2519525f9 [MIRROR] Refactors dbcore and limits the maximum amount of concurrent async queries to a variable amount (#6436)
* Refactors dbcore and limits the maximum amount of concurrent async queries to a variable amount (#59676)

Refactors dbcore to work off a subsystem if executed async and limits the maximum amount of concurrent async queries to 25.

This has been tested locally on a mysql docker image and there were no crashes (as long as you didn't run it with debug extools) + data was getting recorded fine.
Why It's Good For The Game

May or may not resolve terry crashes, however, each query creates a new thread which takes up 2mb, preventing the game from using that 2mb. This can lead to ooms if they stack up, e.g. due to poor connectivity. This solves that issue.

maintainer note: this did not actually resolve the crashes, but has value anyway. Crashes were sidestepped fixed by finding out Large Address Awareness works

cl
refactor: Refactors dbcore.dm to possibly resolve the crashes that happen on Terry.
/cl

* Refactors dbcore and limits the maximum amount of concurrent async queries to a variable amount

Co-authored-by: Watermelon914 <37270891+Watermelon914@users.noreply.github.com>
2021-06-22 00:49:20 +01:00
Watermelon914andGitHub f7117797d4 Refactors dbcore and limits the maximum amount of concurrent async queries to a variable amount (#59676)
Refactors dbcore to work off a subsystem if executed async and limits the maximum amount of concurrent async queries to 25.

This has been tested locally on a mysql docker image and there were no crashes (as long as you didn't run it with debug extools) + data was getting recorded fine.
Why It's Good For The Game

May or may not resolve terry crashes, however, each query creates a new thread which takes up 2mb, preventing the game from using that 2mb. This can lead to ooms if they stack up, e.g. due to poor connectivity. This solves that issue.

maintainer note: this did not actually resolve the crashes, but has value anyway. Crashes were sidestepped fixed by finding out Large Address Awareness works


cl
refactor: Refactors dbcore.dm to possibly resolve the crashes that happen on Terry.
/cl
2021-06-22 09:33:42 +12:00