cc335e7e9e
## About The Pull Request Replaces the asset subsystem's spritesheet generator with a rust-based implementation (https://github.com/tgstation/rust-g/pull/160). This is a rough port of https://github.com/BeeStation/BeeStation-Hornet/pull/10404, but it includes fixes for some cases I didn't catch that apply on TG. (FWIW we've been using this system on prod for over a year and encountered no major issues.) ### TG MAINTAINER NOTE  ### Batched Spritesheets `/datum/asset/spritesheet_batched`: A version of the spritesheet system that collects a list of `/datum/universal_icon`s and sends them off to rustg asynchronously, and the generation also runs on another thread, so the game doesn't block during realize_spritesheet. The rust generation is about 10x faster when it comes to actual icon generation, but the biggest perk of the batched spritesheets is the caching system. This PR notably does not convert a few things to the new spritesheet generator. - Species and antagonist icons in the preferences view because they use getFlatIcon ~~which can't be converted to universal icons~~. - Yes, this is still a *massive* cost to init, unfortunately. On Bee, I actually enabled the 'legacy' cache on prod and development, which you can see in my PR. That's why I added the 'clear cache' verb and the `unregister()` procs, because it can force a regeneration at runtime. I decided not to port this, since I think it would be detrimental to the large amount of contributors here. - It is *technically* possible to port parts of this to the uni_icon system by making a uni_icon version of getFlatIcon. However, some overlays use runtime-generated icons which are ~~completely unparseable to IconForge, since they're stored in the RSC and don't exist as files anywhere~~. This is most noticeable with things like hair (which blend additively with the hair mask on the server, thus making them invisible to `get_flat_uni_icon`). It also doesn't help that species and antag icons will still need to generate a bunch of dummies and delete them to even verify cache validity. - It is actually possible to write the RSC icons to the filesystem (using fcopy) and reference them in IconForge. However, I'm going to wait on doing this until I port my GAGS implementation because it requires GAGS to exist on the filesystem as well. #### Caching IconForge generates a cache based on the set of icons used, all transform operations applied, and the source DMIs of each icon used within the spritesheet. It can compare the hashes and invalidate the cache automatically if any of these change. This means we can enable caching on development, and have absolutely no downsides, because if anything changes, the cache invalidates itself. The caching has a mean cost of ~5ms and saves a lot of time compared to generating the spritesheet, even with rust's faster generation. The main downside is that the cache still requires building the list of icons and their transforms, then json encoding it to send to rustg. Here's an abbreviated example of a cache JSON. All of these need to match for the cache to be valid. `input_hash` contains the transform definitions for all the sprites in the spritesheet, so if the input to iconforge changes, that hash catches it. The `sizes` and `sprites` are loaded into DM. ```json { "input_hash": "99f1bc67d590e000", "dmi_hashes": { "icons/ui/achievements/achievements.dmi": "771200c75da11c62" }, "sizes": [ "76x76" ], "sprites": { "achievement-rustascend": { "size_id": "76x76", "position": 1 } }, "rustg_version": "3.6.0", "dm_version": 1 } ``` ### Universal Icons Universal icons are just a collection of DMI, Icon State, and any icon transformation procs you apply (blends, crops, scales). They can be convered to DM icons via `to_icon()`. I've included an implementation of GAGS that produces universal icons, allowing GAGS items to be converted into them. IconForge can read universal icons and add them to spritesheets. It's basically just a wrapper that reimplements BYOND icon procs. ### Other Stuff Converts some uses of md5asfile within legacy spritesheets to use rustg_hash_file instead, improving the performance of their generation. Fixes lizard body markings not showing in previews, and re-adds eyes to the ethereal color preview. This is a side effect of IconForge having *much* better error handling than DM icon procs. Invalid stuff that gets passed around will error instead of silently doing nothing. Changes the CSS used in legacy spritesheet generation to split `background: url(...) no-repeat` into separate props. This is necessary for WebView2, as IE treats these properties differently - adding `background-color` to an icon object (as seen in the R&D console) won't work if you don't split these out. Deletes unused spritesheets and their associated icons (condiments spritesheet, old PDA spritesheet) ## Why It's Good For The Game If you press "Character Setup", the 10-13sec of lag is now approximately 0.5-2 seconds. Tracy profile showing the time spent on get_asset_datum. I pressed the preferences button during init on both branches. Do note that this was ran with a smart cache HIT, so no generation occurred.  Much lower worst-case for /datum/asset/New (which includes `create_spritesheets()` and `register()`)  Here's a look at the internal costs from rustg - as you can see `generate_spritesheet()` is very fast:  ### Comparison for a single spritesheet - chat spritesheet: **Before**  **After**  ## Changelog 🆑 fix: Fixed lizard body markings and ethereal feature previews in the preference menu missing some overlays. refactor: Optimized spritesheet asset generation greatly using rustg IconForge, greatly reducing post-initialization lag as well as reducing init times and saving server computation. config: Added 'smart' asset caching, for batched rustg IconForge spritesheets. It is persistent and suitable for use on local, with automatic invalidation. add: Added admin verbs - Debug -> Clear Smart/Legacy Asset Cache for spritesheets. fix: Fixed R&D console icons breaking on WebView2/516 /🆑
Unit Tests
What is unit testing?
Unit tests are automated code to verify that parts of the game work exactly as they should. For example, a test to make sure that the amputation surgery actually amputates the limb. These are ran every time a PR is made, and thus are very helpful for preventing bugs from cropping up in your code that would've otherwise gone unnoticed. For example, would you have thought to check that beach boys would still work the same after editing pizza? If you value your time, probably not.
On their most basic level, when UNIT_TESTS is defined, all subtypes of /datum/unit_test will have their Run proc executed. From here, if Fail is called at any point, then the tests will report as failed.
How do I write one?
- Find a relevant file.
All unit test related code is in code/modules/unit_tests. If you are adding a new test for a surgery, for example, then you'd open surgeries.dm. If a relevant file does not exist, simply create one in this folder, then #include it in _unit_tests.dm.
- Create the unit test.
To make a new unit test, you simply need to define a /datum/unit_test.
For example, let's suppose that we are creating a test to make sure a proc square correctly raises inputs to the power of two. We'd start with first:
/datum/unit_test/square/Run()
This defines our new unit test, /datum/unit_test/square. Inside this function, we're then going to run through whatever we want to check. Tests provide a few assertion functions to make this easy. For now, we're going to use TEST_ASSERT_EQUAL.
/datum/unit_test/square/Run()
TEST_ASSERT_EQUAL(square(3), 9, "square(3) did not return 9")
TEST_ASSERT_EQUAL(square(4), 16, "square(4) did not return 16")
As you can hopefully tell, we're simply checking if the output of square matches the output we are expecting. If the test fails, it'll report the error message given as well as whatever the actual output was.
- Run the unit test
Open code/_compile_options.dm and uncomment the following line.
//#define UNIT_TESTS //If this is uncommented, we do a single run though of the game setup and tear down process with unit tests in between
There are 3 ways to run unit tests
-
Run tgstation.dmb in Dream Daemon. Don't bother trying to connect, you won't need to. You'll be able to see the outputs of all the tests. You'll get to see which tests failed and for what reason. If they all pass, you're set!
-
Launch game from VS Code. Launch the game as normal & you will see the output of your unit tests in your fancy chat window. This is preferred as you can use the debugger to step through each line of your unit test & can use the games inbuilt debugging tools to further aid in testing
-
Use VS Code Tgstation Test Explorer Extension. This allows you to run tests without launching the game & can also run focused tests(either a single or a selected group)
How to think about tests
Unit tests exist to prevent bugs that would happen in a real game. Thus, they should attempt to emulate the game world wherever possible. For example, the quick swap sanity test emulates a real scenario of the bug it fixed occurring by creating a character and giving it real items. The unrecommended alternative would be to create special test-only items. This isn't a hard rule, the reagent method exposure tests create a test-only reagent for example, but do keep it in mind.
Unit tests should also be just that--testing units of code. For example, instead of having one massive test for reagents, there are instead several smaller tests for testing exposure, metabolization, etc.
The unit testing API
You can find more information about all of these from their respective doc comments, but for a brief overview:
/datum/unit_test - The base for all tests to be ran. Subtypes must override Run(). New() and Destroy() can be used for setup and teardown. To fail, use TEST_FAIL(reason).
/datum/unit_test/proc/allocate(type, ...) - Allocates an instance of the provided type with the given arguments. Is automatically destroyed when the test is over. Commonly seen in the form of var/mob/living/carbon/human/human = allocate(/mob/living/carbon/human/consistent).
TEST_FAIL(reason) - Marks a failure at this location, but does not stop the test.
TEST_ASSERT(assertion, reason) - Stops the unit test and fails if the assertion is not met. For example: TEST_ASSERT(powered(), "Machine is not powered").
TEST_ASSERT_NOTNULL(a, message) - Same as TEST_ASSERT, but checks if !isnull(a). For example: TEST_ASSERT_NOTNULL(myatom, "My atom was never set!").
TEST_ASSERT_NULL(a, message) - Same as TEST_ASSERT, but checks if isnull(a). If not, gives a helpful message showing what a was. For example: TEST_ASSERT_NULL(delme, "Delme was never cleaned up!").
TEST_ASSERT_EQUAL(a, b, message) - Same as TEST_ASSERT, but checks if a == b. If not, gives a helpful message showing what both a and b were. For example: TEST_ASSERT_EQUAL(2 + 2, 4, "The universe is falling apart before our eyes!").
TEST_ASSERT_NOTEQUAL(a, b, message) - Same as TEST_ASSERT_EQUAL, but reversed.
TEST_FOCUS(test_path) - Only run the test provided within the parameters. Useful for reducing noise. For example, if we only want to run our example square test, we can add TEST_FOCUS(/datum/unit_test/square). Should never be pushed in a pull request--you will be laughed at.
Final Notes
- Writing tests before you attempt to fix the bug can actually speed up development a lot! It means you don't have to go in game and folllow the same exact steps manually every time. This process is known as "TDD" (test driven development). Write the test first, make sure it fails, then start work on the fix/feature, and you'll know you're done when your tests pass. If you do try this, do make sure to confirm in a non-testing environment just to double check.
- Make sure that your tests don't accidentally call RNG functions like
prob. Since RNG is seeded during tests, you may not realize you have until someone else makes a PR and the tests fail! - Do your best not to change the behavior of non-testing code during tests. While it may sometimes be necessary in the case of situations such as the above, it is still a slippery slope that can lead to the code you're testing being too different from the production environment to be useful.