Merge pull request #727 from Cyberboss/V4Release [TGSDeploy] [NugetDeploy]

V4 Release

.github/CONTRIBUTING.md

@@ -143,6 +143,8 @@ There is no strict process when it comes to merging pull requests. Pull requests
 
 * If your pull request is accepted, the code you add no longer belongs exclusively to you but to everyone; everyone is free to work on it, but you are also free to support or object to any changes being made, which will likely hold more weight, as you're the one who added the feature. It is a shame this has to be explicitly said, but there have been cases where this would've saved some trouble.
 
+* Your submission must be tested with 100% code coverage with both unit and integration tests
+
 * Please explain why you are submitting the pull request, and how you think your change will be beneficial to the server. Failure to do so will be grounds for rejecting the PR.
 
 * Commits MUST be properly titled and commented as we only use merge commits for the pull request process

.github/ISSUE_TEMPLATE.md

@@ -4,4 +4,4 @@ Please include:
 
 - Reproduction steps for the issue if possible
 
-- Relevent stack traces, error details, and screenshots if possible
+- Relevent server logs, and screenshots if possible

README.md

@@ -8,19 +8,17 @@
 
 [![forthebadge](http://forthebadge.com/images/badges/built-with-love.svg)](http://forthebadge.com) [![forthebadge](http://forthebadge.com/images/badges/60-percent-of-the-time-works-every-time.svg)](http://forthebadge.com)
 
-
-This is a toolset to manage production BYOND servers. It includes the ability to update the server without having to stop or shutdown the server (the update will take effect on a "reboot" of the server) the ability start the server and restart it if it crashes, as well as systems for fixing errors and merging GitHub Pull Requests locally.
-  
-Generally, updates force a live tracking of the configured git repo, resetting local modifications. If you plan to make modifications, set up a new git repo to store your version of the code in, and point this script to that in the config (explained below). This can be on GitHub or a local repo using file:/// urls.
+This is a toolset to manage production BYOND servers. It includes the ability to update the server without having to stop or shutdown the server (the update will take effect on a "reboot" of the server) the ability start the server and restart it if it crashes, as well as systems for managing code and game files, and merging GitHub Pull Requests locally.  
 
 ### Legacy Servers
+
 * Older server versions can be found in the V# branches of this repository
 
 ## Setup
 
 ### Installation
 
-1. Download and install the [.NET Core Runtime (>= v2.1)](https://www.microsoft.com/net/download) for your system. If you plan to install tgstation-server as a Windows service, you should also ensure that your .NET Framework runtime version is >= v4.7.1 (Download can be found on same page). Enusre that the `dotnet` executable file is in your system's `PATH` variable (or the user's that will be running the server).
+1. Download and install the [.NET Core Runtime (>= v2.1)](https://www.microsoft.com/net/download) for your system. If you plan to install tgstation-server as a Windows service, you should also ensure that your .NET Framework runtime version is >= v4.7.1 (Download can be found on same page). On Windows, ensure that the `dotnet` executable file is in your system's `PATH` variable (or the user's that will be running the server).
 2. [Download the latest V4 release .zip](https://github.com/tgstation/tgstation-server/releases/latest). The ServerService package will only work on Windows. Choose ServerConsole if that is not your target OS or you prefer not to use the Windows service.
 3. Extract the .zip file to where you want the server to run from. Note the account running the server must have write access to the `lib` subdirectory.
 4. If using the ServerService package, run `Tgstation.Server.Host.Service.exe`. It should prompt you to install the service. Click `Yes` and accept a potential UAC elevation prompt. You should now be able to control the service using the Windows service control commandlet.
@@ -37,13 +35,14 @@ Note that tgstation-server has only ever been tested on Linux via it's [docker e
 
 #### Docker
 
-tgstation-server supports running in a docker container and is the recommended deployment method for Linux systems due being the only tested environment. The official image repository is located at https://hub.docker.com/r/tgstation/server. It can also be built locally by running `docker build . -f build/Dockerfile` in the repository root.
+tgstation-server supports running in a docker container and is the recommended deployment method for Linux systems due being the only robustly tested environment. The official image repository is located at https://hub.docker.com/r/tgstation/server. It can also be built locally by running `docker build . -f build/Dockerfile` in the repository root.
 
 To create a container run
-```
+```sh
 docker create \
 	--restart=always \ #if you want maximum uptime
 	--network="host" \ #if your sql server is on the same machine
+	--name="tgs" \ #or whatever else you wanna call it
 	-p <tgs port>:80 \
 	-p 0.0.0.0:<public game port>:<internal game port> \
 	-v /path/to/store/instances:/tgs4_instances \
@@ -55,6 +54,8 @@ with any additional options you desire (i.e. You'll have to expose more game por
 
 Note although `/app/lib` is specified as a volume mount point in the `Dockerfile`, unless you REALLY know what you're doing. Do not mount any volumes over this for fear of breaking your container.
 
+Before starting your container make sure the aforemention `appsettings.Production.json` is configured properly. See below
+
 ### Configuring
 
 Create an `appsettings.Production.json` file next to `appsettings.json`. This will override the default settings in appsettings.json with your production settings. There are a few keys meant to be changed by hosts. Modifying any config files while the server is running will trigger a safe restart (Keeps DreamDaemon's running). Note these are all case-sensitive: 
@@ -79,15 +80,17 @@ Create an `appsettings.Production.json` file next to `appsettings.json`. This wi
 
 If using MySQL, our provider library [recommends you set 'utf8mb4' as your default charset](https://github.com/PomeloFoundation/Pomelo.EntityFrameworkCore.MySql#1-recommended-server-charset) disregard at your own risk.
 
-The user created for the application will need the privilege to create databases on the first run. Once the initial set of migrations is run, the create right may be revoked. The user should maintain DDL rights though for applying future migrations
+The user created for the application will need the privilege to create databases on the first run, do not create the database for it. Once the initial set of migrations is run, the create right may be revoked. The user should maintain DDL rights though for applying future migrations
 
-Note that the ratio of application installations to databases is 1:1. Do not attempt to share a database amongst multiple TGS installations.
+Note that the ratio of application installations to databases is 1:1. Do not attempt to share a database amongst multiple TGS installations. (We know SQLite would be perfect for this, but it does not handle the high level of concurrency the server uses)
 
 ### Starting
 
-For the Windows service version start the `tgstation-server-4` service
+For the Windows service version start the `tgstation-server-4` service. If it fails to start, check the Windows event log under Windows/Application for entries from tgstation-server-4 for errors.
 
-For the console version run `dotnet Tgstation.Server.Host.Console.dll` in the installation directory. The `tgs.bat` and `tgs.sh` shell scripts are shortcuts for this
+For the console version run `dotnet Tgstation.Server.Host.Console.dll` in the installation directory. The `tgs.bat` and `tgs.sh` shell scripts are shortcuts for this. If on Windows and you wish to install byond versions >= 512.1427 you must do this as admin to give the server permission to install the required DirectX dependency
+
+For the docker version run `docker start tgs`
 
 ### Stopping
 
@@ -97,11 +100,13 @@ For the Windows service version stop the `tgstation-server-4` service
 
 For the console version press `Ctrl+C` or send a SIGQUIT to the ORIGINAL dotnet process
 
+For the docker version run `docker stop tgs`
+
 ## Integrating
 
 A breaking change from V3: tgstation-server 4 now REQUIRES the DMAPI to be integrated into any BYOND codebase which plans on being used by it. The integration process is a fairly simple set of code changes.
 
-1. Copy the [DMAPI] files anywhere in your code base. `tgs.dm` can be seperated from the `tgs` folder, but do not modify or move the contents of the `tgs` folder
+1. Copy the [DMAPI](https://github.com/tgstation/tgstation-server/tree/master/src/DMAPI) files anywhere in your code base. `tgs.dm` can be seperated from the `tgs` folder, but do not modify or move the contents of the `tgs` folder
 2. Modify your `.dme`(s) to include the `tgs.dm` and `tgs/includes.dm` files (ORDER OF APPEARANCE IS MANDATORY)
 3. Follow the instructions in `tgs.dm` to integrate the API with your codebase.
 
@@ -155,13 +160,13 @@ var/global/client_count = 0
 
 ## Remote Access
 
-tgstation-server is an [ASP.Net Core](https://docs.microsoft.com/en-us/aspnet/core/) based on the Kestrel web server. This section is meant to serve as a general use case overview, but the entire Kestrel configuration can be modified to your liking with the configuration JSON. See [the official documentation](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel) for details.
+tgstation-server is an [ASP.Net Core](https://docs.microsoft.com/en-us/aspnet/core/) app based on the Kestrel web server. This section is meant to serve as a general use case overview, but the entire Kestrel configuration can be modified to your liking with the configuration JSON. See [the official documentation](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel) for details.
 
-Exposing the builtin kestrel server to the internet directly over HTTP is highly not reccommended due to the lack of security. The recommended way to expose tgstation-server to the internet is to host it through a reverse proxy with HTTPS support. Here are some step by step examples to achieve this for major web servers.
+Exposing the builtin Kestrel server to the internet directly over HTTP is highly not reccommended due to the lack of security. The recommended way to expose tgstation-server to the internet is to host it through a reverse proxy with HTTPS support. Here are some step by step examples to achieve this for major web servers.
 
 System administrators will most likely have their own configuration plans, but here are some basic guides for beginners.
 
-Once complete, test that your configuration worked by visiting your proxy site from a different computer. You should recieve a 401 Unauthorized response.
+Once complete, test that your configuration worked by visiting your proxy site from a browser on a different computer. You should recieve a 401 Unauthorized response.
 
 ### IIS (Reccommended for Windows)
 
@@ -221,13 +226,15 @@ tgstation-server v4 is controlled via a RESTful HTTP json API. Documentation on
 
 ### Users
 
-All actions apart from logging in must be taken by a user. TGS installs with one default user whose credentials can be found [here](https://github.com/tgstation/tgstation-server/blob/master/src/Tgstation.Server.Api/Models/User.cs). If access to all users is lost, the default user can be reset using the `Database:ResetAdminPassword` configuration setting. Users can be enabled/disabled and have a very granular set of rights associated to them that determine the actions they are allowed to take (i.e. Modify the user list or create instances). Users can be _database based_ or _system based_. Database users are your standard web users with a username and password. System users, on the otherhand, are authenticated with the host OS. These users cannot have their password or names changed by TGS as they are managed by the system (and in reverse, login tokens don't expire when their password changes). The benefit to having these users is it allows the use of system ACLs for static file control. More on that later.
+All actions apart from logging in must be taken by a user. TGS installs with one default user whose credentials can be found [here](https://github.com/tgstation/tgstation-server/blob/master/src/Tgstation.Server.Api/Models/User.cs). It is recommended to disable this user ASAP as it is used to create Jobs that are started by the system. If access to all users is lost, the default user can be reset using the `Database:ResetAdminPassword` configuration setting. 
+
+Users can be enabled/disabled and have a very granular set of rights associated to them that determine the actions they are allowed to take (i.e. Modify the user list or create instances). Users can be _database based_ or _system based_. Database users are your standard web users with a username and password. System users, on the otherhand, are authenticated with the host OS. These users cannot have their password or names changed by TGS as they are managed by the system (and in reverse, login tokens don't expire when their password changes). The benefit to having these users is it allows the use of system ACLs for static file control. More on that later.
 
 ### Instances
 
 A TGS deployment is made up with a set of instances, which each represent a production BYOND server. As many instances as desired can be created. Be aware, however, due to the nature of BYOND, this will quickly result in system resource exhaustion.
 
-An instance is stored in a single folder anywhere on a system and is made up of several components: The source code git repository, BYOND, the compiler, the watchdog, chat bots, and static file management systems.
+An instance is stored in a single folder anywhere on a system and is made up of several components: The source code git repository, the BYOND installations, the compiler, the watchdog, chat bots, and static file management systems.
 
 ##### Instance Users
 
@@ -243,7 +250,7 @@ Manual operations on the repository while an instance is running may lead to git
 
 #### Byond
 
-The `Byond` folder contains installations of [BYOND](https://secure.byond.com/) versions. The version which is used by your game code can be changed on a whim (Note that only versions >= 511 have been thouroughly tested. Lower versions should work but if one doesn't function, please open an issue report) and the server will take care of installing it.
+The `Byond` folder contains installations of [BYOND](https://secure.byond.com/) versions. The version which is used by your game code can be changed on a whim (Note that only versions >= 511.1385 have been thouroughly tested. Lower versions should work but if one doesn't function, please open an issue report) and the server will take care of installing it.
 
 #### Compiler
 
@@ -264,20 +271,20 @@ TGS supports creating infinite chat bots for notifying staff or players of thing
 
 More can be added by providing a new implementation of the [IProvider](https://github.com/tgstation/tgstation-server/blob/master/src/Tgstation.Server.Host/Components/Chat/Providers/IProvider.cs) interface
 
-Bots have a set of built-in commands that can be triggered via `!tgs` mentioning, or private messaging them. Along with these, custom commands can be defined using the DMAPI by creating a subtype of the `/datum/tgs_chat_command` type (See `tgs.dm` for details). Invocation for custom commands can be restricted to certain channels.
+Bots have a set of built-in commands that can be triggered via `!tgs`, mentioning, or private messaging them. Along with these, custom commands can be defined using the DMAPI by creating a subtype of the `/datum/tgs_chat_command` type (See `tgs.dm` for details). Invocation for custom commands can be restricted to certain channels.
 
 #### Static Files
 
 All files in game code deployments are considered transient by default, meaning when new code is deployed, changes will be lost. Static files allow you to specify which files and folders stick around throughout all deployments.
 
-The `StaticFiles` folder contains 3 root folders
+The `StaticFiles` folder contains 3 root folders which cannot be deleted and operate under special rules
 	- `CodeModifications`
 	- `EventScripts`
 	- `GameStaticFiles`
 
 These files can be modified either in host mode or system user mode. In host mode, TGS itself is responsible for reading and writing the files. In system user mode read and write actions are performed using the system account of the logged on User, enabling the use of ACLs to control access to files. Database users will not be able to use the static file system if this mode is configured for an instance.
 
-This folder may be freely modified manually just beware this may cause deployments to error if done simulatenously on Windows systems.
+This folder may be freely modified manually just beware this may cause in-progress deployments to error if done on Windows systems.
 
 #### CodeModifications
 
@@ -293,7 +300,7 @@ This folder can contain anything. But, when certain events occur in the instance
 
 #### GameStaticFiles
 
-Any files and folders contained in this folder will be symbolically linked to all deployments at the time they are created. This allows persistent game data (BYOND `.sav`s or code configuration files for example) to persist across all deployments.
+Any files and folders contained in this root level of this folder will be symbolically linked to all deployments at the time they are created. This allows persistent game data (BYOND `.sav`s or code configuration files for example) to persist across all deployments.
 
 ### Updating
 
@@ -320,8 +327,8 @@ Feel free to ask for help at the coderbus discord: https://discord.gg/Vh8TJp9. C
 
 ## Licensing
 
-* The DM API for the project is licensed under the MIT license.
+* The DMAPI for the project is licensed under the MIT license.
 * The /tg/station 13 icon is licensed under [Creative Commons 3.0 BY-SA](http://creativecommons.org/licenses/by-sa/3.0/).
 * The remainder of the project is licensed under [GNU AGPL v3](http://www.gnu.org/licenses/agpl-3.0.html)
 
-See the /src/DMAPI tree for the MIT license
+See the files in the /src/DMAPI tree for the MIT license

appveyor.yml

@@ -73,8 +73,8 @@ after_test:
   - ps: Copy-Item -path "src/Tgstation.Server.Host.Service/bin/$env:CONFIGURATION" -destination artifacts/ServerService -recurse
   - ps: Copy-Item -path "artifacts/ServerHost" -destination artifacts/ServerService/lib/Default -recurse
   - ps: Move-Item -path artifacts/ServerService/lib/Default/appsettings.json -destination artifacts/ServerService/
-  #deploy stuff
   - ps: Remove-Item artifacts/ServerHost/appsettings.json
+  #deploy stuff
   - ps: $env:TGSVersion = [System.Diagnostics.FileVersionInfo]::GetVersionInfo("$env:APPVEYOR_BUILD_FOLDER/artifacts/ServerHost/Tgstation.Server.Host.dll").FileVersion
   - ps: if($env:APPVEYOR_REPO_COMMIT_MESSAGE -match "\[TGSDeploy\]"){ if($env:APPVEYOR_REPO_BRANCH -match "master"){ if($env:CONFIGURATION -match "Release"){ $env:TGSDeploy = "Do it." }}}
   - ps: if($env:APPVEYOR_REPO_COMMIT_MESSAGE -match "\[NugetDeploy\]"){ if($env:APPVEYOR_REPO_BRANCH -match "master"){ if($env:CONFIGURATION -match "Release"){ $env:NugetDeploy = "Do it." }}}

docs/API.dox

@@ -61,7 +61,7 @@ TGS will only every return the response codes listed here
 - 408: Request timeout. The client took to long to continue a request
 - 409: Conflict. Documented in the requests that use them
 - 410: Gone. Attempted to access/modify a resource that ideally should have been ready, but isn't or no longer is
-- 422: Unprocessable Entity: Used specifically when an operation that requires a server restart is unable to be performed due to the @ref Tgstation.Server.Host.Watchdog not being present in the deployment. Blame MSO. Response body contains an @ref Tgstation.Server.Api.Models.ErrorMessage
+- 422: Unprocessable Entity: Used specifically when an operation that requires a server restart is unable to be performed due to the @ref Tgstation.Server.Host.Watchdog not being present in the deployment. Should not happen with a proper server configuration. Response body contains an @ref Tgstation.Server.Api.Models.ErrorMessage
 - 424: Failed Dependency: When a request that depends on the GitHub API fails for a reason other than rate limiting. Check server logs, usually this indicates a bad access token.
 - 426: Upgrade required: Used when the client's API version is not compatible with the server's. Response body contains an @ref Tgstation.Server.Api.Models.ErrorMessage
 - 429: Rate limited. Used with operations that rely on GitHub.com. If a rate limit is hit for an operation this will be returned. Response will contain a Retry-After header
@@ -245,8 +245,6 @@ The @ref Tgstation.Server.Api.Models.Internal.ChatBot.ConnectionString must diff
 For IRC chat bots see @ref Tgstation.Server.Api.Models.IrcConnectionStringBuilder
 For Discord chat bots see @ref Tgstation.Server.Api.Models.DiscordConnectionStringBuilder
 
-For Discord chat bots it should be the <a href="https://discordapp.com/developers/docs/topics/oauth2#bots">bot's Token</a>
-
 A specific bot's settings may be retrieved with:
 
 I GET "/Chat/{ChatBotId}" => @ref Tgstation.Server.Api.Models.ChatBot

docs/Architecture.dox

@@ -42,20 +42,200 @@ The first thing this function does is call @ref Tgstation.Server.Host.Models.Dat
 
 @section arch_db Database and Context
 
-@section arch_security Security
+The database is exposed as a series of DbSet<T> objects through @ref Tgstation.Server.Host.Models.IDatabaseContext . Queries are performed via async LINQ expressions. Inserts, updates, and deletes are done via modifiying the DbSet<T>s and then calling @ref Tgstation.Server.Host.Models.IDatabaseContext.Save . Do some reading on Entity Framework Core for a deeper understanding.
 
 @section arch_controllers Controllers
 
+The webserver operates in an MVC style. All requests are routed through the @ref Tgstation.Server.Host.Controllers . If a route doesn't exist as an action in a controller, a 404 response will be returned. Controllers interact with components via injecting the @ref Tgstation.Server.Host.Components.IInstanceManager interface, access the database with the @ref Tgstation.Server.Host.Controllers.ApiController.DatabaseContext property, and start jobs by injecting the @ref Tgstation.Server.Host.Core.IJobManager interface.
+
+@section arch_security Security
+
+The authentication process begins in @ref Tgstation.Server.Host.Controllers.HomeController.CreateToken . This is where users log in. They must supply their username and password via correct @ref Tgstation.Server.Api.ApiHeaders . The server first attempts to use these credentials to login to the system. If that succeeds it checks if the system user's UID is registered in the database. Failing either of the previous two, it tries to match the username and password to an entry in the database. If either of these methods succeeds the user is considered authenticated and a token is generated and sent back to the user. If the user is a system user, the context of their login is kept for the amount of time until their token expires + 1 minute.
+
+The password hashing used for database users is the standard provided by ASP.Net Core. It utilizes PBKDF2 with HMAC-SHA256, 128-bit salt, 256-bit subkey, with 10000 iterations. Read about it here: https://andrewlock.net/exploring-the-asp-net-core-identity-passwordhasher/
+
+When this token is supplied in the `Authorization` header of a subsequent request, it is first cryptographically validated that it was sent by the current server. The token contain's the user's ID, and, using it, the user's info is retrieved from the database and put into an @ref Tgstation.Server.Host.Security.IAuthenticationContext
+
+Nearly all exposed controller actions are decorated with a @ref Tgstation.Server.Host.Controllers.TgsAuthorizeAttribute . This attribute does 2 things. 1. It ensures the @ref Tgstation.Server.Host.Security.IAuthenticationContext is valid for the request before running the action. 2. If it contains a permission flag specification, it will 403 the request if the user doesn't have one of the listed permissions.
+
 @section arch_jobs Jobs
 
+Long running operations create @ref Tgstation.Server.Host.Models.Job objects which represent information about long running tasks. These objects can be queried to find out who started them, if they've been completed, canceled, who cancelled them, their error message if any, and get their progress percentage in some cases. The job will be created and supplied by the request that started it, but active/all jobs may also be queried.
+
 @section arch_instance Instances
 
+Instances exist in two forms: Their database metadata and their actual class. The class only exists if the instance is set to be @ref Tgstation.Server.Api.Models.Instance.Online . This is where all the actual server management code lives. A single instance is made up of individual components that work with each other through their intefaces
+
 @subsection arch_ifactory Instance Factory
 
+This is responsible for creating the components and weaving them into the final @ref Tgstation.Server.Host.Components.Instance. This happens automatically at server startup if an instance is configured to be online
+
+@section arch_repository Repository Manager
+
+The @ref Tgstation.Server.Host.Components.Repository.IRepositoryManager is the gatekeeper for cloning and accessing a @ref Tgstation.Server.Host.Components.Repository.IRepository . Only one instance of a repository can be in use at a time (due to the single-threaded nature of libgit2), so the repository manager contains a semaphore wait queue which hands out the repository to only one client at a time. All repository operations (aside from cloning and deleting) are performed by the actual repository object. This includes fetching, hard resets, checkouts, synchronizing, etc. Most state put into the @ref Tgstation.Server.Api.Models.Repository object is read directly from libgit2, exceptions being credentials and boolean settings.
+
+@section arch_byond Byond
+
+The BYOND installation setup is largely decoupled from the database. When a byond version is downloaded and installed by the @ref Tgstation.Server.Host.Components.Byond.IByondManager it is extracted to a directory titled with it's version in the `BYOND` folder acompanied by a text document stating which version it is. Platform specific installation steps are handled by specific implementations of @ref Tgstation.Server.Host.Components.Byond.IByondInstaller . When it comes time to use an executable, the manager provides a @ref Tgstation.Server.Host.Components.Byond.IByondExecutableLock which contains absolute paths to the DreamMaker and DreamDaemon executables
+
+@section arch_deployment Compiler and Deployment
+
+The compilation process is a distinct series of steps:
+
+1. Retrieve necessary information from the database in @ref Tgstation.Server.Host.Components.IInstance.CompileProcess
+2. Choose a uniquely named folder in the `Game` directory for deployment
+3. Acquire a @ref Tgstation.Server.Host.Components.Byond.IByondExecutableLock
+4. Build the initial @ref Tgstation.Server.Host.Models.CompileJob object from available data (Byond version, Revision, directory, etc)
+5. Announce the deployment through the chat bot system
+6. Create and copy the repository to the `<target folder>/A` directory
+7. Run the PreCompile hook
+8. Auto detect or check if the configured .dme is present
+9. Copy and apply static code modifications to the environment
+10. Run DreamMaker on the .dme
+11. Start a DreamDaemon instance to validate the DMAPI
+12. Run the PostCompile hook
+13. Copy `<target folder>/A` to `<target folder>/B`
+14. Symlink all `GameStaticFiles` to both the A and B directories
+15. Commit the @ref Tgstation.Server.Host.Models.CompileJob to the database
+
+If any of the above steps fail, the target directory is deleted and the deployment is considered a bust. If all went well, after the @ref Tgstation.Server.Host.Models.Job completes the new CompileJob is loaded into the instance's @ref Tgstation.Server.Host.Components.Compiler.IDmbFactory . 
+
+The DmbFactory is where the @ref arch_watchdog gets the @ref Tgstation.Server.Host.Components.Compiler.IDmbProvider instances to run. Each CompileJob loaded into it is given a lock count. The latest CompileJob holds 1 lock and every DreamDaemon instance running that CompileJob holds another. Loading a new CompileJob releases the initial lock, and when all other locks are released the CompileJob's directory is deleted. Any directories in the `Game` folder not in use are also deleted when the Instance starts. 
+
+@section arch_chat Chat Bot System
+
+The chat system is relatively simple. The @ref Tgstation.Server.Host.Components.Chat.IChat manager creates @ref Tgstation.Server.Host.Components.Chat.Providers.IProvider objects which do the IRC/Discord/etc messaging.
+
+The relationship between providers and the manager is a bit messy at the time of this writing (returned (im)mutable classes that map ids, to ids, to ids...) but it works.
+
+The few built in chat commands query the necessary components to get their results. Custom chat commands are routed to the watchdog's active server and then the response is relayed back. The message information passed to DM code is documented in the `tgs_chat_user` datum in the DMAPI: https://github.com/tgstation/tgstation-server/blob/master/src/DMAPI/tgs.dm#L117
+
+@section arch_static Static File Management
+
+This is largely just a remote file explorer. @ref Tgstation.Server.Host.Controllers send requests to @ref Tgstation.Server.Host.Components.StaticFiles.IConfiguration with an optional @ref Tgstation.Server.Host.Security.ISystemIdentity . If the identity is present @ref Tgstation.Server.Host.Security.ISystemIdentity.RunImpersonated is used to do the reading/writing, otherwise it is done as normal.
+
+The @ref Tgstation.Server.Host.Components.StaticFiles.IConfiguration object is also responsible for things like generating .dme modifications, symlinking static files during deployment, and running hook scripts.
+
 @section arch_watchdog Watchdog
 
-@subsection Communication
+This is the core of tgstation-server, the component that starts, monitors, and updates DreamDaemon. 
+
+At it's core, the watchdog operates using a hot/cold server setup. At any given moment there are two DreamDaemon instances running, only one of which players can see. If anything bad happens to that server, it is killed and the inactive server has its port changed to catch all the connections. If any changes need to be made to the configuration (port, security, compile job), the inactive server is killed and immediately relaunched with the new configuration. Whenever the active server reboots, the two servers change ports so as to minimize load times.
+
+General chat messages and verbose logs can be used to track watchdog state.
+
+That's a high level view of things, now let's get to the nitty gritty.
+
+@subsection arch_wd_launch Launch
+
+First the most recent @ref Tgstation.Server.Host.Components.Compiler.IDmbProvider is retrieved from the @ref Tgstation.Server.Host.Components.Compiler.IDmbFactory twice, adding 2 locks. 
+
+This is used to launch a @ref Tgstation.Server.Host.Components.Watchdog.ISessionController via the watchdog's @ref Tgstation.Server.Host.Components.Watchdog.ISessionControllerFactory in the `A` directory of dmb providers @ref Tgstation.Server.Host.Models.CompileJob . This will be designated the `Alpha` server. 
+
+Whenever DreamDaemon is launched by any part of the watchdog, we try to elevate its process priority to the equivalent of Windows' `Above Normal`
+
+10 seconds are allowed to pass, then the `Bravo` server is launched via the same method except in the `B` directory. We then wait for both servers to finish their initial startup lag and then designate `Alpha` as the `Active` server and pass it to the monitor. The active server will be told to close it's port on reboot
+
+If the watchdog ever enters a failure state it can't recover from, it kills both servers and reruns this process to restart.
+
+@subsection arch_wd_monitor The Monitor
+
+The monitor is responsible for handling every @ref Tgstation.Server.Host.Components.Watchdog.MonitorActivationReason . It sleeps until one of these things happen. If multiple things happen at once, they are processed in their order of declaration. 
+
+The monitor maintains a @ref Tgstation.Server.Host.Components.Watchdog.MonitorState which helps it make descisions on how to handle activation reasons. The @ref Tgstation.Server.Host.Components.Watchdog.MonitorState.NextAction determines how multiple simultaneous events are handled in succession.
+
+@subsubsection mar_activecrash Active Server Crashed/Exited
+
+If there was a graceful shutdown scheduled, exit the watchdog
+
+Otherwise, if the inactive server has critfailed or is still booting, restart the watchdog
+
+Otherwise, try to set the inactive server's port to the active server port and swap their designations. Failing that, restart the watchdog.
+
+Otherwise, attempt to reboot the once active now inactive server with the latest settings, failing that, mark it as critfailed.
+
+Stop processing further activation reasons
+
+@subsubsection mar_inactivecrash Inactive Server Crashed/Exited
+
+Attempt to reboot the inactive server with the latest settings, failing that, mark it as critfailed.
+
+@subsubsection mar_activereboot Active Server Rebooted
+
+Generally, at this point, the active server's port has been closed (unless it isn't for some reason) by the DMAPI
+
+If there was a graceful shutdown scheduled, exit the watchdog
+
+Otherwise, if the inactive server has critfailed or is still booting, restart the watchdog
+
+Otherwise, if the active server needs a new DMB, a graceful restart, or settings update, kill the active server
+
+If the port isn't closed, exit this activation reason. We wanted to keep it open for a reason.
+
+Try to set the inactive server's port to the active server port and swap their designations. Failing that, restart the watchdog.
+
+Set the current active server to close it's port on reboot.
+
+If we didnt kill the now inactive server and got here set it to NOT close it's port on reboot and try and set it's port to be the internal game port (The DMAPI and SessionController have a method for communicating a new port to open on even if it's closed)
+
+Failing the above case, or if we killed the now inactive server attempt to reboot it with the latest settings and stop processing other activation reasons, failing that, mark it as critfailed
+
+Otherwise, skip processing the @ref Tgstation.Server.Host.Components.Watchdog.onitorActivationReason.InactiveServerRebooted
+
+@subsubsection mar_inactivereboot Inactive Server Rebooted
+
+Mark the inactive server as rebooting, tell it to NOT close it's port on reboot.
+
+@subsubsection mar_inactivereboot Inactive Server Startup Complete
+
+Mark the inactive server as ready, tell it to close it's port on reboot.
+
+@subsubsection mar_setting New Dmb Available or Launch Settings Changed
+
+Attempt to reboot the inactive server with the latest settings, failing that, mark it as critfailed
+
+@subsection arch_reattach Reattaching
+
+When TGS reboots for an update or via admin command it keeps the child DreamDaemon processes alive and saves state information to regain control of them when it comes back online. The saved state is entered as the @ref Tgstation.Server.Host.Models.WatchdogReattachInformation for the instance in the database.
+
+When the instance comes online again it will automatically attempt to reattach if said information is present. The information is deleted as soon as it is loaded.
+
+If both servers reattached, the montor starts
+
+If the active server failed to reattach, the watchdog is restarted.
+
+If only the inactive server failed to reattach, it is mocked with a @ref Tgstation.Server.Host.Components.Watchdog.DeadSessionController. Which will immediately trigger @ref Tgstation.Server.Host.Components.Watchdog.MonitorActivationReason.InactiveServerCrashed . Then the monitor starts
+
+@subsection arch_comms Communication
+
+TGS => DM communication is achieved by sending packets which invoke `/world/Topic()` and DM can respond to those in kind. @ref Tgstation.Server.Host.Components.EventType and other command messages are communicated to the active server via this method.
+
+DM => TGS communication was initially meant to use `world.Export()` to access a controller and route the request to the specific instance. Due an issue with inherited handles (https://github.com/tgstation/tgstation-server/issues/71) that was causing problems but went undiagnosed for a long time, this idea was scrapped. It will soon return to replace the current implementation, however (https://github.com/tgstation/tgstation-server/issues/668).
+
+DM => TGS communication currently works like so:
+
+1. DM writes json to a specific file given in the initial launch json then enters a timeout sleep loop
+2. The Host watches for and reads the json
+3. The command is processed via the @ref Tgstation.Server.Host.Components.Interop.ICommHandler for the DreamDaemon instance
+4. The response is sent as a topic
+5. DM writes the topic response into a variable
+6. The sleep loop breaks when it reads this variable and returns the result
+
+Communication is necessary for the watchdog to flow things smoothly from a game client perspective. But realistically, due to how it works, the only absolutely required message is server reboots so it is known when to apply updates.
 
 @section arch_update Host Update Process
 
+There are actually two processes involved in a proper TGS setup. The actual @ref Tgstation.Server.Host .NET core application and the component that runs the @ref Tgstation.Server.Host.Watchdog (at this time, either @ref Tgstation.Server.Host.Service or @ref Tgstation.Server.Host.Console). The directory structure of the setup is like so:
+
+/app_directory
+  - /lib
+    - /Default
+	  - Initial Tgstation.Server.Host executable files
+	- Space for updates to be installed
+  - Host.Watchdog executable files
+
+The Host.Watchdog launches the Host with a designated path in the `/lib` folder. When the Host process wants to update, it extracts the new Host package to this directory and exits with code 1. The watchdog then attempts to rename the current `/lib/Default` directory to something unique, rename the update directory to `/lib/Default` and the launch the new Host process.
+
+The Host.Watchdog serves an additional purpose of automatically restarting the the Host in the case of a fatal crash (which should never happen, but the additional layer of safety is nice).
+
 */

src/DMAPI/tgs.dm

@@ -51,7 +51,21 @@
 #define TGS_EVENT_PORT_SWAP -2	//before a port change is about to happen, extra parameter is new port
 #define TGS_EVENT_REBOOT_MODE_CHANGE -1	//before a reboot mode change, extras parameters are the current and new reboot mode enums
 
-//TODO
+//See the descriptions for these codes here: https://github.com/tgstation/tgstation-server/blob/master/src/Tgstation.Server.Host/Components/EventType.cs
+#define TGS_EVENT_REPO_RESET_ORIGIN 0
+#define TGS_EVENT_REPO_CHECKOUT 1
+#define TGS_EVENT_REPO_FETCH 2
+#define TGS_EVENT_REPO_MERGE_PULL_REQUEST 3
+#define TGS_EVENT_REPO_PRE_SYNCHRONIZE 4
+#define TGS_EVENT_BYOND_INSTALL_START 5
+#define TGS_EVENT_BYOND_INSTALL_FAIL 6
+#define TGS_EVENT_BYOND_ACTIVE_VERSION_CHANGE 7
+#define TGS_EVENT_COMPILE_START 8
+#define TGS_EVENT_COMPILE_CANCELLED 9
+#define TGS_EVENT_COMPILE_FAILURE 10
+#define TGS_EVENT_COMPILE_COMPLETE 11
+#define TGS_EVENT_INSTANCE_AUTO_UPDATE_START 12
+#define TGS_EVENT_REPO_MERGE_CONFLICT 13
 
 //OTHER ENUMS
 

src/Tgstation.Server.Api/Tgstation.Server.Api.csproj

@@ -16,8 +16,8 @@
     <AssemblyVersion>4.0.0.0</AssemblyVersion>
     <FileVersion>4.0.0.0</FileVersion>
     <PackageTags>json web api tgstation-server tgstation ss13 byond</PackageTags>
-    <PackageReleaseNotes>Prototype release</PackageReleaseNotes>
-    <Version>4.0.0.0-preview6007</Version>
+    <PackageReleaseNotes>Initial release</PackageReleaseNotes>
+    <Version>4.0.0.0</Version>
   </PropertyGroup>
 
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
@@ -35,7 +35,7 @@
 
   <ItemGroup>
     <PackageReference Include="Microsoft.AspNetCore.Http.Extensions" Version="2.1.1" />
-    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.1">
+    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.2">
       <PrivateAssets>all</PrivateAssets>
       <IncludeAssets>compile; build; native; contentfiles; analyzers</IncludeAssets>
     </PackageReference>

src/Tgstation.Server.Client/Tgstation.Server.Client.csproj

@@ -3,7 +3,7 @@
   <PropertyGroup>
     <TargetFramework>netstandard2.0</TargetFramework>
     <DebugType>Full</DebugType>
-    <Version>4.0.0.0-preview9116</Version>
+    <Version>4.0.0.0</Version>
     <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
     <Authors>Cyberboss</Authors>
     <Company>/tg/station 13</Company>
@@ -16,7 +16,8 @@
     <PackageIconUrl>https://raw.githubusercontent.com/tgstation/tgstation-server/master/build/tgs.ico</PackageIconUrl>
     <PackageLicenseUrl>https://github.com/tgstation/tgstation-server/blob/master/LICENSE</PackageLicenseUrl>
     <PackageTags>json web api tgstation-server tgstation ss13 byond client</PackageTags>
-    <PackageReleaseNotes>Prototype release</PackageReleaseNotes>
+    <PackageReleaseNotes>Initial release</PackageReleaseNotes>
+    <Copyright>2018</Copyright>
   </PropertyGroup>
 
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
@@ -32,7 +33,7 @@
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.1">
+    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.2">
       <PrivateAssets>all</PrivateAssets>
       <IncludeAssets>compile; build; native; contentfiles; analyzers</IncludeAssets>
     </PackageReference>

src/Tgstation.Server.Host.Console/Properties/launchSettings.json

@@ -1,12 +1,4 @@
 {
-  "iisSettings": {
-    "windowsAuthentication": false,
-    "anonymousAuthentication": true,
-    "iisExpress": {
-      "applicationUrl": "http://localhost:51882/",
-      "sslPort": 0
-    }
-  },
   "profiles": {
     "Tgstation.Server.Host.Console": {
       "commandName": "Project",
@@ -15,8 +7,7 @@
       "launchBrowser": true,
       "environmentVariables": {
         "ASPNETCORE_ENVIRONMENT": "Development"
-      },
-      "applicationUrl": "http://localhost:51885/"
+      }
     }
   }
 }

src/Tgstation.Server.Host.Console/Tgstation.Server.Host.Console.csproj

@@ -5,6 +5,7 @@
     <TargetFramework>netcoreapp2.1</TargetFramework>
     <DebugType>Full</DebugType>
     <FileVersion>4.0.0.0</FileVersion>
+    <AssemblyVersion>4.0.0.0</AssemblyVersion>
   </PropertyGroup>
 
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
@@ -19,7 +20,7 @@
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.1">
+    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.2">
       <PrivateAssets>all</PrivateAssets>
       <IncludeAssets>compile; build; native; contentfiles; analyzers</IncludeAssets>
     </PackageReference>

src/Tgstation.Server.Host.Service/Program.cs

@@ -4,7 +4,6 @@ using System;
 using System.Collections.Specialized;
 using System.Configuration.Install;
 using System.Diagnostics;
-using System.Diagnostics.CodeAnalysis;
 using System.Globalization;
 using System.Linq;
 using System.Reflection;
@@ -18,8 +17,7 @@ namespace Tgstation.Server.Host.Service
 	/// <summary>
 	/// Contains the entrypoint for the application
 	/// </summary>
-	[ExcludeFromCodeCoverage]
-	class Program
+	sealed class Program
 	{
 		/// <summary>
 		/// The --uninstall or -u option
@@ -33,6 +31,18 @@ namespace Tgstation.Server.Host.Service
 		[Option(ShortName = "i")]
 		public bool Install { get; set; }
 
+		/// <summary>
+		/// The --trace or -t option. Enables trace logs
+		/// </summary>
+		[Option(ShortName = "t")]
+		public bool Trace { get; set; }
+
+		/// <summary>
+		/// The --debug or -d option. Enables debug logs
+		/// </summary>
+		[Option(ShortName = "d")]
+		public bool Debug { get; set; }
+
 		/// <summary>
 		/// Check if the running user is a system administrator
 		/// </summary>
@@ -49,32 +59,29 @@ namespace Tgstation.Server.Host.Service
 		/// </summary>
 		public void OnExecute()
 		{
-			if (Environment.UserInteractive)
+			if (Environment.UserInteractive && !IsAdministrator())
 			{
-				if (!IsAdministrator())
+				if (!Install && !Uninstall)
 				{
-					if (!(Install || Uninstall))
-					{
-						var result = MessageBox.Show("You are running the TGS windows service executable directly. It should only be run by the service control manager. Would you like to install the service in this location?", "TGS Service", MessageBoxButtons.YesNo);
-						if (result == DialogResult.No)
-							return;
-						Install = true;
-					}
-
-					//try to restart as admin
-					//its windows, first arg is .exe name guaranteed
-					var exe = Environment.GetCommandLineArgs().First();
-					var startInfo = new ProcessStartInfo
-					{
-						UseShellExecute = true,
-						Verb = "runas",
-						Arguments = Install ? "-i" : "-u",
-						FileName = exe,
-						WorkingDirectory = Environment.CurrentDirectory,
-					};
-					using (Process.Start(startInfo))
+					var result = MessageBox.Show("You are running the TGS windows service executable directly. It should only be run by the service control manager. Would you like to install the service in this location?", "TGS Service", MessageBoxButtons.YesNo);
+					if (result != DialogResult.Yes)
 						return;
+					Install = true;
 				}
+
+				//try to restart as admin
+				//its windows, first arg is .exe name guaranteed
+				var exe = Environment.GetCommandLineArgs().First();
+				var startInfo = new ProcessStartInfo
+				{
+					UseShellExecute = true,
+					Verb = "runas",
+					Arguments = Install ? "-i" : "-u",
+					FileName = exe,
+					WorkingDirectory = Environment.CurrentDirectory,
+				};
+				using (Process.Start(startInfo))
+					return;
 			}
 
 			if (Install)
@@ -109,7 +116,7 @@ namespace Tgstation.Server.Host.Service
 				}
 			else
 				using (var loggerFactory = new LoggerFactory())
-					ServiceBase.Run(new ServerService(new WatchdogFactory(), loggerFactory));
+					ServiceBase.Run(new ServerService(new WatchdogFactory(), loggerFactory, Trace ? LogLevel.Trace : Debug ? LogLevel.Debug : LogLevel.Information));
 		}
 
 		/// <summary>

src/Tgstation.Server.Host.Service/ServerService.cs

@@ -41,7 +41,8 @@ namespace Tgstation.Server.Host.Service
 		/// </summary>
 		/// <param name="watchdogFactory">The <see cref="IWatchdogFactory"/> to create <see cref="watchdog"/> with</param>
 		/// <param name="loggerFactory">The <see cref="ILoggerFactory"/> for <paramref name="watchdogFactory"/></param>
-		public ServerService(IWatchdogFactory watchdogFactory, ILoggerFactory loggerFactory)
+		/// <param name="minumumLogLevel">The minimum <see cref="Microsoft.Extensions.Logging.LogLevel"/> to record in the event log</param>
+		public ServerService(IWatchdogFactory watchdogFactory, ILoggerFactory loggerFactory, LogLevel minumumLogLevel)
 		{
 			if (watchdogFactory == null)
 				throw new ArgumentNullException(nameof(watchdogFactory));
@@ -50,7 +51,8 @@ namespace Tgstation.Server.Host.Service
 
 			loggerFactory.AddEventLog(new EventLogSettings
 			{
-				EventLog = this
+				EventLog = this,
+				Filter = (message, logLevel) => logLevel >= minumumLogLevel
 			});
 
 			ServiceName = Name;

src/Tgstation.Server.Host.Service/Tgstation.Server.Host.Service.csproj

@@ -94,7 +94,7 @@
     <PackageReference Include="McMaster.Extensions.CommandLineUtils">
       <Version>2.2.5</Version>
     </PackageReference>
-    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.1">
+    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.2">
       <PrivateAssets>all</PrivateAssets>
       <IncludeAssets>compile; build; native; contentfiles; analyzers</IncludeAssets>
     </PackageReference>

src/Tgstation.Server.Host.Watchdog/Tgstation.Server.Host.Watchdog.csproj

@@ -20,7 +20,7 @@
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.1">
+    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.2">
       <PrivateAssets>all</PrivateAssets>
       <IncludeAssets>compile; build; native; contentfiles; analyzers</IncludeAssets>
     </PackageReference>

src/Tgstation.Server.Host.Watchdog/Watchdog.cs

@@ -34,7 +34,7 @@ namespace Tgstation.Server.Host.Watchdog
 		public async Task RunAsync(string[] args, CancellationToken cancellationToken)
 		{
 			logger.LogInformation("Host watchdog starting...");
-
+			logger.LogDebug("PID: {0}", Process.GetCurrentProcess().Id);
 			string updateDirectory = null;
 			try
 			{
@@ -228,7 +228,7 @@ namespace Tgstation.Server.Host.Watchdog
 							if (isWindows)
 							{
 								//windows dick sucking resource unlocking
-								GC.Collect();
+								GC.Collect(Int32.MaxValue, GCCollectionMode.Default, true);
 								await Task.Delay(TimeSpan.FromSeconds(5), cancellationToken).ConfigureAwait(false);
 							}
 							var tempPath = Path.Combine(assemblyStoragePath, Guid.NewGuid().ToString());

src/Tgstation.Server.Host.Watchdog/WatchdogFactory.cs

@@ -1,7 +1,5 @@
 using Microsoft.Extensions.Logging;
 using System;
-using System.Diagnostics.CodeAnalysis;
-using System.Runtime.InteropServices;
 
 namespace Tgstation.Server.Host.Watchdog
 {
@@ -9,7 +7,6 @@ namespace Tgstation.Server.Host.Watchdog
 	public sealed class WatchdogFactory : IWatchdogFactory
 	{
 		/// <inheritdoc />
-		[ExcludeFromCodeCoverage]
 		public IWatchdog CreateWatchdog(ILoggerFactory loggerFactory) => new Watchdog(loggerFactory?.CreateLogger<Watchdog>() ?? throw new ArgumentNullException(nameof(loggerFactory)));
 	}
 }

src/Tgstation.Server.Host/Components/Byond/ByondManager.cs

@@ -20,7 +20,13 @@ namespace Tgstation.Server.Host.Components.Byond
 		/// </summary>
 		public const string BinPath = "byond/bin";
 
+		/// <summary>
+		/// The file in which we store the <see cref="VersionKey(Version)"/> for installations
+		/// </summary>
 		const string VersionFileName = "Version.txt";
+		/// <summary>
+		/// The file in which we store the <see cref="VersionKey(Version)"/> for the active installation
+		/// </summary>
 		const string ActiveVersionFileName = "ActiveVersion.txt";
 
 		/// <inheritdoc />
@@ -46,6 +52,11 @@ namespace Tgstation.Server.Host.Components.Byond
 		/// </summary>
 		readonly IByondInstaller byondInstaller;
 
+		/// <summary>
+		/// The <see cref="IEventConsumer"/> for the <see cref="ByondManager"/>
+		/// </summary>
+		readonly IEventConsumer eventConsumer;
+
 		/// <summary>
 		/// The <see cref="ILogger"/> for the <see cref="ByondManager"/>
 		/// </summary>
@@ -61,6 +72,11 @@ namespace Tgstation.Server.Host.Components.Byond
 		/// </summary>
 		readonly SemaphoreSlim semaphore;
 
+		/// <summary>
+		/// Converts a BYOND <paramref name="version"/> to a <see cref="string"/>
+		/// </summary>
+		/// <param name="version">The <see cref="Version"/> to convert</param>
+		/// <returns>The <see cref="string"/> representation of <paramref name="version"/></returns>
 		static string VersionKey(Version version) => new Version(version.Major, version.Minor).ToString();
 
 		/// <summary>
@@ -68,11 +84,13 @@ namespace Tgstation.Server.Host.Components.Byond
 		/// </summary>
 		/// <param name="ioManager">The value of <see cref="ioManager"/></param>
 		/// <param name="byondInstaller">The value of <see cref="byondInstaller"/></param>
+		/// <param name="eventConsumer">The value of <see cref="eventConsumer"/></param>
 		/// <param name="logger">The value of <see cref="logger"/></param>
-		public ByondManager(IIOManager ioManager, IByondInstaller byondInstaller, ILogger<ByondManager> logger)
+		public ByondManager(IIOManager ioManager, IByondInstaller byondInstaller, IEventConsumer eventConsumer, ILogger<ByondManager> logger)
 		{
 			this.ioManager = ioManager ?? throw new ArgumentNullException(nameof(ioManager));
 			this.byondInstaller = byondInstaller ?? throw new ArgumentNullException(nameof(byondInstaller));
+			this.eventConsumer = eventConsumer ?? throw new ArgumentNullException(nameof(eventConsumer));
 			this.logger = logger ?? throw new ArgumentNullException(nameof(logger));
 
 			installedVersions = new Dictionary<string, Task>();
@@ -108,11 +126,12 @@ namespace Tgstation.Server.Host.Components.Byond
 					cancellationToken.ThrowIfCancellationRequested();
 					return;
 				}
+			//okay up to us to install it then
 			try
 			{
+				await eventConsumer.HandleEvent(EventType.ByondInstallStart, new List<string> { versionKey }, cancellationToken).ConfigureAwait(false);
 				var downloadTask = byondInstaller.DownloadVersion(version, cancellationToken);
 
-				//okay up to us to install it then
 				await ioManager.DeleteDirectory(versionKey, cancellationToken).ConfigureAwait(false);
 				await ioManager.CreateDirectory(versionKey, cancellationToken).ConfigureAwait(false);
 
@@ -144,6 +163,8 @@ namespace Tgstation.Server.Host.Components.Byond
 			}
 			catch (Exception e)
 			{
+				if (!(e is OperationCanceledException))
+					await eventConsumer.HandleEvent(EventType.ByondInstallFail, new List<string> { e.Message }, cancellationToken).ConfigureAwait(false);
 				lock (installedVersions)
 					installedVersions.Remove(versionKey);
 				ourTcs.SetException(e);
@@ -154,10 +175,14 @@ namespace Tgstation.Server.Host.Components.Byond
 		/// <inheritdoc />
 		public async Task ChangeVersion(Version version, CancellationToken cancellationToken)
 		{
+			if (version == null)
+				throw new ArgumentNullException(nameof(version));
+			var versionKey = VersionKey(version);
 			await InstallVersion(version, cancellationToken).ConfigureAwait(false);
 			using (await SemaphoreSlimContext.Lock(semaphore, cancellationToken).ConfigureAwait(false))
 			{
-				await ioManager.WriteAllBytes(ActiveVersionFileName, Encoding.UTF8.GetBytes(version.ToString()), cancellationToken).ConfigureAwait(false);
+				await ioManager.WriteAllBytes(ActiveVersionFileName, Encoding.UTF8.GetBytes(versionKey), cancellationToken).ConfigureAwait(false);
+				await eventConsumer.HandleEvent(EventType.ByondActiveVersionChange, new List<string> { ActiveVersion != null ? VersionKey(ActiveVersion) : null, versionKey }, cancellationToken).ConfigureAwait(false);
 				ActiveVersion = version;
 			}
 		}

src/Tgstation.Server.Host/Components/Byond/PosixByondInstaller.cs

@@ -24,11 +24,15 @@ namespace Tgstation.Server.Host.Components.Byond
 		/// </summary>
 		const string ByondCachePath = "~/.byond/cache";
 
-		/// <inheritdoc />
-		public string DreamDaemonName => "DreamDaemon.sh";
+		const string DreamDaemonExecutableName = "DreamDaemon";
+		const string DreamMakerExecutableName = "DreamMaker";
+		const string ShellScriptExtension = ".sh";
 
 		/// <inheritdoc />
-		public string DreamMakerName => "DreamMaker.sh";
+		public string DreamDaemonName => DreamDaemonExecutableName + ShellScriptExtension;
+
+		/// <inheritdoc />
+		public string DreamMakerName => DreamMakerExecutableName + ShellScriptExtension;
 
 		/// <summary>
 		/// The <see cref="IIOManager"/> for the <see cref="PosixByondInstaller"/>
@@ -65,6 +69,10 @@ namespace Tgstation.Server.Host.Components.Byond
 			{
 				await ioManager.DeleteDirectory(ByondCachePath, cancellationToken).ConfigureAwait(false);
 			}
+			catch (OperationCanceledException)
+			{
+				throw;
+			}
 			catch (Exception e)
 			{
 				logger.LogWarning("Error deleting BYOND cache! Exception: {0}", e);
@@ -103,9 +111,6 @@ namespace Tgstation.Server.Host.Components.Byond
 			//need to add $ORIGIN to LD_LIBRARY_PATH
 			const string StandardScript = "#!/bin/sh\nexport LD_LIBRARY_PATH=\"\\$ORIGIN:$LD_LIBRARY_PATH\"\nBASEDIR=$(dirname \"$0\")\nexec \"$BASEDIR/{0}\" \"$@\"\n";
 
-			const string DreamDaemonExecutableName = "DreamDaemon";
-			const string DreamMakerExecutableName = "DreamMaker";
-
 			var dreamDaemonScript = String.Format(CultureInfo.InvariantCulture, StandardScript, DreamDaemonExecutableName);
 			var dreamMakerScript = String.Format(CultureInfo.InvariantCulture, StandardScript, DreamMakerExecutableName);
 

src/Tgstation.Server.Host/Components/Byond/WindowsByondInstaller.cs

@@ -92,6 +92,10 @@ namespace Tgstation.Server.Host.Components.Byond
 			{
 				await ioManager.DeleteDirectory(ioManager.ConcatPath(Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments), "byond/cache"), cancellationToken).ConfigureAwait(false);
 			}
+			catch(OperationCanceledException)
+			{
+				throw;
+			}
 			catch (Exception e)
 			{
 				logger.LogWarning("Error deleting BYOND cache! Exception: {0}", e);
@@ -121,16 +125,26 @@ namespace Tgstation.Server.Host.Components.Byond
 			//after this version lummox made DD depend of directx lol
 			if (version.Major >= 512 && version.Minor >= 1427 && !installedDirectX)
 				using (await SemaphoreSlimContext.Lock(semaphore, cancellationToken).ConfigureAwait(false))
+					//check again because race conditions
 					if (!installedDirectX)
 					{
 						//always install it, it's pretty fast and will do better redundancy checking than us
 						var rbdx = ioManager.ConcatPath(path, ByondDXDir);
 						//noShellExecute because we aren't doing runas shennanigans
-						using (var p = processExecutor.LaunchProcess(ioManager.ConcatPath(rbdx, "DXSETUP.exe"), rbdx, "/silent", noShellExecute: true))
+						IProcess directXInstaller;
+						try
+						{
+							directXInstaller = processExecutor.LaunchProcess(ioManager.ConcatPath(rbdx, "DXSETUP.exe"), rbdx, "/silent", noShellExecute: true);
+						}
+						catch (Exception e)
+						{
+							throw new JobException("Unable to start DirectX installer process! Is the server running with admin privileges?", e);
+						}
+						using (directXInstaller)
 						{
 							int exitCode;
-							using (cancellationToken.Register(() => p.Terminate()))
-								exitCode = await p.Lifetime.ConfigureAwait(false);
+							using (cancellationToken.Register(() => directXInstaller.Terminate()))
+								exitCode = await directXInstaller.Lifetime.ConfigureAwait(false);
 							cancellationToken.ThrowIfCancellationRequested();
 
 							if (exitCode != 0)

src/Tgstation.Server.Host/Components/Chat/Chat.cs

@@ -305,6 +305,10 @@ namespace Tgstation.Server.Host.Components.Chat
 				if (result != null)
 					await SendMessage(result, new List<ulong> { message.User.Channel.RealId }, cancellationToken).ConfigureAwait(false);
 			}
+			catch (OperationCanceledException)
+			{
+				throw;
+			}
 			catch (Exception e)
 			{
 				//error bc custom commands should reply about why it failed

src/Tgstation.Server.Host/Components/Chat/Commands/ByondCommand.cs

@@ -1,8 +1,10 @@
 using System;
 using System.Globalization;
+using System.Linq;
 using System.Threading;
 using System.Threading.Tasks;
 using Tgstation.Server.Host.Components.Byond;
+using Tgstation.Server.Host.Components.Watchdog;
 
 namespace Tgstation.Server.Host.Components.Chat.Commands
 {
@@ -15,26 +17,40 @@ namespace Tgstation.Server.Host.Components.Chat.Commands
 		public string Name => "byond";
 
 		/// <inheritdoc />
-		public string HelpText => "Displays the active Byond version";
+		public string HelpText => "Displays the running Byond version. Use --active for the version used in future deployments";
 
 		/// <inheritdoc />
 		public bool AdminOnly => false;
 
 		/// <summary>
-		/// the <see cref="IByondManager"/> for the <see cref="ByondCommand"/>
+		/// The <see cref="IByondManager"/> for the <see cref="ByondCommand"/>
 		/// </summary>
 		readonly IByondManager byondManager;
 
+		/// <summary>
+		/// The <see cref="IWatchdog"/> for the <see cref="ByondCommand"/>
+		/// </summary>
+		readonly IWatchdog watchdog;
+
 		/// <summary>
 		/// Construct a <see cref="ByondCommand"/>
 		/// </summary>
 		/// <param name="byondManager">The value of <see cref="byondManager"/></param>
-		public ByondCommand(IByondManager byondManager)
+		/// <param name="watchdog">The value of <see cref="watchdog"/></param>
+		public ByondCommand(IByondManager byondManager, IWatchdog watchdog)
 		{
 			this.byondManager = byondManager ?? throw new ArgumentNullException(nameof(byondManager));
+			this.watchdog = watchdog ?? throw new ArgumentNullException(nameof(watchdog));
 		}
 
 		/// <inheritdoc />
-		public Task<string> Invoke(string arguments, User user, CancellationToken cancellationToken) => Task.FromResult(byondManager.ActiveVersion == null ? "None!" : String.Format(CultureInfo.InvariantCulture, "{0}.{1}", byondManager.ActiveVersion.Major, byondManager.ActiveVersion.Minor));
+		public Task<string> Invoke(string arguments, User user, CancellationToken cancellationToken)
+		{
+			if (arguments.Split(' ').Any(x => x.ToUpperInvariant() == "--ACTIVE"))
+				return Task.FromResult(byondManager.ActiveVersion == null ? "None!" : String.Format(CultureInfo.InvariantCulture, "{0}.{1}", byondManager.ActiveVersion.Major, byondManager.ActiveVersion.Minor));
+			if (!watchdog.Running)
+				return Task.FromResult("Server offline!");
+			return Task.FromResult(watchdog.ActiveCompileJob.ByondVersion);
+		}
 	}
 }

src/Tgstation.Server.Host/Components/Chat/Commands/CommandFactory.cs

@@ -76,7 +76,7 @@ namespace Tgstation.Server.Host.Components.Chat.Commands
 			return new List<ICommand>
 			{
 				new VersionCommand(application),
-				new ByondCommand(byondManager),
+				new ByondCommand(byondManager, watchdog),
 				new RevisionCommand(watchdog, repositoryManager, instance),
 				new PullRequestsCommand(watchdog, repositoryManager, databaseContextFactory, instance),
 				new KekCommand()

src/Tgstation.Server.Host/Components/Chat/JsonTrackingContext.cs

@@ -82,6 +82,10 @@ namespace Tgstation.Server.Host.Components.Chat
 					return result;
 				}
 			}
+			catch (OperationCanceledException)
+			{
+				throw;
+			}
 			catch (Exception e)
 			{
 				logger.LogWarning("Error retrieving custom commands! Exception: {0}", e);

src/Tgstation.Server.Host/Components/Chat/Providers/DiscordProvider.cs

@@ -128,6 +128,10 @@ namespace Tgstation.Server.Host.Components.Chat.Providers
 				using (cancellationToken.Register(() => channelsAvailable.SetCanceled()))
 					await channelsAvailable.Task.ConfigureAwait(false);
 			}
+			catch (OperationCanceledException)
+			{
+				throw;
+			}
 			catch (Exception e)
 			{
 				logger.LogWarning("Error connecting to Discord: {0}", e);
@@ -148,6 +152,10 @@ namespace Tgstation.Server.Host.Components.Chat.Providers
 				cancellationToken.ThrowIfCancellationRequested();
 				await client.LogoutAsync().ConfigureAwait(false);
 			}
+			catch (OperationCanceledException)
+			{
+				throw;
+			}
 			catch (Exception e)
 			{
 				logger.LogWarning("Error disconnecting from discord: {0}", e);
@@ -204,6 +212,10 @@ namespace Tgstation.Server.Host.Components.Chat.Providers
 					CancelToken = cancellationToken
 				}) ?? Task.CompletedTask).ConfigureAwait(false);
 			}
+			catch (OperationCanceledException)
+			{
+				throw;
+			}
 			catch (Exception e)
 			{
 				logger.LogWarning("Error sending discord message: {0}", e);

src/Tgstation.Server.Host/Components/Chat/Providers/IrcProvider.cs

@@ -304,9 +304,14 @@ namespace Tgstation.Server.Host.Components.Chat.Providers
 						client.Listen();
 					}, cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Current);
 				}
+				catch (OperationCanceledException)
+				{
+					throw;
+				}
 				catch (Exception e)
 				{
 					logger.LogWarning("Unable to connect to IRC: {0}", e);
+					return false;
 				}
 			return true;
 		}, cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Current);
@@ -332,6 +337,10 @@ namespace Tgstation.Server.Host.Components.Chat.Providers
 				Dispose();
 				await listenTask.ConfigureAwait(false);
 			}
+			catch (OperationCanceledException)
+			{
+				throw;
+			}
 			catch (Exception e)
 			{
 				logger.LogWarning("Error disconnecting from IRC! Exception: {0}", e);

src/Tgstation.Server.Host/Components/Compiler/DmbFactory.cs

@@ -264,6 +264,10 @@ namespace Tgstation.Server.Host.Components.Compiler
 					++deleting;
 					await ioManager.DeleteDirectory(x, cancellationToken).ConfigureAwait(false);
 				}
+				catch (OperationCanceledException)
+				{
+					throw;
+				}
 				catch (Exception e)
 				{
 					logger.LogWarning("Error deleting directory {0}! Exception: {1}", x, e);

src/Tgstation.Server.Host/Components/Compiler/DreamMaker.cs

@@ -317,7 +317,7 @@ namespace Tgstation.Server.Host.Components.Compiler
 					{
 						for (var I = 0; I < 99; ++I)
 						{
-							await Task.Delay(sleepInterval, cancellationToken).ConfigureAwait(false);
+							await Task.Delay(sleepInterval, progressCts.Token).ConfigureAwait(false);
 							progressReporter(I + 1);
 						}
 					}

src/Tgstation.Server.Host/Components/EventType.cs

@@ -1,7 +1,7 @@
 namespace Tgstation.Server.Host.Components
 {
 	/// <summary>
-	/// Types of events 
+	/// Types of events. Mirror in tgs.dm
 	/// </summary>
 	public enum EventType
 	{
@@ -18,7 +18,7 @@
 		/// </summary>
 		RepoFetch = 2,
 		/// <summary>
-		/// Parameters: Pull request number, pull request sha, merger name, merger message
+		/// Parameters: Pull request number, pull request sha, merger message
 		/// </summary>
 		RepoMergePullRequest = 3,
 		/// <summary>
@@ -27,17 +27,17 @@
 		RepoPreSynchronize = 4,
 
 		/// <summary>
-		/// Parameters: Current version, new version
+		/// Parameters: Version being installed
 		/// </summary>
-		ByondChangeStart = 5,
+		ByondInstallStart = 5,
 		/// <summary>
 		/// Parameters: Error string
 		/// </summary>
-		ByondFail = 6,
+		ByondInstallFail = 6,
 		/// <summary>
-		/// No parameters
+		/// Parameters: Old active version, new active version
 		/// </summary>
-		ByondChangeComplete = 7,
+		ByondActiveVersionChange = 7,
 		/// <summary>
 		/// Parameters: Game directory path, origin commit sha
 		/// </summary>
@@ -55,23 +55,14 @@
 		/// </summary>
 		CompileComplete = 11,
 
-		/// <summary>
-		/// Parameters: Exit code
-		/// </summary>
-		DDOtherCrash = 12,
 		/// <summary>
 		/// No parameters
 		/// </summary>
-		DDOtherExit = 13,
-
-		/// <summary>
-		/// No parameters
-		/// </summary>
-		InstanceAutoUpdateStart = 14,
+		InstanceAutoUpdateStart = 12,
 
 		/// <summary>
 		/// Parameters: Base sha, target sha, base reference, target reference
 		/// </summary>
-		RepoMergeConflict = 15,
+		RepoMergeConflict = 13,
 	}
 }

src/Tgstation.Server.Host/Components/Instance.cs

@@ -55,6 +55,11 @@ namespace Tgstation.Server.Host.Components
 		/// </summary>
 		readonly IJobManager jobManager;
 
+		/// <summary>
+		/// The <see cref="IEventConsumer"/> for the <see cref="Instance"/>
+		/// </summary>
+		readonly IEventConsumer eventConsumer;
+
 		/// <summary>
 		/// The <see cref="ILogger"/> for the <see cref="Instance"/>
 		/// </summary>
@@ -88,8 +93,9 @@ namespace Tgstation.Server.Host.Components
 		/// <param name="databaseContextFactory">The value of <see cref="databaseContextFactory"/></param>
 		/// <param name="dmbFactory">The value of <see cref="dmbFactory"/></param>
 		/// <param name="jobManager">The value of <see cref="jobManager"/></param>
+		/// <param name="eventConsumer">The value of <see cref="eventConsumer"/></param>
 		/// <param name="logger">The value of <see cref="logger"/></param>
-		public Instance(Api.Models.Instance metadata, IRepositoryManager repositoryManager, IByondManager byondManager, IDreamMaker dreamMaker, IWatchdog watchdog, IChat chat, StaticFiles.IConfiguration configuration, ICompileJobConsumer compileJobConsumer, IDatabaseContextFactory databaseContextFactory, IDmbFactory dmbFactory, IJobManager jobManager, ILogger<Instance> logger)
+		public Instance(Api.Models.Instance metadata, IRepositoryManager repositoryManager, IByondManager byondManager, IDreamMaker dreamMaker, IWatchdog watchdog, IChat chat, StaticFiles.IConfiguration configuration, ICompileJobConsumer compileJobConsumer, IDatabaseContextFactory databaseContextFactory, IDmbFactory dmbFactory, IJobManager jobManager, IEventConsumer eventConsumer, ILogger<Instance> logger)
 		{
 			this.metadata = metadata ?? throw new ArgumentNullException(nameof(metadata));
 			RepositoryManager = repositoryManager ?? throw new ArgumentNullException(nameof(repositoryManager));
@@ -102,6 +108,7 @@ namespace Tgstation.Server.Host.Components
 			this.databaseContextFactory = databaseContextFactory ?? throw new ArgumentNullException(nameof(databaseContextFactory));
 			this.dmbFactory = dmbFactory ?? throw new ArgumentNullException(nameof(dmbFactory));
 			this.jobManager = jobManager ?? throw new ArgumentNullException(nameof(jobManager));
+			this.eventConsumer = eventConsumer ?? throw new ArgumentNullException(nameof(eventConsumer));
 			this.logger = logger ?? throw new ArgumentNullException(nameof(logger));
 		}
 
@@ -135,7 +142,6 @@ namespace Tgstation.Server.Host.Components
 			var compileJobsTask = databaseContext.CompileJobs
 				.Where(x => x.Job.Instance.Id == metadata.Id)
 				.OrderByDescending(x => x.Job.StoppedAt)
-				.Include(x => x.Job)
 				.Select(x => x.Job.StoppedAt.Value - x.Job.StartedAt.Value)
 				.Take(10)
 				.ToListAsync(cancellationToken);
@@ -205,10 +211,11 @@ namespace Tgstation.Server.Host.Components
 				{
 					await Task.Delay(TimeSpan.FromMinutes(minutes > Int32.MaxValue ? Int32.MaxValue : (int)minutes), cancellationToken).ConfigureAwait(false);
 					logger.LogDebug("Beginning auto update...");
+					await eventConsumer.HandleEvent(EventType.InstanceAutoUpdateStart, new List<string>(), cancellationToken).ConfigureAwait(false);
 					try
 					{
 						Models.User user = null;
-						await databaseContextFactory.UseContext(async (db) => user = await db.Users.FirstAsync(cancellationToken).ConfigureAwait(false)).ConfigureAwait(false);
+						await databaseContextFactory.UseContext(async (db) => user = await db.Users.Where(x => x.CanonicalName == Api.Models.User.AdminName.ToUpperInvariant()).FirstAsync(cancellationToken).ConfigureAwait(false)).ConfigureAwait(false);
 						var repositoryUpdateJob = new Job
 						{
 							Instance = new Models.Instance

src/Tgstation.Server.Host/Components/InstanceFactory.cs

@@ -158,7 +158,7 @@ namespace Tgstation.Server.Host.Components
 				var repoManager = new RepositoryManager(metadata.RepositorySettings, repoIoManager, eventConsumer, credentialsProvider, loggerFactory.CreateLogger<Repository.Repository>(), loggerFactory.CreateLogger<RepositoryManager>());
 				try
 				{
-					var byond = new ByondManager(byondIOManager, byondInstaller, loggerFactory.CreateLogger<ByondManager>());
+					var byond = new ByondManager(byondIOManager, byondInstaller, eventConsumer, loggerFactory.CreateLogger<ByondManager>());
 
 					var commandFactory = new CommandFactory(application, byond, repoManager, databaseContextFactory, metadata);
 
@@ -174,7 +174,7 @@ namespace Tgstation.Server.Host.Components
 						{
 							var dreamMaker = new DreamMaker(byond, gameIoManager, configuration, sessionControllerFactory, dmbFactory, application, eventConsumer, chat, processExecutor, watchdog, loggerFactory.CreateLogger<DreamMaker>());
 
-							return new Instance(metadata.CloneMetadata(), repoManager, byond, dreamMaker, watchdog, chat, configuration, dmbFactory, databaseContextFactory, dmbFactory, jobManager, loggerFactory.CreateLogger<Instance>());
+							return new Instance(metadata.CloneMetadata(), repoManager, byond, dreamMaker, watchdog, chat, configuration, dmbFactory, databaseContextFactory, dmbFactory, jobManager, eventConsumer, loggerFactory.CreateLogger<Instance>());
 						}
 						catch
 						{

src/Tgstation.Server.Host/Components/Interop/CommContext.cs

@@ -121,6 +121,7 @@ namespace Tgstation.Server.Host.Components.Interop
 
 				await (handler?.HandleInterop(command, cancellationToken) ?? Task.CompletedTask).ConfigureAwait(false);
 			}
+			catch (OperationCanceledException) { }
 			catch (Exception ex)
 			{
 				logger.LogDebug("Exception while trying to handle command json write: {0}", ex);

src/Tgstation.Server.Host/Components/Repository/Repository.cs

@@ -299,6 +299,8 @@ namespace Tgstation.Server.Host.Components.Repository
 				}), cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Current).ConfigureAwait(false);
 			}
 
+			await eventConsumer.HandleEvent(EventType.RepoMergePullRequest, new List<string> { testMergeParameters.Number.ToString(), testMergeParameters.PullRequestRevision, testMergeParameters.Comment }, cancellationToken).ConfigureAwait(false);
+
 			return result.Status != MergeStatus.NonFastForward;
 		}
 

src/Tgstation.Server.Host/Components/StaticFiles/Configuration.cs

@@ -112,29 +112,21 @@ namespace Tgstation.Server.Host.Components.StaticFiles
 				var dmeExistsTask = ioManager.FileExists(ioManager.ConcatPath(CodeModificationsSubdirectory, dmeFile), cancellationToken);
 				var headFileExistsTask = ioManager.FileExists(ioManager.ConcatPath(CodeModificationsSubdirectory, CodeModificationsHeadFile), cancellationToken);
 				var tailFileExistsTask = ioManager.FileExists(ioManager.ConcatPath(CodeModificationsSubdirectory, CodeModificationsTailFile), cancellationToken);
+				var copyTask = ioManager.CopyDirectory(CodeModificationsSubdirectory, destination, null, cancellationToken);
 
-				await Task.WhenAll(dmeExistsTask, headFileExistsTask, tailFileExistsTask).ConfigureAwait(false);
+				await Task.WhenAll(dmeExistsTask, headFileExistsTask, tailFileExistsTask, copyTask).ConfigureAwait(false);
 
 				if (!dmeExistsTask.Result && !headFileExistsTask.Result && !tailFileExistsTask.Result)
 					return null;
 
-				var copyTask = ioManager.CopyDirectory(CodeModificationsSubdirectory, destination, null, cancellationToken);
-
 				if (dmeExistsTask.Result)
-				{
-					await copyTask.ConfigureAwait(false);
 					return new ServerSideModifications(null, null, true);
-				}
 
 				if (!headFileExistsTask.Result && !tailFileExistsTask.Result)
-				{
-					await copyTask.ConfigureAwait(false);
 					return null;
-				}
 
 				string IncludeLine(string filePath) => String.Format(CultureInfo.InvariantCulture, "#include \"{0}\"", filePath);
-
-				await copyTask.ConfigureAwait(false);
+			
 				return new ServerSideModifications(headFileExistsTask.Result ? IncludeLine(CodeModificationsHeadFile) : null, tailFileExistsTask.Result ? IncludeLine(CodeModificationsTailFile) : null, false);
 			}
 		}

src/Tgstation.Server.Host/Components/Watchdog/MonitorAction.cs

@@ -10,6 +10,10 @@
 		/// </summary>
 		Continue,
 		/// <summary>
+		/// Skips the next call to HandleMonitorWakeup action 
+		/// </summary>
+		Skip,
+		/// <summary>
 		/// The monitor should kill and restart both servers
 		/// </summary>
 		Restart,

src/Tgstation.Server.Host/Components/Watchdog/MonitorState.cs

@@ -12,11 +12,6 @@ namespace Tgstation.Server.Host.Components.Watchdog
 		/// </summary>
 		public bool RebootingInactiveServer { get; set; }
 
-		/// <summary>
-		/// If the inactive server has a .dmb and needs to be swapped in
-		/// </summary>
-		public bool InactiveServerHasStagedDmb { get; set; }
-
 		/// <summary>
 		/// If the inactive server is in an unrecoverable state
 		/// </summary>

src/Tgstation.Server.Host/Components/Watchdog/SessionController.cs

@@ -401,6 +401,7 @@ namespace Tgstation.Server.Host.Components.Watchdog
 			reattachInformation.Dmb = null;
 			released = true;
 			Dispose();
+			byondLock.DoNotDeleteThisSession();
 			tmpProvider.KeepAlive();
 			reattachInformation.Dmb = tmpProvider;
 			return reattachInformation;
@@ -429,6 +430,10 @@ namespace Tgstation.Server.Host.Components.Watchdog
 					commandString,
 					cancellationToken).ConfigureAwait(false);
 			}
+			catch (OperationCanceledException)
+			{
+				throw;
+			}
 			catch (Exception e)
 			{
 				logger.LogInformation("Send command exception:{0}{1}", Environment.NewLine, e.Message);

src/Tgstation.Server.Host/Components/Watchdog/Watchdog.cs

@@ -284,21 +284,23 @@ namespace Tgstation.Server.Host.Components.Watchdog
 				return true;
 			}
 
-			// Tries to launch inactive server with the latest dmb
-			// Doesn't handle stopping it
+			// Kills and tries to launch inactive server with the latest dmb
 			// falls back to current dmb on failure
 			// Sets critfail on inactive server failing that
 			// returns false if the backup dmb was used successfully, true otherwise
-			async Task<bool> RestartInactiveServer()
+			async Task UpdateAndRestartInactiveServer(bool breakAfter)
 			{
+				activeParametersUpdated = new TaskCompletionSource<object>();
+				monitorState.InactiveServer.Dispose();  //kill or recycle it
+				var desiredNextAction = breakAfter ? MonitorAction.Break : MonitorAction.Continue;
+				monitorState.NextAction = desiredNextAction;
+
 				logger.LogInformation("Rebooting inactive server...");
 				var newDmb = dmbFactory.LockNextDmb(1);
-				bool usedMostRecentDmb;
 				try
 				{
 					monitorState.InactiveServer = await sessionControllerFactory.LaunchNew(ActiveLaunchParameters, newDmb, null, false, !monitorState.ActiveServer.IsPrimary, false, cancellationToken).ConfigureAwait(false);
 					monitorState.InactiveServer.SetHighPriority();
-					usedMostRecentDmb = true;
 				}
 				catch (OperationCanceledException)
 				{
@@ -321,7 +323,6 @@ namespace Tgstation.Server.Host.Components.Watchdog
 
 						monitorState.InactiveServer = await sessionControllerFactory.LaunchNew(ActiveLaunchParameters, dmbBackup, null, false, !monitorState.ActiveServer.IsPrimary, false, cancellationToken).ConfigureAwait(false);
 						monitorState.InactiveServer.SetHighPriority();
-						usedMostRecentDmb = false;
 						await chat.SendWatchdogMessage("Staging newest DMB on inactive server failed: {0} Falling back to previous dmb...", cancellationToken).ConfigureAwait(false);
 					}
 					catch (OperationCanceledException)
@@ -334,33 +335,14 @@ namespace Tgstation.Server.Host.Components.Watchdog
 						logger.LogError("Backup strategy failed! Monitor will restart when active server reboots! Exception: {0}", e2.ToString());
 						monitorState.InactiveServerCritFail = true;
 						await chat.SendWatchdogMessage("Attempted reboot of inactive server failed. Watchdog will reset when active server fails or exits", cancellationToken).ConfigureAwait(false);
-						return true;    //we didn't use the old dmb
+						return;
 					}
 				}
 
 				logger.LogInformation("Successfully relaunched inactive server!");
 				monitorState.RebootingInactiveServer = true;
-				return usedMostRecentDmb;
 			}
 
-			//kills inactive server and tries to relaunch it with the latest dmb
-			async Task UpdateAndRestartInactiveServer(bool breakAfter)
-			{
-				//replace the notification tcs here so that the next loop will read a fresh one
-				activeParametersUpdated = new TaskCompletionSource<object>();
-				monitorState.InactiveServer.Dispose();  //kill or recycle it
-				monitorState.NextAction = breakAfter ? MonitorAction.Break : MonitorAction.Continue;
-
-				var usedLatestDmb = await RestartInactiveServer().ConfigureAwait(false);
-
-				if (monitorState.NextAction == (breakAfter ? MonitorAction.Break : MonitorAction.Continue))
-				{
-					monitorState.ActiveServer.ClosePortOnReboot = false;
-					if (monitorState.InactiveServerHasStagedDmb && !usedLatestDmb)
-						monitorState.InactiveServerHasStagedDmb = false;    //don't try to load it again though
-				}
-			};
-
 			string ExitWord(ISessionController controller) => controller.TerminationWasRequested ? "exited" : "crashed";
 
 			//reason handling
@@ -403,15 +385,15 @@ namespace Tgstation.Server.Host.Components.Watchdog
 				case MonitorActivationReason.ActiveServerRebooted:
 					//ideal goal: active server just closed its port
 					//tell inactive server to open it's port and that's now the active server
+					var rebootState = monitorState.ActiveServer.RebootState;
+					monitorState.ActiveServer.ResetRebootState();   //the DMAPI has already done this internally
 
-					if (FullRestartDeadInactive())
+					if (FullRestartDeadInactive() && rebootState != Components.Watchdog.RebootState.Shutdown)
 						//full restart if the inactive server is being fucky
 						break;
 
 					//what matters here is the RebootState
-					bool restartOnceSwapped = false;
-					var rebootState = monitorState.ActiveServer.RebootState;
-					monitorState.ActiveServer.ResetRebootState();   //the DMAPI has already done this internally
+					var restartOnceSwapped = false;
 
 					switch (rebootState)
 					{
@@ -433,17 +415,13 @@ namespace Tgstation.Server.Host.Components.Watchdog
 					//are both servers now running the same CompileJob?
 					var sameCompileJob = monitorState.InactiveServer.Dmb.CompileJob.Id == monitorState.ActiveServer.Dmb.CompileJob.Id;
 					
-					if (sameCompileJob && monitorState.InactiveServerHasStagedDmb)
-						//both servers now up to date
-						monitorState.InactiveServerHasStagedDmb = false;
-					
 					if (!sameCompileJob || ActiveLaunchParameters != LastLaunchParameters)
 						//need a new launch to update either settings or compile job
 						restartOnceSwapped = true;
 
-					if (restartOnceSwapped && !monitorState.ActiveServer.ClosePortOnReboot)
+					if (restartOnceSwapped)
 						//we need to manually restart active server
-						//it won't listen to us right now because it's port is closed so just kill it
+						//just kill it here, easier that way
 						monitorState.ActiveServer.Dispose();
 
 					var activeServerStillHasPortOpen = !restartOnceSwapped && !monitorState.ActiveServer.ClosePortOnReboot;
@@ -480,10 +458,8 @@ namespace Tgstation.Server.Host.Components.Watchdog
 						//update and reboot
 						await UpdateAndRestartInactiveServer(true).ConfigureAwait(false);
 					else
-						//the one odd case would be getting NewDmbAvailable at this exact moment
-						//in which case, we have the issue that it will never actually deploy until the watchdog restarts completely
-						//TODO: Come up with some way to gracefully handle that
-						monitorState.NextAction = MonitorAction.Break;
+						//only skip checking inactive server rebooted, it's guaranteed InactiveServerStartup complete wouldn't fire this iteration
+						monitorState.NextAction = MonitorAction.Skip;
 					break;
 				case MonitorActivationReason.InactiveServerRebooted:
 					//just don't let the active server close it's port if the inactive server isn't ready
@@ -499,9 +475,6 @@ namespace Tgstation.Server.Host.Components.Watchdog
 					monitorState.NextAction = MonitorAction.Continue;
 					break;
 				case MonitorActivationReason.NewDmbAvailable:
-					//set this and then its the same a settings change
-					monitorState.InactiveServerHasStagedDmb = true;
-					goto case MonitorActivationReason.ActiveLaunchParametersUpdated;
 				case MonitorActivationReason.ActiveLaunchParametersUpdated:
 					//just reload the inactive server and wait for a swap to apply the changes
 					await UpdateAndRestartInactiveServer(true).ConfigureAwait(false);
@@ -534,9 +507,8 @@ namespace Tgstation.Server.Host.Components.Watchdog
 						logger.LogDebug("Alpha is the active server");
 					else
 						logger.LogDebug("Bravo is the active server");
-
-					if (monitorState.InactiveServerHasStagedDmb)
-						logger.LogDebug("Inactive server has staged .dmb");
+					
+					
 					if (monitorState.RebootingInactiveServer)
 						logger.LogDebug("Inactive server is rebooting");
 
@@ -549,13 +521,16 @@ namespace Tgstation.Server.Host.Components.Watchdog
 					if (monitorState.InactiveServer.ClosePortOnReboot)
 						logger.LogDebug("Inactive server will close port on reboot");
 
+					logger.LogDebug("Active server Compile Job ID: {0}", monitorState.ActiveServer.Dmb.CompileJob.Id);
+					logger.LogDebug("Inactive server Compile Job ID: {0}", monitorState.InactiveServer.Dmb.CompileJob.Id);
+
 					//load the activation tasks into local variables
-					var activeServerLifetime = monitorState.ActiveServer.Lifetime;
-					var inactiveServerLifetime = monitorState.InactiveServer.Lifetime;
+					Task activeServerLifetime = monitorState.ActiveServer.Lifetime;
+					Task inactiveServerLifetime = monitorState.InactiveServer.Lifetime;
 					var activeServerReboot = monitorState.ActiveServer.OnReboot;
 					var inactiveServerReboot = monitorState.InactiveServer.OnReboot;
-					var inactiveServerStartup = monitorState.RebootingInactiveServer ? monitorState.InactiveServer.LaunchResult : null;
-					var activeLaunchParametersChanged = activeParametersUpdated.Task;
+					Task inactiveServerStartup = monitorState.RebootingInactiveServer ? monitorState.InactiveServer.LaunchResult : null;
+					Task activeLaunchParametersChanged = activeParametersUpdated.Task;
 					var newDmbAvailable = dmbFactory.OnNewerDmb;
 
 					//cancel waiting if requested
@@ -575,52 +550,36 @@ namespace Tgstation.Server.Host.Components.Watchdog
 					{
 						//always run HandleMonitorWakeup from the context of the semaphore lock
 						//multiple things may have happened, handle them one at a time
-						for (var moreActivationsToProcess = true; moreActivationsToProcess && monitorState.NextAction == MonitorAction.Continue;)
+						for (var moreActivationsToProcess = true; moreActivationsToProcess && (monitorState.NextAction == MonitorAction.Continue || monitorState.NextAction == MonitorAction.Skip);)
 						{
 							MonitorActivationReason activationReason = default; //this will always be assigned before being used
 
 							//process the tasks in this order and call HandlerMonitorWakup for each
 
-							if (activeServerLifetime?.IsCompleted == true)
+							bool CheckActivationReason(ref Task task, MonitorActivationReason testActivationReason)
 							{
-								activationReason = MonitorActivationReason.ActiveServerCrashed;
-								activeServerLifetime = null;
-							}
-							else if (inactiveServerLifetime?.IsCompleted == true)
-							{
-								activationReason = MonitorActivationReason.InactiveServerCrashed;
-								inactiveServerLifetime = null;
-							}
-							else if (activeServerReboot?.IsCompleted == true)
-							{
-								activationReason = MonitorActivationReason.ActiveServerRebooted;
-								activeServerReboot = null;
-							}
-							else if (inactiveServerReboot?.IsCompleted == true)
-							{
-								activationReason = MonitorActivationReason.InactiveServerRebooted;
-								inactiveServerReboot = null;
-							}
-							else if (inactiveServerStartup?.IsCompleted == true)
-							{
-								activationReason = MonitorActivationReason.InactiveServerStartupComplete;
-								inactiveServerStartup = null;
-							}
-							else if (newDmbAvailable?.IsCompleted == true)
-							{
-								activationReason = MonitorActivationReason.NewDmbAvailable;
-								newDmbAvailable = null;
-							}
-							else if (activeLaunchParametersChanged?.IsCompleted == true)
-							{
-								activationReason = MonitorActivationReason.ActiveLaunchParametersUpdated;
-								activeLaunchParametersChanged = null;
-							}
+								var taskCompleted = task?.IsCompleted == true;
+								task = null;
+								if (monitorState.NextAction == MonitorAction.Skip)
+									monitorState.NextAction = MonitorAction.Continue;
+								else if (taskCompleted)
+								{
+									activationReason = testActivationReason;
+									return true;
+								}
+								return false; 
+							};
+
+							if (CheckActivationReason(ref activeServerLifetime, MonitorActivationReason.ActiveServerCrashed)
+								|| CheckActivationReason(ref inactiveServerLifetime, MonitorActivationReason.InactiveServerCrashed)
+								|| CheckActivationReason(ref activeServerReboot, MonitorActivationReason.ActiveServerRebooted)
+								|| CheckActivationReason(ref inactiveServerReboot, MonitorActivationReason.InactiveServerRebooted)
+								|| CheckActivationReason(ref inactiveServerStartup, MonitorActivationReason.InactiveServerStartupComplete)
+								|| CheckActivationReason(ref newDmbAvailable, MonitorActivationReason.NewDmbAvailable)
+								|| CheckActivationReason(ref activeLaunchParametersChanged, MonitorActivationReason.ActiveLaunchParametersUpdated))
+								await HandlerMonitorWakeup(activationReason, monitorState, cancellationToken).ConfigureAwait(false);
 							else
 								moreActivationsToProcess = false;
-
-							if (moreActivationsToProcess)
-								await HandlerMonitorWakeup(activationReason, monitorState, cancellationToken).ConfigureAwait(false);
 						}
 
 						//writeback alphaServer and bravoServer from monitor state in case they changesd
@@ -649,6 +608,10 @@ namespace Tgstation.Server.Host.Components.Watchdog
 										monitorState = new MonitorState();  //clean the slate and continue
 									}
 								}
+								catch (OperationCanceledException)
+								{
+									throw;
+								}
 								catch (Exception e)
 								{
 									launchException = e;
@@ -980,7 +943,10 @@ namespace Tgstation.Server.Host.Components.Watchdog
 
 			long? adminUserId = null;
 
-			await databaseContextFactory.UseContext(async db => adminUserId = await db.Users.Select(x => x.Id).FirstAsync(cancellationToken).ConfigureAwait(false)).ConfigureAwait(false);
+			await databaseContextFactory.UseContext(async db => adminUserId = await db.Users
+			.Where(x => x.CanonicalName == Api.Models.User.AdminName.ToUpperInvariant())
+			.Select(x => x.Id)
+			.FirstAsync(cancellationToken).ConfigureAwait(false)).ConfigureAwait(false);
 			var job = new Models.Job
 			{
 				StartedBy = new Models.User
@@ -1030,13 +996,15 @@ namespace Tgstation.Server.Host.Components.Watchdog
 					return true;
 
 				var builder = new StringBuilder(Constants.DMTopicEvent);
-				builder.Append("&");
+				builder.Append('&');
 				var notification = new EventNotification
 				{
 					Type = eventType,
 					Parameters = parameters
 				};
 				var json = JsonConvert.SerializeObject(notification);
+				builder.Append(byondTopicSender.SanitizeString(Constants.DMParameterData));
+				builder.Append('=');
 				builder.Append(byondTopicSender.SanitizeString(json));
 
 				var activeServer = AlphaIsActive ? alphaServer : bravoServer;

src/Tgstation.Server.Host/Controllers/ApiController.cs

@@ -120,6 +120,7 @@ namespace Tgstation.Server.Host.Controllers
 			if (ModelState?.IsValid == false)
 			{
 				var errorMessages = ModelState.SelectMany(x => x.Value.Errors).Select(x => x.ErrorMessage).ToList();
+				//HACK
 				//do some fuckery to remove RequiredAttribute errors
 				for (var I = 0; I < errorMessages.Count; ++I)
 				{

src/Tgstation.Server.Host/Controllers/ConfigurationController.cs

@@ -49,8 +49,18 @@ namespace Tgstation.Server.Host.Controllers
 		/// If a <see cref="ForbidResult"/> should be returned from actions due to conflicts with one or both of the <see cref="Api.Models.Instance.ConfigurationType"/> or the <see cref="IAuthenticationContext.SystemIdentity"/> or a given <paramref name="path"/> tries to access parent directories
 		/// </summary>
 		/// <param name="path">The path to validate if any</param>
+		/// <param name="systemIdentityToUse">The <see cref="ISystemIdentity"/> to use when calling into <see cref="Components.StaticFiles.IConfiguration"/></param>
 		/// <returns><see langword="true"/> if a <see cref="ForbidResult"/> should be returned, <see langword="false"/> otherwise</returns>
-		bool ForbidDueToModeConflicts(string path) => Instance.ConfigurationType == ConfigurationType.Disallowed || (Instance.ConfigurationType == ConfigurationType.SystemIdentityWrite && AuthenticationContext.SystemIdentity == null) || (path != null && ioManager.PathContainsParentAccess(path));
+		bool ForbidDueToModeConflicts(string path, out ISystemIdentity systemIdentityToUse)
+		{
+			if (Instance.ConfigurationType == ConfigurationType.Disallowed || (Instance.ConfigurationType == ConfigurationType.SystemIdentityWrite && AuthenticationContext.SystemIdentity == null) || (path != null && ioManager.PathContainsParentAccess(path)))
+			{
+				systemIdentityToUse = null;
+				return true;
+			}
+			systemIdentityToUse = Instance.ConfigurationType == ConfigurationType.SystemIdentityWrite ? AuthenticationContext.SystemIdentity : null;
+			return false;
+		}
 
 		/// <inheritdoc />
 		[TgsAuthorize(ConfigurationRights.Write)]
@@ -58,13 +68,13 @@ namespace Tgstation.Server.Host.Controllers
 		{
 			if (model == null)
 				throw new ArgumentNullException(nameof(model));
-			if (ForbidDueToModeConflicts(model.Path))
+			if (ForbidDueToModeConflicts(model.Path, out var systemIdentity))
 				return Forbid();
 
 			var config = instanceManager.GetInstance(Instance).Configuration;
 			try
 			{
-				var newFile = await config.Write(model.Path, AuthenticationContext.SystemIdentity, model.Content, model.LastReadHash, cancellationToken).ConfigureAwait(false);
+				var newFile = await config.Write(model.Path, systemIdentity, model.Content, model.LastReadHash, cancellationToken).ConfigureAwait(false);
 				if (newFile == null)
 					return Conflict(new ErrorMessage
 					{
@@ -99,12 +109,12 @@ namespace Tgstation.Server.Host.Controllers
 		[TgsAuthorize(ConfigurationRights.Read)]
 		public async Task<IActionResult> File(string filePath, CancellationToken cancellationToken)
 		{
-			if (ForbidDueToModeConflicts(filePath))
+			if (ForbidDueToModeConflicts(filePath, out var systemIdentity))
 				return Forbid();
 
 			try
 			{
-				var result = await instanceManager.GetInstance(Instance).Configuration.Read(filePath, AuthenticationContext.SystemIdentity, cancellationToken).ConfigureAwait(false);
+				var result = await instanceManager.GetInstance(Instance).Configuration.Read(filePath, systemIdentity, cancellationToken).ConfigureAwait(false);
 				if (result == null)
 					return StatusCode((int)HttpStatusCode.Gone);
 
@@ -134,12 +144,12 @@ namespace Tgstation.Server.Host.Controllers
 		[TgsAuthorize(ConfigurationRights.List)]
 		public async Task<IActionResult> Directory(string directoryPath, CancellationToken cancellationToken)
 		{
-			if (ForbidDueToModeConflicts(directoryPath))
+			if (ForbidDueToModeConflicts(directoryPath, out var systemIdentity))
 				return Forbid();
 
 			try
 			{
-				var result = await instanceManager.GetInstance(Instance).Configuration.ListDirectory(directoryPath, AuthenticationContext.SystemIdentity, cancellationToken).ConfigureAwait(false);
+				var result = await instanceManager.GetInstance(Instance).Configuration.ListDirectory(directoryPath, systemIdentity, cancellationToken).ConfigureAwait(false);
 				if (result == null)
 					return StatusCode((int)HttpStatusCode.Gone);
 
@@ -166,13 +176,13 @@ namespace Tgstation.Server.Host.Controllers
 			if (model == null)
 				throw new ArgumentNullException(nameof(model));
 
-			if (ForbidDueToModeConflicts(model.Path))
+			if (ForbidDueToModeConflicts(model.Path, out var systemIdentity))
 				return Forbid();
 
 			try
 			{
 				model.IsDirectory = true;
-				return await instanceManager.GetInstance(Instance).Configuration.CreateDirectory(model.Path, AuthenticationContext.SystemIdentity, cancellationToken).ConfigureAwait(false) ? (IActionResult)Json(model) : StatusCode((int)HttpStatusCode.Created, model);
+				return await instanceManager.GetInstance(Instance).Configuration.CreateDirectory(model.Path, systemIdentity, cancellationToken).ConfigureAwait(false) ? (IActionResult)Json(model) : StatusCode((int)HttpStatusCode.Created, model);
 			}
 			catch (NotImplementedException)
 			{
@@ -197,12 +207,12 @@ namespace Tgstation.Server.Host.Controllers
 			if (directory == null)
 				throw new ArgumentNullException(nameof(directory));
 
-			if (ForbidDueToModeConflicts(directory.Path))
+			if (ForbidDueToModeConflicts(directory.Path, out var systemIdentity))
 				return Forbid();
 
 			try
 			{
-				return await instanceManager.GetInstance(Instance).Configuration.DeleteDirectory(directory.Path, AuthenticationContext.SystemIdentity, cancellationToken).ConfigureAwait(false) ? (IActionResult)Ok() : Conflict(new ErrorMessage
+				return await instanceManager.GetInstance(Instance).Configuration.DeleteDirectory(directory.Path, systemIdentity, cancellationToken).ConfigureAwait(false) ? (IActionResult)Ok() : Conflict(new ErrorMessage
 				{
 					Message = "Directory not empty!"
 				});

src/Tgstation.Server.Host/Controllers/InstanceController.cs

@@ -236,7 +236,9 @@ namespace Tgstation.Server.Host.Controllers
 
 			var moveJob = await instanceQuery
 				.SelectMany(x => x.Jobs).
-				Where(x => !x.StoppedAt.HasValue && x.Description.StartsWith(MoveInstanceJobPrefix, StringComparison.Ordinal))
+#pragma warning disable CA1307 // Specify StringComparison
+				Where(x => !x.StoppedAt.HasValue && x.Description.StartsWith(MoveInstanceJobPrefix))
+#pragma warning restore CA1307 // Specify StringComparison
 				.Select(x => new Models.Job
 				{
 					Id = x.Id
@@ -332,7 +334,8 @@ namespace Tgstation.Server.Host.Controllers
 			}
 			catch (Exception e)
 			{
-				Logger.LogError("Error changing instance online state! Exception: {0}", e);
+				if(!(e is OperationCanceledException))
+					Logger.LogError("Error changing instance online state! Exception: {0}", e);
 				originalModel.Online = originalOnline;
 				originalModel.DreamDaemonSettings.AutoStart = oldAutoStart;
 				if (originalModelPath != null)
@@ -376,7 +379,9 @@ namespace Tgstation.Server.Host.Controllers
 
 			var moveJobTasks = query
 				.SelectMany(x => x.Jobs)
-				.Where(x => !x.StoppedAt.HasValue && x.Description.StartsWith(MoveInstanceJobPrefix, StringComparison.Ordinal))
+#pragma warning disable CA1307 // Specify StringComparison
+				.Where(x => !x.StoppedAt.HasValue && x.Description.StartsWith(MoveInstanceJobPrefix))
+#pragma warning restore CA1307 // Specify StringComparison
 				.Include(x => x.StartedBy).ThenInclude(x => x.CreatedBy)
 				.Include(x => x.Instance)
 				.ToListAsync(cancellationToken);
@@ -400,7 +405,9 @@ namespace Tgstation.Server.Host.Controllers
 
 			var moveJobTask = query
 				.SelectMany(x => x.Jobs)
-				.Where(x => !x.StoppedAt.HasValue && x.Description.StartsWith(MoveInstanceJobPrefix, StringComparison.Ordinal))
+#pragma warning disable CA1307 // Specify StringComparison
+				.Where(x => !x.StoppedAt.HasValue && x.Description.StartsWith(MoveInstanceJobPrefix))
+#pragma warning restore CA1307 // Specify StringComparison
 				.Include(x => x.StartedBy).ThenInclude(x => x.CreatedBy)
 				.FirstOrDefaultAsync(cancellationToken);
 			var instance = await query.FirstOrDefaultAsync(cancellationToken).ConfigureAwait(false);

src/Tgstation.Server.Host/Controllers/UserController.cs

@@ -130,7 +130,9 @@ namespace Tgstation.Server.Host.Controllers
 
 			var passwordEditOnly = !AuthenticationContext.User.AdministrationRights.Value.HasFlag(AdministrationRights.WriteUsers);
 
-			var originalUser = passwordEditOnly ? AuthenticationContext.User : await DatabaseContext.Users.Where(x => x.Id == model.Id).FirstOrDefaultAsync(cancellationToken).ConfigureAwait(false);
+			var originalUser = passwordEditOnly ? AuthenticationContext.User : await DatabaseContext.Users.Where(x => x.Id == model.Id)
+				.Include(x => x.CreatedBy)
+				.FirstOrDefaultAsync(cancellationToken).ConfigureAwait(false);
 			if (originalUser == default)
 				return NotFound();
 
@@ -170,7 +172,9 @@ namespace Tgstation.Server.Host.Controllers
 		[TgsAuthorize(AdministrationRights.ReadUsers)]
 		public override async Task<IActionResult> List(CancellationToken cancellationToken)
 		{
-			var users = await DatabaseContext.Users.ToListAsync(cancellationToken).ConfigureAwait(false);
+			var users = await DatabaseContext.Users
+				.Include(x => x.CreatedBy)
+				.ToListAsync(cancellationToken).ConfigureAwait(false);
 			return Json(users.Select(x => x.ToApi(true)));
 		}
 
@@ -184,7 +188,10 @@ namespace Tgstation.Server.Host.Controllers
 			if (!((AdministrationRights)AuthenticationContext.GetRight(RightsType.Administration)).HasFlag(AdministrationRights.ReadUsers))
 				return Forbid();
 
-			var user = await DatabaseContext.Users.Where(x => x.Id == id).FirstOrDefaultAsync(cancellationToken).ConfigureAwait(false);
+			var user = await DatabaseContext.Users
+				.Where(x => x.Id == id)
+				.Include(x => x.CreatedBy)
+				.FirstOrDefaultAsync(cancellationToken).ConfigureAwait(false);
 			if (user == default)
 				return NotFound();
 			return Json(user.ToApi(true));

src/Tgstation.Server.Host/Core/ApplicationBuilderExtensions.cs

@@ -1,6 +1,9 @@
 using Microsoft.AspNetCore.Builder;
+using Microsoft.AspNetCore.Http;
 using Microsoft.AspNetCore.Mvc;
 using Microsoft.EntityFrameworkCore;
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.Logging;
 using System;
 using System.Globalization;
 using Tgstation.Server.Api.Models;
@@ -12,6 +15,13 @@ namespace Tgstation.Server.Host.Core
 	/// </summary>
 	static class ApplicationBuilderExtensions
 	{
+		/// <summary>
+		/// Gets a <see cref="ILogger"/> from a given <paramref name="httpContext"/>
+		/// </summary>
+		/// <param name="httpContext">The <see cref="HttpContext"/> to get the <see cref="ILogger"/> from</param>
+		/// <returns>A new <see cref="ILogger"/></returns>
+		static ILogger GetLogger(HttpContext httpContext) => httpContext.RequestServices.GetRequiredService<ILogger<Application>>();
+
 		/// <summary>
 		/// Return a <see cref="ConflictObjectResult"/> for <see cref="DbUpdateException"/>s
 		/// </summary>
@@ -22,12 +32,14 @@ namespace Tgstation.Server.Host.Core
 				throw new ArgumentNullException(nameof(applicationBuilder));
 			applicationBuilder.Use(async (context, next) =>
 			{
+				var logger = GetLogger(context);
 				try
 				{
 					await next().ConfigureAwait(false);
 				}
 				catch (DbUpdateException e)
 				{
+					logger.LogDebug("Database conflict: {0}", e.Message);
 					await new ConflictObjectResult(new ErrorMessage { Message = String.Format(CultureInfo.InvariantCulture, "A database conflict has occurred: {0}", (e.InnerException ?? e).Message) }).ExecuteResultAsync(new ActionContext
 					{
 						HttpContext = context
@@ -46,11 +58,15 @@ namespace Tgstation.Server.Host.Core
 				throw new ArgumentNullException(nameof(applicationBuilder));
 			applicationBuilder.Use(async (context, next) =>
 			{
+				var logger = GetLogger(context);
 				try
 				{
 					await next().ConfigureAwait(false);
 				}
-				catch (OperationCanceledException) { }
+				catch (OperationCanceledException)
+				{
+					logger.LogDebug("Request cancelled!");
+				}
 			});
 		}
 	}

src/Tgstation.Server.Host/Core/JobManager.cs

@@ -76,6 +76,7 @@ namespace Tgstation.Server.Host.Core
 				{
 					async Task HandleExceptions(Task task)
 					{
+						void LogRegularException() => logger.LogDebug("Job {0} exited with error! Exception: {1}", job.Id, job.ExceptionDetails); 
 						try
 						{
 							await task.ConfigureAwait(false);
@@ -85,10 +86,17 @@ namespace Tgstation.Server.Host.Core
 							logger.LogDebug("Job {0} cancelled!", job.Id);
 							job.Cancelled = true;
 						}
+						catch (JobException e)
+						{
+							job.ExceptionDetails = e.Message;
+							LogRegularException();
+							if (e.InnerException != null)
+								logger.LogDebug("Inner exception for job {0}: {1}", job.Id, e.InnerException);
+						}
 						catch (Exception e)
 						{
-							job.ExceptionDetails = e is JobException ? e.Message : e.ToString();
-							logger.LogDebug("Job {0} exited with error! Exception: {1}", job.Id, job.ExceptionDetails);
+							job.ExceptionDetails = e.ToString();
+							LogRegularException();
 						}
 						finally
 						{

src/Tgstation.Server.Host/Models/User.cs

@@ -53,7 +53,7 @@ namespace Tgstation.Server.Host.Models
 		{
 			AdministrationRights = showDetails ? AdministrationRights : null,
 			CreatedAt = CreatedAt,
-			CreatedBy = recursive ? CreatedBy?.ToApi(false, showDetails) : null,
+			CreatedBy = recursive ? CreatedBy?.ToApi(false, false) : null,
 			Enabled = Enabled,
 			Id = Id,
 			InstanceManagerRights = showDetails ? InstanceManagerRights : null,

src/Tgstation.Server.Host/Security/AuthenticationContext.cs

@@ -55,9 +55,6 @@ namespace Tgstation.Server.Host.Security
 		/// <inheritdoc />
 		public void Dispose() => SystemIdentity?.Dispose();
 
-		/// <inheritdoc />
-		public IAuthenticationContext Clone() => new AuthenticationContext(SystemIdentity.Clone(), User, InstanceUser);
-
 		/// <inheritdoc />
 		public ulong GetRight(RightsType rightsType)
 		{

src/Tgstation.Server.Host/Security/AuthenticationContextFactory.cs

@@ -47,7 +47,9 @@ namespace Tgstation.Server.Host.Security
 			if (CurrentAuthenticationContext != null)
 				throw new InvalidOperationException("Authentication context has already been loaded");
 
-			var userQuery = databaseContext.Users.Where(x => x.Id == userId).FirstOrDefaultAsync(cancellationToken);
+			var userQuery = databaseContext.Users.Where(x => x.Id == userId)
+				.Include(x => x.CreatedBy)
+				.FirstOrDefaultAsync(cancellationToken);
 
 			var instanceUser = instanceId.HasValue ? (await databaseContext.InstanceUsers
 				.Where(x => x.UserId == userId && x.InstanceId == instanceId && x.Instance.Online.Value)

src/Tgstation.Server.Host/Security/IAuthenticationContext.cs

@@ -30,11 +30,5 @@ namespace Tgstation.Server.Host.Security
 		/// The <see cref="ISystemIdentity"/> of <see cref="User"/> if applicable
 		/// </summary>
 		ISystemIdentity SystemIdentity { get; }
-
-		/// <summary>
-		/// Creates a copy of the <see cref="IAuthenticationContext"/>
-		/// </summary>
-		/// <returns>A new <see cref="IAuthenticationContext"/></returns>
-		IAuthenticationContext Clone();
 	}
 }

src/Tgstation.Server.Host/Security/IdentityCache.cs

@@ -1,5 +1,7 @@
-using System;
+using Microsoft.Extensions.Logging;
+using System;
 using System.Collections.Generic;
+using System.Linq;
 using Tgstation.Server.Host.Models;
 
 namespace Tgstation.Server.Host.Security
@@ -7,18 +9,32 @@ namespace Tgstation.Server.Host.Security
 	/// <inheritdoc />
 	sealed class IdentityCache : IIdentityCache, IDisposable
 	{
+		/// <summary>
+		/// The <see cref="ILogger"/> for the <see cref="IdentityCache"/>
+		/// </summary>
+		readonly ILogger<IdentityCache> logger;
+
+		/// <summary>
+		/// The map of <see cref="Api.Models.Internal.User.Id"/>s to <see cref="IdentityCacheObject"/>s
+		/// </summary>
 		readonly Dictionary<long, IdentityCacheObject> cachedIdentities;
 
-		public IdentityCache()
+		/// <summary>
+		/// Construct an <see cref="IdentityCache"/>
+		/// </summary>
+		public IdentityCache(ILogger<IdentityCache> logger)
 		{
+			this.logger = logger ?? throw new ArgumentNullException(nameof(logger));
+
 			cachedIdentities = new Dictionary<long, IdentityCacheObject>();
 		}
 
 		/// <inheritdoc />
 		public void Dispose()
 		{
-			foreach (var I in cachedIdentities)
-				I.Value.Dispose();
+			logger.LogTrace("Disposing...");
+			foreach (var I in cachedIdentities.Select(x => x.Value).ToList())
+				I.Dispose();
 		}
 
 		/// <inheritdoc />
@@ -30,10 +46,17 @@ namespace Tgstation.Server.Host.Security
 				throw new ArgumentNullException(nameof(systemIdentity));
 			lock (cachedIdentities)
 			{
+				var uid = systemIdentity.Uid;
+				logger.LogDebug("Caching system identity {0} of user {1}", uid, user.Id);
+
 				if (cachedIdentities.TryGetValue(user.Id, out var identCache))
+				{
+					logger.LogTrace("Expiring previously cached identity...");
 					identCache.Dispose();   //also clears it out
+				}
 				identCache = new IdentityCacheObject(systemIdentity.Clone(), () =>
 				{
+					logger.LogDebug("Expiring system identity cache for user {1}", uid, user.Id);
 					lock (cachedIdentities)
 						cachedIdentities.Remove(user.Id);
 				}, expiry);

src/Tgstation.Server.Host/Security/PosixSystemIdentityFactory.cs

@@ -8,19 +8,13 @@ namespace Tgstation.Server.Host.Security
 	/// <summary>
 	/// <see cref="ISystemIdentityFactory"/> for posix systems
 	/// </summary>
-	/// <remarks>Blocked by https://github.com/dotnet/corefx/issues/3187</remarks>
+	/// <remarks>TODO: Blocked by https://github.com/dotnet/corefx/issues/3187</remarks>
 	sealed class PosixSystemIdentityFactory : ISystemIdentityFactory
 	{
 		/// <inheritdoc />
-		public Task<ISystemIdentity> CreateSystemIdentity(User user, CancellationToken cancellationToken)
-		{
-			throw new NotImplementedException();
-		}
+		public Task<ISystemIdentity> CreateSystemIdentity(User user, CancellationToken cancellationToken) => throw new NotImplementedException();
 
 		/// <inheritdoc />
-		public Task<ISystemIdentity> CreateSystemIdentity(string username, string password, CancellationToken cancellationToken)
-		{
-			throw new NotImplementedException();
-		}
+		public Task<ISystemIdentity> CreateSystemIdentity(string username, string password, CancellationToken cancellationToken) => throw new NotImplementedException();
 	}
 }

src/Tgstation.Server.Host/Security/WindowsSystemIdentity.cs

@@ -11,6 +11,12 @@ namespace Tgstation.Server.Host.Security
 	/// </summary>
 	sealed class WindowsSystemIdentity : ISystemIdentity
 	{
+		/// <inheritdoc />
+		public string Uid => (userPrincipal?.Sid ?? identity.User).ToString();
+
+		/// <inheritdoc />
+		public string Username => userPrincipal?.Name ?? identity.Name;
+
 		/// <summary>
 		/// The <see cref="WindowsIdentity"/> for the <see cref="WindowsSystemIdentity"/>
 		/// </summary>
@@ -52,17 +58,17 @@ namespace Tgstation.Server.Host.Security
 			}
 		}
 
-		/// <inheritdoc />
-		public string Uid => (userPrincipal?.Sid ?? identity.User).ToString();
-
-		/// <inheritdoc />
-		public string Username => userPrincipal?.Name ?? identity.Name;
-
 		/// <inheritdoc />
 		public ISystemIdentity Clone()
 		{
 			if (identity != null)
-				return new WindowsSystemIdentity((WindowsIdentity)identity.Clone());
+			{
+				//var newIdentity = (WindowsIdentity)identity.Clone();	//doesn't work because of https://github.com/dotnet/corefx/issues/31841
+
+				var newIdentity = new WindowsIdentity(identity.Token);	//the handle is cloned internally
+
+				return new WindowsSystemIdentity(newIdentity);
+			}
 			//can't clone a UP, shouldn't be trying to anyway, cloning is for impersonation
 			throw new InvalidOperationException("Cannot clone a UserPrincipal based WindowsSystemIdentity!");
 		}

src/Tgstation.Server.Host/Security/WindowsSystemIdentityFactory.cs

@@ -1,7 +1,7 @@
-using Microsoft.Win32.SafeHandles;
+using Microsoft.Extensions.Logging;
+using Microsoft.Win32.SafeHandles;
 using System;
 using System.DirectoryServices.AccountManagement;
-using System.Runtime.InteropServices;
 using System.Security.Principal;
 using System.Threading;
 using System.Threading.Tasks;
@@ -14,6 +14,33 @@ namespace Tgstation.Server.Host.Security
 	/// </summary>
 	sealed class WindowsSystemIdentityFactory : ISystemIdentityFactory
 	{
+		/// <summary>
+		/// The <see cref="ILogger"/> for the <see cref="WindowsSystemIdentityFactory"/>
+		/// </summary>
+		readonly ILogger<WindowsSystemIdentityFactory> logger;
+
+		/// <summary>
+		/// Extract the username and domain name from a <see cref="string"/> in the format "username\\domainname"
+		/// </summary>
+		/// <param name="input">The input <see cref="string"/></param>
+		/// <param name="username">The output username</param>
+		/// <param name="domainName">The output domain name. May be <see langword="null"/></param>
+		static void GetUserAndDomainName(string input, out string username, out string domainName)
+		{
+			var splits = input.Split('\\');
+			username = splits.Length > 1 ? splits[1] : splits[0];
+			domainName = splits.Length > 1 ? splits[0] : null;
+		}
+
+		/// <summary>
+		/// Construct a <see cref="WindowsSystemIdentityFactory"/>
+		/// </summary>
+		/// <param name="logger">The value of logger</param>
+		public WindowsSystemIdentityFactory(ILogger<WindowsSystemIdentityFactory> logger)
+		{
+			this.logger = logger ?? throw new ArgumentNullException(nameof(logger));
+		}
+
 		/// <inheritdoc />
 		public Task<ISystemIdentity> CreateSystemIdentity(User user, CancellationToken cancellationToken) => Task.Factory.StartNew(() =>
 		{
@@ -25,26 +52,36 @@ namespace Tgstation.Server.Host.Security
 
 			PrincipalContext pc = null;
 			UserPrincipal principal = null;
-			//machine logon first cause it's faster
-			pc = new PrincipalContext(ContextType.Machine);
-			principal = UserPrincipal.FindByIdentity(pc, user.SystemIdentifier);
-			if (principal == null)
+
+			bool TryGetPrincipalFromContextType(ContextType contextType)
 			{
-				pc.Dispose();
-				//try domain now
 				try
 				{
-					pc = new PrincipalContext(ContextType.Domain);
+					pc = new PrincipalContext(contextType);
+					cancellationToken.ThrowIfCancellationRequested();
 					principal = UserPrincipal.FindByIdentity(pc, user.SystemIdentifier);
 				}
-				catch (PrincipalServerDownException) { }
-
-				if (principal == null)
+				catch (OperationCanceledException)
 				{
-					pc?.Dispose();
-					return null;
+					throw;
 				}
-			}
+				catch (Exception e)
+				{
+					logger.LogWarning("Error loading user for context type {0}! Exception: {1}", contextType, e);
+				}
+				finally
+				{
+					if (principal == null)
+					{
+						pc?.Dispose();
+						cancellationToken.ThrowIfCancellationRequested();
+					}
+				}
+				return principal != null;
+			};
+			
+			if (!TryGetPrincipalFromContextType(ContextType.Machine) && !TryGetPrincipalFromContextType(ContextType.Domain))
+				return null;
 			return (ISystemIdentity)new WindowsSystemIdentity(principal);
 		}, cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Current);
 
@@ -55,11 +92,18 @@ namespace Tgstation.Server.Host.Security
 				throw new ArgumentNullException(nameof(username));
 			if (password == null)
 				throw new ArgumentNullException(nameof(password));
-			var splits = username.Split('\\');
 
-			var res = NativeMethods.LogonUser(splits.Length > 1 ? splits[1] : splits[0], splits.Length > 1 ? splits[0] : null, password, 3 /*LOGON32_LOGON_NETWORK*/, 0 /*LOGON32_PROVIDER_DEFAULT*/, out var token);
+			var originalUsername = username;
+			GetUserAndDomainName(originalUsername, out username, out var domainName);
+
+			var res = NativeMethods.LogonUser(username, domainName, password, 3 /*LOGON32_LOGON_NETWORK*/, 0 /*LOGON32_PROVIDER_DEFAULT*/, out var token);
 			if (!res)
+			{
+				logger.LogTrace("Failed to log in username {0}!", originalUsername);
 				return null;
+			}
+
+			logger.LogTrace("Successfully logged in username {0}!", originalUsername);
 
 			using (var handle = new SafeAccessTokenHandle(token))   //checked internally, windows identity always duplicates the handle when constructed with a userToken
 				return (ISystemIdentity)new WindowsSystemIdentity(new WindowsIdentity(handle.DangerousGetHandle()));   //https://github.com/dotnet/corefx/blob/6ed61acebe3214fcf79b4274f2bb9b55c0604a4d/src/System.Security.Principal.Windows/src/System/Security/Principal/WindowsIdentity.cs#L271

src/Tgstation.Server.Host/Server.cs

@@ -3,7 +3,6 @@ using Microsoft.Extensions.DependencyInjection;
 using System;
 using System.Collections.Generic;
 using System.IO;
-using System.Diagnostics.CodeAnalysis;
 using System.Linq;
 using System.Threading;
 using System.Threading.Tasks;
@@ -68,7 +67,6 @@ namespace Tgstation.Server.Host
 		public void Dispose() => semaphore.Dispose();
 
 		/// <inheritdoc />
-		[ExcludeFromCodeCoverage]
 		public async Task RunAsync(CancellationToken cancellationToken)
 		{
 			using (cancellationTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken))

src/Tgstation.Server.Host/Tgstation.Server.Host.csproj

@@ -22,7 +22,7 @@
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="Byond.TopicSender" Version="1.1.2" />
+    <PackageReference Include="Byond.TopicSender" Version="1.1.4" />
     <PackageReference Include="Cyberboss.AspNetCore.AsyncInitializer" Version="1.2.0" />
     <PackageReference Include="Cyberboss.SmartIrc4net.Standard" Version="0.4.6" />
     <PackageReference Include="Discord.Net.WebSocket" Version="1.0.2" />
@@ -31,9 +31,9 @@
     <PackageReference Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="2.1.2" />
     <PackageReference Include="Microsoft.AspNetCore.Identity" Version="2.1.3" />
     <PackageReference Include="Microsoft.AspNetCore.Mvc" Version="2.1.2" />
-    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.1">
+    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.6.2">
       <PrivateAssets>all</PrivateAssets>
-      <IncludeAssets>compile; build; native; contentfiles; analyzers</IncludeAssets>
+      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
     </PackageReference>
     <PackageReference Include="Microsoft.EntityFrameworkCore" Version="2.1.3" />
     <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="2.1.3" PrivateAssets="All" />

tests/Tgstation.Server.Host.Service.Tests/TestServerService.cs

@@ -18,11 +18,11 @@ namespace Tgstation.Server.Host.Service.Tests
 		[TestMethod]
 		public void TestConstructionAndDisposal()
 		{
-			Assert.ThrowsException<ArgumentNullException>(() => new ServerService(null, null));
+			Assert.ThrowsException<ArgumentNullException>(() => new ServerService(null, null, default));
 			var mockWatchdogFactory = new Mock<IWatchdogFactory>();
-			Assert.ThrowsException<ArgumentNullException>(() => new ServerService(mockWatchdogFactory.Object, null));
+			Assert.ThrowsException<ArgumentNullException>(() => new ServerService(mockWatchdogFactory.Object, null, default));
 			var mockLoggerFactory = new LoggerFactory();
-			new ServerService(mockWatchdogFactory.Object, mockLoggerFactory).Dispose();
+			new ServerService(mockWatchdogFactory.Object, mockLoggerFactory, default).Dispose();
 		}
 
 		[TestMethod]
@@ -40,7 +40,7 @@ namespace Tgstation.Server.Host.Service.Tests
 			var mockLoggerFactory = new LoggerFactory();
 			mockWatchdogFactory.Setup(x => x.CreateWatchdog(mockLoggerFactory)).Returns(mockWatchdog.Object).Verifiable();
 
-			using (var service = new ServerService(mockWatchdogFactory.Object, mockLoggerFactory))
+			using (var service = new ServerService(mockWatchdogFactory.Object, mockLoggerFactory, default))
 			{
 				onStart.Invoke(service, new object[] { args });
 

tests/Tgstation.Server.Host.Watchdog.Tests/TestWatchdogFactory.cs

@@ -1,6 +1,5 @@
 using Microsoft.Extensions.Logging;
 using Microsoft.VisualStudio.TestTools.UnitTesting;
-using Moq;
 
 namespace Tgstation.Server.Host.Watchdog.Tests
 {