Code walkthrough almost everything except component code [DMDeploy]

.github/CONTRIBUTING.md

@@ -46,6 +46,10 @@ The following environment variables aren't required but enable more tests.
 - `TSG4_TEST_DISCORD_TOKEN`: To a valid discord bot token.
 - `TGS4_TEST_DISCORD_CHANNEL`: To a valid discord channel ID that the above bot can access.
 
+### Know your Code
+
+The `/src` folder at the root of this repository contains a series of `README.md` files useful for helping find your way around the codebase.
+
 ## Specifications
 
 As mentioned before, you are expected to follow these specifications in order to make everyone's lives easier. It'll save both your time and ours, by making sure you don't have to make any changes and we don't have to ask you to. Thank you for reading this section!

src/DMAPI/tgs/README.md

@@ -0,0 +1,13 @@
+# DMAPI Internals
+
+This folder should be placed on it's own inside a codebase that wishes to use the TGS DMAPI. Warranty void if modified.
+
+- [includes.dm](./includes.dm) is the file that should be included by DM code, it handles including the rest.
+- The [core](./core) folder includes all code not directly part of any API version.
+- The other versioned folders contain code for the different DMAPI versions.
+    - [v3210](./v3210) contains the final TGS3 API.
+    - [v4](./v4) is the legacy DMAPI 4 (Used in TGS 4.0.X versions).
+    - [v5](./v5) is the current DMAPI version used by TGS4 >=4.1.
+- [LICENSE](./LICENSE) is the MIT license for the DMAPI.
+
+APIs communicate with TGS in two ways. All versions implement TGS -> DM communication using /world/Topic. DM -> TGS communication, called the bridge method, is different for each version.

src/DMAPI/tgs/core/README.md

@@ -0,0 +1,8 @@
+# Core DMAPI functions
+
+This folder contains all DMAPI code not directly involved in an API.
+
+- [_definitions.dm](./definitions.dm) contains defines needed across DMAPI internals.
+- [core.dm](./core.dm) contains the implementations of the `/world/proc/TgsXXX()` procs. Many map directly to the `/datum/tgs_api` functions. It also contains the /datum selection and setup code.
+- [datum.dm](./datum.dm) contains the `/datum/tgs_api` declarations that all APIs must implement.
+- [tgs_version.dm](./tgs_version.dm) contains the `/datum/tgs_version` definition

src/DMAPI/tgs/v3210/README.md

@@ -0,0 +1,6 @@
+# DMAPI V3
+
+This DMAPI implements bridge using file output which TGS monitors for.
+
+- [api.dm](./api.dm) contains the bulk of the API code.
+- [commands.dm](./commands.dm) contains functions relating to `/datum/tgs_chat_command`s.

src/DMAPI/tgs/v4/README.md

@@ -0,0 +1,6 @@
+# DMAPI V4
+
+This DMAPI implements bridge requests using file output which TGS monitors for. It has a safe mode restriction.
+
+- [api.dm](./api.dm) contains the bulk of the API code.
+- [commands.dm](./commands.dm) contains functions relating to `/datum/tgs_chat_command`s.

src/README.md

@@ -0,0 +1,11 @@
+# Guided Code Tour
+
+This is a series of README.md files aimed at directing people around the codebase based on what they want to know/change. Best viewed on the GitHub website.
+
+- To explore the server code navigate to [Tgstation.Server.Host](./Tgstation.Server.Host).
+- To explore the DreamMaker public API navigate to [DMAPI/tgs.dm](./DMAPI/tgs.dm).
+- To explore the DMAPI internals, navigate to [DMAPI/tgs](./DMAPI/tgs).
+- To explore the HTTP API definitions navigate to [Tgstation.Server.Api](./Tgstation.Server.Api).
+- To explore the C# client code navigate to [Tgstation.Server.Client](./Tgstation.Server.Client).
+
+[Tgstation.Server.Host.Watchdog](./Tgstation.Server.Host.Watchdog), [Tgstation.Server.Host.Service](./Tgstation.Server.Host.Service), and [Tgstation.Server.Host.Console](./Tgstation.Server.Host.Console) are related to the Service/Console runners which have the simple task of executing Tgstation.Server.Host and updating it when requested.

src/Tgstation.Server.Api/README.md

@@ -0,0 +1,7 @@
+# TGS API Definitions
+
+This assembly defines models and routes for communicating with TGS.
+
+- Data models are stored in the [Models](./Models) directory.
+- Rights enums are stored in the [Rights](./Rights) directory.
+- API routes are defined in the [Routes.cs](./Routes.cs) file.

src/Tgstation.Server.Host/ClientApp/README.md

@@ -0,0 +1,3 @@
+# Web Control Panel Code
+
+This code is responsible for including the selected [tgstation-server-control-panel](https://github.com/tgstation/tgstation-server-control-panel) version with the build.

src/Tgstation.Server.Host/Controllers/README.md

@@ -0,0 +1,15 @@
+# API Controllers
+
+Most of these controllers map to their specific routes. See the public functions marked with `[HttpXXX]` attributes for where the web meets code.
+
+Some notable exceptions:
+
+- [ApiController](./ApiController.cs) is the base class of nearly all API related controllers. It does the following:
+    - Contains code to deny the request if the instance is not present when it should be.
+    - Contains the `IDatabaseContext` and `ILogger` properties for child controllers.
+    - Returns 426 Upgrade Required if the API version in the headers are incompatible with the request.
+    - Returns 400 Bad Request if the headers or the PUT/POST'd model is invalid.
+    - Returns 401 If an `IAuthenticationContext` could not be created for a request.
+- [BridgeController](./BridgeController.cs) is a special controller accessible only from localhost and is used to receive bridge request from DreamDaemon
+- [HomeController](./HomeController.cs) contains the code to initially log in and generate an API token for a user.
+- [TgsAuthorizeAttribute](./TgsAuthorizeAttribute.cs) is a special attribute applied to controller methods to define which rights are required to run a verb.

src/Tgstation.Server.Host/Core/README.md

@@ -0,0 +1,13 @@
+# Core Services
+
+This is a bag of classes used throughout TGS that don't quite belong anywhere else.
+
+- [Application](./Application.cs) is our main [composition root](https://freecontent.manning.com/dependency-injection-in-net-2nd-edition-understanding-the-composition-root/).
+- [IAsyncDelayer](./IAsyncDelayer.cs) and [implementation](./AsyncDelayer.cs) is a class used to sleep code. It's generally a no-op in test scenarios.
+- [IGitHubClientFactory](./IGitHubClientFactory.cs) and [implementation](./GitHubClientFactory.cs) is a class used to create GitHub API clients using [ocktokit.net](https://github.com/octokit/octokit.net).
+- [IRestartHandler](./IRestartHandler.cs) and [IRestartRegistration](./IRestartRegistration.cs) are a set of interface services use when they want to be aware of a TGS restart/update (i.e. This is how the watchdog know to detach instead of shutdown).
+- [IServerControl](./IServerControl.cs) is an interface used to initiate a restart or update the server.
+- [IServerPortProvider](./IServerPortProvider.cs) and [implementation](./ServerPortProvider.cs) is used by services to determine the local TGS API port. Used mainly for telling DreamDaemon where to make bridge requests.
+- [OpenApiEnumVarNamesExtension](./OpenApiEnumVarNamesExtension) implements the [x-var-names OpenAPI 3.0 extension](https://github.com/OpenAPITools/openapi-generator/blob/master/docs/templating.md#enum) in our generated API json.
+- [SemaphoreSlimContext](./SemaphoreSlimContext.cs) is a helper class for working with [.NET asynchronous sempahores](https://docs.microsoft.com/en-us/dotnet/api/system.threading.semaphoreslim?view=netcore-3.1).
+- [SwaggerConfiguration](./SwaggerConfiguration.cs) configures [Swashbuckle](https://github.com/domaindrivendev/Swashbuckle.AspNetCore) to generate our OpenAPI specification.

src/Tgstation.Server.Host/Database/README.md

@@ -0,0 +1,11 @@
+# Database Interaction Classes
+
+The star of the show here is [IDatabaseContext](./IDatabaseContext.cs) which represents a connection to the database via the ORM across the server. The ORM used is [Entity Framework Core](https://docs.microsoft.com/en-us/ef/core/).
+
+- [IDatabaseCollection](./IDatabaseCollection.cs) and [implementation](./DatabaseCollection.cs) is a wrapper around EF's [DbSet<TModel>](https://docs.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbset-1?view=efcore-3.1) class. Used primarily for abstraction.
+- [IDatabaseConnectionFactory](./IDatabaseConnectionFactory.cs) and [implementation](./DatabaseConnectionFactory.cs) is used for creating raw database connections. Mainly for testing purposes in the setup wizard.
+- [IDatabaseContextFactory](./IDatabaseContextFactory.cs) and [implementation](./DatabaseContextFactory.cs) is used to create connections to the database. Used primarily within a component or job context.
+- [IDatabaseSeeder](./IDatabaseSeeder.cs) and [implementation](./DatabaseSeeder.cs) is used to seed the admin user, reset the admin password if necessary, and clean up known bad data.
+- The various XXXDatabaseContext classes override the abstract [DatabaseContext](./DatabaseContext.cs) class to implement it for the various DB backends.
+- The [Design](./Design) directory contains classes used when generation migrations.
+- The [Migrations](./Migrations) directory contains the migrations for each backend.

src/Tgstation.Server.Host/IO/README.md

@@ -0,0 +1,12 @@
+# IO Related Classes
+
+- [IConsole](./IConsole.cs) and [implementation](./Console.cs) provide methods for writing to the console.
+- [IIOManager](./IIOManager.cs) is the primary method of performing file I/O across the server. It comes in two flavors.
+    - [DefaultIOManager](./DefaultIOManager.cs) implements the interface as expected.
+    - [ResolvingIOManager](./ResolvingIOManager.cs) implements the interface with a custom [working directory](https://en.wikipedia.org/wiki/Working_directory).
+- [IPostWriteHandler](./IPostWriteHandler.cs) dictates a set of actions to take after writing a file.
+    - The [WindowsPostWriteHandler](./WindowsPostWriteHandler.cs) is a no-op.
+    - The [PosixPostWriteHandler](./PosixPostWriteHandler.cs) sets +x on files.
+- [ISymlinkFactory](./ISymlinkFactory.cs) is used to create [symbolic links](https://en.wikipedia.org/wiki/Symbolic_link) across the server.
+- [ISynchronousIOManager](./ISynchronousIOManager.cs) and [implementation](./SynchronousIOManager.cs) is a subset of `IIOManager` functions that are performed in a blocking manner.
+    - The only current use of this is to perform `ISystemIdentity` impersonated operations while keeping the OS security context of the executing thread. This means that `async` and `Task`s cannot be used as they make the active thread non-deterministic.

src/Tgstation.Server.Host/Jobs/README.md

@@ -0,0 +1,5 @@
+# Jobs Subsystem
+
+- [IJobManager](./IJobManager.cs) and [implementation](./JobManager.cs) is where the bulk of the magic happens. The `RegisterOperation()` call is what takes a work unit and sets it to run asynchronously while being tracked through the API.
+- [JobException] is a special .NET Exception implementation that is able to carry API `ErrorCode`s and other additional data.
+- [JobHandler](./JobHandler.cs) carries the [CancellationTokenSource](https://stackoverflow.com/questions/20638952/cancellationtoken-and-cancellationtokensource-how-to-use-it) for a given job in a disposable context.

src/Tgstation.Server.Host/README.md

@@ -0,0 +1,41 @@
+# Tgstation.Server.Host
+
+This assembly is the primary TGS executable and does basically everything.
+
+Before going forward, note that there (usually) is a one-to-one mapping of interface/implementation when it comes to classes. This is for ease of unit testing and facilitating saner dependency injection.
+
+Server startup can be a bit complicated so here's a walkthrough
+
+1. The `Main` method in [Program.cs](./Program.cs) is called.
+1. The default [IServerFactory](./IServerFactory) instance is created via a static method in the [Application](./Core/Application.cs) class.
+    - The `Application` class is what's called the composition root. A good article on this principle can be found [here](https://freecontent.manning.com/dependency-injection-in-net-2nd-edition-understanding-the-composition-root/).
+1. `CreateServer()` is called on the `IServerFactory` to get the `IServer` instance.
+    - The factory pattern is used throughout TGS to construct implementations where the composition root is not sufficient. `ServerFactory` is somewhat of an exception to this because it exists outside of the dependency injection umbrella.
+1. Inside `CreateServer()` we run the [setup code](./Setup) if need be.
+    - This is implemented as a separate [dotnet host](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/generic-host?view=aspnetcore-3.1) to the main server.
+1. Still inside `CreateServer()` we configure the main [dotnet host (IHostBuilder)](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/generic-host?view=aspnetcore-3.1) using the application [Application](./Core/Application.cs) class as the [Startup class](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/startup?view=aspnetcore-3.1#the-startup-class).
+1. The `IHostBuilder` is used to construct the return `Server` implementation.
+1. `Run()` is called on the `IServer` instance.
+1. The DI container is built using the [Application](./Core/Application.cs) class.
+1. The [component services](./Components) are started.
+1. The web server is started.
+
+Here's a breakdown of things in this directory
+
+- [.config](./.config) contains the dotnet-tools.json. At the time of writing, this is only used to set the version of the [dotnet ef tools](https://docs.microsoft.com/en-us/ef/core/miscellaneous/cli/) used to create database migrations.
+- [ClientApp](./ClientApp) contains scripts to build and deploy the web control panel with TGS.
+- [Components](./Components) is where the bulk of the TGS implementation lives.
+- [Configuration](./Configuration) contains classes that partly make up the configuration json files (i.e. [appsettings.json](./appsettings.json)).
+- [Controllers](./Controllers) is where HTTP API code lives and bridges it with component code.
+- [Core](./Core) contains [Application.cs](./Core/Application.cs) and other various helpers that don't belong anywhere else.
+- [Database](./Database) contains all database related code.
+- [Extensions](./Extensions) contains helper functions implemented as C# extension methods.
+- [IO](./IO) contains classes related to interacting with the filesystem.
+- [Jobs](./Jobs) contains the job manager code.
+- [Models](./Models) contains ORM models used to interact with the database.
+- [Properties](./Properties) contains some assembly metadata, mainly used to expose internals to the testing suite.
+- [Security](./Security) contains all the security related classes.
+- [Setup](./Setup) contains code to initiate and run the setup wizard.
+- [System](./System) contains various OS related functions.
+
+As a final note, all project configuration is of course in the [Tgstation.Server.Host.csproj](./Tgstation.Server.Host.csproj) file. This contains package references that are pulled in on build, as well as other things like global warning supressions, and specialized build scripts.

src/Tgstation.Server.Host/Security/README.md

@@ -0,0 +1,10 @@
+# Security Classes
+
+- [IAuthenticationContext](./IAuthenticationContext.cs) and [implementation](./AuthenticationContext.cs) is what contains information about an authenticated user for a request. Includes things like the relevant `InstanceUser` and any associated rights.
+- [IAuthenticationContextFactory](./IAuthenticationContextFactory.cs) and [implementation](AuthenticationContextFactory.cs) is a factory for `IAuthenticationContext`s. It handles things related to the database for a user's authentication. This includes loading their rights/associated instance user. It will also stop the request if the users token was issued before the last time their password or enabled status was updated.
+- [IClaimsInjector](./IClaimsInjector.cs) and [implementation](./ClaimsInjector.cs) is used to associate rights with a request context so that it may properly pass appropriate `TgsAuthorizeAttribute`s.
+- [ICrytopgraphySuite](./ICrytopgraphySuite.cs) and [implementation](./CrytopgraphySuite.cs) is used to generate secure strings and byte arrays. It also contains the password hashing and validation logic.
+- [IIdentityCache](./IIdentityCache.cs) and [implementation](./IdentityCache.cs) is used to store `ISystemIdentity`s for the duration of their associated tokens as [IdentityCacheObject](./IdentityCacheObject.cs)s.
+- [ITokenFactory](./ITokenFactory.cs) and [implementation](./TokenFactory.cs) is used to generate the Json Web Token for a session after a user successfully authenticates.
+- [ISystemIdentity](./ISystemIdentity.cs)s represent a logon session with the operating system for a given user. It contains a method to run code under the security context of said user.
+- [ISystemIdentityFactory](./ISystemIdentityFactory.cs) is used to create `ISystemIdentity`s by attempting to log the user in with the OS with a given username and password.

src/Tgstation.Server.Host/Setup/README.md

@@ -0,0 +1,6 @@
+# Setup Code
+
+This is the code used to facilitate running the [SetupWizard](./SetupWizard.cs).
+
+- [SetupApplication](./SetupApplication.cs) is used to configure services used by the setup wizard. It is the parent class of `Application`.
+- [IPostSetupServices](./IPostSetupServices.cs) and [implementation](./PostSetupServices.cs) is used to provide certain services `Application` depends on at configuration time after the setup wizard has (or has not) run.

src/Tgstation.Server.Host/System/README.md

@@ -0,0 +1,8 @@
+# System Services
+
+- [IAssemblyInformationProvider](./IAssemblyInformationProvider.cs) and [implementation](./AssemblyInformationProvider.cs) is used to provide server build metadata to code.
+- [INetworkPromptReaper](./INetworkPromptReaper.cs) exists mainly to prevent DreamDaemon dialogs from popping up when using `/world/proc/OpenPort()` in DM code.
+    - Currently it's only implemented as the [WindowsNetworkPromptReaper](./WindowsNetworkPromptReaper.cs)
+- [IPlatformIdentifier](./IPlatformIdentifier.cs) and [implementation](./PlatformIdentifier.cs) provides detection of the current operating system.
+- The process-related interfaces and classes are used to provide a nice level of abstration around process execution.
+- [ProgramShutdownTokenSource](./ProgramShutdownTokenSource.cs) generates a `CancellationToken` that triggers when a quit signal is sent to the process.