SimpleLinearAlgebra

📐 Simple Linear Algebra for C#

A small, easy-to-understand matrix library for C# — the foundation of every from-scratch neural network in this repo.

Portability is the whole point: the library targets netstandard2.0, uses C# 7.3 and has zero dependencies, so it runs unchanged in modern .NET, .NET Framework, Mono and Unity (2018.1+). You can reference the project, consume the compiled DLL, or simply copy LinearAlgebra.cs, Tensor4.cs, Convolution.cs, Enums.cs and Helper.cs into a Unity Assets/ folder.

The Matrix type

using LinearAlgebra;

Matrix a = new double[,] { { 10, 20, 30 }, { 1, 3, 4 } }; // implicit conversion
Matrix identity = Matrix.Identy(2);
Matrix random = Matrix.Random(2, 3, new Random(123));

Matrix t = a.T;              // transpose
Matrix sum = a + 1;          // scalar and matrix addition/subtraction
Matrix scaled = a * 10;      // scalar multiplication / division
Matrix product = a.T * a;    // matrix multiplication
Matrix squared = a ^ 2;      // element-wise power

double avg = a.Avg;          // aggregates: Avg, Max, Abs
Matrix flat = a.Flat;        // flatten to a column, Deflat to restore
Matrix slice = a.Slice(0, 1, 2, 3);
Matrix withBias = a.AddColumn(Matrix.Ones(2, 1));
Matrix sums = a.Sumatory(AxisZero.horizontal);

⚠️ One quirk to know: a * b with equal dimensions is the element-wise (Hadamard) product; with compatible-but-different dimensions it is the matrix product. Use Matrix.MatMult/Matrix.DeltaMult to be explicit.

Helper adds persistence: SaveMatrix/LoadMatrix and SaveTensor/LoadTensor (compact dependency-free binary format) and CSV import/export with invariant-culture parsing.

The Tensor4 type and Convolution

Tensor4 is a rank-4 tensor with shape (n, c, x, y) — the natural layout for convolutional networks, where image batches are (batch, channel, height, width) and filter banks are (filter, channel, height, width). It offers elementwise operators (+, -, * scalar and Hadamard), GetMatrix/SetMatrix to move planes to and from Matrix, and Flatten/Unflatten to hand a batch over to a fully connected layer and take its gradient back.

Convolution builds trainable convolutional layers on top of it:

Tensor4 output = Convolution.Forward(input, kernels, bias, pad);     // batched conv, "same" with pad = (k-1)/2
Tensor4 pooled = Convolution.MaxPool(output, size, out var argmax);  // non-overlapping max-pooling

Tensor4 dKernels = Convolution.KernelGradients(input, dOutput, pad); // summed over the batch
double[] dBias   = Convolution.BiasGradients(dOutput);
Tensor4 dInput   = Convolution.InputGradients(dOutput, kernels, pad);
Tensor4 dPooled  = Convolution.MaxPoolBackward(dPooled, argmax, x, y);

All gradients are verified against numerical differentiation in the test suite.

Examples

Three generations of neural networks written directly with this library, from 2018, modernized and runnable on any OS:

Example	What it learns
MonoLayerPerceptron	XNOR, AND, OR and XOR at once — one hidden layer, manual backprop
MultiLayerPerceptron	XOR with an arbitrary stack of layers — the generalized version
ConvolutionalNeuralNetwork	Fashion-MNIST clothing images — two conv + max-pooling stages and a dense softmax head, trained end-to-end with mini-batch SGD + momentum

dotnet run --project examples/MonoLayerPerceptron        # optional arg: epochs
dotnet run --project examples/MultiLayerPerceptron       # optional arg: epochs
dotnet run --project examples/ConvolutionalNeuralNetwork # optional args: epochs, train count, test count
dotnet run --project examples/ConvolutionalNeuralNetwork -- 0    # evaluate the saved model, no retraining

The CNN example trains on Fashion-MNIST (28x28 grayscale clothing images, 10 classes). The dataset is not committed to the repo: on first run a tiny dependency-free downloader fetches the four official IDX files, inflates them into a gitignored data/ folder and parses them directly. The whole batch flows through the network as a Tensor4 — the input channel count comes from the dataset, so an RGB dataset would work unchanged — and backpropagation reaches every parameter: the softmax + cross-entropy head, both max-pooling stages and both convolutions, so the filters themselves learn (in the original 2018 project they stayed as fixed random feature extractors). Defaults train on a 2,000-image subset for a quick CPU demo (~80% test accuracy in a minute); pass bigger counts for better accuracy.

Tests

dotnet test tests/SimpleLinearAlgebra.Tests