CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

Added

• the makefiles build the C++20 module smspp.svmblock when SMSppMOD is not empty (e.g., make SMSppMOD=1), with clang++ 16 or later [see lib/makefile-mod of SMS++]

• the C++20 module smspp.svmblock (src/SVMBlock.cppm), which exports every name that the headers declare at namespace scope and re-exports smspp, so that a source file can write import smspp.svmblock; in place of the #include of the headers; it is built with the CMake option SMSpp_BUILD_MODULE, as the target SMS++::SVMBlock_module

Changed

• the pipeline builds the module also with its makefiles, after CMake, and runs on a change of the makefiles too [see ci/makefile of the umbrella]

• SMSpp_BUILD_MODULE no longer stops the configuration when the C++20 module cannot be built, i.e., with a CMake older than 3.28, a generator other than Ninja or Visual Studio, a compiler other than Clang 16 (with clang-scan-deps) or MSVC 19.34, or a module it imports that is not there: the option is turned off with a warning that says why, and the build goes on with the headers [see cmake/CheckModule.cmake of SMS++]

• the instances of data/ are downloaded and extracted by the targets download_svm_<fmt> and extract_svm_<fmt>, written as in every module that keeps its instances in the Package Registry, and the marker of the extraction carries the format in its name, so that a tree extracted before extracts once more

Fixed

• a build of the module alone, with find_package(), fetches FastFlow, which SVMBlock.cpp includes and which an installed SMS++ does not carry [see cmake/FastFlow.cmake of SMS++]

• the makefiles of the tests and of the tools build also with g++: they take the option that stops at the first error and the one of the debugging information, which clang++ and g++ spell differently (-ferror-limit=1 and -glldb against -fmax-errors=1 and -ggdb), from SMSppSWERR and SMSppSWDBG of lib/makefile-OS of SMS++

0.3.0 - 2026-10-09

Added

• SVMBlock::set_active_features(), which features the kernel reads: the samples stay where they are and an inactive feature counts as a column of zeroes, in the Gram matrix, in the coefficients of the model in the primal and in the weights the dual gives back, so that a selection of the features is a change of the training problem and not a copy of the data set. With the linear kernel switching one off is a rank-one change of the Hessian of the dual, which SMOSolver follows by correcting the gradient in O( N ) operations per feature rather than starting over

• LIBSVMSolver::get_elapsed_iterations(), how many iterations the last call to compute() took: the library has no accessor for the count, which it only prints, hence what is returned is read out of what it prints, which is what makes a comparison able to say whether a difference is in the iterations or in what each of them costs

• SVMBlock::set_sparse_density(), the density below which a sample is read as the list of its nonzeroes rather than as m consecutive doubles: zero reads them always dense and one always sparse, the default being the tenth at which the merge of two lists stops paying. The value of the kernel is the same either way, the entries the merge skips being zeroes, so that this changes the time and not what any algorithm does

Changed

• the data archive is downloaded by version: DATA_VERSION in CMakeLists.txt names the version of the Package Registry to read, and the archive and the marker of its extraction carry it in their name, so that a tree holding an older extraction (the cache of the CI, or a clone extracted before) downloads and extracts again instead of running on the old data; data/upload-libsvm publishes the archive under that version

• a file in the sparse format of LIBSVM may carry any two labels (e.g., 0 and 1, or 2 and 4), as LIBSVM itself accepts: SVCBlock turns the smaller into -1 and the larger into +1, and rejects more than two

• dblLSVMCache 0 gives LIBSVM the budget the SVMBlock gives its own cache of the Gram matrix, so that LIBSVM and SMOSolver can be compared with the same memory; the default stays at the 100 MB of LIBSVM

• the tester reads the groups of Variable and of Constraint of the Block, the vectors of boost::any it used to read not being there any more

• the makefile asks for -O3 -DNDEBUG and nothing else, the macro of the patch for boost::any on macOS having no reason to be there since there is no boost::any left in the core

• whoever links the module keeps it: the classes of a module register themselves in the factory from a static initialiser, and a linker that drops what looks unused takes the registration away with it, so the target now tells whoever links it to keep the symbol that forces the module in, and on ELF, where naming the symbol is not enough, the library as a whole

• a budget that would pay for the whole Gram matrix is no longer a reason to compute it: what decides is how many of its rows the algorithm reads, which is a fraction of them whenever the support set is one, so the rows are computed as they are read and a budget above the matrix simply means that the cache never evicts anything. On rcv1 (20242 samples, whose matrix is 3.3 GB) under a budget of 4 GB this is 45.2 seconds against 90.5, the optimal value being the same; the whole matrix is still built for whoever asks for the whole of it, such as the abstract representation of the dual

• the restore of the active set recomputes the gradient in whichever of the two ways costs less at that moment: a row of each multiplier that is neither zero nor at a bound, read at full width, or a row of each index being restored, read at the width of the active set alone. The count of the kernel evaluations the two ask for decides, the first being one per free multiplier and index of the data set and the second one per index restored and index of the active set

• the changes of the abstract representation that one setter causes travel as a single group: a hyper-parameter can touch the bounds, the sides and the Objective at once, and each of the three used to issue its own, so that whoever receives them now receives the whole change in one pass, with the group of each piece nested into it

• the three Solver of the module answer get_Solution() with the SVMBlockSolution, i.e., the trained model, whatever the Configuration asks for, and say so in is_get_Solution_physical(): a Solver that has the model and no abstract representation to fill has one part of the solution information to give, and whoever wants it in the Variable writes the Solution into the SVMBlock

• the restore of the active set, which the shrinking takes every so often so that a collapsed one cannot grind, costs what it should: the pair is selected out of the whole index space again for one pass rather than for a whole period of them, each of which would read a row of the Gram matrix as wide as the data set, and the gradient it recomputes no longer reads one such row per nonzero multiplier. Those that sit at their upper bound contribute the same thing until they leave it, so their part is kept from one restore to the next and brought up to date with one row per multiplier that has changed side, as LIBSVM does. On w8a under a budget of 200 MB for the rows this is 2.0 billion evaluations of the kernel against 252 million, i.e., 18 seconds against 101

Fixed

• on macOS a program linking the module lost the classes the module registers in the factories when the linker dropped the library, as it does under -dead_strip_dylibs, which conda sets: the target now asks the linker for the symbol that forces the module in (-u), which ld64, unlike the ELF linker, counts as a use of the library

• the step that fetches the data archive of this module says what went wrong when it goes wrong: the download is checked, an archive that did not arrive is removed instead of being left on disk for the build to take for the real one, and the message names the URL. A server that answers with an error page used to leave a file of a few bytes there, which made the next build fail while extracting it, with the message of tar and no mention of the download

• when no multiplier can absorb the movement of the index being learnt or unlearnt, SMOSolver moves the bias in the direction of that movement, i.e., raising the margin of the index when its multiplier is to grow and lowering it when the multiplier is to be driven to zero, and stops early only when learning; when unlearning, the direction was that of learning and the walk could stop as soon as the margin of the index was met, with the multiplier still positive

• the lists of the nonzeroes are built before the threads that fill the Gram matrix are started, and not by whichever of them asks for a kernel first: they all ask at once, and they were all building them, which corrupts the heap. The value of gamma derived from the data was already resolved there for the same reason

0.2.0 - 2026-09-13

Added

• the samples are read as the list of their nonzeroes when the data set is sparse, which is how the kernel is computed there: the dense reading costs m operations per evaluation whatever the data holds, the merge of two such lists costs their nonzeroes, and the second is worth it below about a tenth of density, where the lists are built and above which they are not. The Gaussian kernel goes through the two squared norms and the inner product, the Laplacian one walks the union rather than the intersection, and the value is the very same the dense reading gives, the entries that are skipped being zeroes: not a single iteration changes. On w8a, which is 3.9 per cent full, this is a factor of 4.5 on the solve

0.1.0 - 2026-09-12

Added

• SVMBlock, the abstract base class holding the data set, the hyper-parameters, the kernel and both the training problem and its Wolfe dual, the latter written as the maximisation it customarily is, so that the value of the Objective is the same number whichever is encoded

• SVCBlock and SVRBlock, the classification and the regression variants

• SMOSolver, the ad hoc Sequential Minimal Optimization solver for the dual

• netCDF and text serialization, and the tester

• LIBLINEARSolver, which hands the training problem over to LIBLINEAR: the linear kernel only and the regularised bias only, but both losses and any weight of the regularisation term, which is the side of the problem LIBSVMSolver does not cover. A run that stops on the cap LIBLINEAR has on its own iterations is reported as kLowPrecision

• get_K_row(), which serves one row of the Gram matrix, and set_K_memory(), which says how much memory the matrix may take: under that budget the whole matrix is built as before, over it the rows are computed on demand and kept in a cache of the least recently used ones, so that a data set whose Gram matrix does not fit in memory can be trained anyway. set_K_active() tells the cache which entries are going to be read, and each row records which of its own it has already computed: an algorithm that shrinks its active set therefore pays for the entries it asks for, and completes a row rather than recomputing it when the set grows back

• add_samples() and remove_samples(), which change the data set instead of replacing it, with the eAddSamples and eRmvSamples Modification: the samples that stay are not touched, their multipliers keep their value and the Gram matrix is extended or compacted rather than recomputed. Whatever is indexed over the dual index space is dynamic accordingly, and the abstract representation is extended and shrunk rather than rebuilt. SMOSolver re-optimizes across them, which is what an incremental training, and a k-fold cross-validation done by taking a fold out and putting it back, need

• LIBSVMSolver, which hands the training problem over to LIBSVM: an independent implementation of the very algorithm SMOSolver implements, hence a reference to check it against, and a fast one for a large data set. It reads the physical representation, recovers the multipliers, the bias and the value out of the kernel expansion LIBSVM returns, and refuses the training problems LIBSVM cannot express rather than approximating them. It is built only when LIBSVM is found

• SVMBlockMod, SVMBlockRngdMod and SVMBlockSbstMod, the Modification describing a change of the training problem, and the abstract Modification issued alongside them; the hyper-parameters can therefore now be changed while the abstract representation is constructed and a Solver is attached, and so can the targets, through the new chg_target() and chg_targets()

• SMOSolver re-optimizes: it keeps the multipliers and the gradient of the dual at them across the calls to compute() and, whenever the change the Modification describe leaves the Hessian alone, it scales the former back into their bounds and updates the latter in linear time rather than restarting from the origin

• SMOSolver::get_Solution(), which builds the SVMBlockSolution out of the multipliers the Solver holds, without writing anything into the SVMBlock and therefore without requiring any Variable to exist; any other Solution saves the abstract representation, and is left to the base class. Requires the Solver::get_Solution() of the core

• SVMBlockSolution, the Solution saving the trained model rather than the abstract representation, which is what get_Solution() returns when the Configuration asks for it and what makes the model writable to a file

Changed

• the Gram matrix is filled by as many threads as the data set warrants, one every few hundred rows instead of one per core, and they are started once and reused: on a machine with hundreds of cores starting them costs more than filling the matrix does, and a training that builds many models paid it once per model

• the two dual indices of a sample of a SVRBlock are adjacent, rather than the two sides of the tube being one block each: a sample then adds its multipliers at the end of the dual index space, which is where a dynamic Variable is added. Note that this changes the order in which the multipliers are stored, hence that of a SVMBlockSolution written to a file

• the consensus rewriting is a structure of the SVMBlock, chosen by the new set_structure() out of the number of chunks, rather than a separate Block that a free function assembles: make_consensus_Block() is therefore gone, and with it the AbstractBlock that used to be the father. Since the SVMBlock still holds the whole data set, a Solver reading the physical representation keeps solving the very same training problem whatever the structure, so that SMOSolver can now be attached to the rewriting as well

• the Gram matrix of a data set large enough to be worth a thread is computed in parallel

• the version of the module is the git tag of its repository, or the VERSION.txt of a release tarball, and the shared library carries it: its SONAME is major.minor while the major is 0, and it is installed with an RPATH relative to itself, so that an installed tree keeps working wherever it is moved

Fixed

• the cache of the rows of the Gram matrix did not survive a change of the data set: it was laid out once on the number of samples of the moment, so adding samples wrote past the end of it, and its mask marked as present the bits past the end of a row, which stand for no entry until the row grows into them and then serve an entry that was never computed. Learning one sample at a time under a memory budget therefore either crashed or took a different number of iterations; it now takes the same number as with the whole matrix, which is the only right answer

• SVMBlockSolution::read() only looked at the physical representation, hence it saved nothing at all when the SVMBlock had been solved by a Solver working on the abstract one, which is where the model is then left

• loading a new data set left the SVMBlock with no abstract representation at all, so that a Solver reading it found an empty problem after the NBModification; it is now rebuilt out of the new data set

• the conventional values of gamma, which are derived from the data set, were derived again at each of the O( n^2 ) entries of the Gram matrix

• the library builds with MSVC, which rejects the FastFlow helpers that are declared static and not defined

• the package configuration file finds the libraries the module links, so that a project using the installed module needs nothing more than find_package()