Skip to content

Latest commit

 

History

History
460 lines (346 loc) · 31.3 KB

TESTGUIDE.md

File metadata and controls

460 lines (346 loc) · 31.3 KB

Test Guide for Microsoft.Data.SqlClient

This guide describes how to run the test projects in this repository and how to configure the SQL Server-backed manual tests.

For build prerequisites and general build.proj usage, see BUILDGUIDE.md.

Test Projects

The primary test projects for Microsoft.Data.SqlClient are under src/Microsoft.Data.SqlClient/tests:

Project Path Purpose
Unit tests src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj Unit tests and tests against simulated servers.
Functional tests src/Microsoft.Data.SqlClient/tests/FunctionalTests/Microsoft.Data.SqlClient.FunctionalTests.csproj Functional tests for public and internal behavior. Some tests use simulated servers or local test infrastructure.
Manual tests src/Microsoft.Data.SqlClient/tests/ManualTests/Microsoft.Data.SqlClient.ManualTests.csproj Integration tests that generally require a configured SQL Server or Azure SQL target.

These projects declare net462 and net10.0. The net462 tests require Windows and .NET Framework 4.6.2 or a compatible later runtime; the project framework lists are not automatically reduced on Linux/macOS. Select -p:TestFramework=net10.0 when using build.proj, or --framework net10.0 with dotnet test. The examples below select net10.0 unless stated otherwise. Install the .NET 10 runtime to run modern tests. Existing dual-target companion tests also use net462;net10.0; modern-only projects, including performance tests, use only net10.0. Stress projects use net462;net10.0. PackageCompatibility tool tests remain on net481;net10.0 with xUnit v3, and their Framework target requires Windows and .NET Framework 4.8.1. The historical tools/SniCloseLegacyRepro test matrix is unchanged.

Accessing Driver Internals

The driver grants InternalsVisibleTo access to UnitTests, ManualTests, and Microsoft.Data.SqlClient.TestCommon (the assembly names, not the project file names). Unsigned driver builds grant access in both Project and Package reference modes. Manual tests can use internal types and members directly, without reflection. The manual-test InternalsVisibleToTest verifies this access without SQL Server configuration.

In Package mode, tests that use internals must compile against the implementation assembly in lib/<tfm>/, not the public reference assembly in ref/<tfm>/. The manual-test project selects the implementation assembly explicitly.

Signed drivers grant internal access only in Package mode, to test assemblies signed with the shared test key. Signed Project builds do not grant test access. UnitTests, ManualTests, and their helpers use the sqlclient-test-key.snk secure file. This key is provisioned outside the repository and is not the production signing key.

For signed packages, pass -p:TestSigningKeyPath=<absolute-path-to-sqlclient-test-key.snk> to TestSqlClientManual or BuildSqlClientTestsManual in build.proj. The property is forwarded to the test project and signs its test-helper dependencies too. Package-mode ManualTests consumes the packaged AKV provider, so prepare that package alongside the driver. The internal Managed Instance and Kerberos package-test jobs download the secure file and pass its path explicitly.

FunctionalTests is not a friend assembly. Its linked internal source files are compiled into the test assembly; they do not demonstrate access to the driver's internals through InternalsVisibleTo.

Simulated server tests

The unit project's SimulatedServerTests use local TDS endpoints, not an installed SQL Server. Run this suite with TestSqlClientUnit and a FullyQualifiedName filter containing SimulatedServerTests. Validate discovery and the resulting TRX case counts when narrowing a run; the default category filter excludes quarantined tests.

For new scenarios, use explicit protocol gates rather than sleeps to decide which login phase fails. GatedTdsServer can withhold pre-login responses while keeping its port and existing sessions alive. This separates failover from post-login failures and allows pool clearing to be verified with physical connection identity. Use a numeric loopback address when DNS is not under test.

Declare SimulatedServerPoolScope after the servers and before the connections in pooling scenarios. It assigns a unique pool identity and clears that pool before the ephemeral servers are disposed. Keep retry/error/partner assertions explicit; transport attempt counts are not proof of pool reuse.

The simulator runs blocking handlers on dedicated workers. Fault budgets are reserved before response processing, and disposal must finish owned workers and report unexpected handler failures. Transport initialization and socket closure share a lock so shutdown cannot invalidate a TcpClient while its worker acquires the stream or initializes the parser. The worker-drain wait stays outside that lock, allowing a stopped worker to finish its cleanup. Negative TLS scenarios can set TdsServerArguments.ExpectTlsHandshakeFailure to explicitly expect authentication failure during TLS negotiation. Missing certificates/private keys and exceptions from protocol handlers still fail disposal; the default remains strict. Assert the expected client failure as well, and keep success controls so simulator setup errors cannot masquerade as certificate rejection. ServerLifecycleTests covers these contracts. Validate changes on both supported frameworks and on native and managed SNI, including broader simulator consumers before removing a flaky trait.

The manual CertificateTestWithTdsServer matrix requires an untrusted self-signed certificate. Its setup and cleanup scripts use -SkipTrust to leave root trust stores unchanged, and the test asserts UntrustedRoot before connecting. Installing that certificate as trusted invalidates the negative cases; explicit certificate pinning and TrustServerCertificate provide the success controls.

CI Platform Coverage

PR validation runs tests on Windows and Linux only. The project-reference and package-reference PR pipelines set runMacOSTests: false, excluding macOS SqlClient/AKV, Abstractions, and Azure extension tests and their packaging dependencies. The unified PR pipeline already has a Windows/Linux-only platform matrix.

The shared CI template defaults runMacOSTests to true, so scheduled and push-triggered CI retains macOS coverage. Local macOS testing remains supported. See Pipeline Script Tests for platform-selection regression checks.

Recommended Entry Point

Use build.proj from the repository root:

dotnet build build.proj -t:<test_target> [optional_parameters]

Since build.proj is the only project file in the repo root, it can be omitted when building from the root:

dotnet build -t:<test_target> [optional_parameters]

The command-line examples below will assume that build.proj is selected by default and will omit it from the dotnet build command.

In the default project-reference mode, test targets build the projects they depend on, so a separate build step is not required. In package mode, prepare the referenced packages first; see Package Mode Builds.

Test Targets

Before manual tests or their certificate/account setup, explicitly restore PowerShell modules from the repository root:

dotnet tool restore
dotnet build build.proj -t:RestorePowerShellModules

This restores all versions in powershell-modules.json for the current user and reuses matching installed versions. Test targets do not install modules automatically. The Unix certificate script calls the native openssl CLI; ensure it is available on PATH before certificate tests. See PowerShell module setup for feed and pipeline details.

Target Description
Test Runs the Abstractions, Azure, AKV Provider, and SqlClient test targets below. Excludes tool tests and performance/stress runners.
TestAbstractions Runs Microsoft.Data.SqlClient.Extensions.Abstractions tests.
TestAkvProvider Runs Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider tests.
TestAzure Runs Microsoft.Data.SqlClient.Extensions.Azure tests.
TestSqlClient Runs the SqlClient unit, functional, and manual test projects.
TestSqlClientUnit Runs Microsoft.Data.SqlClient unit tests.
TestSqlClientFunctional Runs Microsoft.Data.SqlClient functional tests.
TestSqlClientManual Runs Microsoft.Data.SqlClient manual tests.

Common Commands

Run the SqlClient unit tests:

dotnet build -t:TestSqlClientUnit -p:TestFramework=net10.0

Run the SqlClient functional tests:

dotnet build -t:TestSqlClientFunctional -p:TestFramework=net10.0

Run the SqlClient manual tests:

dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0

Run only manual test set 2:

dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0 -p:TestSet=2

Run manual test sets 1 and 3:

dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0 -p:TestSet=13

Run Always Encrypted manual tests:

dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0 -p:TestSet=AE

Run a specific target framework:

dotnet build -t:TestSqlClientFunctional -p:TestFramework=net10.0

Run functional tests against an x86 dotnet installation:

dotnet build -t:TestSqlClientFunctional -p:TestFramework=net10.0 -p:DotnetPath='C:\path\to\dotnet\x86\'

Run all Azure extension tests, including interactive tests, while still excluding tests marked failing or flaky:

dotnet build -t:TestAzure -p:TestFramework=net10.0 -p:TestFilters='category!=failing&category!=flaky'

Test Parameters

The most commonly used test parameters are:

Parameter Default Description
-p:Configuration= Debug Build configuration. Use Debug or Release.
-p:DotnetPath= Empty Path to the folder containing the dotnet binary. The path must end with \ or /.
-p:ReferenceType= Project For unit, functional, and manual SqlClient tests, use Project to test the source project or Package to test package references.
-p:TestBlameTimeout= 10m Enables hang blame collection with the specified timeout. Use 0 to disable hang timeouts.
-p:TestCodeCoverage= true Collects code coverage when set to true.
-p:TestFilters= category!=failing&category!=flaky&category!=interactive xUnit filter expression. Use none to disable this filter; test-set selection and conditional skips still apply.
-p:TestFramework= Empty Target framework to run. If omitted, all project frameworks are selected, including Windows-only net462.
-p:TestResultsFolderPath= test_results under the repository root Absolute directory where test results are written.
-p:TestSet= Empty Selects manual test sets. Supported values include 1, 2, 3, AE, and combinations such as 13 or 12AE.

Sequential Unicode Decoding Regression

SqlUnicodeEncodingTests checks raw UTF-16LE code-unit preservation, partial consumption, offsets, zero-capacity conversions, and unchanged input bytes. Run it without a database:

dotnet test src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj \
  --framework net10.0 --filter "FullyQualifiedName~SqlUnicodeEncodingTests" --blame-hang-timeout 10m

With SQL Server test configuration in place, run the Unicode round-trip regression:

dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0 \
  -p:TestFilters=FullyQualifiedName~UnicodeParameterAndReaderRoundTrip_PreservesWorldwideText

The integration test covers buffered and streamed parameters, synchronous and asynchronous readers, single-character and mixed-size reads, destination offsets, zero-length reads, and EOF over packet-spanning nvarchar(max) text. It uses the standard database-object fixtures and skips when connection strings are not configured. SQL Server may run remotely; verifying the big-endian decoder path requires running the test client on a real big-endian runtime (for example, s390x). Passing on x64 verifies only the little-endian path.

Test Filters

build.proj passes TestFilters to dotnet test --filter. By default, tests marked with these categories are excluded:

Category Why it is excluded by default
failing Known failing tests.
flaky Intermittently failing tests.
interactive Tests that require user interaction or external setup not suitable for normal runs.

Unsigned runs also exclude category=signed, unless TestFilters=none is specified. An explicit TestFilters expression replaces the default failing, flaky, and interactive exclusions, so include any you still want to exclude. Trait keys must match the source: use lowercase category and uppercase Set.

Examples:

Run a single test by fully-qualified name:

dotnet build -t:TestSqlClientUnit -p:TestFramework=net10.0 -p:TestFilters=FullyQualifiedName=Namespace.ClassName.MethodName

Run only flaky tests while investigating quarantine failures:

dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0 -p:TestFilters=category=flaky

Disable the default filter:

dotnet build -t:TestSqlClientFunctional -p:TestFramework=net10.0 -p:TestFilters=none

When passing filter expressions that contain shell-sensitive characters such as &, quote or escape the value as required by your shell.

Running Test Projects Directly

build.proj is the recommended entry point because it keeps logging, code coverage, package-reference mode, and common parameters consistent. For quick local investigation, you can run a test project directly:

dotnet test src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj \
  --framework net10.0 \
  -p:Configuration=Debug \
  --filter 'category!=failing&category!=flaky&category!=interactive&category!=signed'

For manual tests, select the Set trait in the filter. TestSet is interpreted by build.proj, not by the test project; passing -p:TestSet=2 directly to dotnet test does not select a set:

dotnet test src/Microsoft.Data.SqlClient/tests/ManualTests/Microsoft.Data.SqlClient.ManualTests.csproj \
  --framework net10.0 \
  -p:Configuration=Debug \
  --filter 'category!=failing&category!=flaky&category!=interactive&category!=signed&Set=2'

Manual Test Prerequisites

Manual tests require SQL Server or Azure SQL resources and a local test configuration file.

For a basic local SQL Server run, prepare:

  • A SQL Server instance that the test machine can reach.
  • Shared Memory, TCP, and Named Pipes protocols enabled when testing local Windows SQL Server scenarios.
  • The NORTHWIND database created from tools/testsql/createNorthwindDb.sql. For Azure SQL, use tools/testsql/createNorthwindAzureDb.sql. Both scripts turn READ_COMMITTED_SNAPSHOT and ALLOW_SNAPSHOT_ISOLATION off, which some transaction tests rely on. Azure SQL Database enables READ_COMMITTED_SNAPSHOT by default, so run the script against the database rather than setting it up by hand.
  • The UdtTestDb database created from tools/testsql/createUdtTestDb.sql if you want UDT tests to run.
  • A login or integrated-security principal with permissions to create and drop the temporary objects used by the tests.

Feature-specific tests require additional resources. If those resources are not configured, the corresponding conditional tests are skipped.

Manual Test Configuration

Edit the source configuration file at src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/config.jsonc. The test utilities project copies that file to the test output directory, where the manual tests load it by default.

The template file is:

src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/config.default.jsonc

config.jsonc is git-ignored. If it does not exist, the test utilities project copies config.default.jsonc to config.jsonc before compile. You can also create it manually:

cp src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/config.default.jsonc \
  src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/config.jsonc

Update config.jsonc for your environment before running manual tests. The most important values for a basic run are TCPConnectionString and NPConnectionString.

{
  "TCPConnectionString": "Data Source=tcp:localhost;Database=Northwind;Integrated Security=true;Encrypt=false;",
  "NPConnectionString": "Data Source=np:localhost;Database=Northwind;Integrated Security=true;Encrypt=false;",
  "EnclaveEnabled": false,
  "TracingEnabled": false,
  "SupportsEntraIntegrated": false,
  "SupportsIntegratedSecurity": true
}

For SQL Server in a Linux container, WSL, or another host where SQL authentication is easier than integrated security, use a TCP connection string like:

{
  "TCPConnectionString": "Data Source=tcp:127.0.0.1;User Id=sa;Password=<password>;Database=Northwind;Encrypt=false;TrustServerCertificate=true"
}

You can override the config file path with the TEST_MDS_CONFIG environment variable:

TEST_MDS_CONFIG=/path/to/config.jsonc dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0 -p:TestSet=2

On PowerShell:

$env:TEST_MDS_CONFIG = "C:\path\to\config.jsonc"
dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0 -p:TestSet=2

Configuration Properties

MDS_TCPConnectionString can override TCPConnectionString after the configuration file is loaded. Keep credentials in local ignored configuration or environment variables, not in committed files.

The Azure extension tests also read this configuration. SupportsEntraIntegrated enables their Entra Integrated authentication test; it is distinct from SupportsIntegratedSecurity. Supported targets include Azure SQL Database, Azure SQL Managed Instance, and SQL Server 2022 or later configured for Microsoft Entra authentication through Azure Arc.

Property Description Example or notes
TCPConnectionString Connection string for a TCP-enabled SQL Server or Azure SQL database. For Azure SQL, also configure UserManagedIdentityClientId or enable SupportsSystemAssignedManagedIdentity so tests can acquire an access token. Data Source=tcp:localhost;Database=Northwind;Integrated Security=true;Encrypt=false; or Data Source=<servername>.database.windows.net;Database=<dbname>;
NPConnectionString Connection string for a Named Pipes-enabled SQL Server instance. Data Source=np:localhost;Database=Northwind;Integrated Security=true;Encrypt=false;
TCPConnectionStringHGSVBS Optional connection string for SQL Server with VBS enclave and HGS attestation. Include Attestation Protocol=HGS and Enclave Attestation Url.
TCPConnectionStringNoneVBS Optional connection string for SQL Server with VBS enclave and no attestation. Include Attestation Protocol=None.
TCPConnectionStringAASSGX Optional connection string for SQL Server with SGX enclave and Microsoft Azure Attestation. Include Attestation Protocol=AAS and Enclave Attestation Url.
EnclaveEnabled Enables tests that require an enclave-configured server. true or false.
TracingEnabled Enables tracing-related tests. true or false.
AADServicePrincipalId Optional application ID for service-principal authentication tests. Former docs may refer to this as a secure principal ID.
AADServicePrincipalSecret Optional application secret for service-principal authentication tests. Keep this only in local, ignored config files or secure pipeline variables.
AzureKeyVaultURL Optional Azure Key Vault URL for Always Encrypted tests. https://<keyvaultname>.vault.azure.net/
AzureKeyVaultTenantId Optional Entra ID tenant ID for Azure Key Vault tests. Tenant ID GUID.
SupportsEntraIntegrated Whether the target supports Entra Integrated authentication for the Windows identity used by the Azure extension tests. true or false; defaults to false. See supported targets above.
SupportsIntegratedSecurity Whether the user running tests has integrated-security access to the target SQL Server. true or false.
LocalDbAppName Optional LocalDB instance name. Empty disables LocalDB testing. MSSQLLocalDB or another local instance.
LocalDbSharedInstanceName Optional shared LocalDB instance name. Used only when testing shared LocalDB.
FileStreamDirectory Directory used for FileStream database setup. Use an escaped absolute path in JSON.
UseManagedSNIOnWindows Enables Managed SNI on Windows test coverage. true or false.
DNSCachingConnString Optional connection string for DNS caching tests. Used with DNS caching server settings.
DNSCachingServerCR Optional DNS caching control-ring server. Feature-specific tests only.
DNSCachingServerTR Optional DNS caching tenant-ring server. Feature-specific tests only.
IsDNSCachingSupportedCR Enables DNS caching control-ring tests. true or false.
IsDNSCachingSupportedTR Enables DNS caching tenant-ring tests. true or false.
EnclaveAzureDatabaseConnString Optional Azure SQL database connection string for enclave tests. Feature-specific tests only.
SupportsSystemAssignedManagedIdentity Whether system-assigned managed identity tests should run. Defaults to false. Requires ManagedIdentitySupported.
ManagedIdentitySupported Whether managed identity tests should run. When omitted, defaults to true in manual tests and false in Azure extension tests. Set explicitly for the target environment.
UserManagedIdentityClientId Optional client ID for user-assigned managed identity tests. Feature-specific tests only.
KerberosDomainUser Optional Kerberos test domain user. Feature-specific tests only.
KerberosDomainPassword Optional Kerberos test domain password. Keep only in local, ignored config files or secure pipeline variables.
IsManagedInstance Marks the target as Azure SQL Managed Instance. Set true for Managed Instance to use non-Azure TVP baseline files in test set 3.
PowerShellPath Full path to PowerShell if it is not on PATH. C:\\escaped\\path\\to\\powershell.exe
AliasName Optional SQL Server alias used by alias-related tests. Feature-specific tests only.

Manual Test Sets

Manual tests are grouped using [Trait("Set", "...")] attributes so large runs can be partitioned. All sets are compiled; build.proj converts TestSet into a runtime filter and combines it with TestFilters.

TestSet Coverage
1 Smaller SQL connectivity and command scenarios.
2 Broad data access coverage, including adapters, bulk copy, retry logic, data reader, schema, DNS caching, and related scenarios.
3 Additional integration coverage, including LocalDB, pooling, parameters, transactions, JSON, Kerberos, UDT, vector, and other SQL feature tests.
AE Always Encrypted tests.

If TestSet is omitted, no set filter is added (the other filters and conditional skips still apply). You can combine sets by concatenating values, for example -p:TestSet=23 or -p:TestSet=12AE.

Results and Diagnostics

Test results are written to the test_results directory by default. Override the location with TestResultsFolderPath:

dotnet build -t:TestSqlClientUnit -p:TestFramework=net10.0 -p:TestResultsFolderPath="$PWD/test_results/unit"

Hang blame collection is enabled by default with a 10m timeout. To increase the timeout:

dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0 -p:TestBlameTimeout=30m

To disable hang blame collection:

dotnet build -t:TestSqlClientManual -p:TestFramework=net10.0 -p:TestBlameTimeout=0

Code coverage is enabled by default. To disable it for a faster local run:

dotnet build -t:TestSqlClientUnit -p:TestFramework=net10.0 -p:TestCodeCoverage=false