Modern Image Formats: let the browser encode client side uploads, and reflect it on the settings screen

[adamsilverstein]

This picks up the verification list from #2662. Short version: the plugin was telling the browser to produce WebP whenever the server could not encode AVIF, and the picture element assumed the server had made the sub-sizes. Both are addressed here, and I ran through each of the scenarios from the issue on WP 7.1 in the editor. There are Playground links below for anyone who wants to try it without a local setup.

Claude wrote the code and this write up, findings and all:

Description

Fixes #2662

With client side media processing (WordPress 7.1), the block editor uploads the original, then the browser generates the sub-sizes in whatever format the image_output_format REST field reports and sideloads them. That field comes from the image_editor_output_format filter, which the plugin only mapped to AVIF when the server's own image editor could encode AVIF. So on a host without AVIF support the browser was told to make WebP even though it could have made AVIF.

Three changes:

• The REST requests of the client side flow are now recognized: attachment creation with generate_sub_sizes set to false, plus the sideload and finalize requests. This is exposed through a new webp_uploads_client_side_media_processing filter, which the plugin toggles for those requests. Within them, webp_uploads_get_upload_image_mime_transforms() no longer downgrades AVIF to WebP, and webp_uploads_filter_image_editor_output_format() maps the source type to the selected modern format without consulting the server's editor. That also applies when "Output fallback images" is on, since the server generates the original format sizes when the upload is finalized (the same path the plugin already uses for AVIF and WebP originals).
• The picture element wrapper assumed the inner <img> was in the original format, which is true for server uploads but not for client side uploads, where the primary sub-size files are AVIF. The fallback <img> now gets its src and srcset pointed at the original format sources whenever the image is in a modern format.
• The settings screen keeps AVIF and WebP selectable when core reports client side media processing is enabled, and a small inline script mirrors the editor's runtime feature detection to swap in the right notice. The four cases from the issue:

Server encodes AVIF	Browser can process	What the user sees
yes	yes	AVIF selectable, no notice
yes	no	AVIF selectable, "uploaded images are converted on the server"
no	yes	AVIF selectable, "AVIF images are created by your browser", with a note that Media Library uploads are converted to WebP on the server (or not converted, if the server lacks WebP too)
no	no	the existing "AVIF support is not available" notice

The detection skips the SharedArrayBuffer check on purpose: it only passes on cross-origin isolated documents, which the editor sets up for itself and the settings screen does not. Everything else (WebAssembly, Workers, device memory, cores, data saver, the blob worker CSP probe) matches the upload-media package.

Verification of the items in the issue

Tested on wp-env with WP 7.1, PHP 8.3, Imagick with AVIF and WebP, in Chrome 154 with cross-origin isolation active in the editor. A "no AVIF" server was simulated with a wp_image_editors filter swapping in an editor subclass that refuses image/avif. Uploading leaves.jpg (1080x720, under the big image threshold) from an image block:

1. AVIF selected, server can encode it: the server converts the full size to AVIF during the create request as before, the browser produces the AVIF sub-sizes, and sources is recorded for every size.
2. AVIF selected, server cannot encode it: before this change the browser was told to make WebP. Now it makes AVIF sub-sizes. The full size stays JPEG, since the server cannot convert it and the browser only sideloads a scaled full size when the image is over the threshold.
3. "Output fallback images" on, server cannot encode AVIF: the browser makes the AVIF sub-sizes and the server generates the JPEG fallbacks for the supported sizes when the upload is finalized, eg. leaves-1024x683.avif plus leaves-1024x683-jpg.jpg. So the fallback sizes are still generated, by the server, and "Generate all fallback image sizes" governs them as usual.
4. Picture element: <source type="image/avif"> carries the browser made AVIF files and the fallback <img> now carries the JPEG files. Before the picture element fix the <img> pointed at the AVIF files.
5. sources metadata: the finalize request runs wp_generate_attachment_metadata, so webp_uploads_create_sources_property() records the browser made files and adds the server made fallbacks. Deleting the attachment removed every file, fallbacks included.

A Media Library upload on the simulated no-AVIF server still falls back to WebP on the server, so the non-editor path is unchanged.

Worth noting

• The "discard larger generated images" rule also applies to the server made fallbacks: if a JPEG fallback comes out larger than the browser's AVIF, it is discarded and the picture element loses its fallback for that size. It did not trigger in testing because the AVIF files from vips were larger than the JPEGs, but it is the same pre-existing behavior the plugin has for AVIF and WebP originals with fallback enabled. Probably worth a follow up to exempt original format fallbacks from that rule.
• Related to the above, the AVIF sub-sizes produced by the browser were consistently larger than the equivalent JPEGs (251 KB vs 239 KB at 1024px). The plugin only pins WebP quality via wp_editor_set_quality; the AVIF quality the editor uses may be worth a look separately.
• The settings screen keys off core's wp_is_client_side_media_processing_enabled(), so it stays server-only on 6.9 with the Gutenberg plugin. The REST detection works there too, since Gutenberg's controller extends the core one.
• For the 7.1 window the notice says editor uploads only; once the media library gets client side processing the wording needs a revisit.

How has this been tested

• New and existing PHPUnit tests for webp-uploads, single site and multisite, via npm run test-php:webp-uploads.
• Manual testing in the block editor as described above, in both server modes and with fallback output on and off.
• Settings screen checked in all four cases, using the WebAssembly check to simulate an incapable browser.

Testing instructions

Two Playground links, both WordPress 7.1 on PHP 8.3 with the plugin installed from this branch, logged in, landing on Settings > Media. Playground's PHP can encode AVIF on its own, so the first link covers the "server encodes AVIF" row of the table above. The second link adds a small mu-plugin that swaps in image editors refusing to encode AVIF, which covers the "server cannot encode AVIF" row. Use a recent Chrome, Firefox or Safari, since the client side pipeline needs WebAssembly and workers.

• Playground: server encodes AVIF
• Playground: server cannot encode AVIF

Steps, for either link:

1. On the Media Settings landing page, under "Modern Image Formats", AVIF should be selected. On the first link there is no notice below the format select. On the second link the "AVIF images are created by your browser" notice appears, mentioning that Media Library uploads are converted to WebP on the server instead.
2. Go to Posts > Add New, add an Image block and upload a JPEG from your computer. Anything under 2560px wide keeps the full size untouched, which makes the next step easier to read.
3. Once the upload finishes, open Media > Library and click the new image. On the first link the file name ends in .avif, since the server converted the full size. On the second link it is still the .jpg, and the sub-sizes are AVIF anyway.
4. To see the sub-sizes, paste /wp-json/wp/v2/media?per_page=1&orderby=date&order=desc into the Playground address bar. Every entry in media_details.sizes should report image/avif with an .avif file, and each should carry a sources entry for image/avif. Before this change, on the second link, those came out as WebP.
5. Fallback images and the picture element: back in Settings > Media, turn on "Output fallback images" and "Use Element", save, and upload another JPEG from an Image block. In the REST response each size should now list both image/avif and image/jpeg under sources, with JPEG files named like example-1024x640-jpg.jpg. Publish the post and view it: the <picture> has a <source type="image/avif"> pointing at the AVIF files, and the <img> src and srcset point at the JPEG files.
6. On the second link, upload a JPEG through Media > Add New Media File. That path still goes through the server, so its sub-sizes should be WebP, unchanged from before.

Steps 1 through 4 were checked in Playground in Chrome for both links. Steps 5 and 6 come from the wp-env testing described above and were not re-run in Playground.

The mu-plugin used by the second link, for local testing

<?php
/**
 * Plugin Name: Simulate a server without AVIF support
 * Description: Testing aid for WordPress/performance#2663. Swaps in image editors that refuse to encode AVIF.
 */

add_filter(
	'wp_image_editors',
	static function ( array $editors ): array {
		if ( ! class_exists( 'No_AVIF_Image_Editor_GD' ) ) {
			class No_AVIF_Image_Editor_GD extends WP_Image_Editor_GD {
				public static function supports_mime_type( $mime_type ) {
					return 'image/avif' !== $mime_type && parent::supports_mime_type( $mime_type );
				}
			}
		}
		if ( ! class_exists( 'No_AVIF_Image_Editor_Imagick' ) ) {
			class No_AVIF_Image_Editor_Imagick extends WP_Image_Editor_Imagick {
				public static function supports_mime_type( $mime_type ) {
					return 'image/avif' !== $mime_type && parent::supports_mime_type( $mime_type );
				}
			}
		}
		return array( 'No_AVIF_Image_Editor_Imagick', 'No_AVIF_Image_Editor_GD' );
	}
);

Types of changes

• Add webp_uploads_is_client_side_media_processing(), webp_uploads_is_client_side_media_processing_request(), webp_uploads_is_client_side_media_processing_enabled() and the webp_uploads_client_side_media_processing filter.
• Track the client side flow around REST callbacks and use the selected modern format within it.
• Point the picture element fallback <img> at the original format when the image is in a modern format.
• Update the settings screen notices and options for client side media processing.

AI Use

Code and description both written with 🤖 Claude Code. I will review and test.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 95.19231% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.87%. Comparing base (c080864) to head (8fbb1da).
⚠️ Report is 15 commits behind head on trunk.

Files with missing lines	Patch %	Lines
plugins/webp-uploads/hooks.php	87.50%	2 Missing ⚠️
plugins/webp-uploads/settings.php	95.23%	2 Missing ⚠️
plugins/webp-uploads/helper.php	94.73%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##            trunk    #2663      +/-   ##
==========================================
+ Coverage   70.50%   70.87%   +0.36%     
==========================================
  Files          91       91              
  Lines        7870     7931      +61     
==========================================
+ Hits         5549     5621      +72     
+ Misses       2321     2310      -11     

Flag	Coverage Δ
multisite	70.87% <95.19%> (+0.36%)	⬆️
single	35.19% <25.96%> (-0.15%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>
Co-authored-by: westonruter <westonruter@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[adamsilverstein]

This is a first pass at leveraging client side media for modern image support. The big caveat with 7.1 is the feature client side media is only available in the editor. For 7.2 I hope to land this in the media library as well, so maybe its worth waiting for 7.2 to try to land this feature?

[westonruter]

We should not be printing script tags manually as this violates CSP compat.

This should instead be:

	wp_print_inline_script_tag(
		<<<'JS'
		/* ... */
		JS
	);

See Core-59446 and WordPress/wordpress-develop#13319

[adamsilverstein]

Thanks!

I also wonder looking at this whether we could leverage the existing checks somehow (even though they don't typically run here) instead of duplicating them.

[adamsilverstein]

Switched to wp_print_inline_script_tag() in 8fbb1da.

Claude checked whether the core detection could be reused here:

wp.uploadMedia.isClientSideMediaSupported() is available on this screen, but it bails on the missing SharedArrayBuffer because the settings page is not cross-origin isolated, so it would report unsupported in every browser. Until the package can skip that check, the mirrored detection seems the practical option.