Skip to content

fix: calculation of content upload size - #2533

Merged
joelkeener merged 1 commit into
develfrom
AUTOTOOL-5386-content-chunks
Sep 15, 2026
Merged

joelkeener merged 1 commit into
develfrom
AUTOTOOL-5386-content-chunks

Conversation

@joelkeener

@joelkeener joelkeener commented Sep 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes an upload failure in upload_file() where files provided as StringIO (such as SSL certificates and keys) failed with a byte count mismatch between Content-Range and Content-Length. In Python 3, StringIO.tell() returns the character count rather than the UTF-8 byte count, causing uploads with multibyte UTF-8 characters to be rejected by BIG-IP iControl REST.

Changes

Module Utils

  • icontrol.py: When src is an instance of StringIO, encode the contents to UTF-8 bytes into a BytesIO stream and derive the upload size from byte length (len(content_bytes)), ensuring Content-Range and Content-Length match.

Tests

  • test_icontrol_upload.py: Added test_stringio_multibyte_utf8_content_range_matches_actual_bytes to verify StringIO inputs containing multibyte characters calculate Content-Range matching the exact transmitted byte length.

API Endpoints

Method Endpoint Description
POST /mgmt/shared/file-transfer/uploads/{filename} Uploads chunked binary data with RFC 7233 byte-range validation

Testing

  • Added unit test test_stringio_multibyte_utf8_content_range_matches_actual_bytes verifying byte count vs character count disparity.
  • Confirmed test fails without the fix (AssertionError: assert 7 == 10) and passes when the fix is applied.
  • Ran the full module_utils test suite (37/37 passing).

Checklist

  • Unit tests pass
  • Code follows project conventions
  • No security concerns identified

Closes AUTOTOOL-5386

@crosbygw crosbygw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary: fix: calculation of content upload size (AUTOTOOL-5386)

File: ansible_collections/f5networks/f5_modules/plugins/module_utils/icontrol.py — upload_file()

The bug being fixed: StringIO content is str, not bytes. The old code computed size via os.stat()/seek-tell without accounting for encoding, so multibyte UTF-8 characters in the content (e.g., PEM certs/keys with non-ASCII chars) would cause size and the Content-Range byte length to mismatch character count vs. byte count.

The fix: StringIO input is now explicitly encoded to UTF-8 into a fresh BytesIO before computing size/reading chunks, guaranteeing byte-accurate Content-Range headers.

Verdict: Correct, well-scoped fix. No bugs found.

  • Traced all current callers (bigip_ssl_certificate, bigip_ssl_key, bigip_ssl_key_cert, bigip_iapp_template, bigip_imish_config, bigip_config, bigip_data_group, bigip_user, plus file-path callers like bigip_lx_package, bigip_software_image, bigip_file_copy, bigip_asm_policy_import) — all consistent with the new branching logic.
  • BytesIO path (used only in bigip_user.py for ciphertext) and plain file-path path are functionally unchanged — no behavior regression for existing callers.
  • Tests pass (5/5).

Minor, non-blocking notes:

  1. Some duplicated seek/tell boilerplate across the BytesIO and file-path branches (byproduct of splitting one try/except into three branches) — could be factored into a helper later, not worth blocking on.
  2. The new test file's module docstring describes the previous commit's off-by-one Content-Range fix, not this commit's UTF-8 encoding fix — only one of the five tests actually covers this change. Cosmetic/misleading but doesn't affect correctness.

@joelkeener
joelkeener merged commit 94d9679 into devel Sep 15, 2026
9 checks passed
@joelkeener
joelkeener deleted the AUTOTOOL-5386-content-chunks branch September 16, 2026 01:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants