Skip to content

feat: send cert-key-chain files to icr w/o enforcing encodin - #2540

Merged
joelkeener merged 1 commit into
develfrom
AUTOTOOL-5428-cert-key-encoding
Oct 14, 2026
Merged

joelkeener merged 1 commit into
develfrom
AUTOTOOL-5428-cert-key-encoding

Conversation

@joelkeener

Copy link
Copy Markdown
Collaborator

Summary

Enable bigip_ssl_key_cert, bigip_ssl_key, and bigip_ssl_certificate modules to accept and upload certificate/key content with non-UTF-8 character encodings (Latin-1, CP1252, etc.). Fixes UTF-8 codec errors when uploading certificates with extended metadata while maintaining full backward compatibility with UTF-8 content. Also resolves Ansible 2.24+ deprecation warnings for module imports.

Changes

Core Encoding Fix (3 modules)

  • bigip_ssl_key_cert.py, bigip_ssl_key.py, bigip_ssl_certificate.py
    • Replaced StringIO with BytesIO for raw byte handling
    • Added _to_bytes() method to ModuleParameters class to convert strings to bytes while preserving surrogate characters using surrogateescape error handler
    • Updated _get_hash() to use BytesIO without UTF-8 encoding
    • Updated create_on_device() and update_on_device() to use BytesIO for key/cert content

Deprecation Fixes

  • common.py: Updated import from ansible.module_utils._text to ansible.module_utils.common.text.converters
  • bigip_data_group.py: Updated import from ansible.module_utils._text to ansible.module_utils.common.text.converters
  • loader.py: Updated imports for to_bytes and to_text from new location

Documentation Updates

  • bigip_ssl_key_cert.py, bigip_ssl_key.py, bigip_ssl_certificate.py
    • Added notes describing support for non-UTF-8 encoded content
    • Added example showing certificate import with metadata encoding

Testing

  • Manual integration testing: Playbook successfully uploads certificates with Latin-1 and CP1252 encoded metadata
  • Unit tests: Existing tests pass (10/11 for ssl_key_cert, 3/5 for ssl_certificate, 1/2 for ssl_key; pre-existing infrastructure failures unrelated to changes)
  • Verified existing UTF-8 behavior preserved (backward compatible)
  • Verified deprecation warnings eliminated with updated imports

Checklist

  • Unit tests pass (confirmed no regressions from encoding changes)
  • Code follows project conventions (matches existing patterns)
  • Backward compatible with UTF-8 content
  • Manual integration testing completed
  • Deprecation warnings resolved for Ansible 2.24+
  • Documentation updated (description and examples)
  • All three SSL modules fixed consistently

Ref:AUTOTOOL-5428

@crosbygw crosbygw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary
The delegated review reported no definite bugs. It called out a caveat:

  1. The new documentation claim about supporting content with "any character encoding" was broader than what the subagent could verify.
    I then reviewed the branch against devel directly and checked the full context of the changed files. The changes are limited to:
  • switching deprecated ansible.module_utils._text imports to ansible.module_utils.common.text.converters
  • changing SSL cert/key content handling from StringIO to BytesIO
  • hashing and upload paths now operating on raw bytes via surrogateescape
    That implementation is internally consistent:
  • the collection declares requires_ansible: '>=2.16.0', so the new import path matches the supported Ansible floor
  • upload_file() already has an explicit BytesIO code path in plugins/module_utils/icontrol.py, so the stream type change fits existing helper behavior
  • the SHA1 comparison logic now hashes uploaded bytes directly instead of re-encoding chunked text, which is the right direction for preserving non-UTF-8-originated content

@joelkeener

joelkeener commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator Author

While it is true that Ansible now accepts any character encoding in cert/key/chain metadata, iControl currently replaces some characters, if they are non-utf-8 chars, which means this documentation is accurate before and after that is fixed.

We have tested these 27 encodings:

  • Latin-based (8):
  1. UTF-8 - Café résumé naïve ✓
  2. CP1252 - "Smart quotes" and Euro € ✓
  3. ISO-8859-15 - Café with Euro € symbol ✓
  4. CP1250 - Zażółw gęślą jaźń ✓
  5. CP1254 - Merhaba dünya çalışma ✓
  6. ISO-8859-9 - Merhaba dünya ✓
  7. UTF-16 - Café résumé ✓
  8. UTF-32 - Café résumé ✓
  • Cyrillic (2):
  1. Russian - Привет мир ✓
  2. Ukrainian - Привіт світе ✓
  • Middle East (3):
  1. Arabic - مرحبا بالعالم ✓
  2. Hebrew - שלום עולם ✓
  3. CP1256 - مرحبا بالعالم ✓
  • European (8):
  1. Greek - Καλημέρα κόσμε ✓
  2. CP1253 - Καλημέρα κόσμε ✓
  3. CP1257 - Sveiki pasaulē ✓
  4. CP1258 - Xin chào thế giới ✓
  5. ISO-8859-10 - Halló heimur ✓
  6. ISO-8859-13 - Sveikas pasaule ✓
  7. ISO-8859-14 - Halló saoghal ✓
  8. CP1255 - שלום עולם ✓
  • Asian (10):
  1. Thai - สวัสดีชาวโลก ✓
  2. ISO-8859-11 - สวัสดีชาวโลก ✓
  3. Japanese - こんにちは世界 ✓
  4. Chinese Simplified - 你好世界 ✓
  5. Big5 - 你好世界 ✓
  6. Korean - 안녕하세요 세계 ✓

@joelkeener
joelkeener merged commit 61b65d3 into devel Oct 14, 2026
9 checks passed
@joelkeener
joelkeener deleted the AUTOTOOL-5428-cert-key-encoding branch October 28, 2026 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants