Add blog post: Kafka bucket notifications in Ceph RGW - Part 1 by sujay-d07 · Pull Request #1183 · ceph/ceph.io

sujay-d07 · 2026-08-11T17:04:21Z

Really excited to share Part 1 of my blog on Kafka Bucket Notifications in Ceph RGW! :)

Part 1 includes the plaintext, SASL_PLAINTEXT (PLAIN and SCRAM), SSL, SASL_SSL, and mTLS mechanisms.

Each section includes the commands needed to reproduce the setup and steps to verify that events actually make it to the Kafka broker.

anthonyeleven

Good stuff. Was AI used here?

yuvalif · 2026-08-12T10:17:55Z

@sujay-d07

your audience are ceph users not developers. they dont build from source, use vstart etc.
the main issue that they faced is our lack of support of kafka security mechanism, not how to do the setup (though, keep the security conf bits)
please use more toned down language, it should read a as somewhat boring user docs rather than marketing material :-)

anthonymicmidd · 2026-08-12T13:48:18Z

I would like to promote this blog on social media through the Ceph LinkedIn and Twitter accounts. I'll wait until the changes have been made.

yuvalif · 2026-08-25T14:02:10Z

+  - "security"
+---
+
+Bucket notifications to a Kafka broker over plaintext have worked for a long time.


the whole paragraph is incorrect. we supported SASL over SSL.
so, user/password over encrypted and authenticated connection was supported.
mtls only adds client side (not user) authentication by the broker.

yuvalif · 2026-08-25T14:02:54Z

+unsecured listener and no notifications at all.
+
+That gap is now closed. RGW's Kafka endpoint supports SASL/PLAIN,
+SASL/SCRAM-SHA-256 and SCRAM-SHA-512, one-way TLS, SASL over TLS, mutual TLS, and —


IIRC, SCRAM was also supported (but probably not tested)

yuvalif · 2026-08-25T14:14:59Z

+| SCRAM     | SASL_PLAINTEXT               | 9095     | Identity, without sending the password |
+| SSL       | SSL                          | 9093     | The channel is encrypted               |
+| SASL_SSL  | SASL_SSL                     | 9094     | Both: credentials inside TLS           |
+| mTLS      | SSL + `client.auth=required` | 9096     | Both sides prove identity, no password |


the meaning of "identity" here an in the SASL related lines of the table is different.
SASL mecahnism allow the broker to know identity of the user (a person)
TLS allows the TCP server to know the identity of the TCP client (a machine)
(mTLS allow also for the inverse)

please use the correct terms. note that PLAIN SASL (user/password) can work over PLAINTEXT connection, TLS connection or mTLS connection.
and the same is true for SCRAM user identifications

yuvalif · 2026-08-25T14:17:04Z

+Every section creates a bucket and attaches a topic to it, and the JSON differs only
+in two names. This helper keeps the rest of the post shorter:
+
+```bash


this section should be informational, not a toolset
please avoid bach functions, utilities etc.

the actual user may not use aws CLI at all (they can use java/ golang etc as client).
so, this section should provide a clear example to how they should use it in their system.
a bash function just obscures that

yuvalif · 2026-08-25T14:35:18Z

+| ----------------------- | ------- | ----------------------------------------------- |
+| `y-ca.crt`              | RGW     | CA certificate — the `ca-location` attribute    |
+| `server.keystore.jks`   | Broker  | Broker private key and signed certificate       |
+| `server.truststore.jks` | Broker  | Trusts the client CA (needed for mTLS)          |


what is "Trusts" ?

yuvalif · 2026-08-25T14:43:14Z

+An SNS topic on its own does nothing until a bucket points at it:
+
+```bash
+attach_topic test-bucket plaintext-notifications


what is attach_topic ?

yuvalif · 2026-08-25T14:51:30Z

+- **`rgw_allow_notification_secrets_in_cleartext` is not set.** RGW rejects topic
+  credentials over a cleartext connection without it, so the SASL_PLAINTEXT sections
+  do not work.
+- **The CA certificate or client key is not readable by `radosgw`.** The paths in the


not sure it is just a matter of being readable.
the file's permision has to be set correctly (IIRC, setting to 777 still make it readable, but would fail since the permissions are too wide)

yuvalif · 2026-08-25T14:53:18Z

+
+## Summary
+
+| Mechanism              | Encryption   | Authentication | RGW topic attributes                                                     |


the term "Authentication" here is misleading. both SSL and mTLS has authentication (one side, or both sides) but no user identification

yuvalif · 2026-08-25T14:54:35Z

+the `kafka_security_test` marker, in
+[`src/test/rgw/bucket_notification/`](https://github.com/ceph/ceph/tree/main/src/test/rgw/bucket_notification).
+
+For a staged migration, start from a plaintext topic, confirm that events are


we already support SASL over SSL. just remove this section (lines 711 - 714)

yuvalif · 2026-08-25T14:56:03Z

@sujay-d07 veside the comments the rest looks good (the majority of the technical content).
just verifying that this is tested?

Signed-off-by: sujay-d07 <sujaydongre07@gmail.com>

anthonymicmidd · 2026-09-22T21:23:27Z

It would be great to get this blog series out before the GSoC showcase. Is there anything I can help with here? @sujay-d07 @yuvalif

anthonyeleven

AM: I think it’s ready?

sujay-d07 requested a review from a team as a code owner August 11, 2026 17:04

anthonyeleven reviewed Aug 12, 2026

View reviewed changes

yuvalif reviewed Aug 12, 2026

View reviewed changes

Comment thread src/en/news/blog/2026/kafka-bucket-notifications-part-1/index.md Outdated

yuvalif reviewed Aug 12, 2026

View reviewed changes

Comment thread src/en/news/blog/2026/kafka-bucket-notifications-part-1/index.md Outdated

yuvalif reviewed Aug 12, 2026

View reviewed changes

Comment thread src/en/news/blog/2026/kafka-bucket-notifications-part-1/index.md Outdated

sujay-d07 force-pushed the rgw-kafka-bucket-notification-blog-1 branch 2 times, most recently from b185b08 to 9c26fd4 Compare August 23, 2026 11:26

sujay-d07 mentioned this pull request Aug 23, 2026

Add blog post: Kafka bucket notifications in Ceph RGW Part 2 #1191

Open

yuvalif reviewed Aug 25, 2026

View reviewed changes

Comment thread src/en/news/blog/2026/kafka-bucket-notifications-part-1/images/security_ladder.png Outdated

yuvalif reviewed Aug 25, 2026

View reviewed changes

Add blog post: Kafka bucket notifications in Ceph RGW Part 1

dad4c55

Signed-off-by: sujay-d07 <sujaydongre07@gmail.com>

sujay-d07 force-pushed the rgw-kafka-bucket-notification-blog-1 branch from 9c26fd4 to dad4c55 Compare August 30, 2026 17:46

anthonyeleven approved these changes Sep 23, 2026

View reviewed changes


		## Summary

		\| Mechanism \| Encryption \| Authentication \| RGW topic attributes \|

Conversation

sujay-d07 commented Aug 11, 2026

Uh oh!

anthonyeleven left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yuvalif commented Aug 12, 2026

Uh oh!

anthonymicmidd commented Aug 12, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yuvalif commented Aug 25, 2026

Uh oh!

anthonymicmidd commented Sep 22, 2026

Uh oh!

anthonyeleven left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants