Skip to content
View rangareddy's full-sized avatar

Block or report rangareddy

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
rangareddy/README.md

Hi πŸ‘‹, I'm Ranga Reddy

Data engineer working on open lakehouse systems Β Β·Β  Apache Hudi contributor Β Β·Β  Bengaluru, India

I spend my time on the storage layer of the lakehouse: how a table applies a stream of changes efficiently, what it costs to read afterwards, and which knob is actually responsible when a job slows down. Most of that is Apache Hudi, Apache Iceberg and Apache Spark, at @onehouse.


🧰 What I work on

Area What that means day to day
Apache Hudi Indexes, the timeline, compaction and table services. Where most of my open-source time goes
Apache Spark Tuning the write and read paths, memory and shuffle behaviour, and diagnosing jobs that got slower without changing
Open table formats Hudi, Iceberg and Delta Lake, plus Apache XTable for reading one copy of the data as any of them
Platform Kafka and Flink ingestion, catalogs, and the maintenance jobs a lakehouse quietly needs

🌱 Open source

I also answer a fair number of Spark and Hudi questions on Stack Overflow.

✍️ Writing

I write long-form guides and reference sheets at rangareddy.github.io. The ones people come back to:

Apache Spark cheat sheet Execution model, memory, joins, shuffle, AQE, Catalyst, streaming and tuning, with the default and version for every config
Apache Hudi cheat sheet Table types, timeline, indexes, the metadata table, concurrency, table services and migration
Apache Iceberg cheat sheet Snapshot tree, hidden partitioning, row-level modes, procedures and maintenance
Apache Hudi: a complete guide What Hudi is, how a write and a read actually work, and what is genuinely different about it
Open table formats in practice What actually differs between Iceberg, Hudi and Delta Lake, past the feature checklist

Everything in those is read from the release tag rather than recalled, which is the whole point of them.

πŸ”§ Projects worth a look

Repository What it is
ranga_spark_experiments Runnable Spark experiments, the scratchpad behind a lot of the writing
spark-history-server-docker A Spark History Server you can start with one command, for reading event logs locally
spark-logs-collector Pulls application and event logs off a cluster so you can debug a finished job
spark_project_template_generator Generates a Spark project skeleton so you skip the boilerplate
ranga-java-oom Reproducible Java OutOfMemoryError examples, one per flavour

πŸ›  Tech

Core: Apache Spark Apache Hudi Apache Iceberg Delta Lake Apache Flink Apache Kafka

Languages: Java Scala Python SQL

Platform: AWS Docker Kubernetes Apache Airflow Apache Hadoop Apache Hive

πŸ“« Reach me


Pinned Loading

  1. spark_project_template_generator spark_project_template_generator Public

    Used to generate Sample Spark Project Template

    Java 7 4

  2. spark-logs-collector spark-logs-collector Public

    Spark Application and Event Logs extractor

    Shell 7 4

  3. ranga-java-oom ranga-java-oom Public

    Java OutOfMemory Example

    Java 11 9

  4. ranga_spark_experiments ranga_spark_experiments Public

    My Spark Experiments

    Scala 8 6

  5. thread-dumps-collector thread-dumps-collector Public

    Used to collect the Thread Dumps by passing the Container ID

    Shell 3

  6. install_apps_on_mac install_apps_on_mac Public

    Simple application is used to install the development related apps on Mac

    Shell 1