Skip to content

perf(graph-analytics): make C++ faster than Python by swapping slow maps for contiguous vectors #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Phantom0299 wants to merge 3 commits into
base: main
Choose a base branch
from
Open

perf(graph-analytics): make C++ faster than Python by swapping slow maps for contiguous vectors #42

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
41a01f5
perf(graph-analytics): make C++ faster than Python by swapping slow m…
Phantom0299 Apr 12, 2026
46d849e
style: remove whitespace to fix CI
Phantom0299 Apr 12, 2026
48ba0e7
perf: use reserved unordered_map for O(1) lookups and fix collisions
Phantom0299 Apr 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 55 additions & 34 deletions chuck/tasks/graph_analytics/native_cpp/binding.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

#include <algorithm>
#include <cmath>
#include <map>
#include <string>
#include <vector>
#include <unordered_map>

@Aaryan-Dadu Aaryan-Dadu Apr 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are using unordered_map, how will you avoid hash collisions?

#include <utility>

namespace py = pybind11;

py::dict solve(py::object payload_obj) {
std::map<std::string, std::vector<std::string>> graph = payload_obj.cast<std::map<std::string, std::vector<std::string>>>();
auto graph = payload_obj.cast<std::map<std::string, std::vector<std::string>>>();

std::vector<std::string> nodes;
nodes.reserve(graph.size());
for (const auto& [node, _] : graph) {
Expand All @@ -27,54 +29,73 @@ py::dict solve(py::object payload_obj) {
return output;
}

constexpr int iterations = 16;
constexpr double damping = 0.85;
std::map<std::string, double> rank;
std::map<std::string, std::vector<std::string>> outgoing;
for (const auto& node : nodes) {
rank[node] = 1.0 / static_cast<double>(nodes.size());
auto found = graph.find(node);
if (found == graph.end() || found->second.empty()) {
outgoing[node] = nodes;
} else {
outgoing[node] = found->second;
const int n = static_cast<int>(nodes.size());

// Fix: .reserve(n) ensures the load factor stays low, preventing hash collisions
std::unordered_map<std::string, int> name_to_idx;
name_to_idx.reserve(n);
Comment on lines +34 to +36

@coderabbitai coderabbitai Bot Apr 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

In C++, does std::unordered_map::reserve prevent hash collisions, or does it only reduce rehashes and average bucket occupancy?

💡 Result:

std::unordered_map::reserve does not prevent hash collisions. It only reduces rehashes and average bucket occupancy (i.e., lowers the load factor). Hash collisions occur when distinct keys have the same hash value, determined solely by the hash function. Collisions are handled by chaining (linked lists per bucket) regardless of the number of buckets. reserve cannot change hash values or prevent collisions; it only allocates more buckets. reserve(n) sets bucket_count to at least ceil(n / max_load_factor), ensuring at least n elements can be inserted without exceeding max_load_factor and triggering automatic rehash. This pre-allocates buckets to avoid incremental rehashes during insertion and keeps average elements per bucket low, improving performance by reducing chain lengths for collided elements. No source indicates reserve affects collision occurrence; it manages bucket count for load factor control.

Citations:

Correct the misleading comment on unordered_map::reserve.

.reserve(n) reduces rehashing and keeps average bucket occupancy low, but does not prevent hash collisions. Collisions are determined by the hash function and cannot be prevented by adjusting bucket count. Update the comment to reflect what the method actually guarantees.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@chuck/tasks/graph_analytics/native_cpp/binding.cpp` around lines 34 - 36, The
comment about unordered_map::reserve on the name_to_idx local is misleading;
change it to state that reserve(n) pre-allocates buckets to reduce rehashing and
keep the average load factor lower (improving performance) but does not prevent
hash collisions, which are determined by the hash function; update the comment
near name_to_idx and the reserve call to reflect this accurate behavior.

for (int i = 0; i < n; ++i) {
name_to_idx[nodes[i]] = i;
}

@Aaryan-Dadu Aaryan-Dadu Apr 12, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please share the A/B comparison workflow for the changes


std::vector<std::vector<int>> adj(n);
for (int u = 0; u < n; ++u) {
auto found = graph.find(nodes[u]);
if (found != graph.end() && !found->second.empty()) {
adj[u].reserve(found->second.size());
for (const auto& target : found->second) {
adj[u].push_back(name_to_idx.at(target));
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
}
}

const double base = (1.0 - damping) / static_cast<double>(nodes.size());
constexpr int iterations = 16;
constexpr double damping = 0.85;
const double base = (1.0 - damping) / static_cast<double>(n);

std::vector<double> rank(n, 1.0 / static_cast<double>(n));
std::vector<double> new_rank(n, 0.0);

for (int step = 0; step < iterations; ++step) {
std::map<std::string, double> new_rank;
for (const auto& node : nodes) {
new_rank[node] = base;
std::fill(new_rank.begin(), new_rank.end(), base);

double dangling_share = 0.0;
for (int u = 0; u < n; ++u) {
if (adj[u].empty()) {
dangling_share += damping * rank[u] / static_cast<double>(n);
continue;
}
const double share = (damping * rank[u]) / static_cast<double>(adj[u].size());
for (int v : adj[u]) {
new_rank[v] += share;
}
}
for (const auto& node : nodes) {
const auto& edges = outgoing[node];
const double share = rank[node] / static_cast<double>(edges.size());
for (const auto& target : edges) {
new_rank[target] += damping * share;

if (dangling_share > 0.0) {
for (double& value : new_rank) {
value += dangling_share;
}
}
rank = std::move(new_rank);
std::swap(rank, new_rank);
}

std::string top_node;
double top_score = -1.0;
for (const auto& node : nodes) {
double score = rank[node];
if (score > top_score || (std::abs(score - top_score) < 1e-15 && node > top_node)) {
top_score = score;
top_node = node;
int top_idx = 0;
double top_score = rank[0];
for (int i = 1; i < n; ++i) {
if (rank[i] > top_score || (std::abs(rank[i] - top_score) < 1e-15 && nodes[i] > nodes[top_idx])) {
top_score = rank[i];
Comment on lines +84 to +86

@coderabbitai coderabbitai Bot Apr 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Remove the epsilon from the top-node tie-break.

The Python implementation only falls back to node-name ordering when the scores are exactly equal. With std::abs(rank[i] - top_score) < 1e-15, a slightly smaller score can still win here, so top_node can diverge from chuck/tasks/graph_analytics/task.py.

💡 Suggested fix 
-        if (rank[i] > top_score || (std::abs(rank[i] - top_score) < 1e-15 && nodes[i] > nodes[top_idx])) {
+        if (rank[i] > top_score || (rank[i] == top_score && nodes[i] > nodes[top_idx])) {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for (int i = 1; i < n; ++i) {
if (rank[i] > top_score || (std::abs(rank[i] - top_score) < 1e-15 && nodes[i] > nodes[top_idx])) {
top_score = rank[i];
for (int i = 1; i < n; ++i) {
if (rank[i] > top_score || (rank[i] == top_score && nodes[i] > nodes[top_idx])) {
top_score = rank[i];
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@chuck/tasks/graph_analytics/native_cpp/binding.cpp` around lines 84 - 86, The
tie-break in the top-node selection loop uses an epsilon check allowing
near-equal scores to fall back to node ordering; update the condition in the
loop that iterates over rank[] (the for loop using rank, top_score, nodes,
top_idx) to only use exact equality for the tie case (i.e., replace the
fabs(rank[i] - top_score) < 1e-15 check with a direct equality comparison
rank[i] == top_score) so that node-name ordering is applied only when scores are
exactly equal, matching the Python implementation.

top_idx = i;
}
}

double checksum = 0.0;
for (std::size_t index = 0; index < nodes.size(); ++index) {
checksum += static_cast<double>(index + 1) * rank[nodes[index]];
for (int i = 0; i < n; ++i) {
checksum += static_cast<double>(i + 1) * rank[i];
}

py::dict output;
output["node_count"] = py::int_(nodes.size());
output["top_node"] = py::str(top_node);
output["node_count"] = py::int_(n);
output["top_node"] = py::str(nodes[top_idx]);
output["top_score"] = py::float_(std::round(top_score * 1000000.0) / 1000000.0);
output["checksum"] = py::float_(std::round(checksum * 1000000.0) / 1000000.0);
return output;
Expand Down
Loading