In postgres.c,exports.c and utils/postgres.c:
The only command that is checked for errors is PQexec(m->conn_master, m->cpycmd);
Which is the only place where we can notice that something about the postgres connection fails.
This leads to the loss of data in many constellations.
Furthermore, as we batch 2000 lines for copy, but have the messages already destroyed in the queue, we need a solution to retain the messages in case of copy error (possibly a ring buffer).
In postgres.c,exports.c and utils/postgres.c:
The only command that is checked for errors is
PQexec(m->conn_master, m->cpycmd);Which is the only place where we can notice that something about the postgres connection fails.
This leads to the loss of data in many constellations.
Furthermore, as we batch 2000 lines for copy, but have the messages already destroyed in the queue, we need a solution to retain the messages in case of copy error (possibly a ring buffer).