Description
The elasticsearch source can't currently scan OpenSearch clusters, even though OpenSearch
shares Elasticsearch's REST API surface and is a common target.
Pointing the Trufflehog with the latest cersion at an OpenSearch cluster fails for several reasons at once:
- OpenSearch's JWT auth plugin expects a raw Authorization: Bearer
header. None of the existing auth options (basic, API key, service token) produce that.
- go-elasticsearch client rejects any response
missing the X-Elastic-Product header. Real Elasticsearch always sends it while OpenSearch
doesn't reliably send it
- OpenSearch exposes PIT creation at
POST /{index}/_search/point_in_time, not Elasticsearch's POST /{index}/_pit, so the
vendored esapi.OpenPointInTimeRequest (hardcoded to the ES8 path) 404s.
- Elasticsearch's PIT tiebreaker sort field is
_shard_doc;
OpenSearch doesn't recognize it and rejects the search with a query_shard_exception,
but accepts the equivalent _doc.
- When no
--query-json/--since-timestamp is set, the search
body's query field is {}. Elasticsearch tolerates this as an implicit match-all;
OpenSearch's stricter parser rejects it with illegal_argument_exception: query malformed, empty clause found, silently zeroing out every scan.
Preferred Solution
A new --bearer-token flag / ELASTICSEARCH_BEARER_TOKEN env var on the existing
elasticsearch command that:
- Sets the
Authorization: Bearer header directly for JWT-based auth.
- Wraps the HTTP transport to inject
X-Elastic-Product on successful responses when
bearer-token auth is used, so the client's product check passes against OpenSearch.
- Uses an OpenSearch-specific PIT-open call and
_doc tiebreaker when bearer-token auth
indicates an OpenSearch target.
- Sends an explicit
match_all instead of an empty query clause (valid and
behavior-preserving on real Elasticsearch too, so this fix isn't even OpenSearch-gated).
All of the above only activate when --bearer-token is set, so existing Elasticsearch
behavior (basic/API-key/service-token auth) is unchanged.
References
(none yet. Will link the PR here once opened)
Description
The elasticsearch source can't currently scan OpenSearch clusters, even though OpenSearch
shares Elasticsearch's REST API surface and is a common target.
Pointing the Trufflehog with the latest cersion at an OpenSearch cluster fails for several reasons at once:
header. None of the existing auth options (basic, API key, service token) produce that.
missing the
X-Elastic-Productheader. Real Elasticsearch always sends it while OpenSearchdoesn't reliably send it
POST /{index}/_search/point_in_time, not Elasticsearch'sPOST /{index}/_pit, so thevendored esapi.OpenPointInTimeRequest (hardcoded to the ES8 path) 404s.
_shard_doc;OpenSearch doesn't recognize it and rejects the search with a
query_shard_exception,but accepts the equivalent
_doc.--query-json/--since-timestampis set, the searchbody's
queryfield is{}. Elasticsearch tolerates this as an implicit match-all;OpenSearch's stricter parser rejects it with
illegal_argument_exception: query malformed, empty clause found, silently zeroing out every scan.Preferred Solution
A new
--bearer-tokenflag /ELASTICSEARCH_BEARER_TOKENenv var on the existingelasticsearchcommand that:Authorization: Bearerheader directly for JWT-based auth.X-Elastic-Producton successful responses whenbearer-token auth is used, so the client's product check passes against OpenSearch.
_doctiebreaker when bearer-token authindicates an OpenSearch target.
match_allinstead of an empty query clause (valid andbehavior-preserving on real Elasticsearch too, so this fix isn't even OpenSearch-gated).
All of the above only activate when
--bearer-tokenis set, so existing Elasticsearchbehavior (basic/API-key/service-token auth) is unchanged.
References
(none yet. Will link the PR here once opened)