Problem
Isthmus cannot convert struct-field access rooted at a Calcite RexLambdaRef.
For an array of structs:
transform(events, event -> event.name)
the canonical Calcite lambda has one parameter representing the array element:
lambda(event: ROW<name, attributes>) -> event.name
However, converting RexFieldAccess(RexLambdaRef(event), name) fails with:
UnsupportedOperationException:
RexFieldAccess for SqlKind LAMBDA_REF not supported
Observed with 0.89.1 and current main at ade72bc2.
Cause
RexExpressionConverter.visitLambda() and visitLambdaRef() support lambda expressions and references, but visitFieldAccess() only accepts:
ITEM
INPUT_REF
FIELD_ACCESS
CORREL_VARIABLE
It does not accept LAMBDA_REF.
This prevents canonical list-function lambdas over struct elements. Consumers must instead model every struct child as a separate lambda parameter and repair the generated Substrait protobuf afterward.
Expected behavior
Isthmus should convert:
RexFieldAccess(RexLambdaRef(event), name)
into a lambda parameter field reference equivalent to:
lambda_parameter[0].field[0]
Multi-level access should also work:
employee.profile.address.city
Suggested changes
- Handle
SqlKind.LAMBDA_REF in RexExpressionConverter.visitFieldAccess().
- Preserve lambda-reference scope when calling
FieldReference.dereferenceStruct().
- Support chained field segments for
LAMBDA_PARAMETER_REFERENCE in ProtoExpressionConverter.
- Add forward and round-trip tests for:
employee.name
employee.profile.address.city
Related work
Problem
Isthmus cannot convert struct-field access rooted at a Calcite
RexLambdaRef.For an array of structs:
the canonical Calcite lambda has one parameter representing the array element:
However, converting
RexFieldAccess(RexLambdaRef(event), name)fails with:Observed with
0.89.1and currentmainatade72bc2.Cause
RexExpressionConverter.visitLambda()andvisitLambdaRef()support lambda expressions and references, butvisitFieldAccess()only accepts:ITEMINPUT_REFFIELD_ACCESSCORREL_VARIABLEIt does not accept
LAMBDA_REF.This prevents canonical list-function lambdas over struct elements. Consumers must instead model every struct child as a separate lambda parameter and repair the generated Substrait protobuf afterward.
Expected behavior
Isthmus should convert:
into a lambda parameter field reference equivalent to:
Multi-level access should also work:
Suggested changes
SqlKind.LAMBDA_REFinRexExpressionConverter.visitFieldAccess().FieldReference.dereferenceStruct().LAMBDA_PARAMETER_REFERENCEinProtoExpressionConverter.employee.nameemployee.profile.address.cityRelated work
steps_out = 0