Skip to content

isthmus: support struct field access from RexLambdaRef #1322

Description

@vinaykpud

Problem

Isthmus cannot convert struct-field access rooted at a Calcite RexLambdaRef.

For an array of structs:

transform(events, event -> event.name)

the canonical Calcite lambda has one parameter representing the array element:

lambda(event: ROW<name, attributes>) -> event.name

However, converting RexFieldAccess(RexLambdaRef(event), name) fails with:

UnsupportedOperationException:
RexFieldAccess for SqlKind LAMBDA_REF not supported

Observed with 0.89.1 and current main at ade72bc2.

Cause

RexExpressionConverter.visitLambda() and visitLambdaRef() support lambda expressions and references, but visitFieldAccess() only accepts:

  • ITEM
  • INPUT_REF
  • FIELD_ACCESS
  • CORREL_VARIABLE

It does not accept LAMBDA_REF.

This prevents canonical list-function lambdas over struct elements. Consumers must instead model every struct child as a separate lambda parameter and repair the generated Substrait protobuf afterward.

Expected behavior

Isthmus should convert:

RexFieldAccess(RexLambdaRef(event), name)

into a lambda parameter field reference equivalent to:

lambda_parameter[0].field[0]

Multi-level access should also work:

employee.profile.address.city

Suggested changes

  1. Handle SqlKind.LAMBDA_REF in RexExpressionConverter.visitFieldAccess().
  2. Preserve lambda-reference scope when calling FieldReference.dereferenceStruct().
  3. Support chained field segments for LAMBDA_PARAMETER_REFERENCE in ProtoExpressionConverter.
  4. Add forward and round-trip tests for:
    • employee.name
    • employee.profile.address.city

Related work

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions