[NopeCharon]

What changes were proposed in this pull request?

The Airflow provider package catalog connector determines the modules that contain operators from the provider info. To get it, the connector searched the extracted package for a file named get_provider_info.py and called its get_provider_info() function. Airflow doesn't require this name: a provider package declares the function that returns its provider info as the provider_info entry point in the apache_airflow_provider group (Airflow documentation), and the function can be defined in any module. Custom provider packages that use a different module or function name couldn't be added to a catalog. The connector logged that the archive "contains 0 file(s) named 'get_provider_info.py'" and returned no components.

This PR adds _locate_provider_info(), which:

1. reads the provider_info entry point from the package's *.dist-info/entry_points.txt with importlib.metadata.Distribution.at(...), and resolves its module to a script (my_provider/provider.py) or a package (my_provider/provider/__init__.py) in the extracted archive;
2. falls back to the previous lookup of get_provider_info.py / get_provider_info() if the package doesn't declare the entry point, so existing behavior is preserved for such archives.

The rest of the analysis is unchanged. The error message for a missing function now names the function from the entry point, which also drops a stray space from ' get_provider_info'.

How was this pull request tested?

New tests in elyra/tests/pipeline/airflow/test_airflow_provider_package_connector.py build small provider wheels and load them through file:// URLs:

• test_locate_provider_info: the provider info is found through the entry point in a module with a custom name, through the entry point in a package (__init__.py, with another entry point group present), and through get_provider_info.py without an entry point. In each case the connector returns the operator script. The two entry point cases fail on main.
• test_locate_provider_info_errors: an error is logged and no entries are returned if the entry point module isn't in the archive, the entry point function isn't defined, or there's neither an entry point nor get_provider_info.py.

pytest elyra/tests/pipeline/airflow/test_airflow_provider_package_connector.py: 12 passed, including the existing tests with the apache-airflow-providers-http 2.0.2 package, which declares the entry point. I also ran the connector against apache-airflow-providers-http 6.3.0: the provider info is located through the entry point on this branch, and the result is the same as on main. flake8 and black --check are clean on the changed files.

Fixes #3152

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.89%. Comparing base (be38427) to head (fbdca08).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3447      +/-   ##
==========================================
+ Coverage   84.82%   84.89%   +0.06%     
==========================================
  Files         159      159              
  Lines       19297    19351      +54     
  Branches      500      500              
==========================================
+ Hits        16369    16428      +59     
+ Misses       2889     2884       -5     
  Partials       39       39              

Flag	Coverage Δ
integration-editors	48.21% <ø> (+0.06%)	⬆️
integration-misc	25.64% <ø> (ø)
integration-pipeline	47.68% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...ctor/airflow_provider_package_catalog_connector.py	83.98% <100.00%> (+4.09%)	⬆️
...airflow/test_airflow_provider_package_connector.py	100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lresende]

Thanks for picking this up! Reading the provider_info entry point from the dist-info metadata is the right approach and matches Airflow's provider contract, and the parametrized tests cover the main paths well. There are a few things I'd like to see before merging. First, the package case (my_provider/provider/__init__.py) only works if that __init__.py doesn't import from its own package. The module is still run with exec() in an empty namespace, so from my_provider.version import ... and from .version import ... both fail. The error then lands in the generic outer except with a message that doesn't help the user. Could you catch ImportError around the exec() and log a specific message, or load the module with its package context, and add a test for a provider-info module that has imports? Second, namespace[entry_point.attr] doesn't handle the dotted form (module:Info.get) or an entry point with no :attr (it logs "cannot invoke 'None' method"), so please handle or reject those cases explicitly. Third, it would help to log a deprecation warning when we fall back to get_provider_info.py, since Airflow itself only finds providers through the entry point. Finally, please review docs/source/user_guide/pipeline-components.md for inconsistencies. The intro (line 68) says this connector is "only supported for Airflow < 2", and the provider package section (lines 459–463) marks it "Deprecated in Elyra 4" and "not working in Elyra >=4". But the apache_airflow_provider entry point is an Airflow 2 mechanism, and both the existing and new tests use Airflow 2 provider packages. Please update that section to say which Airflow and Elyra versions the connector actually supports, and mention that it now finds the provider info through the entry point.

[NopeCharon]

@lresende Updated in fbdca08: the module now runs with a name so both absolute and relative imports raise ImportError, which is logged with a specific message, module:Info.get is resolved and an entry point without :attr is rejected, and the get_provider_info.py fallback logs a deprecation warning, all with tests. In the docs I added how the provider info is located, but I left the Airflow/Elyra version statements alone because I couldn't confirm that custom Airflow components work end to end on Elyra 4, so I'd rather you say which versions it should claim.