How to re-run only failed mini-batches in AML pipeline with prompt flow component ?

[Megha Patil (meghapatilcode)]

We are using prompt flow as a parallel component in our AML pipelines. Prompt flow is run for each input row from a JSONL file in mini-batches. The mini-batch workloads are set to fail after reaching the "mini batch max retries". The AML pipeline is however set to not fail despite failed mini-batches by keeping the default value for "mini batch error threshold" as -1. We are unable to find a way to re-run the pipeline for only the failed mini-batches. Also, we are unable to find a direct way to isolate and download the records for which mini-batch workloads have failed. How can we re-run the AML pipeline such that only the failed mini-batches are run again?

[Xingzhi Zhang (elliotzh)]

Alain LI (@alainli0928) do we have such support for parallel component?