Batch sizing: get recommendations for a list of models in one run

[aditisaluja5]

Today ConfigIQ sizes one model at a time. Sizing a set of models (e.g. for an architecture doc or a customer deployment plan) means running the flow once per model and manually collecting the results into a table.

Request: Let a user submit a list of models and get sizing back for all of them in one run.

Proposed shape:

• Input a list of models (paste HuggingFace IDs, or multi-select from the model picker), plus one shared workload profile (GPU, ISL, OSL, TTFT/concurrency target, precision).
• ConfigIQ runs the recommend flow for each model and returns a single results table: model, GPUs/replica (TP), replicas for target, memory/replica, TTFT, TPOT, throughput.
• Export the table to CSV.
• Handle unsupported models gracefully in the output (mark "not supported in AIC" rather than failing the whole batch).

Why: Real sizing tasks are almost always multi-model (a customer runs a fleet, an arch doc lists 10+ models). One-at-a-time is slow and error-prone when assembling comparisons.

Notes:

• Per-model workload overrides could come later; a shared workload profile covers the common case.
• Pairs well with the existing CSV export on the performance page.

[mazam-lab]

Adding some quick notes on ideas for approaching this feature @natoscott if you want to take a look

1. The model dropdown we have can be replaced by a multiselect dropdown that simply lists the models names that have been selected by the user. To avoid obscuring information for non-batch runs, if the user has less than two currently selected models, it will just show the same details of the selected model that we currently have

2. For displaying the results; I think the cleanest solution is to keep the core panel appearance we currently have, but, once the results are finished, add buttons that allow the user to switch between the results of different models. This can also be used so that the user is able to view results for models whose data has been collected while the rest of the batch is still being processed (any models that don't have complete data can simply be greyed out until they're done)

3. We can add a setting or toggle for a "compact mode" where, instead of needing to switch between the results of different models, they can see a singular table, where each model in the batch is given a row, with its respective data values filling up the columns. This can also serve as the basis for how the data will be structured when exporting as a CSV or other format

[natoscott]

Good ideas there @mazam-lab - I like it. Multi-select sounds like a good solution, and the results display ideas (if I understand correctly) are strong IMO.

@aditisaluja5 originally mentioned results tables - if I follow your approach of using buttons to switch between the results, this is effectively achieving that by displaying (essentially) one "row" at a time. I'm imagining a kind of "button" something like this...

| Search Results (Showing 1 of N)            |

(regular UI)

| [Prev]  Result 1 of N  [Next]  [Jump to N] |

Is that along the lines of what you're thinking?

I was recently pondering how we could show top N results too (which the REST API already supports I think, from memory) - it'd be great if we could use this mechanism in that case too. Perhaps for a single model request we should top N results this way, for a multi-model request we show top-1 result for N models? (changing the text above "Result" to "Model" for in the latter case I guess)