Support for Image Inputs and Outputs for Multimodal Models

[byfb]

Hi agenta community,

I’m exploring Agenta’s support for multimodal models and noticed that the core litellm backend supports image inputs via URLs/base64. However, I couldn’t find an option to submit images in the frontend UI.

Could you clarify:

Is this feature intentionally not implemented yet?

Or am I missing where the image input option is located?

agenta v0.48.9

[dosubot[bot]]

Hi @sbbfung! I'm Dosu and I’m helping the agenta team.

You’re not missing anything—image input support isn’t available in the Agenta frontend UI as of v0.48.9. While the backend (litellm) can handle image inputs, there’s currently no way to upload or submit images from the UI. There is an unused image upload component in the codebase, which suggests this feature may be planned, but it’s not wired into any user-facing flows yet. There’s also no documentation or public discussion indicating a timeline or intentional omission for this feature, so you’re not overlooking a hidden option or setting reference.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[mmabrouk]

Hey @sbbfung

We don't support image input in the playground yet. We are adding this support this week. You will be able to add images in the chat messages (but not part of the prompt template [like you are showing in the screenshot]).

We did not find any use case where the prompt template would include images. Do you have such use case?

[byfb]

Hi @mmabrouk

Thanks for the update! Use case as follow:
Rapidly A/B test prompts across images via the playground
Tweak prompt variables (e.g., tone, focus areas) while keeping the image fixed
Test the same prompt against multiple images (e.g., swapping charts/scenes)

I’m building workflows for tasks like visual storytelling (e.g., "Generate a story based on this scene") and image-based text extraction (e.g., "Pull key metrics from this chart")

[mmabrouk]

Thanks for the explanation of the user case @sbbfung

I think the feature in work would solve this issue. You can have your images as part of the test set for the prompt and then A/B test prompt playground on the same images, play with the prompts and rerun it on the same image, test the prompt on different images, etc..

[byfb]

hi @mmabrouk

Thanks for the update on image inputs

Will the frontend UI also support displaying image outputs for some model that can generate images on playground?

This would be super helpful for end-to-end prototyping of image-generation workflows!

[mmabrouk]

Hello @sbbfung,

We have this on our roadmap. But the current feature that is being worked on only includes using images as inputs.

[mmabrouk]

@sbbfung we have officially launched this feature today (at least for inputs)