I am trying to reproduce the ASID-Captioner-3B using the provided training scripts. Specifically, when running train_qwen2.5-omi-stage1-2.sh for Stage 2, I noticed that a significant number of samples from the single_attribute_0_30_s_youtube_v0_1.jsonl dataset are being filtered out or truncated because they exceed the defined max_length.
Details:
In train_qwen2.5-omi-stage1-2.sh, the configuration is:
VIDEO_TOTAL_PIXELS=20070400 (~20M pixels)
max_length 16384
Problem Analysis: Based on the Qwen2.5-Omni architecture:
The combination of visual tokens + structured text + system prompts often exceeds 16,384, causing the trainer to skip these valuable samples.
Interestingly, I noticed that in train_qwen2.5-omi-stage3.sh, the configuration is more relaxed regarding length but stricter on pixels:
VIDEO_TOTAL_PIXELS=12042240 (~12M pixels)
max_length 22384
Questions:
Is it intended for Stage 2 to have such a high pixel budget (20M) with a relatively small max_length (16384)?
Does this filtering significantly impact the model's ability to learn fine-grained attributes during the early stages?
For a successful reproduction, should I:
Increase max_length to 22384 for Stage 2?
Or decrease VIDEO_TOTAL_PIXELS to 12M (similar to Stage 3)?
Environment:
Model: Qwen2.5-Omni-3B
Framework: ms-swift v3.9.3
Script: train_qwen2.5-omi-stage1-2.sh
I am trying to reproduce the ASID-Captioner-3B using the provided training scripts. Specifically, when running train_qwen2.5-omi-stage1-2.sh for Stage 2, I noticed that a significant number of samples from the single_attribute_0_30_s_youtube_v0_1.jsonl dataset are being filtered out or truncated because they exceed the defined max_length.
Details:
In train_qwen2.5-omi-stage1-2.sh, the configuration is:
VIDEO_TOTAL_PIXELS=20070400 (~20M pixels)
max_length 16384
Problem Analysis: Based on the Qwen2.5-Omni architecture:
The combination of visual tokens + structured text + system prompts often exceeds 16,384, causing the trainer to skip these valuable samples.
Interestingly, I noticed that in train_qwen2.5-omi-stage3.sh, the configuration is more relaxed regarding length but stricter on pixels:
VIDEO_TOTAL_PIXELS=12042240 (~12M pixels)
max_length 22384
Questions:
Is it intended for Stage 2 to have such a high pixel budget (20M) with a relatively small max_length (16384)?
Does this filtering significantly impact the model's ability to learn fine-grained attributes during the early stages?
For a successful reproduction, should I:
Increase max_length to 22384 for Stage 2?
Or decrease VIDEO_TOTAL_PIXELS to 12M (similar to Stage 3)?
Environment:
Model: Qwen2.5-Omni-3B
Framework: ms-swift v3.9.3
Script: train_qwen2.5-omi-stage1-2.sh