Hyper-parameters used for reproducibility

[dmgcsilva]

Hello, thanks for the great work and contribution to the community!

Would it be possible to share more details regarding the hyperparameters used for each stage to ensure reproducibility?
For example the batch size, max sequence length, learning rate and scheduler, warmup ration, etc.

Many thanks!

[yiyexy]

Sure, you can find the scripts for each stage in the repo. We used the default values for training.