Variants order might change in package.py when doing partial builds/releases

[herronelou]

Describe the bug here.

Environment

• OS - Windows 11 / Rocky 9
• Rez version - 2.112.0
• Rez python version - Python 3.7.9

To Reproduce

• Create a package.py with multiple variants
• Build the variants using the --variants arg, and build index 1, then repeat with index 0
  Resulting built package has variants out of order, which is problematic when the commands refer to a variant by index.

Expected behavior
Ideally preserve order, but might require padding the variants list with something until the other variants are built.

Actual behavior
Variant index changes

Why are we trying to do this
We're having problems building packages with variants for multiple OSs, we have to build the linux variants on linux, and the windows variants on windows. The order in which we do so is currently affecting the final order in the package.

[nrusch]

This would be a nice improvement indeed. 👍

Just in case it's useful for you, we are also in a multi-platform environment where we're coalescing variants built in parallel on different hosts into a single package deployment. Originally, our plan was to rez-cp new variants into a package that's currently ignored, but that just doesn't work right now (reported in #1957). There are also some hairier issues (race conditions, etc.) with trying to approximate coherent file locking on shared filesystems from simultaneous Windows and Linux clients, but that's a more fundamental problem with the de facto "filesystem-as-a-database" approach to rez package repositories.

Anyway... Our workaround for now is to build each variant to a separate directory (essentially its own package repository), and then use a (mostly) straightforward post-process to merge the package.py files (which basically involves taking one of them and replacing its variants data with the array from the input package.py) and move/copy the variant payloads into their final location.

[JeanChristopheMorinPerso]

Hey @herronelou and @nrusch , I don't think that we give a guarantee on the variants order. I believe the proper way to fix this would be to introduce named variants, as proposed in #1503.

[herronelou]

Agreed @JeanChristopheMorinPerso this would likely be the way to go.

We've been using variant indices in the commands in the past for things like OS or DCC variations, I'm not sure where we picked this habit from, but we probably want to use another way instead.