Regex decoding support

[Jiayang Wu (sheepymeh)]

Is regex decoding or a "JSON mode" available?

[Yufeng Li (yufenglee)]

Could you please explain more on this? What is regex decoding?

[Jiayang Wu (sheepymeh)]

It's decoding that follows a specific format (typically described by a regex). Some examples are:

Outlines: https://github.com/outlines-dev/outlines

SGLang: https://lmsys.org/blog/2024-02-05-compressed-fsm/

This allows the model to generate according to a fixed format that can later be decoded, such as ensuring that output is valid JSON.

[Nat Kershaw (MSFT) (natke)]

Thank you for the suggestion! I am adding it to our new Features discussions so that we can look into this further

[kunal-vaishnavi]

All examples have been updated in this PR. Please see the READMEs in each example for more information on grammars and tool calling.