wassname/vllm
1
0 Fork 0
mirror of https://github.com/wassname/vllm.git synced 2026-09-30 11:45:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Frontend] Add backend-specific options for guided decoding (#13505)

Browse Source 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
This commit is contained in:
Joe Runde authored and GitHub committed 2025-02-20 15:07:58 -05:00
1 parent 6a417b8600
commit bfbc0b32c6
8 files changed
+123 -42
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/source/features/structured_outputs.md
+1 -1
View File
@@ -16,7 +16,7 @@ The following parameters are supported, which must be added as extra parameters:
- `guided_json`: the output will follow the JSON schema.
- `guided_grammar`: the output will follow the context free grammar.
- `guided_whitespace_pattern`: used to override the default whitespace pattern for guided json decoding.
- `guided_decoding_backend`: used to select the guided decoding backend to use.
- `guided_decoding_backend`: used to select the guided decoding backend to use. Additional backend-specific options can be supplied in a comma separated list following a colon after the backend name. For example `"xgrammar:no-fallback"` will not allow vLLM to fallback to a different backend on error.
You can see the complete list of supported parameters on the [OpenAI-Compatible Server](#openai-compatible-server)page.