[Model] Add user-configurable task for models that support both generation and embedding (#9424)

This commit is contained in:
Cyrus Leung authored and GitHub committed 2024-10-18 11:31:58 -07:00
1 parent 7dbe738d65
commit 051eaf6db3
33 files changed
+451 -201

No files matched your search

@@ -89,6 +89,7 @@ def run_test(
# max_model_len should be greater than image_feature_size
with vllm_runner(model,
task="generate",
max_model_len=4096,
max_num_seqs=2,
dtype=dtype,