6 Commits

Author	SHA1	Message	Date
Woosuk Kwon	311490a720	Add script for benchmarking serving throughput (#145)	2023-06-14 19:55:38 -07:00
Zhuohan Li	4298374265	Add docstrings for LLMServer and related classes and examples (#142)	2023-06-07 18:25:20 +08:00
Woosuk Kwon	e38074b1e6	Support FP32 (#141)	2023-06-07 00:40:21 -07:00
Zhuohan Li	1a956e136b	Fix various issues of async servers (#135)	2023-06-05 23:44:50 +08:00
Woosuk Kwon	655a5e48df	Introduce LLM class for offline inference (#115)	2023-05-21 17:04:18 -07:00
Woosuk Kwon	c3442c1f6f	Refactor system architecture (#109)	2023-05-20 13:06:59 -07:00