wassname/alpaca_convert
1
0
Fork 0
mirror of https://github.com/wassname/alpaca_convert.git synced 2026-06-27 17:47:04 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix some issues.

Browse Source
This commit is contained in:
Andy Barry
2023-04-05 23:29:10 -04:00
parent 417eba372a
commit 1af4023883
2 changed files with 6 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+4 -8
View File
@@ -7,8 +7,8 @@ Based on https://github.com/johnsmith0031/alpaca_lora_4bit
Can run real-time LLM chat using alpaca on a 8GB NVIDIA/CUDA GPU (ie 3070 Ti mobile)
## Requirements
- linux with docker
- nvidia GPU
- Docker
- NVIDIA GPU
## Installation
@@ -19,13 +19,9 @@ docker run -p 7086:7086 alpaca_lora_4bit
Point your browser to http://localhost:7086
## Results
It's fast on a 3070 Ti.
### Discussion
The model isn't all that good, sometimes it goes crazy. But hey, "when 4-bits _you reach_ look this good you will not."
But it is fast (on my 3070 Ti mobile at least)
It's fast on a 3070 Ti mobile. Uses 5-6 GB of GPU RAM.
The model isn't all that good, sometimes it goes crazy. But hey, as I always say, "when 4-bits _you reach_ look this good you will not."
## References