mirror of
https://github.com/wassname/discovering_latent_knowledge.git
synced 2026-09-10 12:00:13 +08:00
tidy
This commit is contained in:
deep1
@@ -1,6 +1,7 @@
|
||||
lightning_logs/
|
||||
.pkl_cache/
|
||||
.ds/
|
||||
/notebooks/old/
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
|
||||
+174
@@ -527,3 +527,177 @@ OK so I tried to do it with 3 type of prompt: lie, true and simple. The simple o
|
||||
|
||||
Oh it was because I asked it to say negative, but it REALLY wanted to say Negative. OK.
|
||||
So measuring a simple prompt: 95%, and measuring the complex true prompt: 94%, so not worth the 2x slowdown.
|
||||
****
|
||||
# 2023-07-13 18:21:20
|
||||
|
||||
- [ ] do the triple label experiment
|
||||
- [ ] do mc ranking for deception?
|
||||
|
||||
Where were we? Oh I now have paired lie and no lie. The no lie is 94% so it should give us some idea of when it knows it.
|
||||
|
||||
Now I can try that 4 class one. Actually I will merge the two unknown classes as we do not have the information to **distinguish**
|
||||
|
||||
|
||||
| | knows | unknown |
|
||||
| ----- | ------- | ------- |
|
||||
| right | correct | unknown |
|
||||
| wrong | lie | unknown |
|
||||
|
||||
|
||||
And in terms of putting them into the model. Do I want
|
||||
- MC ranking: model(x)>model(y)
|
||||
- this is really ranking loss, and I can use montecarlo dropouts
|
||||
- model(x, y)>0, which seems to work better but might it be cheating?
|
||||
- model(x)=class?
|
||||
- which is the simplest.
|
||||
- I'm really saying, given these hidden states is the model going to be
|
||||
- unsure
|
||||
- I can get this for when I know it's wrong. That means internal and external uncertainty, but the model can probobly only do internal
|
||||
- Or I could use something like MCDropout but then we have further subclasses
|
||||
- unconfident wrong
|
||||
- unconfident right
|
||||
- confident wrong
|
||||
- deceptive
|
||||
- sure
|
||||
- or
|
||||
- truth
|
||||
- lie
|
||||
- unsure/mistake/random
|
||||
-
|
||||
|
||||
|
||||
I could do just the ones where it is given examples lie? but sometimes it does and sometimes it doesn't
|
||||
|
||||
|
||||
# exp 3 classes: truth, deception, unkown. lie only
|
||||
|
||||
Results:
|
||||
- 37% f1 deception detection on weight cross entropy loss
|
||||
|
||||
precision recall f1-score support
|
||||
truth 0.873932 0.976911 0.922556 3768.000000
|
||||
deception 0.493878 0.297297 0.371166 407.000000
|
||||
unkown 0.558140 0.073846 0.130435 325.000000
|
||||
accuracy 0.850222 0.850222 0.850222 0.850222
|
||||
macro avg 0.641983 0.449351 0.474719 4500.000000
|
||||
weighted avg 0.816751 0.850222 0.815477 4500.000000
|
||||
|
||||
|
||||
# exp 2 classes. lie only
|
||||
|
||||
notebooks/018_mjc_2cls.ipynb
|
||||
|
||||
Hmm maybe I should make a deception vs not class? 44% f1 score
|
||||
|
||||
|
||||
# experiment mc dropout, ranking?
|
||||
|
||||
Can I use a ranking loss?
|
||||
|
||||
" Ranking Losses is to predict relative distances between inputs. This task if often called metric learning."
|
||||
|
||||
usuall you input a similar and disimilar pair. So in that case I don't have that data.
|
||||
But I do have 2 and I know the direciton
|
||||
|
||||
|
||||
We could have true>unsure>lie
|
||||
|
||||
|
||||
We could use https://pytorch.org/docs/stable/generated/torch.nn.MarginRankingLoss.html#torch.nn.MarginRankingLoss with -1 and 1 losses
|
||||
|
||||
|
||||
|
||||
Tasks
|
||||
|
||||
| type | max auc_roc |
|
||||
| ------- | ----------- |
|
||||
| ranking | 82% |
|
||||
| cls_2 | 80% |
|
||||
| cls_3 | |
|
||||
| mse | 74% |
|
||||
|
||||
|
||||
- exp: OK so if we use ranking ~30%... no 82
|
||||
- exp: if we use distance, and mse or smoothl1loss then we do a bit better ~50% notebooks/019_mjc_ranking_distance.ipynb
|
||||
- and with better hparams we get 86%!
|
||||
- this kind of makes sense? now what if we normalize? 65% notebooks/019_mjc_distance_mse_norm.ipynb
|
||||
- subtract and norm? 72% notebooks/019_mjc_distance_mse_subt_norm.ipynb
|
||||
- exp: what about just classify direction? 66%
|
||||
- OH it turn out the loss curve is weird, as the modedl si too small...
|
||||
|
||||
- [ ] Not very good? What if we normalize the hidden states in one of a few ways
|
||||
- [ ] each neurons
|
||||
- [ ] the total magnitude
|
||||
|
||||
ideas:
|
||||
- normalize
|
||||
- only wory about direction, nothing else. so it's a class
|
||||
- actually remove the 4% of confusion, it might significantly overlap with the 10% of lies!... but oh wait we are looking at direction right now
|
||||
- for that matter we have ans1 and ans2 and one might be a lie and one migth not in another ~4% of cases
|
||||
- maybe I should dropout the first few layers and measure the next?
|
||||
- hs1-hs2
|
||||
|
||||
# 2023-07-20 08:53:24
|
||||
|
||||
I would like a better score than 82% but when independant models are getting similar rates then meh.
|
||||
|
||||
- [ ] Exp: clean data, more data
|
||||
- [x] Only the ones where it knows the answer
|
||||
- [x] Only the ones with significant permuations
|
||||
- [x] More data (using map to transform)
|
||||
- [ ] results...?... it's broken lol
|
||||
- Exp: test generalization other prompts
|
||||
- what's the accuracy with multiple dropouts? does it help?
|
||||
|
||||
150*6
|
||||
|
||||
Oh when I limited it to answers that moved by more than 5%, it did poorly
|
||||
maybe if I divide by that?
|
||||
|
||||
|
||||
what about `(hs1-hs2)/dProb`? And then cls direction?
|
||||
|
||||
|
||||
Ah found the bug! I shuffled the dataset for X, but then drew y from the unshuffled lol! FML
|
||||
|
||||
:poop: :poop: :poop:
|
||||
|
||||
|
||||
Hmm so a linear model gets 70%, and all my models get only 73% lol. This is with all "accident" rows removed...
|
||||
|
||||
So what next? A differen't way o toiew the data?
|
||||
|
||||
Oh I can rerun my norm ones..
|
||||
|
||||
|
||||
# 2023-07-21 21:44:17
|
||||
|
||||
Good result. notebooks/020_mjc_ranking_loss_w_scaling_big_moves_94% copy.ipynb
|
||||
|
||||
Here is get 71% with a linear prob. But 89% with a ranking loss model!
|
||||
|
||||
I might be able to restrict it to large dprobs and tune to get an even better result!
|
||||
|
||||
Perhaps I can do linear probes on a subset to explore some dims?
|
||||
|
||||
:notebook: ranking loss performs better, learning more, managing deeper networks, not overfitting.
|
||||
|
||||
This makes sense for several reasons:
|
||||
- the network has no realitive information it can use to overfit
|
||||
- it has absolute information on activations, which may be importanst as it's operating on a multidimensional optimisation surface, where absolute position may give important informaiton. As an analogy imagine you are on a gold course, which is more usefull, knowledge that two balls are 2 meters apart and a 30deg incline. Or that that 2 meters is between the top of a small hill and the other a sandpit, with the 30 deg incline between them. Absolute information seems important!
|
||||
|
||||
|
||||
exp
|
||||
- [ ] how does a change in min dDrop change things? maybe with lienar
|
||||
- [ ] use UQA dataset... oh wait that's a type of dataset, and am odel
|
||||
- [ ] does result generalzie between datasets?
|
||||
- [ ] can I get above 89% with hyperopt?
|
||||
- [ ] can I get above 89% with mcdropout?
|
||||
- [ ] Triplet loss? I just need to make more mcdropouts
|
||||
|
||||
|
||||
TODO
|
||||
- test with diff prompt e.g please lie, e.g. please tell truth, e.g. give random answer
|
||||
- can we do this interactivly? or a very small dataset with random prompt the model comes up with?
|
||||
- test with truthfullqa https://huggingface.co/datasets/EleutherAI/truthful_qa_binary
|
||||
- maybe generate dataset?
|
||||
|
||||
@@ -1,407 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Let's implement CCS from scratch.\n",
|
||||
"This will deliberately be a simple (but less efficient) implementation to make everything as clear as possible."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm.auto import tqdm\n",
|
||||
"import copy\n",
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"from matplotlib import pyplot as plt\n",
|
||||
"\n",
|
||||
"import torch\n",
|
||||
"import torch.nn as nn\n",
|
||||
"import torch.nn.functional as F\n",
|
||||
"from torch import Tensor\n",
|
||||
"from torch import optim\n",
|
||||
"\n",
|
||||
"import pickle\n",
|
||||
"import hashlib\n",
|
||||
"from pathlib import Path\n",
|
||||
"import os\n",
|
||||
"# os.environ[\"HF_DATASETS_OFFLINE\"] = \"0\"\n",
|
||||
"from datasets import load_dataset\n",
|
||||
"import datasets\n",
|
||||
"from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, AutoModelForMaskedLM, AutoModelForCausalLM\n",
|
||||
"from transformers import LlamaTokenizer, LlamaForCausalLM\n",
|
||||
"from sklearn.linear_model import LogisticRegression\n",
|
||||
"\n",
|
||||
"import lightning.pytorch as pl\n",
|
||||
"from dataclasses import dataclass\n",
|
||||
"from torch.utils.data import random_split, DataLoader, TensorDataset\n",
|
||||
"from transformers.models.auto.modeling_auto import AutoModel\n",
|
||||
"# from scipy.stats import zscore\n",
|
||||
"from sklearn.metrics import f1_score, roc_auc_score, accuracy_score\n",
|
||||
"from sklearn.preprocessing import RobustScaler\n",
|
||||
"import gc\n",
|
||||
"\n",
|
||||
"from loguru import logger\n",
|
||||
"logger.add(os.sys.stderr, format=\"{time} {level} {message}\", level=\"INFO\")\n",
|
||||
"\n",
|
||||
"import os"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Model"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "95967022bbfd4f268f6470cc17b3c6b7",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading tokenizer.model: 0%| | 0.00/500k [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "dfe0b9f4ce8f46d0b208cf142d99927f",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)cial_tokens_map.json: 0%| | 0.00/411 [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "133d92009e8b4beca07d5f69f179a9cf",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)okenizer_config.json: 0%| | 0.00/727 [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "bd3f4d8521034734906339f6587ca368",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)lve/main/config.json: 0%| | 0.00/502 [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "a6694e8377c240e2afa67d660cb8789a",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)fetensors.index.json: 0%| | 0.00/50.1k [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "46ee6330036947ae95624b7227372d7d",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading shards: 0%| | 0/7 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "f13d7bb9be9f4837933906ebb1e1dfd7",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)of-00007.safetensors: 0%| | 0.00/9.82G [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "dbd3c95f1eb64d3f8d1827c0ba6f014a",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)of-00007.safetensors: 0%| | 0.00/9.96G [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "b13c88c85eba421bafc30573ca7f3af2",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)of-00007.safetensors: 0%| | 0.00/9.90G [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "40fa7d1f69ec4c5fb30c2391f2cb101b",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)of-00007.safetensors: 0%| | 0.00/9.87G [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "4472fae2f74944f9a2ccedaba9a554cf",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)of-00007.safetensors: 0%| | 0.00/9.87G [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "a8d62732bb3a4a98802b70886a08878e",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)of-00007.safetensors: 0%| | 0.00/9.96G [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "7241a271ba7e49cea3fd6ea5c84b10ab",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)of-00007.safetensors: 0%| | 0.00/5.69G [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"===================================BUG REPORT===================================\n",
|
||||
"Welcome to bitsandbytes. For bug reports, please run\n",
|
||||
"\n",
|
||||
"python -m bitsandbytes\n",
|
||||
"\n",
|
||||
" and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues\n",
|
||||
"================================================================================\n",
|
||||
"bin /home/ubuntu/mambaforge/envs/dlk2/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda117.so\n",
|
||||
"CUDA SETUP: CUDA runtime path found: /home/ubuntu/mambaforge/envs/dlk2/lib/libcudart.so\n",
|
||||
"CUDA SETUP: Highest compute capability among GPUs detected: 8.6\n",
|
||||
"CUDA SETUP: Detected CUDA version 117\n",
|
||||
"CUDA SETUP: Loading binary /home/ubuntu/mambaforge/envs/dlk2/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda117.so...\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/home/ubuntu/mambaforge/envs/dlk2/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/home/ubuntu/mambaforge/envs/dlk2/lib/libcudart.so'), PosixPath('/home/ubuntu/mambaforge/envs/dlk2/lib/libcudart.so.11.0')}.. We'll flip a coin and try one of these, in order to fail forward.\n",
|
||||
"Either way, this might cause trouble in the future:\n",
|
||||
"If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.\n",
|
||||
" warn(msg)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "68cf34d0be6a44498b9596423c532c64",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Loading checkpoint shards: 0%| | 0/7 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "bb4895dd76054731876ad58fc4dd15ce",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)neration_config.json: 0%| | 0.00/132 [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "285efdf077ce46919fba89f4255880eb",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading (…)/adapter_config.json: 0%| | 0.00/429 [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "4809b959221f4f12914e19cc74a1d5b3",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Downloading adapter_model.bin: 0%| | 0.00/205M [00:00<?, ?B/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model_options = dict(\n",
|
||||
" device_map=\"auto\", \n",
|
||||
" # load_in_8bit=True,\n",
|
||||
" load_in_4bit=True,\n",
|
||||
" torch_dtype=torch.float16,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# 7B\n",
|
||||
"# model_repo = \"Neko-Institute-of-Science/LLaMA-7B-HF\"\n",
|
||||
"# lora_repo = \"chansung/gpt4-alpaca-lora-7b\"\n",
|
||||
"\n",
|
||||
"# 13B\n",
|
||||
"model_repo = \"Neko-Institute-of-Science/LLaMA-13B-HF\"\n",
|
||||
"lora_repo = \"chansung/gpt4-alpaca-lora-13b\"\n",
|
||||
"\n",
|
||||
"model_repo = \"TheBloke/Wizard-Vicuna-13B-Uncensored-HF\"\n",
|
||||
"lora_repo = None\n",
|
||||
"\n",
|
||||
"# 30B\n",
|
||||
"model_repo = \"TheBloke/OpenAssistant-SFT-7-Llama-30B-HF\"\n",
|
||||
"model_repo = \"ausboss/llama-30b-supercot\"\n",
|
||||
"# model_repo= \"timdettmers/guanaco-33b-merged\"\n",
|
||||
"lora_repo = None\n",
|
||||
"\n",
|
||||
"model_repo = \"Neko-Institute-of-Science/LLaMA-30B-HF\"\n",
|
||||
"lora_repo = \"chansung/gpt4-alpaca-lora-30b\"\n",
|
||||
" \n",
|
||||
"tokenizer = LlamaTokenizer.from_pretrained(model_repo)\n",
|
||||
"model = AutoModelForCausalLM.from_pretrained(model_repo, **model_options)\n",
|
||||
"\n",
|
||||
"if lora_repo is not None:\n",
|
||||
" # https://github.com/tloen/alpaca-lora/blob/main/generate.py#L40\n",
|
||||
" from peft import PeftModel\n",
|
||||
" model = PeftModel.from_pretrained(\n",
|
||||
" model, \n",
|
||||
" lora_repo, \n",
|
||||
" torch_dtype=torch.float16,\n",
|
||||
" device_map='auto'#{'': 0}\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "dlk2",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.16"
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user