mirror of
https://github.com/wassname/lora-lite.git
synced 2026-07-13 17:44:07 +08:00
types, review
This commit is contained in:
wassname
@@ -0,0 +1,186 @@
|
||||
# V4 Variant Review — per-component vs reference + smoke/probe validity
|
||||
|
||||
You are an expert ML engineer reviewing a from-scratch PEFT library
|
||||
(`lora-lite`, ~500 LOC) that re-implements 8 LoRA variants. Three prior
|
||||
reviews already happened (V1 paper-vs-code, V2 with refs provided, V3
|
||||
per-component). Your job is V4: re-run the per-component check and
|
||||
additionally validate the test harness.
|
||||
|
||||
# Part A — per-variant audit (re-do, more rigorous)
|
||||
|
||||
8 variants live in `src/lora_lite/variants/`:
|
||||
- lora.py
|
||||
- pissa.py
|
||||
- delora.py
|
||||
- ia3.py (registers `ia3` and `ia3_ff`)
|
||||
- dora.py
|
||||
- hra.py
|
||||
- eva.py
|
||||
- antipasto.py
|
||||
|
||||
Plus runtime in `src/lora_lite/{adapter.py,variant.py,target.py,config.py}`.
|
||||
Reference implementations are in `docs/refs/` and URLs are pasted in each
|
||||
variant's module docstring.
|
||||
|
||||
## For EACH variant, in this order, every time:
|
||||
|
||||
1. **REFERENCE EXISTS** — verify the variant has a real, citeable
|
||||
reference. Required:
|
||||
- a paper (arxiv/conference) link, AND
|
||||
- either an upstream peft implementation OR the original author's
|
||||
code (GitHub).
|
||||
If the variant has NO paper, NO reference code, OR the references
|
||||
are dead/missing/clearly wrong, FLAG IT as `NO REFERENCE` -- this
|
||||
is severity HIGH because it means there's nothing to validate
|
||||
against.
|
||||
|
||||
2. **PARAMS** — every spec from `param_specs`: shape, dtype, trainable,
|
||||
as_buffer. Match against the reference. Buffers vs Parameters
|
||||
chosen correctly?
|
||||
|
||||
3. **INIT** — what does `init()` (and `group_init()` if defined) do?
|
||||
Match the reference exactly? Walk gradient flow at t=0: which
|
||||
trainable params actually receive non-zero gradient on step 1?
|
||||
|
||||
4. **DTYPE** — trace dtype through init -> storage -> forward.
|
||||
Silent precision loss? Identity-at-init survive bf16?
|
||||
|
||||
5. **FORWARD** — write the math the forward implements vs the math
|
||||
in the reference. Term-by-term comparison. Common mistakes:
|
||||
- wrong scale (alpha/r vs 1/r vs alpha vs 1)
|
||||
- missing/doubled normalization
|
||||
- wrong basis (rotating U vs V; gating input vs output)
|
||||
- dropout placement (we have NO dropout by design — flag if any
|
||||
code path depends on one)
|
||||
|
||||
6. **LINK SANITY** — actually open the URLs. Verify:
|
||||
- paper arxiv link goes to the right paper
|
||||
- github link points to a real file (not 404)
|
||||
- offline `docs/refs/` snapshot still matches what the URL serves
|
||||
today (snapshots may be stale; flag drift)
|
||||
|
||||
## Per-variant output (≤60 lines each):
|
||||
|
||||
## <variant>
|
||||
|
||||
### references
|
||||
- paper: <url> -- OK / WRONG / DEAD / MISSING
|
||||
- peft ref: <url> -- OK / DEAD / MISSING
|
||||
- author ref (if any): <url> -- OK / DEAD / MISSING
|
||||
- offline snapshot (`docs/refs/...`): NONE / MATCH / DRIFT
|
||||
- VERDICT: HAS_REFERENCE / NO_REFERENCE
|
||||
|
||||
### params
|
||||
- <one bullet per ParamSpec; flag bug if any>
|
||||
|
||||
### init / group_init
|
||||
- <bullets; identify GRADIENT FLOW at t=0 explicitly>
|
||||
|
||||
### dtype
|
||||
- <bullets>
|
||||
|
||||
### forward
|
||||
Math (ours): <one-line equation>
|
||||
Math (ref): <one-line equation>
|
||||
Match? YES / NO + one-line reason
|
||||
|
||||
### verdict
|
||||
CORRECT / PARTIAL / BUGGY -- one-sentence reason
|
||||
|
||||
# Part B — validate the smoke test (`tests/smoke.py`)
|
||||
|
||||
Read `tests/smoke.py` end-to-end. For each per-variant SHOULD claim,
|
||||
answer:
|
||||
|
||||
1. **Distinguishing power** — would a SILENT FAILURE (e.g. forward
|
||||
returning `y` unchanged, or training only the bias term, or
|
||||
loading an empty state dict) STILL pass this check? If yes,
|
||||
the check is WEAK -- name a stronger one.
|
||||
|
||||
2. **Tolerance sanity** — the bf16/fp16 tolerances are computed
|
||||
from `base_scale`. Are they too loose? Too tight? Could they
|
||||
pass on noise alone?
|
||||
|
||||
3. **Coverage** — what mechanisms are NOT tested? (e.g. multi-step
|
||||
convergence on real targets, dtype mismatch between attach and
|
||||
load, mixing variants, calibration data of len < r for EVA)
|
||||
|
||||
Output:
|
||||
|
||||
## smoke.py validity
|
||||
|
||||
### per-variant SHOULD checks
|
||||
| check | distinguishes silent failure? | tolerance ok? | notes |
|
||||
| ... |
|
||||
|
||||
### gaps
|
||||
- bullets
|
||||
|
||||
### must-add tests
|
||||
- bullets
|
||||
|
||||
# Part C — validate the qwen overfit probe (`scripts/qwen_train_probe.py`)
|
||||
|
||||
Read `scripts/qwen_train_probe.py` end-to-end. Same questions as Part B
|
||||
but for the Qwen probe specifically:
|
||||
|
||||
1. Does `assert_only_lora_trainable` actually catch a leaked base
|
||||
parameter, given the way `requires_grad` is set in `adapter.py`?
|
||||
|
||||
2. `perturb_first_adapter` only perturbs ONE param per variant. Does
|
||||
`perturb_delta > 1e-7` distinguish "the variant uses that param in
|
||||
forward" from "the variant ignores that param"?
|
||||
|
||||
3. `loss_last < loss0` after 8 steps with lr=5e-3 -- could this pass
|
||||
purely from optimizer noise? What's the right held-out / validation
|
||||
check to add?
|
||||
|
||||
4. The reload check uses `args.reload_tol` (default 2e-2 in bf16). Is
|
||||
that loose enough to mask a real save/load bug?
|
||||
|
||||
5. Targets are restricted to `model.layers.0.self_attn.{q,v}_proj` --
|
||||
does this exercise the full attach path or hide bugs that only
|
||||
appear with multi-layer / FFN / lm_head edge cases?
|
||||
|
||||
Output:
|
||||
|
||||
## qwen_train_probe.py validity
|
||||
|
||||
### claim-by-claim
|
||||
| assertion | catches silent failure? | notes |
|
||||
| ... |
|
||||
|
||||
### gaps
|
||||
- bullets
|
||||
|
||||
### must-add tests
|
||||
- bullets
|
||||
|
||||
# Final summary
|
||||
|
||||
After parts A, B, C, write:
|
||||
|
||||
## summary
|
||||
|
||||
### variant verdicts
|
||||
| variant | has_ref | params | init | dtype | forward | verdict |
|
||||
|
||||
### MUST-FIX (severity HIGH, blocks correctness claim)
|
||||
1. ...
|
||||
2. ...
|
||||
|
||||
### NICE-TO-HAVE
|
||||
- ...
|
||||
|
||||
# Hard rules
|
||||
|
||||
- Be specific. Cite line numbers (`src/lora_lite/variants/foo.py:NN`)
|
||||
for every claim.
|
||||
- Do NOT propose redesigns. Only flag correctness issues against
|
||||
references and validity issues in the test harness.
|
||||
- If an issue is intentional and documented in the docstring, say so
|
||||
and move on -- don't re-flag known deviations.
|
||||
- If you can't tell whether something is a bug, say "AMBIGUOUS" with
|
||||
the question you'd need answered.
|
||||
- For Part B/C, focus on whether checks have DISTINGUISHING power
|
||||
(would a silent failure still pass?) -- not just whether they run.
|
||||
File diff suppressed because it is too large
Load Diff
+2
-1
@@ -7,6 +7,7 @@ requires-python = ">=3.10"
|
||||
dependencies = [
|
||||
"torch>=2.1",
|
||||
"einops>=0.7",
|
||||
"jaxtyping>=0.2.34",
|
||||
]
|
||||
keywords = ["lora", "pytorch", "peft", "adapters", "llm"]
|
||||
classifiers = [
|
||||
@@ -24,7 +25,7 @@ Issues = "https://github.com/wassname/lora-lite/issues"
|
||||
|
||||
[project.optional-dependencies]
|
||||
build = ["twine>=6"]
|
||||
test = ["pytest", "tabulate"]
|
||||
test = ["pytest", "tabulate", "beartype>=0.18"]
|
||||
hf-test = ["accelerate>=1.6", "safetensors>=0.5", "transformers>=4.51"]
|
||||
bnb-test = ["bitsandbytes>=0.46"]
|
||||
|
||||
|
||||
@@ -1,3 +1,11 @@
|
||||
import os as _os
|
||||
|
||||
# Optional runtime shape/dtype checking via jaxtyping + beartype.
|
||||
# Set BEARTYPE=1 for smoke tests / debugging; off by default for zero overhead.
|
||||
if _os.environ.get("BEARTYPE"):
|
||||
from beartype.claw import beartype_this_package as _bt
|
||||
_bt()
|
||||
|
||||
from .config import LoraLiteConfig
|
||||
from .adapter import attach, detach, save, load
|
||||
from .variant import REGISTRY, register, ParamSpec, Variant
|
||||
|
||||
+9
-10
@@ -1,17 +1,19 @@
|
||||
from dataclasses import dataclass, field, asdict
|
||||
from typing import Any
|
||||
from typing import Any, Literal
|
||||
import torch
|
||||
|
||||
Role = Literal["reader", "writer", "inner"]
|
||||
|
||||
|
||||
@dataclass
|
||||
class LoraLiteConfig:
|
||||
variant: str = "lora"
|
||||
r: int = 8
|
||||
alpha: float = 16.0
|
||||
alpha: float | int = 16.0
|
||||
dtype: torch.dtype = torch.bfloat16
|
||||
|
||||
# targeting
|
||||
target_roles: tuple[str, ...] = ("reader", "writer")
|
||||
target_roles: tuple[Role, ...] = ("reader", "writer")
|
||||
target_names: tuple[str, ...] = ()
|
||||
exclude_names: tuple[str, ...] = ("lm_head", "embed_tokens")
|
||||
layers: tuple[int, ...] | None = None
|
||||
@@ -26,12 +28,9 @@ class LoraLiteConfig:
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, d: dict) -> "LoraLiteConfig":
|
||||
# to_dict always serializes dtype as str; torch.save preserves tuples.
|
||||
# If you build the dict by hand, pass the right types -- fail loud otherwise.
|
||||
d = dict(d)
|
||||
if isinstance(d.get("dtype"), str):
|
||||
d["dtype"] = getattr(torch, d["dtype"])
|
||||
if isinstance(d.get("layers"), list):
|
||||
d["layers"] = tuple(d["layers"])
|
||||
for k in ("target_roles", "target_names", "exclude_names"):
|
||||
if isinstance(d.get(k), list):
|
||||
d[k] = tuple(d[k])
|
||||
d["dtype"] = getattr(torch, d["dtype"])
|
||||
return cls(**d)
|
||||
|
||||
|
||||
@@ -34,12 +34,12 @@ WHICH BASIS IS ROTATED:
|
||||
|
||||
REQUIRES even rank divisible by `block_size` (default 4). r=8, bs=4 -> 2 blocks.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import math
|
||||
|
||||
import torch
|
||||
from einops import einsum
|
||||
from torch import nn
|
||||
from jaxtyping import Float
|
||||
from torch import nn, Tensor as T
|
||||
|
||||
from ..variant import register, ParamSpec
|
||||
|
||||
@@ -96,7 +96,7 @@ class AntiPaSTO:
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def init(layer: nn.Linear, cfg) -> None:
|
||||
def init(layer: nn.Module, cfg) -> None:
|
||||
if type(layer) is not nn.Linear:
|
||||
raise TypeError(
|
||||
"AntiPaSTO mutates layer.weight into W_res (like PiSSA), so v1 "
|
||||
@@ -116,7 +116,11 @@ class AntiPaSTO:
|
||||
layer.weight.data.copy_(W_res)
|
||||
|
||||
@staticmethod
|
||||
def forward(layer: nn.Linear, x, y):
|
||||
def forward(
|
||||
layer: nn.Module,
|
||||
x: Float[T, '*B i'],
|
||||
y: Float[T, '*B o'],
|
||||
) -> Float[T, '*B o']:
|
||||
cfg = layer._lora_cfg
|
||||
bs = int(cfg.variant_kwargs.get("block_size", 4))
|
||||
max_angle = float(cfg.variant_kwargs.get("max_rotation_angle", 0.5))
|
||||
|
||||
@@ -33,7 +33,8 @@ Reference implementations:
|
||||
"""
|
||||
import torch
|
||||
from einops import einsum
|
||||
from torch import nn
|
||||
from jaxtyping import Float
|
||||
from torch import nn, Tensor as T
|
||||
|
||||
from ..variant import register, ParamSpec
|
||||
|
||||
@@ -57,7 +58,7 @@ class DeLoRA:
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def init(layer: nn.Linear, cfg) -> None:
|
||||
def init(layer: nn.Module, cfg) -> None:
|
||||
# Reading layer.weight only works for plain Linear; for bnb layers this
|
||||
# dequantizes via .float() round-trip if available, or fails cleanly.
|
||||
with torch.no_grad():
|
||||
@@ -67,7 +68,11 @@ class DeLoRA:
|
||||
return
|
||||
|
||||
@staticmethod
|
||||
def forward(layer: nn.Linear, x, y):
|
||||
def forward(
|
||||
layer: nn.Module,
|
||||
x: Float[T, '*B i'],
|
||||
y: Float[T, '*B o'],
|
||||
) -> Float[T, '*B o']:
|
||||
cfg = layer._lora_cfg
|
||||
A = layer.lora_A # (r, d_in)
|
||||
B = layer.lora_B # (d_out, r)
|
||||
|
||||
@@ -19,9 +19,9 @@ Reference implementations (for review/cross-check):
|
||||
(offline: docs/refs/peft_lora_dora.py)
|
||||
"""
|
||||
import torch
|
||||
import torch.nn.functional as F
|
||||
from einops import einsum
|
||||
from torch import nn
|
||||
from jaxtyping import Float
|
||||
from torch import nn, Tensor as T
|
||||
|
||||
from ..variant import register, ParamSpec
|
||||
|
||||
@@ -40,7 +40,7 @@ class DoRA:
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def init(layer: nn.Linear, cfg) -> None:
|
||||
def init(layer: nn.Module, cfg) -> None:
|
||||
if type(layer) is not nn.Linear:
|
||||
raise TypeError(
|
||||
"DoRA needs ||W||_c, so v1 only supports plain nn.Linear. "
|
||||
@@ -52,7 +52,11 @@ class DoRA:
|
||||
layer.lora_m.data.copy_(col_norm)
|
||||
|
||||
@staticmethod
|
||||
def forward(layer: nn.Linear, x, y):
|
||||
def forward(
|
||||
layer: nn.Module,
|
||||
x: Float[T, '*B i'],
|
||||
y: Float[T, '*B o'],
|
||||
) -> Float[T, '*B o']:
|
||||
cfg = layer._lora_cfg
|
||||
scale = cfg.alpha / cfg.r
|
||||
# V = W + scale * B @ A
|
||||
|
||||
@@ -31,14 +31,17 @@ References:
|
||||
https://github.com/huggingface/peft/blob/main/examples/eva_finetuning/eva_finetuning.py
|
||||
(offline: docs/refs/peft_eva_finetuning.py)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import torch
|
||||
from einops import einsum
|
||||
from torch import nn
|
||||
from jaxtyping import Float
|
||||
from torch import nn, Tensor as T
|
||||
from typing import Iterable
|
||||
|
||||
from ..variant import register, ParamSpec
|
||||
|
||||
CalibrationBatch = dict | tuple | list | T
|
||||
CalibrationData = Iterable[CalibrationBatch]
|
||||
|
||||
|
||||
@register
|
||||
class EVA:
|
||||
@@ -55,12 +58,12 @@ class EVA:
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def init(layer: nn.Linear, cfg) -> None:
|
||||
def init(layer: nn.Module, cfg) -> None:
|
||||
# No-op; group_init does the data-driven SVD across all targets at once.
|
||||
return
|
||||
|
||||
@staticmethod
|
||||
def group_init(model: nn.Module, targets, cfg, calibration_data) -> None:
|
||||
def group_init(model: nn.Module, targets, cfg, calibration_data: CalibrationData | None) -> None:
|
||||
# adapter.load() passes _skip_group_init=True so this is only called on
|
||||
# the live attach path where calibration_data is required.
|
||||
if calibration_data is None:
|
||||
@@ -72,7 +75,7 @@ class EVA:
|
||||
)
|
||||
# Collect input activations per target via forward hooks.
|
||||
layers = {name: layer for name, layer, _ in targets}
|
||||
captured: dict[str, list[torch.Tensor]] = {n: [] for n in layers}
|
||||
captured: dict[str, list[T]] = {n: [] for n in layers}
|
||||
|
||||
def make_hook(name):
|
||||
def _h(module, args, kwargs):
|
||||
@@ -115,7 +118,11 @@ class EVA:
|
||||
layer.lora_A.copy_(A)
|
||||
|
||||
@staticmethod
|
||||
def forward(layer: nn.Linear, x, y):
|
||||
def forward(
|
||||
layer: nn.Module,
|
||||
x: Float[T, '*B i'],
|
||||
y: Float[T, '*B o'],
|
||||
) -> Float[T, '*B o']:
|
||||
cfg = layer._lora_cfg
|
||||
scale = cfg.alpha / cfg.r
|
||||
h = einsum(x, layer.lora_A, "... i, r i -> ... r")
|
||||
|
||||
@@ -30,7 +30,8 @@ Reference implementations (for review/cross-check):
|
||||
"""
|
||||
import torch
|
||||
from einops import einsum
|
||||
from torch import nn
|
||||
from jaxtyping import Float
|
||||
from torch import nn, Tensor as T
|
||||
|
||||
from ..variant import register, ParamSpec
|
||||
|
||||
@@ -53,7 +54,7 @@ class HRA:
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def init(layer: nn.Linear, cfg) -> None:
|
||||
def init(layer: nn.Module, cfg) -> None:
|
||||
# Symmetric init per peft (docs/refs/peft_hra_layer.py:101-108):
|
||||
# half = kaiming(r//2, d_in); U = repeat_interleave(half, 2, dim=0)
|
||||
# Adjacent pairs (H_2k H_2k+1) cancel since H^2 = I, so R = I exactly,
|
||||
@@ -66,7 +67,10 @@ class HRA:
|
||||
return
|
||||
|
||||
@staticmethod
|
||||
def forward_input(layer: nn.Linear, x: torch.Tensor) -> torch.Tensor:
|
||||
def forward_input(
|
||||
layer: nn.Module,
|
||||
x: Float[T, '*B i'],
|
||||
) -> Float[T, '*B i']:
|
||||
"""Apply Rx where R = prod_i H_i, H_i = I - 2 u_i u_i^T / ||u_i||^2."""
|
||||
U = layer.lora_U # (r, d_in)
|
||||
Rx = x
|
||||
|
||||
@@ -25,7 +25,8 @@ Reference implementation:
|
||||
https://github.com/huggingface/peft/blob/main/src/peft/tuners/ia3/layer.py
|
||||
"""
|
||||
import torch
|
||||
from torch import nn
|
||||
from jaxtyping import Float
|
||||
from torch import nn, Tensor as T
|
||||
|
||||
from ..variant import register, ParamSpec
|
||||
|
||||
@@ -39,11 +40,15 @@ class IA3:
|
||||
return {"lora_g": ParamSpec((d_out,), init="ones", trainable=True)}
|
||||
|
||||
@staticmethod
|
||||
def init(layer: nn.Linear, cfg) -> None:
|
||||
def init(layer: nn.Module, cfg) -> None:
|
||||
return
|
||||
|
||||
@staticmethod
|
||||
def forward(layer: nn.Linear, x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:
|
||||
def forward(
|
||||
layer: nn.Module,
|
||||
x: Float[T, '*B i'],
|
||||
y: Float[T, '*B o'],
|
||||
) -> Float[T, '*B o']:
|
||||
return y * layer.lora_g
|
||||
|
||||
|
||||
@@ -56,9 +61,12 @@ class IA3FF:
|
||||
return {"lora_g": ParamSpec((d_in,), init="ones", trainable=True)}
|
||||
|
||||
@staticmethod
|
||||
def init(layer: nn.Linear, cfg) -> None:
|
||||
def init(layer: nn.Module, cfg) -> None:
|
||||
return
|
||||
|
||||
@staticmethod
|
||||
def forward_input(layer: nn.Linear, x: torch.Tensor) -> torch.Tensor:
|
||||
def forward_input(
|
||||
layer: nn.Module,
|
||||
x: Float[T, '*B i'],
|
||||
) -> Float[T, '*B i']:
|
||||
return x * layer.lora_g
|
||||
@@ -10,7 +10,8 @@ Reference implementations (for review/cross-check):
|
||||
(see docs/refs/peft_lora_layer.py for offline copy)
|
||||
"""
|
||||
from einops import einsum
|
||||
from torch import nn
|
||||
from jaxtyping import Float
|
||||
from torch import nn, Tensor as T
|
||||
import torch
|
||||
|
||||
from ..variant import register, ParamSpec
|
||||
@@ -28,12 +29,16 @@ class LoRA:
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def init(layer: nn.Linear, cfg) -> None:
|
||||
def init(layer: nn.Module, cfg) -> None:
|
||||
# B is zeros => delta=0 at t=0; identity invariant holds.
|
||||
return
|
||||
|
||||
@staticmethod
|
||||
def forward(layer: nn.Linear, x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:
|
||||
def forward(
|
||||
layer: nn.Module,
|
||||
x: Float[T, '*B i'],
|
||||
y: Float[T, '*B o'],
|
||||
) -> Float[T, '*B o']:
|
||||
cfg = layer._lora_cfg
|
||||
scale = cfg.alpha / cfg.r
|
||||
h = einsum(x, layer.lora_A, "... i, r i -> ... r")
|
||||
|
||||
@@ -22,7 +22,8 @@ Reference implementations (for review/cross-check):
|
||||
"""
|
||||
import torch
|
||||
from einops import einsum
|
||||
from torch import nn
|
||||
from jaxtyping import Float
|
||||
from torch import nn, Tensor as T
|
||||
|
||||
from ..variant import register, ParamSpec
|
||||
|
||||
@@ -39,7 +40,7 @@ class PiSSA:
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def init(layer: nn.Linear, cfg) -> None:
|
||||
def init(layer: nn.Module, cfg) -> None:
|
||||
if type(layer) is not nn.Linear:
|
||||
raise TypeError(
|
||||
"PiSSA mutates layer.weight into W_res, so v1 only supports plain nn.Linear. "
|
||||
@@ -63,7 +64,11 @@ class PiSSA:
|
||||
layer.weight.data.copy_((W - scale * BA).to(layer.weight.dtype))
|
||||
|
||||
@staticmethod
|
||||
def forward(layer: nn.Linear, x, y):
|
||||
def forward(
|
||||
layer: nn.Module,
|
||||
x: Float[T, '*B i'],
|
||||
y: Float[T, '*B o'],
|
||||
) -> Float[T, '*B o']:
|
||||
cfg = layer._lora_cfg
|
||||
scale = cfg.alpha / cfg.r
|
||||
h = einsum(x, layer.lora_A, "... i, r i -> ... r")
|
||||
|
||||
@@ -7,7 +7,7 @@ resolution-markers = [
|
||||
]
|
||||
|
||||
[options]
|
||||
exclude-newer = "2026-04-21T09:27:46.246831625Z"
|
||||
exclude-newer = "2026-04-21T11:53:16.039887908Z"
|
||||
exclude-newer-span = "P5D"
|
||||
|
||||
[[package]]
|
||||
@@ -61,6 +61,15 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b9/fa/123043af240e49752f1c4bd24da5053b6bd00cad78c2be53c0d1e8b975bc/backports.tarfile-1.2.0-py3-none-any.whl", hash = "sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34", size = 30181, upload-time = "2024-05-28T17:01:53.112Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "beartype"
|
||||
version = "0.22.9"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c7/94/1009e248bbfbab11397abca7193bea6626806be9a327d399810d523a07cb/beartype-0.22.9.tar.gz", hash = "sha256:8f82b54aa723a2848a56008d18875f91c1db02c32ef6a62319a002e3e25a975f", size = 1608866, upload-time = "2025-12-13T06:50:30.72Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/71/cc/18245721fa7747065ab478316c7fea7c74777d07f37ae60db2e84f8172e8/beartype-0.22.9-py3-none-any.whl", hash = "sha256:d16c9bbc61ea14637596c5f6fbff2ee99cbe3573e46a716401734ef50c3060c2", size = 1333658, upload-time = "2025-12-13T06:50:28.266Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bitsandbytes"
|
||||
version = "0.49.2"
|
||||
@@ -585,6 +594,36 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/fd/c4/813bb09f0985cb21e959f21f2464169eca882656849adf727ac7bb7e1767/jaraco_functools-4.4.0-py3-none-any.whl", hash = "sha256:9eec1e36f45c818d9bf307c8948eb03b2b56cd44087b3cdc989abca1f20b9176", size = 10481, upload-time = "2025-12-21T09:29:42.27Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jaxtyping"
|
||||
version = "0.3.7"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
resolution-markers = [
|
||||
"python_full_version < '3.11'",
|
||||
]
|
||||
dependencies = [
|
||||
{ name = "wadler-lindig", marker = "python_full_version < '3.11'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/38/40/a2ea3ce0e3e5f540eb970de7792c90fa58fef1b27d34c83f9fa94fea4729/jaxtyping-0.3.7.tar.gz", hash = "sha256:3bd7d9beb7d3cb01a89f93f90581c6f4fff3e5c5dc3c9307e8f8687a040d10c4", size = 45721, upload-time = "2026-01-30T14:18:47.409Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/78/42/caf65e9a0576a3abadc537e2f831701ba9081f21317fb3be87d64451587a/jaxtyping-0.3.7-py3-none-any.whl", hash = "sha256:303ab8599edf412eeb40bf06c863e3168fa186cf0e7334703fa741ddd7046e66", size = 56101, upload-time = "2026-01-30T14:18:45.954Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jaxtyping"
|
||||
version = "0.3.9"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
resolution-markers = [
|
||||
"python_full_version >= '3.11'",
|
||||
]
|
||||
dependencies = [
|
||||
{ name = "wadler-lindig", marker = "python_full_version >= '3.11'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c2/be/00294e369938937e31b094437d5ea040e4fd1a20b998ebe572c4a1dcfa68/jaxtyping-0.3.9.tar.gz", hash = "sha256:f8c02d1b623d5f1b6665d4f3ddaec675d70004f16a792102c2fc51264190951d", size = 45857, upload-time = "2026-02-16T10:35:13.263Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/94/05/3e39d416fb92b2738a76e8265e6bfc5d10542f90a7c32ad1eb831eea3fa3/jaxtyping-0.3.9-py3-none-any.whl", hash = "sha256:a00557a9d616eff157491f06ed2e21ed94886fad3832399273eb912b345da378", size = 56274, upload-time = "2026-02-16T10:35:11.795Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jeepney"
|
||||
version = "0.9.0"
|
||||
@@ -630,6 +669,8 @@ version = "0.0.1"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "einops" },
|
||||
{ name = "jaxtyping", version = "0.3.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
|
||||
{ name = "jaxtyping", version = "0.3.9", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
|
||||
{ name = "torch" },
|
||||
]
|
||||
|
||||
@@ -646,6 +687,7 @@ hf-test = [
|
||||
{ name = "transformers" },
|
||||
]
|
||||
test = [
|
||||
{ name = "beartype" },
|
||||
{ name = "pytest" },
|
||||
{ name = "tabulate" },
|
||||
]
|
||||
@@ -653,8 +695,10 @@ test = [
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "accelerate", marker = "extra == 'hf-test'", specifier = ">=1.6" },
|
||||
{ name = "beartype", marker = "extra == 'test'", specifier = ">=0.18" },
|
||||
{ name = "bitsandbytes", marker = "extra == 'bnb-test'", specifier = ">=0.46" },
|
||||
{ name = "einops", specifier = ">=0.7" },
|
||||
{ name = "jaxtyping", specifier = ">=0.2.34" },
|
||||
{ name = "pytest", marker = "extra == 'test'" },
|
||||
{ name = "safetensors", marker = "extra == 'hf-test'", specifier = ">=0.5" },
|
||||
{ name = "tabulate", marker = "extra == 'test'" },
|
||||
@@ -1802,6 +1846,15 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wadler-lindig"
|
||||
version = "0.1.7"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/1e/67/cbae4bf7683a64755c2c1778c418fea96d00e34395bb91743f08bd951571/wadler_lindig-0.1.7.tar.gz", hash = "sha256:81d14d3fe77d441acf3ebd7f4aefac20c74128bf460e84b512806dccf7b2cd55", size = 15842, upload-time = "2025-06-18T07:00:42.843Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/8d/96/04e7b441807b26b794da5b11e59ed7f83b2cf8af202bd7eba8ad2fa6046e/wadler_lindig-0.1.7-py3-none-any.whl", hash = "sha256:e3ec83835570fd0a9509f969162aeb9c65618f998b1f42918cfc8d45122fe953", size = 20516, upload-time = "2025-06-18T07:00:41.684Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zipp"
|
||||
version = "3.23.1"
|
||||
|
||||
Reference in New Issue
Block a user