 Phil Wang
|
d26acbcae6
|
more skip connections, as in guided diffusion
|
2022-06-27 13:23:32 -07:00 |
|
|
Phil Wang
|
9939a48139
|
make sure all versions of torch supported
|
2022-06-23 12:28:09 -07:00 |
|
|
Phil Wang
|
75ea49a7ef
|
pass parameter for Trainer to EMA properly
|
2022-06-21 07:37:43 -07:00 |
|
|
Phil Wang
|
8c3609a6e3
|
move EMA logic out of the repository for clarity
|
2022-06-20 13:17:51 -07:00 |
|
|
Phil Wang
|
1586d1a8a0
|
just pluck the image size off the gaussian diffusion class
|
2022-06-17 13:54:41 -07:00 |
|
|
Phil Wang
|
b4fb8804d2
|
conditioning on final resnet block
|
2022-06-17 10:38:17 -07:00 |
|
|
Phil Wang
|
9fd05f1b1f
|
switch to learned sinsuoidal pos emb for the continuous case
|
2022-06-17 09:24:51 -07:00 |
|
|
Phil Wang
|
ec2397f0ba
|
add one more residual
|
2022-06-16 11:08:42 -07:00 |
|
|
Phil Wang
|
844e557dfb
|
fix a missing residual needed at the top most resolution in the unet
|
2022-06-15 19:10:05 -07:00 |
|
|
Phil Wang
|
8b30be8042
|
add p2 loss reweighting for default ddpm as an option
|
2022-06-14 10:49:13 -07:00 |
|
|
Phil Wang
|
ecc6f30901
|
for https://github.com/lucidrains/denoising-diffusion-pytorch/issues/36
|
2022-06-11 10:51:13 -07:00 |
|
|
Phil Wang
|
f900f40f14
|
allow for turning off horizontal flip augmentation
|
2022-06-09 20:59:59 -07:00 |
|
|
Phil Wang
|
479f60c178
|
add p2 loss weighting to SNR version of denoising diffusion, brought up by @Mut1nyJD, paper is https://arxiv.org/abs/2204.00227
|
2022-06-09 08:25:05 -07:00 |
|
|
Phil Wang
|
96bb2ff310
|
alpha cosine noise schedule is now working for continuous time gaussian diffusion
|
2022-06-08 23:07:01 -07:00 |
|
|
Phil Wang
|
582bfe275b
|
successfully did some basic math and clipped the predicted x0 intermediate for the continuous time case
|
2022-06-08 17:59:41 -07:00 |
|
|
Phil Wang
|
4284c8840d
|
clipping for continuous time diffusion not working
|
2022-06-08 16:26:18 -07:00 |
|
|
Phil Wang
|
c44d3ea01d
|
learned noise schedule seems to be working, allow for one to make the monotonic net learn a bit more slowly than the unet
|
2022-06-08 12:34:07 -07:00 |
|
|
Phil Wang
|
c4991f576f
|
allow for configuring the hidden dimension of the monotonic mlp parameterizing the noise schedule
|
2022-06-08 11:18:54 -07:00 |
|
|
Phil Wang
|
a19331aa59
|
fix learned noise schedule
|
2022-06-08 10:19:02 -07:00 |
|
|
Phil Wang
|
94eabaca1a
|
complete learned noise schedule for variational ddpm paper, still need to finish cosine alpha schedule in log(snr) form
|
2022-06-08 09:47:09 -07:00 |
|
|
Phil Wang
|
eaf9d9fdc4
|
unet needs to be conditioned on log(snr) in p_mean_variance for continuous time gaussian diffusion
|
2022-06-08 00:41:41 -07:00 |
|
|
Phil Wang
|
3bf5e768c2
|
use a non-sinusoidal embedded condition for continuous time gaussian diffusion conditioned on log(snr)
|
2022-06-07 21:15:27 -07:00 |
|
|
Phil Wang
|
532178a6a3
|
assume when sampling all batch samples are at the same time, and do not noise for the last time step
|
2022-06-07 16:12:44 -07:00 |
|
|
Phil Wang
|
3bbb6ebf16
|
get working version of gaussian diffusion with continuous time (only beta linear schedule for now, but will eventually contain alpha cosine schedule as well as parameterized, learned monotonic MLP)
|
2022-06-07 15:59:29 -07:00 |
|
|
Phil Wang
|
a291da5098
|
bring back linear noise schedule, but default to cosine
|
2022-05-27 19:13:05 -07:00 |
|
|
Phil Wang
|
e5a18bb25c
|
switch over to film like conditioning, used by both openai and google at this point
|
2022-05-24 23:47:34 -07:00 |
|
|
Phil Wang
|
fc8e4547aa
|
higher default learning rate
|
2022-05-16 13:39:55 -07:00 |
|
|
Phil Wang
|
cae9f4a71f
|
whoops
|
2022-05-14 13:59:21 -07:00 |
|
|
Phil Wang
|
91f03fb88b
|
optimize for simplicity and clarity - researcher does not need to worry about normalizing and unnormalizing now
|
2022-05-14 11:38:43 -07:00 |
|
|
Phil Wang
|
60128257c5
|
use tqdm pbar during training
|
2022-05-13 20:25:49 -07:00 |
|
|
Phil Wang
|
cf6db71985
|
add gaussian diffusion where model predicts both noise and x_start, with a learned weighting between the two (experimental)
|
2022-05-13 13:56:32 -07:00 |
|
|
Phil Wang
|
84ebb9ad13
|
offer predict_x0 objective
|
2022-05-13 10:15:54 -07:00 |
|
|
Phil Wang
|
caa5af170d
|
final cleanup
|
2022-05-12 13:58:16 -07:00 |
|
|
Phil Wang
|
e0f26677d6
|
make sure predicted mean is actually detached for all of the kl loss calculations
|
2022-05-12 11:12:34 -07:00 |
|
|
Phil Wang
|
e147839d74
|
make sure to clip when sampling from gaussian diffusion with learned variance
|
2022-05-12 10:08:53 -07:00 |
|
|
Phil Wang
|
62e8490385
|
complete the gaussian diffusion with hybrid loss (learned variance) as in the improved ddpm paper
|
2022-05-12 08:54:47 -07:00 |
|
|
Phil Wang
|
402b7c26df
|
calculate noise schedule with float64 for numerical accuracy
|
2022-05-10 15:23:34 -07:00 |
|
|
Phil Wang
|
989f0fcb8e
|
remove convnext blocks, they do not work well, validated in video diffusion repository
|
2022-05-05 07:03:55 -07:00 |
|
|
Phil Wang
|
84731bb03d
|
groupnorm groups should be actually configurable
|
2022-05-04 10:38:29 -07:00 |
|
|
Phil Wang
|
c6ecca555b
|
allow for configuring expansion factor in convnext
|
2022-05-04 10:33:23 -07:00 |
|
|
Phil Wang
|
1f5c233072
|
bring back resnet blocks, make convnext blocks an experimental option
|
2022-05-04 10:30:09 -07:00 |
|
|
Phil Wang
|
e274fb305a
|
give an initial conv
|
2022-05-01 08:49:38 -07:00 |
|
|
Phil Wang
|
f39b3b1d3f
|
make sure time embedding dimension is kept at 4 x dimension (thanks @borisdayma)
|
2022-04-29 14:55:12 -07:00 |
|
|
Phil Wang
|
782c904d3b
|
fix cosine beta schedule, thanks to @Zhengxinyang
|
2022-04-19 20:51:50 -07:00 |
|
|
Phil Wang
|
71953ebd22
|
fix bug, thanks to @jihoonerd
|
2022-04-15 06:37:31 -07:00 |
|
|
Phil Wang
|
0b8cdb4c8b
|
remove outdated apex in favor of native pytorch AMP
|
2022-04-13 08:59:18 -07:00 |
|
|
Phil Wang
|
bd1e3b676e
|
get rid of numpy
|
2022-04-12 11:58:46 -07:00 |
|
|
Phil Wang
|
f4615599bc
|
use full attention at the center of the unet
|
2022-04-04 09:03:41 -07:00 |
|
|
Phil Wang
|
eb6e1b508e
|
greater kernel size in convnext blocks
|
2022-01-31 17:13:27 -08:00 |
|
|
Phil Wang
|
91cff45939
|
replace resnets with convnext blocks
|
2022-01-25 09:02:45 -08:00 |
|