103 Commits
Author SHA1 Message Date
Phil Wang b4fb8804d2 conditioning on final resnet block v0.20.2 v0.20.1 2022-06-17 10:38:17 -07:00
Phil Wang 9fd05f1b1f switch to learned sinsuoidal pos emb for the continuous case v0.20.0 2022-06-17 09:24:51 -07:00
Phil Wang ec2397f0ba add one more residual v0.19.2 2022-06-16 11:08:42 -07:00
Phil Wang 844e557dfb fix a missing residual needed at the top most resolution in the unet v0.19.1 2022-06-15 19:10:05 -07:00
Phil Wang 8b30be8042 add p2 loss reweighting for default ddpm as an option v0.18.4 2022-06-14 10:49:13 -07:00
Phil Wang f2f3994b92 link to Letitia 2022-06-12 14:57:44 -07:00
Phil Wang 8ec4ea56a5 link to yannic 2022-06-12 14:56:55 -07:00
Phil Wang 99cf9b5b96 link to ai educator 2022-06-12 14:54:43 -07:00
Phil Wang ecc6f30901 for https://github.com/lucidrains/denoising-diffusion-pytorch/issues/36 v0.18.3 2022-06-11 10:51:13 -07:00
Phil Wang f900f40f14 allow for turning off horizontal flip augmentation v0.18.2 2022-06-09 20:59:59 -07:00
Phil Wang 479f60c178 add p2 loss weighting to SNR version of denoising diffusion, brought up by @Mut1nyJD, paper is https://arxiv.org/abs/2204.00227 v0.18.0 2022-06-09 08:25:05 -07:00
Phil Wang 96bb2ff310 alpha cosine noise schedule is now working for continuous time gaussian diffusion v0.17.7 2022-06-08 23:07:01 -07:00
Phil Wang 582bfe275b successfully did some basic math and clipped the predicted x0 intermediate for the continuous time case v0.17.6 2022-06-08 17:59:41 -07:00
Phil Wang 4284c8840d clipping for continuous time diffusion not working v0.17.4 2022-06-08 16:26:18 -07:00
Phil Wang d4ffa3fced link to annotated ddpm 2022-06-08 12:55:15 -07:00
Phil Wang c44d3ea01d learned noise schedule seems to be working, allow for one to make the monotonic net learn a bit more slowly than the unet v0.17.3 2022-06-08 12:34:07 -07:00
Phil Wang c4991f576f allow for configuring the hidden dimension of the monotonic mlp parameterizing the noise schedule v0.17.2 2022-06-08 11:18:54 -07:00
Phil Wang a19331aa59 fix learned noise schedule v0.17.1 2022-06-08 10:19:02 -07:00
Phil Wang 94eabaca1a complete learned noise schedule for variational ddpm paper, still need to finish cosine alpha schedule in log(snr) form v0.17.0 2022-06-08 09:47:09 -07:00
Phil Wang eaf9d9fdc4 unet needs to be conditioned on log(snr) in p_mean_variance for continuous time gaussian diffusion v0.16.7 2022-06-08 00:41:41 -07:00
Phil Wang 3bf5e768c2 use a non-sinusoidal embedded condition for continuous time gaussian diffusion conditioned on log(snr) v0.16.5 2022-06-07 21:15:27 -07:00
Phil Wang 532178a6a3 assume when sampling all batch samples are at the same time, and do not noise for the last time step v0.16.4 2022-06-07 16:12:44 -07:00
Phil Wang 3bbb6ebf16 get working version of gaussian diffusion with continuous time (only beta linear schedule for now, but will eventually contain alpha cosine schedule as well as parameterized, learned monotonic MLP) v0.16.3 2022-06-07 15:59:29 -07:00
Phil Wang 6b93fa48f6 fix comment 2022-06-06 17:30:46 -07:00
Phil Wang a291da5098 bring back linear noise schedule, but default to cosine v0.16.1 2022-05-27 19:13:05 -07:00
Phil Wang e5a18bb25c switch over to film like conditioning, used by both openai and google at this point v0.16.0 2022-05-24 23:47:34 -07:00
Phil Wang fc8e4547aa higher default learning rate 0.15.7 2022-05-16 13:39:55 -07:00
Phil Wang cae9f4a71f whoops 0.15.6 2022-05-14 13:59:21 -07:00
Phil Wang 91f03fb88b optimize for simplicity and clarity - researcher does not need to worry about normalizing and unnormalizing now 0.15.4 2022-05-14 11:38:43 -07:00
Phil Wang 60128257c5 use tqdm pbar during training 0.15.3 2022-05-13 20:25:49 -07:00
Phil Wang cf6db71985 add gaussian diffusion where model predicts both noise and x_start, with a learned weighting between the two (experimental) 0.15.2 2022-05-13 13:56:32 -07:00
Phil Wang 84ebb9ad13 offer predict_x0 objective 0.15.0 2022-05-13 10:15:54 -07:00
Phil Wang caa5af170d final cleanup 0.14.3 2022-05-12 13:58:16 -07:00
Phil Wang 55c658b967 cleanup unused 2022-05-12 11:52:06 -07:00
Phil Wang e0f26677d6 make sure predicted mean is actually detached for all of the kl loss calculations 0.14.2 2022-05-12 11:12:34 -07:00
Phil Wang e147839d74 make sure to clip when sampling from gaussian diffusion with learned variance 0.14.1 2022-05-12 10:08:53 -07:00
Phil Wang 62e8490385 complete the gaussian diffusion with hybrid loss (learned variance) as in the improved ddpm paper 0.14.0 2022-05-12 08:54:47 -07:00
Phil Wang d412d8816b first pass at ddpm with learned variance 2022-05-11 17:38:29 -07:00
Phil Wang 402b7c26df calculate noise schedule with float64 for numerical accuracy 0.12.1 2022-05-10 15:23:34 -07:00
Phil Wang 09613a40f3 cleanup 2022-05-07 05:47:21 -07:00
Phil Wang c6966ae95a Merge pull request #24 from kashif/patch-1
updated citation in README
2022-05-07 05:32:51 -07:00
Kashif Rasul 73591cf1ad updated citation in README 2022-05-07 11:23:45 +02:00
Phil Wang 989f0fcb8e remove convnext blocks, they do not work well, validated in video diffusion repository 0.12.0 2022-05-05 07:03:55 -07:00
Phil Wang 84731bb03d groupnorm groups should be actually configurable 0.11.2 2022-05-04 10:38:29 -07:00
Phil Wang c6ecca555b allow for configuring expansion factor in convnext 0.11.1 2022-05-04 10:33:23 -07:00
Phil Wang 1f5c233072 bring back resnet blocks, make convnext blocks an experimental option 0.11.0 2022-05-04 10:30:09 -07:00
Phil Wang de378158e5 readme 2022-05-01 13:16:06 -07:00
Phil Wang e274fb305a give an initial conv 0.10.1 2022-05-01 08:49:38 -07:00
Phil Wang f39b3b1d3f make sure time embedding dimension is kept at 4 x dimension (thanks @borisdayma) 0.10.0 2022-04-29 14:55:12 -07:00
Phil Wang 782c904d3b fix cosine beta schedule, thanks to @Zhengxinyang 0.9.2 2022-04-19 20:51:50 -07:00