Commit Graph
15 Commits
Author SHA1 Message Date
aayush 03c4a2cd15 Fixed self attention, local tests passed 2019-02-15 02:10:18 +05:30
Aayush 8fe2891402 Make callable through bicls_head 2019-01-07 00:02:39 +05:30
Aayush d0c15472d6 Self attention for pooling linear classifier
This PR will introduce a `BiAttentionPoolingClassifier` as in [Attention is all you need](https://arxiv.org/abs/1706.03762) following the discussion with @sebastianruder in Teams.

I ran out of memory on my 1060 while testing the attention module, but was able to at least verify that it is functionally correct. Some changes might be required to ensure that the tensor passed to `self.layers` is of the right shape (but I'm not quite sure as of now).

I'll shift all the stuff to Collab for testing and see if it's any help.
2019-01-06 23:58:09 +05:30
Piotr Czapla 514a9e6b86 Fix BiLM training after update to newest fastai 2018-12-31 12:13:53 +01:00
Piotr Czapla ff30fb5642 Fastai upgrade 2018-12-27 14:57:15 +01:00
Piotr Czapla 2338618563 Make biclasifier head a hyperparameter. 2018-12-09 00:39:27 +01:00
Piotr Czapla 591393b96c Change AvgPooling to BiPooling as a default 2018-12-08 23:48:02 +01:00
Piotr Czapla 8da47324c2 Clean ups and fixes 2018-11-22 15:32:40 +01:00
Piotr Czapla 6e3ef21b1f Add Avg BiClassifier 2018-11-21 18:44:49 +01:00
Piotr Czapla 7f1f8efcc3 Working version of biclassfier 2018-11-19 12:58:58 +01:00
Piotr Czapla c821d2e783 first version of bi classifier 2018-11-19 09:59:08 +01:00
Piotr Czapla 33f9eb2cc7 Reuse RNNCore in implementation of BiLM, add accuracy 2018-11-14 21:16:16 +01:00
Piotr Czapla 5ba83b1d4e Working version of BILM - probably won't train well yet 2018-11-14 14:55:56 +01:00
Piotr Czapla a85800610b WIP Working Backward LM using our new LangaugeModelLoader 2018-11-14 13:22:03 +01:00
Sebastian cba86ab5d4 Added initial classes and changes for BiLM implementation 2018-11-13 17:46:28 +00:00