wassname/keras-language-modeling
1
0
Fork 0
mirror of https://github.com/wassname/keras-language-modeling.git synced 2026-09-10 12:15:18 +08:00
Code Issues Packages Projects Releases Wiki Activity

updated README

Browse Source
This commit is contained in:
codekansas
2016-04-30 02:48:32 -04:00
parent 51223f2ec2
commit eb1564f856
1 changed files with 6 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+6 -5
View File
@@ -1,16 +1,17 @@
# keras-language-modeling
Some code for doing language modeling with Keras, in particular for question-answering tasks. The original paper, which provided inspiriation for this project, can be found [here](http://arxiv.org/pdf/1511.04108.pdf).
Some code for doing language modeling with Keras, in particular for question-answering tasks. I wrote a very long blog post that explains how a lot of this works, which can be found [here](http://benjaminbolte.com/blog/2016/keras-language-modeling.html).
### Stuff that might be of interest
Most of this stuff is just a project for a class of mine, but in the process of writing it I wrote a few files that might be of interest to other people.
- `attention_lstm.py`: Attentional LSTM, based on the referenced paper, and others. One application used it for [image captioning](http://arxiv.org/pdf/1502.03044.pdf).
- `keras_attention_model.py`: Implementation of an attentional LSTM architecture for question-answer matching, which closely mimics one of the architectures described in the referenced paper. In particular, the model uses the attentional LSTM implementation in `attention_lstm.py` to generate a sentence embedding for each answer given a particular question, and fits the answer embedding to have a high cosine similarity with the question embedding.
- `attention_lstm.py`: Attentional LSTM, based on one of the papers referenced in the blog post and others. One application used it for [image captioning](http://arxiv.org/pdf/1502.03044.pdf). It is initialized with an attention vector which provides the attention component for the neural network.
- `insurance_qa_eval.py`: Evaluation framework for the InsuranceQA dataset. To get this working, clone the [data repository](https://github.com/codekansas/insurance_qa_python) and change the `data_path` to the cloned repository. Changing `config` will adjust how the model is trained.
- `keras-language-model.py`: The `LanguageModel` class uses the `config` settings to generate a training model and a testing model. The model can be trained by passing a question vector, a ground truth answer vector, and a bad answer vector to `fit`. Then `predict` calculates the similarity between a question and answer. Override the `build` method with whatever language model you want to get a trainable model. Examples are provided at the bottom, including the `EmbeddingModel`, `ConvolutionModel`, and `RecurrentModel`.
- `word_embeddings.py`: A Word2Vec layer that uses the embeddings generated by Gensim's word2vec model to provide vectors in place of the Keras `Embedding` layer, which could help improve convergence, since fewer parameters need to be learned. Note that this requires generating a separate file with the word2vec weights, so it doesn't fit in very nicely with the Keras architecture.
### Data
- L6 from [Yahoo Webscope](http://webscope.sandbox.yahoo.com/)
- [InsuranceQA data](https://github.com/shuzi/insuranceQA)
- [Pythonic version](https://github.com/codekansas/insurance_qa_python)