# AlignmentSearch

This project creates embeddings for every set of a few paragraphs from the source dataset, in order to do real-time semantic search and question answering on them.

The very barebones of the project is currently in testing.ipynb.

## TODO:
- Getting data:
    - Figure out the right format for the dataset
    - Get entirety of data
    - Searches for new posts/papers/etc and scrape them, runs once a day
- Semantic search:
    - Test out other techniques than just vector similarity (e.g. LSH-index, see Dense Retrieval methods (here)[https://medium.com/@aikho/deep-learning-in-information-retrieval-part-ii-dense-retrieval-1f9fecb47de9])
    - Test other embeddings models ((SimCSE)[https://github.com/princeton-nlp/SimCSE] possibly SOTA?)
- Question answering:
    - Test out other models prompts to see which is best
- Summarization:
    - Test out other models and prompts to see which is best (Forefront?)
- Info extraction from PDF:
    - Specifically mentioned by Anson. Look into methods by Mely.ai to extract tables from PDFs maybe?
    - Test various techniques to make it more performant
- Finetuning:
    - Finetune embeddings model
    - Finetune q&a model
    - Finetune summarization model
    - Finetune info extraction model
- Create website/other. Not sure what would be most helpful here
    - Find someone that can figure this out

Source dataset: Kirchner, J. H., Smith, L., Thibodeau, J., McDonnell, K., and Reynolds, L. "Understanding AI alignment research: A Systematic Analysis." arXiv preprint arXiv:2022.4338861 (2022).