UCL COMP0084 Information Retrieval and Data Mining (2023/24)

This repository contains the first coursework I completed for my MSc module COMP0084 Information Retrieval and Data Mining.

The two coursework implemented a retrieval system using different retrieval algorithms.

Tasks for coursework 1

1. Investigate the application of Zipf's law on our corpus.
2. Compute the inverted index of our corpus.
3. Implement the retrieval algorithm based on the cosine similarity between the query and passage TFIDF vectors
4. Implement the BM25 algorithm.
5. Implement and analyse the following query likelihood language model
   • Laplace smoothing model
   • Lidstone correction model
   • Dirichlet smoothing model

Tasks for coursework 2

1. Implement ranking evaluation metrics including mNDCG and mAP.
2. Implement a logistic regression model from scratch.
3. Employ the LambdaMART model for ranking using XGBoost.
4. Design and implement a neural network for ranking.