We have created domain-adaptable rankers fine-tuned using knowledge distillation in order to re-rank the passages retrieved using BM25. We propose a novel difficulty prediction heuristic which dynamically determines the number of paragraphs to be fed to the reader by utilising the ranker scores and the remaining time. Finally, we use signals from reader, ranker as well as the retriever to determine the answerability of the question.