A NLP based summarizer which can summarize news article. Using T5TokenizerFast I have trained a custom model on t5-base architecture and the dataset was collected from kaggle.
To get started with this project, follow these steps:
Clone the repository:
git clone https://github.com/marufc36/news_article_summarizer_using_custom_model
pip3 install requirements.txt
Data-set is collected from Kaggle. The dataset consists of 4515 examples and contains Author_name, Headlines, Url of Article, Short text, Complete Article. I have only worked with Short text and Complete Article and other columns were dropped from the data-set.
Using t5-base pretrained model, T5TokenizerFast, pytorch lighting the model was trained to three epoch on the collected data-set.Short text was used as summary and complete article was used as text.
Colab Notebook
https://drive.google.com/file/d/1qbCqMXjv5aqyfkJ6ntQMfGwYoXXATqWM/view?usp=sharing
I have inferenced the using a gradio app. I have to load the best checkpoint using NewsSummaryModel. I have created model.py module which includes NewsSummaryModel.
The model was deployed to HuggingFace Spaces. Linkn is given below.
https://huggingface.co/spaces/mmchowdhury/News_Summary_With_Custom_Model