list of efficient attention modules
-
Updated
Aug 23, 2021 - Python
list of efficient attention modules
Master thesis with code investigating methods for incorporating long-context reasoning in low-resource languages, without the need to pre-train from scratch. We investigated if multilingual models could inherit these properties by making it an Efficient Transformer (s.a. the Longformer architecture).
Abstractive and Extractive Text summarization using Transformers.
[제 13회 투빅스 컨퍼런스] YoYAK - Yes or Yes, Attention with gap-sentence for Korean long sequence
Convert pretrained RoBerta models to various long-document transformer models
Industrial Text Scoring using Multimodal Deep Natural Language Processing 🚀 | Code for IEA AIE 2022 paper
A hyperpartisan news article classification system using BERT-based techniques. The goal was to leverage state-of-the-art transformer models like BERT, ROBERTa, and Longformer to accurately classify news articles as hyperpartisan or non-hyperpartisan.
Project as part of COMP34812: Natural Language Understanding
using transformers to do text classification.
A WebApp to summarize research papers using HuggingFace Transformers.
A summarization website that can generate summaries from either YouTube videos or PDF files.
Focus - Understanding contextual retrievability.
This project was developed for a Kaggle competition focused on detecting Personally Identifiable Information (PII) in student writing. The primary objective was to build a robust model capable of identifying PII with high recall. The DeBERTa v3 transformer model was chosen for this task after comparing its performance with other transformer models.
Fine-tuned Longformer for Summarization of Machine Learning Articles
Factuality check of the SemRep Predications
An attempt of creating a model and pipeline for retrieving italian legal documents given a prompt from the user.
Longformer Encoder Decoder model for the legal domain, trained for long document abstractive summarization task.
Add a description, image, and links to the longformer topic page so that developers can more easily learn about it.
To associate your repository with the longformer topic, visit your repo's landing page and select "manage topics."