Skip to content

[ACL 2022] JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection

Notifications You must be signed in to change notification settings

BinLiang-NLP/JointCL

 
 

Repository files navigation

Introduction

This repository is used in our paper:

JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection
Bin Liang, Qinlin Zhu, Xiang Li, Min Yang, Lin Gui, Yulan He, Ruifeng Xu*. Proceedings of ACL 2022

Please cite our paper and kindly give a star for this repository if you use this code.

Requirements

  • Python 3.6
  • PyTorch 1.6.0
  • faiss-gpu 1.7.1
  • transformers 2.5.1

Usage

Training

  • Train with command, optional arguments could be found in run_semeval.py & run_vast.py & run_wtwt.py

  • Run Semeval dataset: python ./run_semeval.py

  • Run VAST dataset: python ./run_vast.py

  • Run WTWT dataset: python ./run_wtwt.py

Reproduce & Peformance

  • Due to the small number of dataset samples (especially SEM16), the performance gap between different seeds will vary greatly, please tune the parameter of --seed for better performance.
  • We have provided checkpoints that are superior or equal to the performance reported in the paper.
  • Please Run python files in run_checkpoints, you can use the trained model for prediction, and the model can be downloaded from Google drives.
  • We also use 5 random seeds to run the code directly without any other tuning parameters. The performance is as follows:
Task Dataset Target Reported Checkpoint seed1 seed2 seed3 seed4 seed5 Mean Max Gap
Zero-shot VAST - 72.372.470.671.372.472.071.371.572.4+0.1
SEM16 DT 50.550.946.040.645.648.450.246.250.2-0.3
HC 54.856.4 50.7 56.4 45.7 55.9 51.3 52.0 56.4 +1.6
FM 53.8 54.2 50.9 51.1 49.1 49.8 49.4 50.1 51.1 -2.7
LA 49.5 55.5 54.8 54.3 55.5 51.3 47.1 52.6 55.5 +6
A 54.5 54.6 55.1 48.0 60.0 55.4 55.2 54.7 60.0 +5.5
CC 39.7 40.7 31.9 36.9 39.7 40.2 28.3 35.4 40.2 +0.5
WTWT CA 72.4 73.6 72.5 71.4 73.3 73.4 74.9 73.1 74.9 +2.5
CE 70.2 70.9 70.1 71.4 70.4 70.3 70.3 70.1 71.4 +1.2
AC 76.0 76.5 75.0 74.3 77.3 73.3 75.6 75.1 77.3 +1.3
AH 75.2 76.5 76.2 76.1 76.0 77.9 78.0 76.8 78.0 +2.8
Few-shot VAST - 71.5 71.6 71.6 71.9 68.4 66.1 69.5 69.5 71.9 +0.4
Cross-target SEM16 HC->DT 52.8 54.6 42.9 46.9 48.1 53.7 54.2 49.2 54.2 +1.4
DT->HC 54.3 55.4 52.1 55.8 54.6 47.8 38.6 49.8 55.8 +1.5
FM->LA 58.8 60.0 49.8 58.0 58.3 46.7 45.7 51.7 60.0 -0.5
LA->FM 54.5 54.8 45.8 41.8 54.1 36.2 47.9 45.2 54.1 -0.4

Credits

About

[ACL 2022] JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%