This release is the tokenization/sentence segmentation part of the code we used for the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies.
The system is described in:
Miryam de Lhoneux, Yan Shao, Ali Basirat, Eliyahu Kiperwasser, Sara Stymne, Yoav Goldberg, and Joakim Nivre. 2017. From raw text to Universal Dependencies - look, no tags! In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics.