PoeSC is a compositional and live-coding interface that uses plain text as a coding grammar to generate musical phrases. Currently english is supported.
It is developed with Python and Supercollider.
After text is input via Python, NLP analysis is performed and data is sent to SuperCollider via OSC where it is mapped and transformed into sound.
It is a "slow-coding" experience and most of the text/sound isomorphisms are product of my arbitrary/subjective taste. This is not an attempt to a meaningful sonification of language, rather a speculation towards a musically creative use of "natural" textual structures. An attempt to an esolang of sorts.
The development of this project was possible through the support of the Musikfonds "Neustart Kultur" Stipendienprogramm 2020/2021 scholarship.
PoeSC started as an endeavour to create musical meso-structures through live coding.
In my practice up to the beginning of the project I would struggle with the fact that my output would tend towards repetitive structures. I was interested in developing a method of generating a varied rhythmical narrative that could evolve easily into complex sonic constructions which could span numerous seconds.
I focused my research into Formal Grammars as a way of understanding basic parsing and articulate the problem programmatically. Out of this investigation, Curtis Roads' paper "Composing Grammars" (1978) (accessible here) turned out to be very influential, as it presented in a systematic way a "structural/linguistic" model of developing a composing ecosystem. In parallel, I discovered the current work of Allison Parrish in computational poetry and the use of programming languages in creating poems.
This fusion of perspectives established for me the main pivotal point of the project. I realized that common speech/text includes in its metrical structures the variation of rhythm and meter I was looking for and a potential for creative complexity through NLP.
All this, of course, would not have been possible without the latest developments in Natural Language Processing, so I'm indebted to the work of the people active in that field. The project is heavily based on the Python libraries spacy and prosodic, for NLP and metrical analysis.
A Model-View-Controller design pattern is loosely applied, with:
- the View being the text-input interface,
- the Controller being classes of OSCResponder and SimpleController in SuperCollider
- and the Model are dictionaries of sonic parameters under control
The user inserts text through a simple Tkinter window in Python. This is analysed:
- Metrically by the prosodic python extension.
- POS (Part Of Speech) analysis by spacy
- Word "score" calculation (which later affects pitch) based on Letter Frequency analysis
- OSC communication between Python and SuperCollider is implemented using pyliblo3
After the analysis, all data are transmitted to SuperCollider as an OSC Bundle.
Each new block of text entered by the user creates a Stanza player in Supercollider. The number of lines contained in each block of text create equal Phrase players. The role of the Stanza player is to include and sequence these Phrase players. There can be only 1 Stanza player for each text block and as many Phrase players as the lines of text entered.
As soon as the OSC messages are received in SuperCollider they are mapped to numerous parameters.
4 Servers are used in SuperCollider and whenever a new Phrase is initialized a Server is chosen according to its idleness. This is done in order to avoid audio dropouts from server overloading. In addition, each Phrase is initialized within a new Group and with its own FX. As soon as the Phrase is done playing, Group, FX and Pattern proxies are cleaned from memory.
The interface is functional with the following steps already established:
- Basic core and communication between Python and Supercollider is established, using pyliblo3
- Metrical analysis of text is fully developed, solely based on the python library prosodic
- NLP analysis ( using spacy ) is at a primary stage:
- Sentence-splitting established
- Punctuation detected and mapped
- Adjectives detected and mapped
- Each letter is assigned a score, based on Letter Frequncy analysis
- Basic GUI for text input using tkinter
- Stanza dictionary & player established
- Phrase dictionary & player established
- Completed OSC receivers and corresponding Controllers:
- Stanza trigger
- Stanza score
- Stanza mode
- Syllable length
- Syllable weight
- Syllable stress
- Syllable text
- Syllable score
- Phrase punctuation
- Phrase Adjectives
- Update the current README regarding:
- Dependencies
- How to use
- Detailed table of current rules of the program
- Develop in SuperCollider:
- Automatic initialization document
- Repetitions Model
- TempoGrid Modulation
- Pitch Root Modulation (mtranspose)
- Develop Sample Based SynthDefs
- Instrument selection Model
- Further POS mapping (nouns, verbs etc)
- Develop in Python:
- Further POS Models
- a better GUI implementation
Author: Thomas Chousos / Tadklimp
This is a documentation repository of my personal compositional process.
Please feel free to fork and analyze and I hope you'll find inspiration.
The development of this project was possible through the support of the Musikfonds "Neustart Kultur" Stipendienprogramm 2020/2021 scholarship.