Javascript implementation of REPL + Web Audio API

[kylestetz]

I am loosely aware of the ability, via clojurescript, to get clojure running in native javascript. If it's possible for the parser to be ported to javascript I would be happy to build the website that surrounds it and plug in the web audio API so folks can aldajam in the browser.

Any idea what would be involved in porting the parser to JS?

[daveyarwood]

Whoops -- didn't see this issue before I made #73 :P

I'm definitely interested in doing what I'm proposing in #73, for the full-on Alda experience in the browser, but this would probably be a little easier.

As luck would have it, the Clojure parsing library that we're using, Instaparse, has a cljs port, so it would at least be very easy to port the actual grammar/parsing part to ClojureScript. Clojure and ClojureScript are very similar, so I think we could reuse a lot of the same code.

This is a really interesting idea... I might take a stab at this over the next few days, time permitting.

[kylestetz]

Awesome, let me know if I can provide any assistance! When it comes to matters of timing you'll have to rewrite some parts to use the dance of the clocks, which I would be more than happy to tackle with you.

[daveyarwood]

I'm no stranger to the dance of the clocks -- I've actually already written a ClojureScript library that is essentially a port of WAAClock, a scheduling library that uses a combination of setTimeout and the WebAudio API clock to get more precise scheduling.

I'll get cracking on this soon -- will keep you posted!

[kylestetz]

Heh. 👌

[daveyarwood]

I went ahead and took an initial stab at this: https://github.com/alda-lang/alda-cljs

It doesn't do much yet, but I've got a basic parser set up with Instaparse, and a public function that will parse a string of Alda code and return & print the parse tree:

The [:stuff [:that :looks] [:like [:this]]] is the data in ClojureScript format being printed to the console -- it translates to nested arrays of strings in JavaScript, which is what you see in the return values.

It occurred to me when I got to this stage that ClojureScript doesn't have eval, which is a crucial part of how the Alda (Clojure version) parser works -- it takes the parse tree and uses it to generate Clojure code, which it then evaluates in the context of a namespace that has a bunch of functions defined for dealing with different types of nodes in the parse tree. This isn't a total blocker -- I'll keep hacking on it.

(Aside from not being able to use eval, I think I should be able to just re-use pretty much all of the code in the alda.lisp namespace!)

[crisptrutski]

For replacing eval, you could use the bootstrapped compiler in cljs.js (as long as the state is isolatable).

Should be able to execute without eval for regular alda.lisp forms (no expressions besides alda function calls and primates) with a simple multimethod though (dispatch on #(when (list? %) (first %))), to get rid of eval on both platforms

[ondras]

This looks awesome! Once the parser (emitting JS code) is in place, I would be happy to hook this up with http://mudcu.be/midi-js/ in order to provide a MIDI playback. Midi.js supports multiple outputs, including Web MIDI and Web Audio (via mp3/ogg samples).

[crisptrutski]

@daveyarwood what do you think about creating a .cljc based alda-lib so we can keep both platforms in sync more easily?

[daveyarwood]

@ondras Awesome! This is really exciting, and makes me want to get the score-emitting piece in place as soon as possible. Once that's done, you and @kylestetz can team up on hooking it up to MIDI.js.

@crisptrutski I'm very much in favor of that idea. It would be good for the two versions to share as much code as possible.

[ondras]

Awesome! This is really exciting, and makes me want to get the score-emitting piece in place as soon as possible. Once that's done, you and @kylestetz can team up on hooking it up to MIDI.js.

If you have some sample data in the form of those emitted JS arrays (i.e. a JSON representation of the AST), feel free to post it somewhere. I can start transforming those into a set of midi.js commands, without having an access to the full cljs alda/parser port...

[crisptrutski]

@ondras here you go, bonus points if you identity it :P

["score",["global-attribute-change","tempo",["number","126"]],["part",["calls",["name","violin"],["nickname","violin-1"]],["octave-set",["number","4"]],["note",["pitch","g"],["duration",["note-length",["number","4"]]]],["note",["pitch","f"],["duration",["note-length",["number","4"]],["note-length",["number","8"]]]],["note",["pitch","d"]],["note",["pitch","f"]],["note",["pitch","a-"],["duration",["note-length",["number","24"]]]],["note",["pitch","b-"]],["note",["pitch","a-"]],["note",["pitch","g"],["duration",["note-length",["number","8"]]]],["note",["pitch","f"],["duration",["note-length",["number","4"]]]],["note",["pitch","g"],["duration",["note-length",["number","8"]]]],["note",["pitch","d"],["duration",["note-length",["number","4"]]]],["note",["pitch","d"]],["note",["pitch","b-"],["duration",["note-length",["number","8"]]]],["rest"],["note",["pitch","g"],["duration",["note-length",["number","4"]],["note-length",["number","8"]]]],["note",["pitch","b-"]],["note",["pitch","g"],["duration",["note-length",["number","12"]]]],["note",["pitch","b-"]],["octave-up"],["note",["pitch","d"]],["octave-down"],["note",["pitch","b-"],["duration",["note-length",["number","8"]]]],["rest"],["note",["pitch","g"],["duration",["note-length",["number","4"]],["note-length",["number","8"]]]],["note",["pitch","b-"]],["note",["pitch","g"],["duration",["note-length",["number","12"]]]],["note",["pitch","b-"]],["octave-up"],["note",["pitch","d"]]],["part",["calls",["name","violin"],["nickname","violin-2"]],["octave-set",["number","3"]],["note",["pitch","b-"],["duration",["note-length",["number","4"]]]],["octave-up"],["note",["pitch","c"],["duration",["note-length",["number","4"]],["note-length",["number","8"]]]],["octave-down"],["note",["pitch","b-"],["duration",["note-length",["number","8"]]]],["octave-up"],["note",["pitch","c"]],["note",["pitch","c"]],["octave-down"],["note",["pitch","b-"]],["octave-up"],["note",["pitch","c"],["duration",["note-length",["number","4"]]]],["octave-down"],["note",["pitch","b-"],["duration",["note-length",["number","8"]]]],["note",["pitch","b-"],["duration",["note-length",["number","4"]]]],["octave-up"],["note",["pitch","c"]],["octave-down"],["note",["pitch","b-"],["duration",["note-length",["number","8"]]]],["rest"],["note",["pitch","b-"],["duration",["note-length",["number","2"]],["note-length",["number","12"]]]],["octave-up"],["note",["pitch","d"]],["note",["pitch","g"]],["note",["pitch","d"],["duration",["note-length",["number","8"]]]],["rest"],["octave-down"],["note",["pitch","b-"],["duration",["note-length",["number","4"]],["note-length",["number","8"]]]],["octave-up"],["note",["pitch","d"]],["octave-down"],["note",["pitch","b-"],["duration",["note-length",["number","12"]]]],["octave-up"],["note",["pitch","d"]],["note",["pitch","g"]]],["part",["calls",["name","viola"]],["octave-set",["number","3"]],["note",["pitch","d"],["duration",["note-length",["number","4"]]]],["note",["pitch","d"],["duration",["note-length",["number","4"]],["note-length",["number","8"]]]],["note",["pitch","f"]],["note",["pitch","d"]],["note",["pitch","d"]],["note",["pitch","d"]],["note",["pitch","d"],["duration",["note-length",["number","4"]]]],["note",["pitch","g"],["duration",["note-length",["number","8"]]]],["note",["pitch","f"],["duration",["note-length",["number","4"]]]],["note",["pitch","f+"]],["note",["pitch","g"],["duration",["note-length",["number","8"]]]],["rest"],["note",["pitch","f+"],["duration",["note-length",["number","4"]]]],["note",["pitch","f"]],["note",["pitch","e"],["duration",["note-length",["number","12"]]]],["note",["pitch","g"]],["note",["pitch","b-"]],["note",["pitch","g"],["duration",["note-length",["number","8"]]]],["rest"],["note",["pitch","f+"],["duration",["note-length",["number","4"]]]],["note",["pitch","f"]],["note",["pitch","e"],["duration",["note-length",["number","12"]]]],["note",["pitch","g"]],["note",["pitch","b-"]]],["part",["calls",["name","cello"]],["octave-set",["number","2"]],["note",["pitch","g"],["duration",["note-length",["number","4"]]]],["note",["pitch","a-"],["duration",["note-length",["number","4"]],["note-length",["number","8"]]]],["note",["pitch","b-"]],["note",["pitch","a-"]],["note",["pitch","f"]],["note",["pitch","g"]],["note",["pitch","a-"],["duration",["note-length",["number","4"]]]],["note",["pitch","g"],["duration",["note-length",["number","8"]]]],["note",["pitch","b-"],["duration",["note-length",["number","4"]]]],["note",["pitch","a-"]],["note",["pitch","g"],["duration",["note-length",["number","1"]]]],["note",["pitch","g"]]]]

Will add some notes at the alda-cljs repo so you can export stuff easilly yourself (see there are actually decent instructions already around the main steps, i just hadn't checked)

Using http://www.freeformatter.com/javascript-escape.html#ad-output is quite convenient to just paste example scores into the console, then just used JSON.stringify.

[ondras]

@ondras here you go, bonus points if you identity it :P

Thanks a lot! I will give it a shot hopefully tomorrow, if I find some time. As far as identification goes, the JSON is too abstract for me :-) I will have to wait until I have something that actually plays music before I can identify it.

Also my general MIDI knowledge is not great, so I will have to study some more (the timing is still puzzling me - I am supposed to periodically schedule playback of some notes, wait a bit, check the time, schedule more notes, rinse & repeat ... ?).

[kylestetz]

@ondras the timing can fortunately all be dealt with in advance... the way the alda REPL works is by executing a line at a time, so you'd be able to schedule that entire line to play. The return key triggers a line to play, so in that sense it will be event-based.

[ondras]

Right, I understand this. But once I have a JS state machine that interprets those arrays, I am not sure if I am free to buffer all notes to midi.js at once (before the playback actually starts), or if I shall batch those "play on"/"play off" commands using some time windows and setTimeout calls.

(Of course, this question belongs more to the midi.js project, but I was curious how this stuff is handled in general.)

[kylestetz]

I would advocate for sending it all in advance and letting midi.js deal with the timing (since it does that already).

[ondras]

I will definitely try that as a first attempt :)

[ondras]

So, @crisptrutski , I have a very first testing version available at http://tmp.zarovi.cz/midi/. It kinda works in latest FF and Chrome, other browsers are untested.

I apparently somehow screwed with timing -- the default tempo seems to be too fast. That's why I introduced -- for debugging purposes -- the Duration multiplier input. When slowed 5x (or 10x), the sound sounds appropriate, although I am still unable to identify the piece :-)

Also, many features are not supported or implemented at all. Volume/velocity is weird, only one instrument (grand piano) is supported, only default rest durations are available.

What next? Shall I create an alda-lang/... repo and push my player code there? I would appreciate some more consultations re. alda syntax / semantics to fix my code as well; do we have an IRC / discussion group?

[crisptrutski]

@ondras exciting! Will check it out later, bit tied up right now. I'very also been thinking a it would be nice to get a discussion group going, but my preference is for gitter or slack. This repo is accumulating stars quickly, and curious to hear more voices 😄

[crisptrutski]

For a mailing list, might be worth trying ou clojureverse

[ondras]

Ha, the tempo issues has been probably solved -- the tempo attribute defines a number of beats per minute, which translates to the number of quarter notes per minute (right?). My original code assumed that it is a number of whole notes per minute, thus playing four times faster...

[crisptrutski]

@ondras checked it out, sounding good!

@daveyarwood what do you think of a PR to bring the JS straight into https://github.com/alda-lang/alda-cljs for now, and we can port it to CLJS "in the open" for people who're interested in seeing the process

[ondras]

I would suggest creating a separate repo instead: this JS player has little relation to the alda language itself; it also depends on the midi.js code (preferrably as a git submodule).

Finally, the player repo could leverage GitHub pages hosting, providing a live demo/player page...

[daveyarwood]

@ondras That sounds right! Looks and sounds awesome, btw! 👍

I like the sound of what you're proposing -- we could make this a separate repo within the alda-lang org. We should make it clear that this feature is in alpha, since it will be improving a lot in the near future.

I think you'll be able to greatly simplify your JS player code once we port more of Alda to ClojureScript. The parser will basically hand you a JavaScript object representing the score, which will contain an array of instruments with config info (the MIDI patch number, etc.) and an array of objects representing note events with precise pitches, durations, etc. which should be easier to hook up to MIDI.js, I think.

[kylestetz]

✋ Officially declaring my interest in designing and helping to build the demo site.

Can't do it right this minute, but over the next month I can put it together (collaboratively, with lots of help!).

[ondras]

I like the sound of what you're proposing -- we could make this a separate repo within the alda-lang org. We should make it clear that this feature is in alpha, since it will be improving a lot in the near future.

Cool, what repo name do you suggest? webplayer? I would create it and upload the initial code (along with the midi.js submodule).

I think you'll be able to greatly simplify your JS player code once we port more of Alda to ClojureScript. The parser will basically hand you a JavaScript object representing the score, which will contain an array of instruments with config info (the MIDI patch number, etc.) and an array of objects representing note events with precise pitches, durations, etc. which should be easier to hook up to MIDI.js, I think.

Yes, this might simplify things. There are still several points (both in alda and midi.js) which are not completely clear to me:

1. does the alda track number somehow correspond to the midi output channel number?
2. how does the volume work, generally speaking? Specifically, volume changes on a per-note basis.
3. why does the track-volume attribute default to 100/127?

✋ Officially declaring my interest in designing and helping to build the demo site.

Can't do it right this minute, but over the next month I can put it together (collaboratively, with lots of help!).

I will be attending the Web Audio Hackday as part of the jsconf.eu extravaganza. This will give me one full day to hack on this project, potentially improving the web playback client -- adding support for more instruments, working on performance, visualizing the score using traditional notation via <canvas> and so on.

[ondras]

One more question:

4. are rests with duration supported?

Having some kitchen-sink example of all language features would be very cool for testing and improving on the player.

[daveyarwood]

@ondras:

Maybe web-demo would be a good repo name? I'm open to other ideas.

To answer your questions:

1. What do you mean by the Alda track number? Do you mean the mapping in *midi-channels*? This establishes a direct link between Alda instrument instances and MIDI channels (one instrument per channel)

2. Volume is slightly confusing because there are two attributes in Alda -- volume and track-volume. volume corresponds to MIDI velocity, which is sort of like volume, but technically it's how hard you hit a note. For most MIDI implementations, when you set the velocity higher, it both makes it louder and makes it sound like the note is being hit harder -- it will sound sharper, brighter, etc. depending on the instrument. track-volume corresponds to MIDI volume, which you can think of as like a slider on a mixing board -- regardless of how aggressively the instrument is being played, you can control how loud that instrument is in the mix. Recommended usage in Alda is the same as it is with MIDI in general -- track volume is typically something you set once for each instrument (if at all), and then you can adjust the volume of individual notes to make your music sound more dynamic.

3. Strangely, this is the default track volume for MIDI instruments, at least in the JVM. I was going for compatibility there.

4. Yes -- rests are essentially just "silent notes", so you can do things like r4, r1~1~2., etc.

If I have a minute, I might throw together a little demo .alda file that uses all of the available features -- would be handy for testing new implementations like this.

[ondras]

Maybe web-demo would be a good repo name? I'm open to other ideas.

Why not; I will wait till Monday (some other names might come in) and create the repo then.

1. What do you mean by the Alda track number?

If a song has four tracks (as in the sample JS data), these are all to be played via the same MIDI channel? (I am really not sure what the MIDI channel really means.) Consider no *midi-channels* mapping present.

Recommended usage in Alda is the same as it is with MIDI in general -- track volume is typically something you set once for each instrument (if at all), and then you can adjust the volume of individual notes to make your music sound more dynamic.

Understood. Is the velocity "set once, use for all further notes" (similar to duration)?

4. Yes -- rests are essentially just "silent notes", so you can do things like r4, r112., etc.

Is the rest duration propagated to all following rests? Or even to all following notes? Are rest lengths also influenced by duration of the notes played before rests?

[daveyarwood]

Re: *midi-channels*: This is something generated in the Clojure environment -- I guess for the purpose of the ClojureScript/JavaScript port, we could include this sort of information in the JS "score object" that the parser will create for you. Basically, the workflow looks like this:

• Parser does its thing, creates an intermediate thing where instruments are represented as simple strings like "piano," "bassoon," etc.
• There is a MIDI-channel-determining step, where each name is looked-up in a master list of stock instruments. If it's a MIDI instrument (currently all of them), it will look up the patch number and store that in *midi-channels* along with a channel number, so that Alda knows which channel to use for which instrument, and which patch number to use for that channel.

Patches have to be set per-channel -- to my knowledge, it is not possible to have multiple notes played by different instruments in the same channel, at the same time.

Re: velocity, yes, it is a "set once, use for all further notes until you set it again" kind of thing, just like pitch and duration. In terms of the JS score object, it will contain pitch, duration, volume, etc. information for every note, which should simplify things once we're at that stage.

Re: rests: they work interchangeably with notes. They are influenced by the duration of notes and rests played before them, and they influence the duration of the notes and rests following.

[daveyarwood]

Actually, the JavaScript score object will also simplify things significantly so that we don't have to include the *midi-channels* information. It's basically already included in the instrument information that you get back.

If you run alda parse --map --file /path/to/some/file.alda, it should give you an idea of what information you will have available about the instruments in a score, and about each note in the score.

[ondras]

Patches have to be set per-channel -- to my knowledge, it is not possible to have multiple notes played by different instruments in the same channel, at the same time.

Makes sense. And what about multiple parts/voices using the same instrument? Can these fit into one channel, or is it always "one channel per part/voice"?

Generally, having a completely prepared score/note data sounds cool, but I do not require it. I believe some level of abstraction -- such as a JSONified AST -- might be generally useful, not particularly for my use case. Also, midi.js already provides the instrument <-> patch number mapping, so I can do this myself. Having "acoustic grand piano" in the intermediate format looks far more readable than "1" :-)

[daveyarwood]

Cool! I think what I'll do is make it so that the exported "parse" can take
some Alda code and output either the intermediate AST or the compiled score
object, depending on what options you give it. This is sort of what the
command line "alda parse" allows you to do, so we can replicate that.

Channels are divided up by instrument part, not voice. So if you have a
single piano part with multiple voices, that's one channel. But if you add
a second piano, it will use a different channel. This is so that MIDI "note
off" events for one instrument instance don't inadvertently affect another
instrument instance.

On Sat, Sep 12, 2015, 1:40 PM Ondřej Žára [email protected] wrote:

Patches have to be set per-channel -- to my knowledge, it is not possible
to have multiple notes played by different instruments in the same channel,
at the same time.

Makes sense. And what about multiple parts/voices using the same
instrument? Can these fit into one channel, or is it always "one channel
per part/voice"?

Generally, having a completely prepared score/note data sounds cool, but I
do not require it. I believe some level of abstraction -- such as a
JSONified AST -- might be generally useful, not particularly for my use
case. Also, midi.js already provides the instrument <-> patch number
mapping, so I can do this myself. Having "acoustic grand piano" in the
intermediate format looks far more readable than "1" :-)

—
Reply to this email directly or view it on GitHub
#72 (comment).

[daveyarwood]

*the exported "parse" function

On Sat, Sep 12, 2015, 2:21 PM Dave Yarwood [email protected] wrote:

Cool! I think what I'll do is make it so that the exported "parse" can
take some Alda code and output either the intermediate AST or the compiled
score object, depending on what options you give it. This is sort of what
the command line "alda parse" allows you to do, so we can replicate that.

Channels are divided up by instrument part, not voice. So if you have a
single piano part with multiple voices, that's one channel. But if you add
a second piano, it will use a different channel. This is so that MIDI "note
off" events for one instrument instance don't inadvertently affect another
instrument instance.

On Sat, Sep 12, 2015, 1:40 PM Ondřej Žára [email protected]
wrote:

Patches have to be set per-channel -- to my knowledge, it is not possible
to have multiple notes played by different instruments in the same channel,
at the same time.

Makes sense. And what about multiple parts/voices using the same
instrument? Can these fit into one channel, or is it always "one channel
per part/voice"?

Generally, having a completely prepared score/note data sounds cool, but
I do not require it. I believe some level of abstraction -- such as a
JSONified AST -- might be generally useful, not particularly for my use
case. Also, midi.js already provides the instrument <-> patch number
mapping, so I can do this myself. Having "acoustic grand piano" in the
intermediate format looks far more readable than "1" :-)

—
Reply to this email directly or view it on GitHub
#72 (comment).

[ondras]

The initial code has been uploaded to https://github.com/alda-lang/web-demo. I suggest closing this issue and using the repo's issue tracker for further progress.

[ondras]

Added support for multiple (different) instruments. Not sure if the sample data sounds right ;-)

[daveyarwood]

@ondras That sounds right to me -- awesome! 🎻 🎶

I think since we've gotten cracking on this, we should close this issue and move this discussion to issues on the web-demo and alda-cljs libraries.