Easier use without Processing + Mavenizing

[noncom]

I have created a fork where I:

1. added the classes to easily create Minim object without Processing (e.g. from Clojure) that provide the basic implementation of the required methods.
2. mavenized the project and all deps, so that the libraries that you have in the "libraries" folder are now fetched from Maven central repo (the original ones still remain in the folder just in case)

Are you interested in such a pull request?

BTW: uploaded the latest commit currently available (b8afdc1) to Clojars: https://clojars.org/ddf.minim/versions/2.2.1-b8afdc1-SNAPSHOT wow! so far "586 Downloads" - for the previous versions - pretty cool :) People love Minim!

[stevenpetryk]

👍

Currently working on a Clojure project that could use the Minim library, this would be incredibly useful. I saw your comments in the other issue, and you are doing really good work.

[ddf]

I'd like to get 2.2.1 properly out the door and then I will be able to take your pull request. I know basically nothing about Clojure and Maven, so I may hit you up if I wind up with questions. I dig the FileSystemHandler idea and the interface, tho will likely modify it with a constructor for setting customSketch path, I would not expect those values to change over the course of an application, so don't see why they should be public fields. Also, my preference for using it would be to explicitly construct one and pass it to Minim, rather than introducing the static method.

PS sorry it has taken me so g-d long to respond to this.

[noncom]

Of course! Yeah, surely you can make a better structured solution than the small patch, that'd be great. Actually, PApplet encloses quite of a facility hidden behind the resource stream acquisition single method. It could be great if this facility is somehow present there at the most possible degree.

As for changing the custom sketch path idk, but why not? I do not recall exactly what Processing uses it for, but it is pointless to cling to any restriction on relative path file access. If we need the application path we still have System.getProperty("user.dir");.

And yeah, life is such a thing, many things are happening all the time, have to manage.. BTW, congrats with the success of Waking Mars! Just finished my copy of the game and saw your name in the titles! So cool, I was really enjoying the soundscape of the game and the music. Did not know you took part in it!

[ddf]

The sketch path is used when saving AudioRecordings to disk. I now have regrets about simply using the method names from Processing, but at the time I was feeling lazy and only wanted to write the reflection code. I think what I will probably do is give the IFileSystemHandler interface method names that are more descriptive for how Minim uses them and then have an implementation that specifically looks for the Processing methods. I finally got around to creating a pure-Java test case for a particular bug, so I should be paying more attention to ease of use outside of Processing.

Thanks about Waking Mars. I did audio programming and wrote a few songs for that one. If you enjoyed that game, you might also like Spider: Rite of the Shrouded Moon, which was just released by the same studio.

[noncom]

Very cool! Looking forward for the new version. I am always around if you will have to ask something about Maven or Clojure, as you've said before!

Thanks for the recommendation, yeah, indeed, the game looks very inviting, gonna be my next play since I really enjoyed waking the Mars! :D

[ddf]

Please feel free to send the pull request whenever you get the chance!

[noncom]

@ddf You mean the pull request for the mavenization? As I understood, you are going to implement the independence from Processing, so could you please tell me what could I do, and I will take on that.

[ddf]

Ah, yes, I was thinking it'd be easier for you to send a pull request with the changes you made and then I can modify from there as I see fit, which I think would make for a cleaner merge on your end if you want to pull my changes into your fork?

A question about Maven: does it preclude me from using the Ant build file I already have?

[noncom]

Oh, sure. I will revise my repository this weekend and check if there's any mess to clear up before making a PR. I will also have to verify my repo changes to correspond to the Maven transperancy policy, that is - that all dependencies are already available from the official Maven Central repo (can sometimes be tricky). Currently I have mavenized Minim through the Clojars repo - the one erected by the Clojure people. It has much milder requirements for uploading an artifact.

When you take on this, I do not think I will keep maintaining my fork since there'll be no need. I would prefer to use the true Minim. Aside from that I am likely to create a Clojuric wrapper sometime not too long in the future, and if I get to get this task going and done, I'll share.

As for using an Ant script with Maven - it is perfectly doable. An Ant script can be included as a task into Maven project file ( https://maven.apache.org/guides/mini/guide-using-ant.html ). It can also work the other way around: http://www.mkyong.com/maven/how-do-create-an-ant-build-file-from-maven-pomxml/ .

[noncom]

@ddf Now working on this, and there's a quesion: what is the prodoc project that is bundled with Minim? Is it considered an essential part of Minim? Can it be integrated into Minim, or can it be fetched as a dependancy?

(I have gathered some general knowledge on prodoc here http://creativecomputing.cc/p5libs/prodoc/ but those "structural" questions are still not answered)

[ddf]

The prodoc.jar that is in ext/prodoc is exported from my fork of prodoc (https://github.com/ddf/prodoc), which has several changes that make it generate the "easy" documentation how I'd prefer (http://code.compartmental.net/minim/). ext/prodoc/documentation is not actually required, since it is simply the generated documentation for prodoc itself. ext/prodoc/templates contains the html template files used in documentation generation, so that should stay. the .classpath and .project files in ext/prodoc I think are leftover from me just copying an existing eclipse project of the library into the folder, they should probably be removed.

I'm not sure what the general placement of these kinds of things are, so happy to consider your recommends.

[ddf]

Oh, you can check out the doc tag in build.xml to see how I'm using prodoc and javadoc.

[noncom]

So, ProDoc is not a part of Minim. But it generates the JavaDoc for Minim...

I guess, it would be correct to first mavenize ProDoc and make it a dependency of Minim, thus, removing it from Minim. The javadoc will be generated for Minim using the Maven Javadoc plugin. It allows specifying a custom doclet, thus, with ProDoc already in maven, it'll be a piece of cake:

https://maven.apache.org/plugins/maven-javadoc-plugin/examples/alternate-doclet.html

Effectively this presumes that the building process will be given to maven instead of Ant. There's actually no problem with moving from Ant to Maven, the Ant script is very simple. However, if you still prefer using Ant for some reason, it can still be used within a mavenized project: https://maven.apache.org/ant-tasks/examples/dependencies.html (but I do not see a reason to do so, except if you want to stay compliant to something that requires Ant).

All it looks like there are these steps to be done:

1. Pick a group id. For example, ddf or compartmental or something else ( https://maven.apache.org/guides/mini/guide-naming-conventions.html )
2. Mavenize ProDoc as <groupId>/prodoc
3. Specify it as a dependency of Minim, setup the javadoc plugin with the ProDoc doclet
4. Mavenize Minim

If all this looks too invasive and a big change you would like to avoid, then there are 2 options:

• Leave ProDoc as it is, but it will be not functional in Maven build. Users will have to be aware of it and how to use it OR the Maven script will reference your Ant script, which feels quite patchy, but maybe this is the easiest way to leave as they are as many things as possible.
• Simply ignore all that and only publish to Clojars (as already it is in my fork). The Clojars repository is kept up by Clojure people and has much milder rules of submission. Minim is already available there, we'll only have to update it to verson 2.2.2. So, all that I said above is required only for submission into Maven Central. And if you feel you want to go full-scale on Maven since Maven is the future (looks pretty inevitable), then we can do it.

What's the difference between Maven Central and Clojars, you ask? Well, not much except that for fetching artifacts from Clojars, one, if he's not on Clojure himself, has to specify the repository link in his Maven pom. And also Clojars is less known as of yet among non-Clojure people.

I'm sorry, this might seem as a lot of info to consider, but there's no other way to act on this.

What do you think? Which of these ways do you like the most?

[ddf]

I think my only problem with Mavenizing my fork of prodoc is that I treat it essentially as private code. Like, I wouldn't want anyone finding that in Maven Central and deciding to use it for their own project and then coming to ask me questions about why it behaves a certain way. I modify it when I see fit in order to have the documentation generated how I'd like and don't bother with any kind of versioning. When I do a release of Minim, I do not include prodoc in the release.

So, I dunno, if you think it's not such a big deal to throw my version of prodoc into Maven Central, then I guess I don't really have a problem with it, especially if it helps others build from source and incorporate Minim into their projects more easily.

I'm not particularly attached to keeping the Ant build file around, as long as Maven can accomplish all the same tasks. Ideally Maven is able to help automate some of the release process, outlined here: https://github.com/ddf/Minim/wiki/Doing-a-Release

[noncom]

Good. Artifacts in Maven are not required to contain information sufficient for building from sources. The user simply receives a jar that just works. It contains the javadoc already generated by ProDoc. So, no ProDoc there.

However, if we are to migrate the automation to Maven (which is functionally superior to Ant, so it is a win), ProDoc be turned into a dependency and be itself available through Maven. It will help to those who build from sources. I think it will bring no problems of it being that way.

Thanks for answering the questions! Okay, now I have enough understanding, doubts are eliminated, and am starting the work on this. I will take the release process list you linked into account and try to automate the steps. For example, Processing is already there: http://mvnrepository.com/artifact/org.processing

[ddf]

Thanks very much! You won't be able to automate all of the steps since some of them require access to my website ftp, but if creating zip files with proper names and so forth can be automated by Maven that would be amaze. Keep in mind there are essentially two releases we are talking about here: one that pushes a new version of Minim with updated documentation to Maven Central (for people who are developing non-processing projects) and one that generates the files I need to update the release I host for Processing (for people working in the Processing IDE and using their Library manager).

To answer your question about what groupId to use: given the naming guidelines, it probably makes the most sense to use net.compartmental.code, since the online docs for minim are hosted at code.compartmental.net/minim

[noncom]

Okay. Also:

1. You have to register here: https://issues.sonatype.org/projects/OSSRH and tell me your username so that I include you in the repository registration request to enable you to upload artifacts in the repository.
2. We have to create an identity (name and email) for the PGP key. The artifacts will be signed with it, so I think that putting mine is inappropriate. Usually either a developers personal name / email are used for that, or some identity, like Minim Association or like is created to be specified in this data.

I think these issues (esp the 2nd) are better discussed privately. Please email me at deleted. If you already know what would that be, just write them to me. It has to be 1) name, 2) email, 3) some short comment

OR if you already have a PGP key, then it is fine, I can just leave the blank field in the maven pom file that you can fill-in yourself.

[ddf]

I've registered at sonatype as ddf. I will make a PGP key to put into the pom file, so you can leave the field blank.

[noncom]

Okay, sounds good. I've opened the issue: https://issues.sonatype.org/browse/OSSRH-18230

Also, maybe you find this document useful for workings with PGP: http://blog.sonatype.com/2010/01/how-to-generate-pgp-signatures-with-maven/#.Vh0QsfntlBc

[noncom]

Uhh, I am sorry for the delays... things are happening in life that require much time and attention... But finally I have managed to make progress on the task. It turned to be trickier than I thought in some places, but I got it. Here is the Minim project on mavenization progress: https://github.com/noncom/Minim and the fork of your fork of ProDoc: https://github.com/noncom/prodoc .

I would ask you to fetch them and check up with the pom.xml files, the resulting jar and all stuff. I hope I did not break anything. All Maven lifecycle phases (https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html) up to install are passing on my end. Tasks further in the lifecycle, like deployment, require fullfillment of the GPG and signature data, which you could perform when it is ready to be released to the repo.

Noticeable changes also include: the directory layout is changed to correspond to Maven standard, the Eclipse projects are updated correspondingly.

There's one peculiarity in the ProDoc pom.xml - I specify an exact path to my JDK, since for some reason JAVA_HOME is not getting picked up. I will try to deal with this in the next iteration. We require this because ProDoc relies on com.sun.javadoc which is prohibited for redistribution and thus is not available from Maven repos but has to be taken directly from the JVM distribution.

As for the other custom build phases specified in your build.xml, they are currenly inactive because: 1) for javadoc generation - I have included the step, but am still playing with setting correct parameters, 2) I have not yet got to the rest of them. But it is a start, never the less..

[ddf]

Hi hi, I've been pretty busy myself. I'm looking at these now. The ProDoc repo is busted for me because the absolute path to tools.jar doesn't work on my Mac. But I think I'm just gonna merge that stuff and try to figure it out on my end. Once I get that sorted I will fetch your changes to Minim. Should I fetch your master branch or your mavenize branch?

[noncom]

The branch of interest is mavenize.

I would advice the following approach: install maven, clone the repo (not merge with your repo), try maven on it. For example, the mvn install command should compile everything, pack and publish to the local repository, which you can find somewhere around ~/.m2. Once you have Minim successfully available in that folder, it is all ok to go on with the PGP and publishing stuff. Also, at that point you could better see how to adopt the required information into your repo.

Be sure to examine the contents of the jar so that they satisfy you. For example, I have noticed that source folders of Minim with *.java files, where there are some prodoc-oriented stuff, are retained in the jar, though empty.

The ~/.m2 folder is the local maven repo, which stores all maven dependencies that you have ever required in any of your maven projects, so to say, a local subset copy of the online maven repository. If Minim is there, then all is ok except the signature and upload.

About the ProDoc repo problem: yeah, a nasty one, well, you can replace the absolute path with anything. For example, by default, in the example that I have found, they were using JAVA_HOME, but it did not work on my windows pc. For an example, you could look like here http://stackoverflow.com/questions/13288735/maven-not-picking-java-home-correctly or some other similar googing. What is said in that question should enable you to use the dependency ok.

[ddf]

Ok, thanks. I pushed some changes to my prodoc repo. You might like to pull them and see if I managed to fix the tool.jar dependency in a way that works for you. I was able to run mvn install on prodoc and that all seemed to work fine.

So, if I understand correctly, I should be able to run mvn install on your mavenized version of Minim and it will work because I now have prodoc in my local maven repo. But when we get to the point of deploying Minim we will also need to deploy prodoc, correct? Does the issue you opened on Sonatype for creating a Minim project there also cover prodoc, or will we need to make another request?

[noncom]

Wow, you're so genius! So the solution was to make the platform profiles... And it really works for me now, yes. That's cool! :)

There's another minor problem though. When I clone the repo and import it as an existing project, then Eclipse source path problem happens. Somehow, the source folder is included twice.. To fix it, I have to do this one thing: http://joxi.ru/LmGVaByIyO3orl However, that's just Eclipse problem, all Maven things are working ok!

As for your question - yes, now that ProDoc successfully gets into the local repo dir, it should be ok for the mavenized Minim to compile locally on your Mac/PC! ProDoc will have to be deployed on Maven Central too, correct. Once both the artifacts are deployed, they should be available worldwide for Maven projects.

However, there's one thing: since people will fetch Minim to work with it, they most often will not be compiling it from sources, just using the jar. Therefore, they will need no ProDoc because ProDoc is only required on compilation stage, as a doclet plugin. Minim does not need ProDoc in runtime/usage. So, there's that alternative variant: if you want, you can only deploy ProDoc to your local repo and Minim will compile and deploy to the Central, even if ProDoc is not there. But for the sake of simplicity and transperancy, I would recommend deploying ProDoc too.

The ticket that we have created and that was greenlit covers the entire group of "net.compartmental.code" where you can deploy ProDoc, Minim, and any other stuff that is made by you under the "net.compartmental.code" group id. That being said, since I am not really belong to this group, I will resign my rights to participate in the group once we finish this process with Minim, so that you will have your very own account to publish your artifacts :)

[noncom]

Also, one more note on deployment: you can freely deploy on Sonatype as many times as you want, experimenting with the stuff and how it works, but for that your version has to have the -SNAPSHOT suffix, like 2.2.5-SNAPSHOT. This way, it will be overwritten with each new deployment. That suits well for testing things. However, if you set 2.2.5 or 2.2.6, then you won't be able to overwrite it in the Sonatype repo without altering the version.

For the SNAPSHOT versions, dependencies too have to specify the -SNAPSHOT suffix, not just the version.

Here's the relevant link: http://central.sonatype.org/pages/ossrh-guide.html in the bottom it describes the repositories.

[ddf]

Ok I had a look at your mavenize branch and everything works fine for me. I saw the empty directories in the compiled jars you mentioned and figured out those can be eliminated by adding <exclude>**/*.html</exclude> to resources section of the pom. I've also followed all the pgp instructions and have a key generated and deployed to a keyserver and everything. I haven't tried deploying a snapshot to sonatype yet because I was curious about the difference between the current distributionManagement section and the one shown here:

http://central.sonatype.org/pages/apache-maven.html

Basically I'm wondering if we need to have that snapshotRepository tag or if it suffices to set the version in to like 2.3.0-SNAPSHOT. When you mention that all the dependencies have to specify the -SNAPSHOT suffix do you mean mp3spi and so forth? So I have to change all of those until ready to do the proper release?

I'd be ok with pulling your mavenize branch changes into my master branch and moving forward from there, or would you like to work a bit more in that branch before I do so?

[noncom]

Yes, I think I have to make some improvement to the branch first. You are right that some sections of the pom.xml can be improved. Give me a couple of days, I will finish it and answer your questions too.

[noncom]

Damien, I am very sorry for the delay, the end of the year is pressing hard on my free time, recently been working 1.5-2 shifts a day. I have made some progress on our task, I need to ensure it is ok and I plan to upload the thing to my repo in the upcoming holidays, at the beginning of the January. Happy New Year! :)

[ddf]

No worries on the delay, I've been plenty busy myself! Thanks for the update and enjoy your holidays!

[noncom]

Thanks!

So finally I have pulled everything up together. Seems it looks much better now. I have brought everything up in correspondence with what the manual says. So:

1. The pom.xml is now according to the manual, all maven plugins are updated to their latest current versions.
2. There's now settings.xml which contains fields for you to fill in if you gonna deploy (both either release or -SNAPSHOT)
3. Currently I have specified -SNAPSHOT for the artifact version so there's no trouble in attempting the put it to the Central for training. And when you sure everything's fine and 2.2.2 is sealed, just remove this (for details on deploying snapshot and requiring it as a dependency from a users project pom.xml, refer to http://central.sonatype.org/pages/apache-maven.html section Performing a Snapshot Deployment)
4. The GPG maven plugin is commented in pom.xml now. This enables to run mvn install to ensure that locally everything works.
5. When you ready to deploy your first try to the Central, I advice to keep the -SNAPSHOT prefix at first, and be sure that you have filled in settings.xml and uncommented the GPG plugin.
    
   ATTENTION: Your actual settings.xml with the fields filled in is not to be commited to Minim repo - copy the file from the repo to ~/.m2 and fill in the fields in that file. Here's the reference just in case: https://maven.apache.org/settings.html
6. When all is ok, and Minim 2.2.2 is stabile and out, remove -SNAPSHOT in pom.xml and deploy as described in http://central.sonatype.org/pages/apache-maven.html section Performing a Release Deployment

Seems like it. I hope I did not forget anything, looks pretty complete.

[ddf]

Yes, hello, thanks! Time has really gotten away from me. I hope to get my head back in this soon.

[leif81]

Cool, I'm keen to try out minim, but lack of Maven dependency is a showstopper. So I'm glad it sounds like most of the hard work on this is mostly done! Any progress on this lately? @noncom maybe open a PR from your branch and the remaining bits can be sorted out there?

[noncom]

@leif81 Well, it's all ready, actually, it's just that Damien did not yet take on it. There's just all that hassle with the GPG and the push left. Not much else that I can do here... if you want, you still can use it with Maven -- just add the Clojars repository and fetch it from there (see my first post, it has the link). The version there is slightly outdated... if you need the latest one, I can attempt to rebase and resync, but there were no critical commits.

[leif81]

if you need the latest one, I can attempt to rebase and resync, but there were no critical commits.

Ya that'd be useful. Thank you!

Once it's rebased you might as well open a pull request for it here and then all @ddf needs to do is click merge.

[ddf]

Hi friends, I am finally in a place where I can devote significant time to getting this working.

@noncom can you send me a pull request so I can be sure to be integrating from the correct branch?

[ddf]

@noncom if I don't hear from you before Monday I'm planning to simply pull your mavenize branch

[ddf]

I have just released Minim 2.2.2 in Maven Central. If one of y'all could try to use it in a project to verify that it all worked properly, that'd be great. Then we can finally close this issue!

[bjakke]

Hello,

I assume it is this one: https://mvnrepository.com/artifact/net.compartmental.code/minim/2.2.2 ?

It would be nice if you would mention the latest release in the README that it would show:

<dependency>
    <groupId>net.compartmental.code</groupId>
    <artifactId>minim</artifactId>
    <version>2.2.2</version>
</dependency>

My initial test would indicate that it works, or at least by code from https://github.com/bjakke/graffathon2016 works if I change the pom to point to the above coordinates (as a quick test, not in GH).

One minor thing/error is that the org.apache.maven.plugins » maven-javadoc-plugin is listed as compile time dependency. Probably it should not be a dependency. Anyway that can be excluded, so it is not that big of a problem, however it might be something that you improve in the next release maybe.

I can probably give better test result in a few days as I will probably use the same technology for 2017 version of the event.

[bjakke]

Also thank you so much for making this library to be available from Maven Central 👍 , makes things much easier :)

[ddf]

@bjakke thanks for the feedback, I will fix things up in the next release.

I don't exactly understand your comment about the README. Are you talking about the README in the repository here or a different one?

[noncom]

@ddf Hello again! I appologize for not responding to your query when you were about to go on with Maven. I was so out these days. Now it's better! :) And I am glad that it went ok for the mavenization too, you did everything just right and that's cool! I think that it's ok now.

As for the bjakkes comment on README, I guess that yes, he meant to include the Maven coords in the repo README. That's a rather common pattern for mavenized projects, for example: https://github.com/perwendel/spark , and some even specify Gradle coords too since only hardcore people use Maven in its pure form today (like here: http://sparkjava.com/documentation#getting-started) but it's not mandatory and essentially just repeats the same information that is accessible via clicking the maven-central tag... so it's up to your taste. All in all, as I feel it, more gladness arises when you directly see colorful Maven/Gradle/<other_build_systems> coords on a projects GitHub page...

Also, this page has outdated info on version number and download options: http://code.compartmental.net/tools/minim/

As for the maven-javadoc-plugin, indeed it can be removed from the <dependencies> section of the pom.xml (I mean the one on line 58), since it is not a compile-time dependency at all. I don't remember how it got there, however, but iirc a part of the build process included building a special kind of docs for the website, which were based on javadoc and hence it was written this way... but I might be wrong and imagining all this. Anyway, I've just tried running the mvn install command with the dependency on lines 58-62 being commented and it went all ok, so it should be... safe?

Thanks for the wonderful library and for taking care!

PS Also, when was looking for library details on your website and found out about the new (for me) Everything experience, I bought it and played, that's amazing!