Support creating a playlist of files we haven't seen exported from Sw…

…insian in x minutes. Massive performance improvements on playlist export due to using the Swinsian database for track info rather than AppleScript.

README.md

@@ -2,10 +2,10 @@
 A command line tool to export [Swinsian](https://swinsian.com/) playlists to iTunes/Music and sync back playcounts and ratings.
 * Converts FLAC files to mp3 (requires sox and lame) as needed. Tries (not too hard) to find cover images (cover.jpg, etc), and include them (requires metaflac).
 * Reads song data from the iTunesLibrary Framework (macOS 10.15+). (Thanks to code from the [idbcl](https://github.com/jmkerr/idbcl/) project.)
-* Keeps track of files being removed from iTunes, so that we can keep proper count of playcount, and remove files from disk that we created.
+* Keeps track of files being removed from iTunes, so that we can keep proper count of playcount, and remove files from disk that we transcoded.
 ### Install:
-* Download `sw2iphone.zip` from the latest [release](https://github.com/akda5id/sw2iphone/releases/tag/v0.2.0) and extract it and place in your $PATH (or reference it directly).
-* If you have FLAC files you want to sync, make sure you have sox, lame, and metaflac installed in /usr/local/bin/. Recommend installing with homebrew for ease: `brew install flac`  `brew install lame` and `brew install sox`.
+* Download `sw2iphone.zip` from the latest [release](https://github.com/akda5id/sw2iphone/releases/tag/v0.3.0) and extract it and place it somewhere in your $PATH (or reference it directly).
+* If you have FLAC files you want to sync, make sure you have sox, lame, and metaflac installed in /usr/local/bin/. Recommend installing with [homebrew](https://brew.sh/) for ease: `brew install flac`  `brew install lame` and `brew install sox`.
 #### Usage:
 `sw2iphone -h` will give the commands, but here is what you need to know:
 * By default we will use the directory `~/Music/sw2iphone` as our export folder for playlists, and any mp3s we create. You can change this by passing in a path after `--path`. If you do change the path after mp3s have been created in the old path, be aware that we don't do anything to move them, and they will be recreated next time you export the playlist, and things will get funky (duplicated tracks in iTunes). Recommend you set the path to what you want from the start, and if you do change it later, either manually move over the files, or start fresh in iTunes. We do save the path, you don't have to pass it each time, just when you want to change it.
@@ -16,9 +16,11 @@ A command line tool to export [Swinsian](https://swinsian.com/) playlists to iTu
 #### Status and future features:
 This is a tool I hacked up to fill a need of mine (to get music from Swinsian to my iPhone, and sync back playcounts and ratings). It is pretty simple, and may break in bad ways if you are doing something other than my specific workflow (see workflows below). That said I think other people may have needs similar to mine, so I am releasing it here under the MIT license. And I do intend to continue work to clean it up, make it more robust, and add a few features. Send me mail if you have problems, requests or just to let me know you are using it.
 ##### Todo:
-* Provide support for an alternate workflow where we remove files if they are not exported from Swinsian in a while.
-* Better status messages and logging.
-* Potential performance improvements. I am using AppleScript to get track info out of Swinsian, as it doesn't persist smart playlists to its' database. This is kinda slow, so may be worth enabling a workflow where you would export a (smart) playlist manually from Swinsian, then I could run on that faster, perhaps. But really, the speed hit from AppleScript doesn't much matter if you are converting any FLACs. On that note, I am intentionally single threaded, as I am on a laptop and a prefer not spinning my fans up, and I don't mind waiting a bit for my music to be ready. I suppose this is one of the first feature requests I am going to get however :p
+* Better progress messages and logging.
+* There are some files that iTunes silently refuses to import, related to characters in the path name. I don't think it is my fault, as even dragging the file directly into iTunes doesn't work. But I should warn you when this happens, on sync, if there are files I have never seen in iTunes.
+* An interactive mode to handle sync conflicts with differing ratings rather than just warning about them and skipping?
+* Potential performance improvements. I am using AppleScript to push changes back to Swinsian, to get out of having to deal with the UI and other challenges of tag editing. This is kinda slow, so may be worth hitting the files directly and inserting changes into Swinsian's database. Also, I am intentionally single threaded, as I am on a laptop and a prefer not spinning my fans up, and I don't mind waiting a bit for my music to be ready. I suppose this is one of the first feature requests I am going to get however :p
+* I only copy over the basic tags when we do a transcode from flac. I bet there are some more that people might want?
 * I guess some configurability around lame and sox options would be nice to have. Though I think I have good defaults, I suppose a switch for -V 2 rather than -V 0 would be good. If you care you can just edit the code ;)
 ### Workflows:
 For now, just one supported, mine :p
@@ -29,7 +31,10 @@ For now, just one supported, mine :p
 5) Sync iTunes to your phone again to pick up playcounts, ratings.
 6) Run `sw2iphone -s` to sync that info back into Swinsian.
 7) Before you run `sw2iphone -s`  again, you must quit Swinsian, as it doesn't persist changes to its' database until quit.
+8) Run `sw2iphone -e` on all the playlists again, and then add an `--age x` on your last, or on a separate run, where x is a number of minutes that is larger than the time since you started your round of playlist exports. This will produce a playlist of tracks that had previously been exported by us, but are no-longer in the last batch of exports. You can then import this playlist called `to_Delete` into iTunes, select all of the tracks in it, right click and choose `Delete from Library`.
+9) Alternatively to 8, you can also track and delete tracks from iTunes on your own, perhaps with a smart playlist. Either method works fine, on the next `sw2iphone -s` we will detect the removed tracks. 
+10) Optionally after 8, or 9, you can run `sw2iphone -s --clean`. This will remove any transcoded (mp3 conversions of FLAC files) that we created from disk.
 
-When you have files you no longer need in iTunes, you should delete them from iTunes (recommend a smart playlist to help you find these). On the next `sw2iphone -s`  we will detect they have been removed, and handle them, including removing from disk if you include the `-c` flag.
+iTunes ignores files that it already knows about when you import a playlist, so feel free to export playlists using `-e` as many times over as you like, even if most of the files are duplicates. (iTunes will create a duplicate play**list** though, so you will need to clean up old playlists on iTunes before bringing it in again.) I have a smart playlist in Swinsian for unplayed and unrated songs, and continuously export that, and bring it to my phone. Then as they get played and rated, the `-s` syncs back to Swinsian dropping those files from that playlist, which I export back to iTunes again, ad nauseam.
 
-iTunes ignores files that it already knows about when you import a playlist, so feel free to export playlists using `-e` as many times over as you like, even if most of the files are duplicates. (iTunes will create a duplicate playlist, so you will need to clean up old ones before bringing it in again.) I have a smart playlist in Swinsian for unplayed and unrated songs, and continuously export that, and bring it to my phone. Then as they get played and rated, the `-s` syncs back to Swinsian drop those files from that playlist, which I export back to iTunes again, ad nauseam. However this workflow doesn't remove files from iTunes when they are dropped from the Swinsian playlist. That is a feature I plan to add, but in the meantime, you can add a smart playlist in iTunes to search for the files that would be dropped on the Swinsian side (for myself with playcount > 0 or rating > 0) and then manually delete from iTunes library all files that match. Then the next `-s` will pick those up.
+Note that if you delete files from iTunes, you **must** run a `sw2iphone -s` before adding them back, otherwise we will potentially lose playcounts.

sw2iphone.xcodeproj/project.pbxproj

@@ -293,7 +293,7 @@
 			buildSettings = {
 				CODE_SIGN_STYLE = Automatic;
 				CREATE_INFOPLIST_SECTION_IN_BINARY = NO;
-				CURRENT_PROJECT_VERSION = 0.2.0;
+				CURRENT_PROJECT_VERSION = 0.3.0;
 				PRODUCT_NAME = "$(TARGET_NAME)";
 				SWIFT_VERSION = 5.0;
 			};
@@ -304,7 +304,7 @@
 			buildSettings = {
 				CODE_SIGN_STYLE = Automatic;
 				CREATE_INFOPLIST_SECTION_IN_BINARY = NO;
-				CURRENT_PROJECT_VERSION = 0.2.0;
+				CURRENT_PROJECT_VERSION = 0.3.0;
 				PRODUCT_NAME = "$(TARGET_NAME)";
 				SWIFT_VERSION = 5.0;
 			};

sw2iphone/clean.swift

@@ -41,3 +41,52 @@ func cleanTracks() -> Void {
         exit(1)
     }
 }
+
+func checkForStaleSWTracks(_ age: Int) {
+    let threshold = run_started - Double(age * 60)
+    l.trace("threshold for stale swinsian tracks is \(threshold)")
+    do {
+        try OurDB.write { db in
+            let stale = OurTracks.select(OurTracks.Columns.our_track_id, OurTracks.Columns.our_track_path).filter((OurTracks.Columns.our_lastseen_sw < threshold) && (OurTracks.Columns.our_lastseen_it > 1.0))
+            let count = try stale.fetchCount(db)
+            if count == 0 {
+                l.info("No tracks older than \(age) minutes found")
+                return
+            } else {
+                guard let mp3_url = mp3_url else {
+                    l.error("problem with mp3 url")
+                    exit(1)
+                }
+                let playlist_loc = mp3_url.appendingPathComponent("to_Delete.m3u8")
+                try "#EXTM3U\n".write(to: playlist_loc, atomically: true, encoding: String.Encoding.utf8)
+                var fileHandle: FileHandle = try FileHandle(forWritingTo: playlist_loc)
+                fileHandle.seekToEndOfFile()
+                defer { fileHandle.closeFile() }
+
+                for track in try stale.fetchAll(db) {
+                    let path = track.our_track_path
+                    let id = track.our_track_id
+                    if dry_run {
+                        l.info("would mark \(id) \(path) as stale")
+                        continue
+                    }
+                    l.debug("marking \(id) \(path) stale")
+                    track.our_lastseen_sw = 1.0
+                    try track.update(db)
+                    //write playlist
+                    fileHandle.write("#EXTINF: ,  - \n".data(using: .utf8)!)
+                    fileHandle.write("\(path)\n".data(using: .utf8)!)
+                }
+                l.warning("\(count) tracks were added to \(playlist_loc). You should load that into iTunes, and then delete all tracks in the playlist from library, and then run --sync")
+            }
+        }
+    } catch let error as DatabaseError {
+        l.error("got db error \(error.message ?? "")")
+        exit(1)
+    } catch {
+        l.error("got unknown error \(error)")
+        exit(1)
+    }
+    exit(1)
+}
+

sw2iphone/database.swift

@@ -144,32 +144,94 @@ class PlaylistTable: Record {
     }
 }
 
-class SwTrackTable: Record {
+class SwTrack: Record {
     var sw_track_id: Int64
     var sw_lastplayed: Double?
     var sw_playcount: Int64
     var sw_rating: Int64
-
-    init(sw_track_id: Int64, sw_lastplayed: Double?, sw_playcount: Int64, sw_rating: Int64) {
+    let artist: String?
+    let title: String?
+    let album: String?
+    let year: Int32?
+    let tracknumber_pre: Int32?
+    let totaltracks: Int32?
+    let comment: String?
+    let duration: Int32
+    var path: String
+    var we_created_it: Bool?
+
+    init(sw_track_id: Int64, sw_lastplayed: Double?, sw_playcount: Int64, sw_rating: Int64, artist: String?, title: String?, album: String?, year: Int32?, tracknumber_pre: Int32?, totaltracks: Int32?, comment: String?, duration: Int32, path: String, we_created_it: Bool? = nil) {
         self.sw_track_id = sw_track_id
         self.sw_lastplayed = sw_lastplayed
         self.sw_playcount = sw_playcount
         self.sw_rating = sw_rating
+        self.artist = artist
+        self.title = title
+        self.album = album
+        self.year = year
+        self.tracknumber_pre = tracknumber_pre
+        self.totaltracks = totaltracks
+        self.comment = comment
+        self.duration = duration
+        self.path = path
+        self.we_created_it = we_created_it
         super.init()
     }
 
     override class var databaseTableName: String { "track" }
 
     enum Columns: String, ColumnExpression {
-        case track_id, lastplayed, playcount, rating
+        case track_id, lastplayed, playcount, rating, artist, title, album, year, tracknumber, totaltracknumber, comment, length, path
     }
 
     required init(row: Row) {
         sw_track_id = row[Columns.track_id]
         sw_lastplayed = row[Columns.lastplayed]
         sw_playcount = row[Columns.playcount]
         sw_rating = row[Columns.rating]
+        artist = row[Columns.artist]
+        title = row[Columns.title]
+        album = row[Columns.album]
+        year = row[Columns.year]
+        tracknumber_pre = row[Columns.tracknumber]
+        totaltracks = row[Columns.totaltracknumber]
+        comment = row[Columns.comment]
+        duration = row[Columns.length]
+        path = row[Columns.path]
+        we_created_it = nil
         super.init(row: row)
     }
+
+    var kind: String {
+        let ext = URL(fileURLWithPath: self.path).pathExtension.lowercased()
+        if ext == "mp3" {
+            return "MP3"
+        } else if ext == "flac" {
+            return "FLAC"
+        } else {
+            return "OTHER"
+        }
+    }
+
+    var id: Int {
+        return Int(self.sw_track_id)
+    }
+
+    var tracknumber: String? {
+        guard let tracknumber_pre = tracknumber_pre else {
+            return nil
+        }
+        var output = ""
+        if let totaltracks = totaltracks {
+            if totaltracks != 0 {
+                output = "\(String(tracknumber_pre))/\(String(totaltracks))"
+            } else {
+                output = (String(tracknumber_pre))
+            }
+        } else {
+            output = (String(tracknumber_pre))
+        }
+        return output
+    }
 }