{"id":875,"date":"2014-04-06T12:56:01","date_gmt":"2014-04-06T16:56:01","guid":{"rendered":"http:\/\/www.visophyte.org\/blog\/?p=875"},"modified":"2014-04-06T12:56:01","modified_gmt":"2014-04-06T16:56:01","slug":"webpd-a-polymer-based-web-ui-for-the-beets-music-library-manager","status":"publish","type":"post","link":"https:\/\/www.visophyte.org\/blog\/2014\/04\/06\/webpd-a-polymer-based-web-ui-for-the-beets-music-library-manager\/","title":{"rendered":"webpd: a Polymer-based web UI for the beets music library manager"},"content":{"rendered":"<p><a href=\"http:\/\/beets.radbox.org\/\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-878\" alt=\"beets webpd filtered artists list\" src=\"http:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2014\/04\/beets-webpd-filtered-artists.png\" width=\"863\" height=\"713\" srcset=\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2014\/04\/beets-webpd-filtered-artists.png 863w, https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2014\/04\/beets-webpd-filtered-artists-600x495.png 600w, https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2014\/04\/beets-webpd-filtered-artists-300x247.png 300w\" sizes=\"auto, (max-width: 863px) 100vw, 863px\" \/><\/a><\/p>\n<p><a href=\"http:\/\/beets.radbox.org\/\">beets<\/a> is the extensible music database tool every programmer with a music collection has dreamed of writing.\u00a0 At its simplest it&#8217;s a clever tagger that can normalize your music against the <a href=\"https:\/\/musicbrainz.org\/\">MusicBrainz<\/a> database and then store the results in a searchable SQLite database.\u00a0 But with plugins it can fetch album art, use the <a href=\"http:\/\/www.discogs.com\/\">Discogs<\/a> music database for tagging too, calculate ReplayGain values for all your music, integrate meta-data from <a href=\"http:\/\/the.echonest.com\/\">The Echo Nest<\/a>, etc.\u00a0 It even has a <a href=\"http:\/\/mpd.wikia.com\/wiki\/Music_Player_Daemon_Wiki\">Music Player Daemon<\/a> server-mode (<a href=\"http:\/\/beets.readthedocs.org\/en\/v1.3.3\/plugins\/bpd.html\">bpd<\/a>) and a simple HTML interface (<a href=\"http:\/\/beets.readthedocs.org\/en\/v1.3.3\/plugins\/web.html\">web<\/a>) that lets you search for tracks and play them in your browse using the HTML5 audio tag.<\/p>\n<p>I&#8217;ve tried a lot of music players through the years (alphabetically: <a href=\"http:\/\/amarok.kde.org\/\">amarok<\/a>, <a href=\"http:\/\/banshee.fm\/\">banshee<\/a>, <a href=\"http:\/\/www.exaile.org\/\">exaile<\/a>, <a href=\"http:\/\/code.google.com\/p\/quodlibet\/\">quodlibet<\/a>,\u00a0<a href=\"https:\/\/wiki.gnome.org\/Apps\/Rhythmbox\">rhythmbox<\/a>).\u00a0 They all are great music players and (at least!) satisfy the traditional Artist\/Album\/Track hierarchy use-case, but when you exceed 20,000 tracks and you have a lot of compilation cd&#8217;s, that frequently ends up not being enough. Extending them usually turned out to be too hard \/ not fun enough, although sometimes it was just a question of time and seeking greener pastures.<\/p>\n<p>But enough context; if you&#8217;re reading my blog you probably are on board with the web platform being the greatest platform ever.\u00a0 The notable bits of the implementation are:<\/p>\n<ul>\n<li>Server-wise, it&#8217;s a mash-up of beets&#8217; MPD-alike plugin bpd and its web plugin.\u00a0 Rather than needing to speak the MPD protocol over TCP to get your server to play music, you can just hit it with an HTTP POST and it will enqueue and play the song.\u00a0 <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Server-sent_events\/Using_server-sent_events\">Server-sent events<\/a>\/<a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/API\/EventSource\">EventSource<\/a> are used to let the web UI hypothetically update as things happen on the server.\u00a0 Right now the client can indeed tell the server to play a song and hear an update via the EventSource channel, but there&#8217;s almost certainly a resource leak on the server-side and there&#8217;s a lot more web\/bpd interlinking required to get it reliable.\u00a0 (Python&#8217;s <a href=\"http:\/\/flask.pocoo.org\/\">Flask<\/a> is neat, but I&#8217;m not yet clear on how to properly manage the life-cycle of a long-lived request that only dies when the connection dies since I&#8217;m seeing the appcontext get torn down even before the generator starts running.)<\/li>\n<li>The client is implemented in <a href=\"http:\/\/www.polymer-project.org\/\">Polymer<\/a> on top of some simple <a href=\"http:\/\/backbonejs.org\/\">backbone.js<\/a> collections that build on the existing logic from the beets web plugin.\n<ul>\n<li>The artist list uses the <a href=\"https:\/\/github.com\/PolymerLabs\/polymer-list\">polymer-virtual-list<\/a> element which is important if you&#8217;re going to be scrolling through a ton of artists.\u00a0 The implementation is page-based; you tell it how many pages you want and how many items are on each page.\u00a0 As you scroll it fires events that compel you to generate the appropriate page.\u00a0 It&#8217;s an interesting implementation:\n<ul>\n<li>Pages are allowed to be variable height and therefore their contents are too, although a fixedHeight mode is also supported.<\/li>\n<li>In variable-height mode, scroll offsets are translated to page positions by guessing the page based on the height of the first page and then walking up\/down from there based on cached page-sizes until the right page size is found.\u00a0 If there is missing information because the user managed to trigger a huge jump, extrapolation is performed based on the average item size from the first page.<\/li>\n<li>Any changes to the contents of the list regrettably require discarding all existing pages\/bindings.\u00a0 At this time there is no way to indicate a splice at a certain point that should simply result in a displacement of the existing items.<\/li>\n<\/ul>\n<\/li>\n<li>Albums are loaded in batches from the server and artists dynamically derived from them.\u00a0 Although this would allow for the UI to update as things are retrieved, the virtual-list invalidation issue concerned me enough to have the artist-list defer initialization until all albums are loaded.\u00a0 On my machine a couple thousand albums load pretty quickly, so this isn&#8217;t a huge deal.<\/li>\n<li>There&#8217;s filtering by artist name and number of albums in the database by that artist built on <a href=\"https:\/\/github.com\/jmorrell\/backbone-filtered-collection\">backbone-filtered-collection<\/a>.\u00a0 The latter one is important to me because scrolling through hundreds of artists where I might only own one cd or not even one whole cd is annoying.\u00a0 (Although the latter is somewhat addressed currently by only using the albumartist for the artist generation so various artists compilations don&#8217;t clutter things up.)<\/li>\n<li>If you click on an artist it plays the first track (numerically) from the first album (alphabetically) associated with the artist.\u00a0 This does limit the songs you can listen to somewhat&#8230;<\/li>\n<li>visualizations are done using <a href=\"http:\/\/d3js.org\/\">d3.js<\/a>; one svg per visualization<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-880\" alt=\"beets webpd madonna and morrissey\" src=\"http:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2014\/04\/beets-webpd-madonna-morissey-smooshified.png\" width=\"709\" height=\"81\" srcset=\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2014\/04\/beets-webpd-madonna-morissey-smooshified.png 709w, https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2014\/04\/beets-webpd-madonna-morissey-smooshified-600x68.png 600w, https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2014\/04\/beets-webpd-madonna-morissey-smooshified-300x34.png 300w\" sizes=\"auto, (max-width: 709px) 100vw, 709px\" \/><\/p>\n<p>&#8220;What&#8217;s with all those tastefully chosen colors?&#8221; is what you are probably asking yourself.\u00a0 The answer?\u00a0 Two things!<\/p>\n<ol>\n<li>A visualization of albums\/releases in the database by time, heat-map style.\n<ul>\n<li>We bin all of the albums that beets knows about by year.\u00a0 In this case we assume that 1980 is the first interesting year and so put 1979 and everything before it (including albums without a year) in the very first bin on the left.\u00a0 The current year is the rightmost bucket.<\/li>\n<li>We vertically divide the albums into &#8220;albums&#8221; (red), &#8220;singles&#8221; (green), and &#8220;compilations&#8221; (blue).\u00a0 This is accomplished by taking the MusicBrainz <a href=\"https:\/\/musicbrainz.org\/doc\/Release_Group\/Type\">Release Group \/ Types<\/a> and mapping them down to our smaller space.<\/li>\n<li>The more albums in a bin, the stronger the color.<\/li>\n<\/ul>\n<\/li>\n<li>A scatter-plot using <a href=\"http:\/\/the.echonest.com\/\">the echo nest<\/a>&#8216;s <a href=\"http:\/\/musicbrainz.org\/doc\/Release_Group\/Type\">acoustic attributes<\/a> for the tracks where:\n<ul>\n<li>the x-axis is &#8220;danceability&#8221;.\u00a0 Things to the left are less danceable.\u00a0 Things to the right are more danceable.<\/li>\n<li>the y-axis is &#8220;valence&#8221; which they define as &#8220;the musical positiveness conveyed by a track&#8221;.\u00a0 Things near the top are sadder, things near the bottom are happier.<\/li>\n<li>the colors are based on the type of album the track is from.\u00a0 The idea was that singles tend to have remixes on them, so it&#8217;s interesting if we always see a big cluster of green remixes to the right.<\/li>\n<li>tracks without the relevant data all end up in the upper-left corner.\u00a0 There are a lot of these.\u00a0 The echo nest is extremely generous in allowing non-commercial use of their API, but they limit you to 20 requests per minute and at this point the beets echonest plugin needs to upload (transcoded) versions of all my tracks since my music collection is apparently more esoteric than what the servers already have fingerprints for.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>Together these visualizations let us infer:<\/p>\n<ul>\n<li>Madonna is more dancey than Morrissey!\u00a0 Shocking, right?<\/li>\n<li>I bought the Morrissey singles box sets. And I got ripped off because there&#8217;s a distinct lack of green dots over on the right side.<\/li>\n<\/ul>\n<p>Code is currently in the <a href=\"https:\/\/github.com\/asutherland\/beets\/tree\/webpd\">webpd branch of my beets fork<\/a> although I should probably try and split it out into a separate repo.\u00a0 You need to enable the webpd plugin <a href=\"http:\/\/beets.readthedocs.org\/en\/v1.3.3\/plugins\/index.html\">like you would any other plugin<\/a> for it to work.\u00a0 There&#8217;s still a lot lot lot more work to be done for it to be usable, but I think it&#8217;s neat already.\u00a0 It definitely works in Firefox and Chrome.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>beets is the extensible music database tool every programmer with a music collection has dreamed of writing.\u00a0 At its simplest it&#8217;s a clever tagger that can normalize your music against the MusicBrainz database and then store the results in a &hellip; <a href=\"https:\/\/www.visophyte.org\/blog\/2014\/04\/06\/webpd-a-polymer-based-web-ui-for-the-beets-music-library-manager\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[119,13],"tags":[120,122,123,121,124],"class_list":["post-875","post","type-post","status-publish","format-standard","hentry","category-music","category-shiny","tag-beets","tag-d3-js","tag-echonest","tag-polymer","tag-webpd"],"_links":{"self":[{"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/posts\/875","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/comments?post=875"}],"version-history":[{"count":13,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/posts\/875\/revisions"}],"predecessor-version":[{"id":893,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/posts\/875\/revisions\/893"}],"wp:attachment":[{"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/media?parent=875"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/categories?post=875"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/tags?post=875"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}