Friday, October 10, 2008

Pandora App 1.2 Update

Pandora recently released their second version of their iPhone app, and many of the concerns I had with the original app have been addressed. I consider the feature set in the same order as I did in my review of the original app.

I was wrong about the original app not buffering songs. Both the original app and the updated version buffer the current song in its entirety as long as the network connection allows it to. Pandora delivers music well into the Oakland tunnel on BART, and there have been several times that a longer song has completed after we've emerged to the other side.

Pandora still does not display the originating station when playing songs via quickmix. I still want this feature. However, you can now edit the list of stations that will be played by quickmix. One minor niggle with this new feature is that it does not load the current selections, and so if you wish to add or subtract a station or two, you have to reselect all of the stations that you want in your quickmix.

The big news is that the individual stations are far more editable than they were in the Sprint version of the mobile player I used prior to the iPhone. The full ASUD (Artist, Songs, thumbs-Up, and thumbs-Down) are fetched when you edit a station, and you can delete any entry and add new artist and song seeds. Adding new seeds turns out to be quite reasonable with the iPhone keypad, and Pandora tries to auto-complete your entries via its search engine. That feature was well beyond my expectations at this point. The only real downside right now is that the lists are only sorted by recency with the least recent on the bottom. Thus, it's really difficult to cull the thumbs-up you've given a particular artist in a station. Nor can you edit any station description you may have written for a station page. Nevertheless, the editing features are a major leap forward.

You still cannot scroll back through previously played songs from the current session as you can in all other versions of the player to provide feedback. Thus you must still react while a song plays to provide feedback. You cannot add songs to either the lists of thumbed-up or thumbed-down stations which might have been a work-around for this issue.

The app does now remember which station you were playing and defaults to the most recently played station (including your quickmix).

The new app has some miscellanious new features as well. There is a new button allowing you to purchase a track from iTunes. I'd still rather buy DRM-free cd's, and so I'll probably never use that feature. You can now access and edit your bookmarks. I used to print out lists of my song bookmarks from my Pandora page when I went music shopping, and now the list is always in my pocket, and so this feature is actually useful to me. Lastly, there is a settings page that gives you the option to lower your sound quality to reduce any skips and hesitations. I find that all the skips and hesitations seem to be iPhone/ATT network problems. Several times a day I must toggle airport mode to restart the 3G. On the worst days I have to go twenty minutes without music while the iPhone tries to link up. Usually, however, it just takes a couple of seconds to reset. None of these issues are Pandora's fault, though, because Safari is equally blocked when the network is having issues. Thus, I keep the audio quality on high and do not have much us for this feature.

A final new "feature" is, inevitably, advertising. A small bar with advertising now appears above the feedback bar blocking out maybe a quarter inch of the album art. I have no objection to this feature. Pandora has kept ads out of the audio stream, and, further, from interfering with any of the controls. That fact is great - particularly in the face of the pressure they are facing from SoundExchange to monetize internet radio in the same fashion as terrestrial radio. I hope that they will be able to continue to hold the line against obtrusive advertising.

I also continue to pray that the royalty negotiations resolve successfully and that Pandora will be able to grow and prosper. The Pandora app 1.2 is another wonderful step forward, and I looking forward to future versions.

Friday, August 15, 2008

August Listening Test Results

I have just completed my August listening tests, and I must say that the current version of the song selection algorithm is MUCH more to my taste. Check out the charts:



With the exception of the new station, "Corny Schlocky Sappy Songs", (which is the title of one of many delightful Carla Ulbrich songs used as a seed for this station) all the ratings are up to new heights of satisfaction. Part of the reason appears to be that the selection algorithm seems to be doing a bit less exploration:



Note in particular that "Bright Electra of the Seven Sisters" had never played more than eight previously thumbed-up tracks out of forty. This month all forty songs played in the test had been previously thumbed up.

I even had a momentary concern that the stations might not be exploring enough new music, but I've concluded, no, this is pretty much exactly how I've always wished Pandora would perform. I feel much less need for any advanced controls at all under the current state of the selection algorithm.

Consider the seemingly impossible goal of building a novelty song station in Pandora. If we could filter on the focus trait "humorous lyrics", such a station could have be perfected shortly after its birth. However, "The Best Medicine" has performed terribly since I created it in Feb. '07 with satisfaction scores as low as 3/10. This month, however, it played 27 previously thumbed up tracks, and found four new humorous tracks for a satisfaction rating of 8.125/10. I only thumbed down 6 tracks which is significantly less weeding than usual.

Thus, you'll be hearing much less whining from me for advanced controls under the current selection algorithm. This version is approaching being the Pandora I've always wished it would become. In fact, Pandora could probably continue to push the selection algorithm in this direction, and I'd be even happier.

I've sent a version of this post as an e-mail to Tom Conrad to say how pleased I am with the changes.

Wednesday, August 6, 2008

Pandora App: One Week Later Or How I've Learned to Hate EDGE

After a week of use, I am still enjoying my iPhone and the Pandora app. However, it's become increasingly clear that all the woes I've encountered have occurred only when the phone slips out of wifi or 3G and into the EDGE network. It turns that the vast majority of RFI noise occurs when the iPhone receives data over EDGE. Furthermore, the streaming of music on EDGE can become unlistenably intermittent (two to five bars of music intersticed with silence of similar lengths broken only by the RFI bursts). I do not know whether the iPhone or the app is to blame for these problems, but I've learned to switch to the iPod app whenever the RFI noise appears.

Thus, I've ended up playing the iPod more than I've expected or wanted to. I now only play Pandora on the commute when I'm east of downtown Oakland, and I often have to switch the iPod on for lunch. You'd expect that San Francisco, of all places, would be thoroughly covered by the 3G network, but even sitting stationary at my desk, the iPhone can switch to EDGE. Thus, I am not sure whether it's a coverage issue or a capacity issue. Turning the iPhone off and rebooting when I'm at my desk seems to consistently recover the 3G connection, but doing so takes a half a minute or so of my precious Pandora time. Fortunately, I've only had to do so at most twice in one day and some days require no such fiddling at all.

I had one moment of panic this past Sunday when the iPhone got stuck in the favorites page. The home button would not take the phone back to the home page, nor would the phone turn off. Eventually, it white-screened, and I had visions of having to visit a genius bar. However, when I got home, I learned how to force a reset (press and hold the home and sleep buttons for ten seconds), and it's been fine since then.

Thursday, July 31, 2008

July Listening Test Results

I have not posted the updated charts for my station tests since September of last year. As you can see below, my satisfaction peaked towards the beginning of the year. There was a marked drop in May and June, and the ratings have come back quite well. The player seemed to deliver fewer previously thumb-up songs in June as can be seen in the second chart.

Once it became clear that the stations were exploring more than usual I wrote Tim and Tom. Unusually, neither relied (normally both or quite responsive but my e-mail would have been during their crunch time in creating the iPhone App), but as part of Tom's reply on my review of the iPhone app he confirmed my impression. The chart below shows the number of previously thumbed up tracks (out of 40 possible tracks in each test). As you can see, the player has recovered substantially and was exploring even less than usual this past month.

Of course, the right amount of exploration that the player does is a hard thing to determine. If the player went to only playing previously thumbed-up tracks, then by these tests my satisfaction ratings would go to 10 for each station, and the previously thumbed-up counts would all go to 40. There's no doubt that is exactly the behavior that some people want and even expect from the player. However, I do like hearing new music.

For me the ideal player would maximize my satisfaction while minimizing the previously thumbed up tracks. It would take an infinite amount of music for the player to be perfect under that criteria. However, from the drop in performance in May and June, it's clear that the selection algorithm can not tolerate much more exploration at this point. Additional exploration is particularly hard on stations which try to do something other than deliver genres which are easily identified by the genome (like my novelty song station, "The Best Medicine", for instance). Thus, given the current genome pushing the player towards less exploration is probably better (though unsatisfying). Of course, if we could tailor the exploration to each station, that would be best of all.

Monday, July 28, 2008

The Pandora App For The iPhone, A Complete and Utter Review

[Updated including feedback from Pandora's CTO, Tom Conrad (see end of article)]

So I managed to run my Sprint cellphone through the laundry two weeks ago. I immediately checked my options at Pandora.com to see what phones could now run mobile Pandora. The iPhone firmware 2.0 and the iPhone 3G had launched that Friday, and to my shock and delight the iPhone was suddenly an option. I would have to pay the penalty to get out of my Sprint contract. But, come on, Pandora was available for the iPhone! Of course, it helped that Sprint billing was terrible (they changed plans on me in the middle of the year without my authorization, upping the bill to over $100/mo which they reversed, but still..), and Sprint coverage did not, unfortunately, cover my house in the Oakland hills.

I ordered my 3G the following day at an ATT store two blocks from work. It arrived this past Friday, ten days after I placed my order. And so here's your full, in depth comparison to the mobile and Flash versions of Pandora.

Welcome to the World of iTunes

iTunes? What would iTunes have to do with Pandora? Well, if you want to download any apps for your iPhone (including Pandora) you'll need to have an iTunes account. It's free, but it does mean that you must download the iTunes software onto some computer, and create an account. If my recollection is correct, you could also use an AOL account.

The installation and set-up of Pandora was fast and easy after iTunes was set up. You have to key in your Pandora account which is whatever e-mail address you used to create your account. The iPhone touch keyboard works reasonably well, and certainly much easier than the usual cellphone text input.

The Hopes

The night I received my notification that the iPhone was winging its way from Texas I made a list of the features I hoped the app would have. I hoped that the app would buffer songs so that at least the current tune would complete when leaving coverage. (BART goes under ground through downtown Oakland and under the Bay which would cause the mobile version to immediately stop playing). I hoped that when playing Pandora's quickmix, the station which generated the song would be displayed. I hoped that the station web pages would be accessible and editable while the music played. In short, I hoped that the iPhone app would be closer to the Flash version of the player rather than the mobile version of the player.

The Reality

As it turns out, the iPhone app has even fewer features than the mobile version, and absolutely none of those hopes were realized. There is no buffering like the mobile version. [ETA: Strangely, this morning Pandora continued through the 19th St. Oakland BART station whereas Sprint never came close to that. I do not know whether is a matter of buffering or coverage. Sprint seems to do better around West Oakland, but ATT seems to perform better in some other locations.] Quickmix does not display the station, and you cannot access the Pandora station pages even if you happen to know the direct URL.

In all previous versions of the player you can scroll back through the songs that have played in the current listening session and assign feedback (thumbs up or down) to those songs. That feature is not available in the app for the iPhone. Thus, you can only give a song feedback while the song is playing. It is unclear whether thumbing down a song in the app initializes a new set, but it has been unclear for the past half year whether songs are still being generated in sets of three or four even on the Flash player.

Furthermore, the app does not remember which station you were playing on start up. Every time the program initializes, you must select a station. However, after the program initializes, you can exit to go to the iPhone homepage which will stop the music, but the app will then continue the song and the station when you return. This feature is new and superior to the mobile version which had to be re-initialized whenever you exited it.

There is no way to add new artists or songs to a station in the iPhone app. This feature is another one which is available on both the Flash and mobile version of the player, but is lacking on the iPhone app. However, to be fair, the feature was pretty much useless in the mobile version since inputing text was difficult. I never used that feature because of that difficulty; however, I might have on the iPhone since inputting text is easier with the touch interface.

In all other respects, the Pandora app does what you would expect it to in comparison to the mobile and Flash versions. Selecting stations is MUCH easier using the touch interface than either of previous versions of the player. It turns out that sweeping your finger across the screen is a great way to scroll through a list. On the other hand, adjusting the volume with the app's slider is a nightmare, but the volume adjustment on the side of the iPhone is available and much more effective.

The Sound

The audio quality of the iPhone is superior to that of my old M1 as one might hope given its iPod lineage. The highs are a bit brittle, but certainly less sibilant than the M1. The M1 still had, strangely, the wider and deeper sound-stage. The iPhone jack is 3.5mm rather than 2.5mm, but it is a TRRS jack rather than the TRS jack used by most headphones. The extra ring supports a microphone, and so I'll need to get an adapter to use my other headphones. The iPhone puts out a heck of lot of radio interference which apparently can induce noise in head amps. I've already heard that noise in my GoVibe 4.0. The noise also makes the CRT on my home computer flicker a bit if the iPhone is near and roughly pointed toward the screen.

And so I purchased the $10 Griffin iPhone headphone adapter. I had heard rumors on the head-fi.org forums that the iPhone puts out a lot of radio-frequency interference (RFI, as they say). Indeed, my CRT at home jumps around a bit when the iPhone is near pointed at it; however, my iPhone sits happily in the same pocket as my GoVibe v4 headamp with no discernible noise. (Although I did get strange electronic bleeping coming up the stairs at the BART station this morning which I never had before.) The iPhone sound is distinctly better than my old M1 on the same rig and headphones. Furthermore, I was able to use the iPhone as a source for my home stereo with a simple 3.5 mm to RCA cable which is was never able to do with the Sprint phone because of the lack of coverage. The sound was quite good.

[Update on the RFI. When I did my usual walk about for lunch today, I got a bit more of the RFI. It's not loud or unpleasent. It's sort of an electronic buzzing chatter. Pulling the iPhone away from the headamp definitely eliminates it. It's intermittent, only tends to occur when you're moving around, and only occurs when you are using a head-amp. You can stop it immediately by moving the iPhone away from the amp or plugging the headphones directly into the adapter. I still think the rig sounds better through the head-amp, and so I plan to live with it.]

The Interface

The design aesthetic of the app is that of the iPod interface implemented on the iPhone rather than that of the mobile or Flash versions of the Pandora player. The huge album art is lovely to see, and it's much more easy to see who the artist is by glancing at the cover than trying to read the tiny font listing that information on both the mobile and iPhone apps. Songs for which there is no available album art get a white square with a couple of gray eighth-notes just like any song not purachased from iTunes playing on the iPod app. (Of the three default images, I guess I like the gradient blue field of the mobile player the best.) The icon indicating that a song has already been thumbed up is tiny and weirdly placed beside the button which calls up the focus traits for the song. There is a cool animation of the album flipping over when you press that button, however.

Switching to the iPod and back during the tunnels of my commute was reasonable, but a bit cumbersome. Going to the iPod requires clicking the home button, and then three selection touches. Going from there to Pandora requires essentially the some number of inputs, but you have to wait for the app to load before a mandatory selection of the source if there are any wifi signals around you. Neither the mobile or the Flash version of the player require you to select a station at start-up since they remember the last station you were playing, but you must select a station when switching back to Pandora on the iPhone since that information is not retained nor is a there a default station. (To be fair, the iPod app does not remember what playlist you were last playing or have a default playlist either.)

Speaking of wifi, it should also be noted that the iPhone manages its connections to the internet far better than my old cellphone. As I've said, Sprint missed covering my house by about half a block. The ATT signal is quite strong at home, but that does not even matter because the iPhone connects to my home wifi the moment I step in the door.

Conclusions

And so, given that the Pandora app has fewer features than the mobile app, am I disappointed in moving to the iPhone? Not at all. Pandora still sounds and works great on the iPhone. Furthermore, the interface and display on the iPhone in general is substantially better than most cellphones. I could play Mp3's on my old M1 but I never did because it was comparatively difficult to move music onto it and switch in and out of the apps. The iPhone does chain you to the world of iTunes, but within those fetters moving music to the iPhone and switching between apps is relatively effortless and intuitive.

Thus, as a Pandora power user I strongly recommend the iPhone app despite the missing features. It's a true pleasure to use. That being said, I do hope that Pandora does continue to develop the app and incorporate more of the mobile and Flash features over time. I also hope that Pandora receives a cut of the monthly fess going to ATT and Apple since the iPhone app does not require a subscription to Pandora which the mobile version did.

Update

I sent the link to this article to Tom Conrad and Tim Westergren. Tom acknowledged the lac of features, but noted that the app had been put together in five weeks. Five weeks. In that case, the app is truly amazing. Tom also mentioned a number of features they are planning to add in forthcoming versions of the app. (Imagine what they'll have after, say, ten weeks of development!) I will not include that list here because they should not be held to promises made to a completely obscure blogger; however, I will pull the following quote:
Rest assured over time this will become the best on-the-go Pandora listening experience there is.

Whooooo!

I will also note some observations about the player after a day of typical use at work. The app seems more stable than the mobile player on the M1. I typically had to restart the mobile version two to four times per day (after providing feedback usually). Perhaps, providing feedback on previous played songs is trickier and caused network problems. In any case, the iPhone app required no rebooting yesterday at all. The M1 phone would also need to be turned off and restarted entirely maybe once every couple of months indicating, perhaps, a small memory leak. That behavior, of course, is not possible to detect for my iPhone usage to date, but early indications are that the current app is quite stable.

On the other hand, there are times during the day when there are noticeably longer gaps between songs on the Sprint network than on the ATT 3G network. Whether this an app issue or a network issue, I cannot tell. Nevertheless, there are times when the Pandora app fails to provide music for up to a minute.

Wednesday, January 2, 2008

The Similar Albums Feature

Every Backstage album page has a list of similar albums. The feature does not work all that well, and, of course, there's no real documentation on how its generated. I can make some guesses, however.

First, we know that some sort of averaging of the genomes is done at least one place in Pandora. We know that the list of thumbed-up tracks for any station is averaged to create a single seed. I suspect that the averaging is done as simply as possible. I suspect that the genomes are, in the end, long binary strings: does this song have this quality (yes/no). [Pandora is currently playing DeGarmo's "Boy Like You" just to taunt me right now.] I suspect that when Pandora needs an average, it simply takes the straight numerical average of this binary string.

Second, the secret-sauce of Pandora is the distance metrics that it has developed on the genomes to assess "nearness". The creation of the metric on a binary vector space will have been done with a bit of statistics and a lot of tweaking. I suspect that all the genomes have been adjusted to have the same length with a lot of entries zeroed out for attributes which were not assessed at the time of analysis. (That is, when analyzing a harpsichord performance of a Scarlatti sonata the question of whether the track has "trip-hop roots" will not be considered.) Distance metrics would then be constructed on that binary state-space so that nearness can be measured across genomes.

The metric (if all the previous speculation is true) would be a simple linear vector in the n-dimensional state space defined by the n-digit uber-genome which essentially supplies weights indicating which characteristics (like tempo) are more important than other characteristics (like the presence or absence of "thin synth textures").

There might only be one such distance metric used throughout Pandora and there might be several. The most important one is the one which selects songs. I suspect that they calculate a list of a hundred or so nearest neighbors for each song in a database, and that the Player picks a track and then a random set of three other songs from that song's list to make a set and then applies all sorts of mandated constraints replacing tracks as necessary.

I would imagine that the lists of similar artists and similar albums are generated in a similar fashion to the algorithm used to generate the play-lists. Each Artist and and Album will have been reduced to an average of their respective tracks, and the distance between albums and artist may be computed using a different distance metric than used in the player, but the nearest-neighbor algorithm applied will likely be the same.

The big difference between the Player and the Backstage lists is, I suspect, that the Player databases are updated frequently and the Backstage lists are probably generated when the album or the artist is initially published to the Backstage database and are likely never updated. They might update the similar artists lists whenever a new album for that artist is analyzed, but I've seen no evidence that they do (not that I've observed the artist lists that closely over time). Thus, there might be "nearer" artists now available on the database, but the pages are never updated to reflect that fact.

Of course, the real problem with this method is the initial averaging of all the tracks for the artist or album. A diverse artist (or album) gets mushed to a single point in space where the very fact that they are diverse is probably a key characteristic in many people minds when comparing artists. Tom Conrad is pretty smart, and so they probably could measure the width of the footprint of the tracks in n-space via some second-order statistics beyond simple averages and use that in their nearness calculation, but the fact is that I don't think anyone cares all that much about the similar artist and album lists.

Monday, December 10, 2007

Assessing the Development of a Station: ASUD

Nowadays, the inputs that are used by a player for a particular station are almost entirely public. As long as, the person who created the station has not set their profile to private, you can read the lists of artists, song seeds, thumbed up tracks and thumbed down tracks.

Back in the early days, a few of us gathered at pandora-forums.com (now defunct and cybersquated). We came up with the idea of measuring the development of a station as a list of the number of Artist seeds, Song seeds, Ups and Downs (ASUD). Tim, IIRC, when he created the first version of pandorastations.com proposed a measure D which was equal to the sum of the numbers of Ups and Downs, but the rest of us had already grown used to ASUD. And I think it's still a useful place to start.

Artists It's good to keep the number of artists in the station separate from the other numbers because the number of tracks called upon by an artist seed is the one thing about a station which can change without the user's input. Pandora analyzes and makes available for play new tracks by an artist on an on-going basis. The number of tracks available for popular artist who is still recording will always tend to increase over time. The number is more static for more obscure artists. (Interestingly, the founder of Pandora, Tim Westergren, was in the band YellowWood Junction which has exactly three songs available for play and that number has not changed in the past year at least.)

Songs Each song which is added to the list of song seeds for a station becomes available to form the basis for a set of four songs to play on the player. As similar songs are added to the list the station generally becomes more and more focused. Thus, because the number of tracks represented by an artist seed is fairly random, dynamic and difficult to assess, the number of songs seeds is probably the second most important measure of a station's development (after the number of thumbs down).

Ups The entire list of thumbed up songs serves as a single possible basis for a set. I used to believe that all the thumbed up tracks could individually form the basis for a set, but the Pandora FAQ now definitively states that that is not the case. Thus, the number of Ups gives a good idea of how much the creator has listened to a station, but it's the least important measure of the development of a station.

Downs On the other hand, the number of thumbed down tracks is an absolutely crucial indication of how well developed a station is. No thumbed down track will ever play on the station again, and, as long as no track by an artist appears in any of the other three lists (artist seeds, song seeds, or thumbed up tracks), two Downs ban all the tracks by that artist from the station. Thus, unless you are consciously moving tracks into the list of song seeds, cutting out songs one by one via the thumbs-down is the most usual way to incrementally improve a station.

The best measure of a station's development would be a count of all the songs which can be the basis for a set and a count of all the tracks that cannot play on that station. Unfortunately, those two numbers are difficult to calculate and until they become readily available ASUD is the best surrogate we have.