Node-debug - better node.js debugging GUI

I often find myself working on node apps, and missing the robust runtime debugging that I get when working on Objective-C apps.

Node has some pretty solid debugging tools built right in, but they’re not the easiest to use.

There’s a fantastic project called node-inspector that solves most of the issues around the missing debug UI, but there are a few things that I find unwieldy about it. The basic way to get a debugger up with node-inspector would be the following:

  1. Open a terminal window, fire up node-inspector.
  2. Go back to what you’re working on, and start your app using the -debug flag.
  3. Copy paste the url output from node-inspector, open up a browser window, paste it in.
  4. Do your debugging
  5. Close your browser window, kill & close the node-inspector if you’re done debugging for the day.

I’m only interested in step 4 - so I figured I’d write some shell script wrappers to remove the other steps.

The result of that work is a simple bash script, packaged into an npm module. Here’s the gist, for your forking pleasure:

This script allows you to run:

 node-debug somescript.js


 node-debug -brk somescript.js

Both commands will start an inspector session, background it, start your script, and open an incognito Chrome window pointed to the inspector. When you ^C your app, it traps the exit and closes the opened window (along with any other tabs on that same window), and kills the inspector.

node-debug is available on npm, and only compatible with OSX and Chrome at the moment, as it relies on some pretty hard coded osascript stuff. However, if your dev box looks like mine, you can just:

 npm install -g node-debug-osx node-inspector

and node-debug will be available in your shell. Note that node-inspector & Google Chrome must both already be installed.

Note that there is another library on npm called node-debug, which is a less full featured wrapper around the same node-inspector. You’ll need to uninstall that if you’ve got it on your box, or just clone this repo and npm link it to another name.

As always, feature requests & forks are welcome. I’ll be adding to this little script when more robustness becomes necessary in my workflow!

A democratic music player for offices

I’ve been playing a lot of music in the We Heart It office lately.

I usually just put on my or add stuff to a Spotify playlist as I go - however, I have a wide range of music taste, some of which is not office appropriate. I’ve gotten a reputation for playing music nobody has heard of, and sometimes songs come on that people don’t like. I’m more than happy to skip them, but usually people are too nice to let me know that they’re not digging my current song.

We use Heartbot, a Hubot clone in our campfire, so I added a quick script this afternoon to allow people to ask heartbot what I’m playing (it just does some regexing on my now page) . This is cool, but didn’t really solve the issue of people not liking the music, so I decided to write something a little more complex to allow people to skip the current song if they’re not into it.

Enter remote-skip, a package of scripts & servers that allows people to request a skip from campfire. Here’s how it works:

  • A user clones the repo (no fork needed), and sets up a heroku server for it (or connects it to an existing one).
  • The user optionally pushes the repo to heroku, which runs a server that listens for clients and handles requests from heartbot.
  • The user then runs a local server, which proxies via ngrok to a public url. This server also pings the heroku server to let it know that the client is ready to accept skip requests.
  • When heartbot hits the skip url, it then sends requests to all of the connected clients asking them to skip.
  • When a client receives a skip request, it pops up a dialog asking the user if they’d like to accept or deny the skip request.
  • If the request is approved, the client uses node-applescript to select the skip menu item from any running music players.
  • Once a decision is made, heartbot reports back to the user wether or not the skip was approved.

Here’s what that process looks like in practice:

It currently supports, Spotify and iTunes, but can be easily extended to support any application.

Like any good democracy, the person playing the music has veto power if they really want to hear the song, or if someone is just being annoying in campfire. However, the dialog expires and the song is skipped automatically after 15 seconds, in case I’m away from my desk and some At The Gates comes on.

So far it’s working smoothly - because the remote server can support multiple clients, any number of people can point to the same remote app, and it will send skip requests to any connected clients. If a client disconnects or restarts, the server automatically drops old clients after 30 seconds, so it’s not necessary to babysit it, just start the server and forget.

Here’s the repo for remote-skip:

Give it a whirl, setup only takes a few minutes (including adding the script to your hubot), and it’ll work with any client - so in theory, you could arduino a big red skip button and stick it at someone’s desk - it’s just HTTP, so anything that can speak internet can now skip your music!

In addition, this combination of a publicly exposed local applescript runner has all sorts of cool application. Nearly anything on your mac can be automated remotely using the same concepts - I’d love to hear any cool ideas you think of, or fork away and give your hubot a little more power.

Questions? Stick them in the comments or file a github issue, and I’ll do my best to help get your office music setup better for everyone.

Clean(ish) mobile web deep linking for iOS and Android

Recently I needed to redirect a mobile web version of an app to an installed app.

This proved somewhat challenging, as there's no standard for checking if a user has an app installed. You can of course just redirect to a custom scheme, but that would be a pretty crappy experience for users who actually want to use the mobile web version of the site, or don't have the app installed.

Apple and Android each have their own distinctive way to handle custom schemes, which I won't be covering in this article. There's a wealth of resources on these topics, and as always, a good place to start is the docs:

For Android, these are handled as intent filters with the BROWSABLE category in the AndroidManifest.xml file. Here's the docs for intent filters.
On iOS, they are handled in your app's Info.plist file. Here's the docs for implementing custom URL schemes.

This article is about trying to create a smooth transition from mobile web to app, without breaking any existing user expectations and minimizing weirdness.


If you're just looking for some copy pasta, here's a scoped function that you can drop in, change the variables at the top, and open your custom scheme.

// success/failure functions
var success = function(){
// found the app
alert('Redirected to the app')
var failure = function(){
// app not installed
alert('Failed to find app')
var cookieName = "myapp_install_check"
var domainRegex = /myhostname/i
var scheme = "myapp"
if(domainRegex.test(document.referrer) && new RegExp(cookieName + '=','i').test(document.cookie)) return
var __onload = win.onload
var done = function(found){
doc.cookie = cookieName + '=yes;path=/;'
;(found ? success : failure)()
win.onload = function(){
var iframe = doc.createElement("iframe") = "none" = "1px" = "1px"
var t = setTimeout(function() {
}, 1000);
iframe.onload = function () { clearTimeout(t); done(true) }
iframe.src = scheme + '://' + (doc.location.pathname.replace(/^\//,'') || '/')
if(__onload) __onload.apply(this,arguments)

  • replace myapp_install_check with a name of a cookie to use to not redirect a user we've already tried to redirect until they open a new window.

  • replace myhostname with your hostname. This will be something like mysite\.com

  • replace myapp with your custom scheme - something you've set in your mobile app to respond to, minus the ://.

  • add any code that you'd like to execute if the user has the app (and we're doing a redirect to it) to the success function. For instance, if you don't want to leave the window open, this would be a good place for a window.close().

  • add any code that you want to execute if the user does not have the app installed to the failure function. For instance, this would be a good place to show a popup asking the user to install your app.

If you're interested in a more customizable piece of code that can handle multiple schemes, or what this is doing, read on.


Very little backend is necessary for this - however, because this is a hardware based change, it's one of the few cases where I'd consider it an OK practice to use a user-agent header to include or omit a javascript snippet. In my case, we already have UA detection in place, something like this should do the trick (this is untested node pseudocode, but you can do this in any backend):
var ua = request.headers['User-Agent']
var mobile = /(ip(hone|od|ad)|android)/i.test(ua)
// mobile is now true for devices.

If you felt like it, you could also do this check on the client by testing window.navigator.userAgent.

Client Side

To start, we'll add a new window.onload handler to do stuff when we load the page:
var __onload = window.onload
window.onload = function(){
// this is where we'll attempt to open the app.
if(__onload) __onload.apply(this,arguments)

Next, we'll want to write a handler that will prevent this script from running more than once on the same session. In my case, I'm defining a session as an open tab on a browser. I deal with this by enclosing my script in a Self Executing Anonymous Function (SEAF), and running a cookie check before we add the onload handler to make sure we don't add it more than once in the same session. We'll also check to see if the user is visiting from either a non-local domain, or an empty domain. If either of those are true, this is a new session, so we'll want to try to redirect it to the app.

To achieve these things, I'll need a quick method for manipulating cookies, and a check. Here's what our code looks like now:

  var cookieName = 'my_app_tried_redirect'
  var hostRegex = /myhostname/
  // cookie reading method
  var readCookie = function(){
    var parts = document.cookie.split(';')
    var i=0,len=parts.length
      var cName = cookieName + '='
      var part = parts[i]
      // trim leading whitespace
      while(part.charAt(0) == ' ') part = part.substring(1,part.length)
      if(part.indexOf(cName)) return part.substring(cName.length,part.length)
    return null
  // don't fire this twice in the same session
  if(hostRegex.test(document.referrer) && readCookie()) return

  // redirect on load
  var __onload = window.onload
    window.onload = function(){
      // this is where we'll do android specific stuff
    // this is where we'll do stuff that works elsewhere
    if(__onload) __onload.apply(this,arguments)

Now, we'll want to write a function that we'll call once we've completed a check for the app. This will set the cookie and perform any actions that you want to do depending on if the user has the app installed or not. This will look something like this:

var appFound
var complete = function(found){
  appFound = found ? true : appFound
  // set a cookie so we don't redirect again in this session
  doc.cookie = cookieName + '=yes;path=/;'
    // user has app installed
  } else {
    // user does not have app

Put that somewhere in your SEAF. Now we'll need to write something to call that function with a true or false callback. To try to detect if the user has the app, we'll inject an iFrame pointing to a custom scheme url. Depending on if we get an onload callback from the frame, we'll know if the app successfully opened or not. In case we want to do this with multiple schemes, we'll put it in a function:

var injectiFrame = function(path,callback){
  var iframe = document.createElement("iframe") = "none" = "1px" = "1px"
  var t = setTimeout(function() {
  }, 1000);
  iframe.onload = function () { clearTimeout(t); callback(true) }
  iframe.src = path

Basically after 1 second, if the frame hasn't reported success, we assume failure. This approach is cleaner than others because it's mostly invisible to the user (outside of an iOS caveat), and will leave them alone to continue their browsing session (or for instance, you could pop up a dismissable "install the app!" banner) without further disturbance.

We'll now add calls to our various schemes inside the previously created window.onload handler. This is what that looks like:

  // redirect on load
  var __onload = window.onload
    window.onload = function(){
    // attempt to open a custom scheme url
    injectiFrame('yourcustomscheme://' + (document.location.pathname.replace(/^\//,'') || 'home'),complete)
    if(__onload) __onload.apply(this,arguments)

You'll notice I'm passing the pathname (sans leading slash) over to the app. If you handle similar urls in both the app and the mobile site, you should be able to pick up where the user left off in your app by responding to the same paths.

Lastly, I'll want to make sure I don't run the appFound conditional block more than once, as we could potentially annoy the user with multiple popups or similar if we try multiple schemes. So, I'll add a global appChecks variable, and increment it whenever I call injectIFrame. Then, I'll add a line to the top of the complete function:

if(--appChecks > 0) return

This will skip the body of complete when we still have one or more outstanding iFrames to load.

Here's an example of the full code to do a clean redirect to an app:

  // local vars
  var cookieName = 'my_app_tried_redirect'
  var domainRegex = /myhostname/
  var scheme = "mycustomscheme"
  var appFound = false
  var appChecks = 0

  // functions
  var injectiFrame = function(path,callback){
    var iframe = doc.createElement("iframe") = "none" = "1px" = "1px"
    var t = setTimeout(function() {
    }, 1000);
    iframe.onload = function () { clearTimeout(t); callback(true) }
    iframe.src = path
  var complete = function(found){
    appFound = found ? true : appFound
    // wait for all checks to complete
    if(--appChecks > 0) return
    // completed all checks
    // set a cookie so we don't redirect again in this session
    doc.cookie = cookieName + '=yes;path=/;'
      // user has app installed
    } else {
      // user does not have app
  var readCookie = function(name) {
    var nameEQ = name + "="
    var ca = document.cookie.split(';')
    for(var i = 0; i < ca.length; i++) {
        var c = ca[i]
        while (c.charAt(0) == ' ') c = c.substring(1, c.length)
        if (c.indexOf(nameEQ) == 0) return c.substring(nameEQ.length, c.length)
    return null
  // don't fire this twice in the same session
  if(domainRegex.test(document.referrer) &amp;&amp; readCookie(cookieName)) return
  // add some iframes to attempt to pop open the app
  var __onload = win.onload
  win.onload = function(){
    injectiFrame(scheme + '://' + (doc.location.pathname.replace(/^\//,'') || 'home'),complete)
    if(__onload) __onload.apply(this,arguments)


This should be somewhat straightforward and easily modifiable for your needs. The one caveat I know of using this approach is that when used in iOS and the user does not have the app installed, the user will get a popup saying "Cannot Open Page - Safari cannot open the page because the address is invalid." The only option the user has is to dismiss the message, and once they do, they'll be able to continue their session uninterrupted. This can be confusing for some users though, and I would expect a certain amount of CS issues associated with it. This could be treated by using a popup when the user doesn't have the app addressing the fact that they just saw that notice. Some apps, like quora, just don't acknowledge that you got the message at all, but prompt you to download the app. This is annoying, but unfortunately unavoidable in iOS as of this writing.

Governments building scalable websites

I recently was asked via email why government can't build a scalable website. The question was prompted by this doomsaying article about the new insurance programs that are being taken to the cloud in Oregon:

Here's my take:

I can't say anything too concrete about the challenges here, as I haven't seen the backend or talked to the architects, but I'll do my best to help shed some light on it:

There are 3 things about this problem that make it difficult to solve. One is an engineering challenge, another is an interoperability challenge, and the other is a management challenge. I have little faith in the Federal government to solve any of these problems gracefully, but here are my thoughts on them.

On a technical (engineering) side, probably the most difficult (if straightforward) challenge is to retrofit old technology meant for a low amount of concurrency to work at scale. A simple way to think about it is in the context of a basic database. Traditionally, these systems would talk to a single machine, and all the updates are atomic, in that updates need to be made based on information that is contingent on other possible updates. (For instance, let's pretend a certain health care plan's availability is dependent on the number of people in it. By adding a person, you could in effect change that availability.) While this works well when you have a few hundred employees talking to the machine, once you start to scale things start to become less operable. You both introduce the possibility of data inconsistency for the clients, and start to see the system slow down as it "locks" the database for each update to guarantee atomicity. This type of problem can't be solved by simply adding more of the same machines, because at some level they all rely on the same guarantee. So the options you have are to change the fundamental architecture to be able to scale horizontally (where each new machine can add equal value), or you scale it vertically (you make the single machine faster, & bigger, the cost of doing so increasing exponentially with each upgrade). Any new systems also need to be horizontally scalable, to be able to just "add more machines", but even most modern architecture is not the best at making this possible. Something that works well for a small test group can easily be fundamentally broken at scale.

All that said, 4mm uniques is not especially difficult to handle. The quotes in the article inspire little faith in the actual understanding of user behavior, but the truth is that this site probably won't see that much traffic in the first 2 months, much less on the first day. The demographic expected to log on is not one that currently rarely has a personal computer, they're all mobile or shared. Also the program is not targeted for every person it Oregon, it's for people below the poverty level (I certainly hope Oregon doesn't have 3.9MM people in poverty...) Drive bys (the "I'm just checking it out" people mentioned in the article) don't normally break sites built to scale (at least not for an extended period of time). For some perspective, right this second my servers, which are backed by a single vertically scaled MySQL database, are serving about 21k concurrent uniques. This month we'll see well over 20mm uniques, and we'll probably have about 5 minutes total of downtime. Most months, none of those users will experience a single bug related to scaling. This is not rocket science, we have 2 people responsible for making sure we continue to scale.

The interoperability challenge is a bit trickier. The concern here is that the bottleneck isn't the controllable hardware, it's the external APIs (experian, etc) that can't be touched. If they start falling down or haven't been tested at scale, the client can't do anything about it. Of course it's entirely possible to build with this in mind, but if the right care hasn't been taken, this can cause issues. That said, some of the mentioned external APIs are already running at scale. Like I said before, 4mm is not a big number, they won't see that much traffic, and if they do,it will never happen all at the same time.

The last challenge is the one that will cause all of the problems. Government has a reputation for poor management, and engineers have a reputation for bad estimates and over engineering. The article quotes make it sound like they don't know what their performance expectations are, and that they expect failure. If their management acts the same way, they will get very little done, and the FUD will spread through the organization. Solving a wide problem like this means they need to have a well oiled machine that is constantly communicating to make sure the little pieces are fitting in to the big puzzle, and that it's getting done on time and in budget. Over management will guarantee failure regardless of the skills of the engineers. In addition, the management must also make the correct hiring decisions, and frankly the best minds in tech don't work in government. There are some amazing charitable engineers that spend time there, but primarily I would expect this to be a situation where the blind is leading the blind. If you were starting a project to do work that was somewhat unenviable and you had no one in the organization that could tell you the difference between a person up to the challenge and a person who will write bad code and screw you (despite their best intentions), how would you hire a person to build that team? Who would they hire? If the first people in can't vett the next person properly, your chances of building a strong team get smaller with each new hire.

To make this more difficult, this is the worst time in history to find tech talent. Why would I work on an antiquated system that is worried about scaling to 4mm users and can't pay me well or offer me any vested share in the project when I can work for 5-10x the salary reaching 10x more people and if I do a good job and stick around for a year, I'll have vested shares in a company potentially worth billions, which is millions in my pocket just for showing up. It's not exactly a dream job, and probably won't attract dream applicants.

So I guess the short answer is that no, it's not hard to build this. It's probably pretty easy if you could approach it with the right team and the ability to do whatever you want. But there's red tape. There's old systems. There's outside APIs that you need to cover for. And there's the trenches of the project where you can't see the big picture because you're behind schedule, over budget, and you're expecting your work to fail (also, really? They will go on record saying their product sucks?!?), so it probably will.

Setting up heroku-like git push deployment

I often find myself spinning up projects that I want to continuously deploy to a production-like environment. One of the easier ways to deploy code is to add an origin and push to it to deploy. It's already in your workflow, so a setup like heroku makes a lot of sense.

However, I'm often doing things on the server that require the ability to jump on the box and mess with it. I also want to be able to host this anywhere I like, and not deal with dynos going to sleep or paying per-dyno on my projects.

A simple way to set this up is to set up a git server on your box, and push to it. Here's how to do that:


  • git-core - probably already on your box, but just in case it's not, you'll (obviously) need it.
  • whatever other shit you need to run your server. In my case it's usually ruby or node.
  • some understanding of bash scripts. I'll have examples, but starting and stopping your app is usually somewhat app specific.

Setting up your server

First, make a directory where your repo will live. Note that this is just the git resources needed to push to the box, not the app itself. You'll put that somewhere else. I have a repos folder in my home that I stick all my repos in.

 mkdir -p ~/repos/my-app
 cd ~/repos/my-app
 git init --bare

Next set up a folder for your app. You can put it anywhere you like, as long as your user has access to it:

 mkdir -p ~/apps/my-app

Now you have a shiny new git repo set up! You'll want this repo to do something when pushed to, so you'll need to add a post-receive hook. You can find your hooks in the my-app/hooks directory:

 vi ~/repos/my-app/hooks/post-receive

Post receive should checkout the latest over your app on the server when you push, so add something similar to this:

 export GIT_WORK_TREE=/home//apps/my-app
 git checkout -f

This exports the GIT_WORK_TREE variable. Git will perform operations in this directory regardless of where your user is.
Then, it does a forced checkout. This will overwrite whatever is in my-app with whatever you push to this git repo.

You'll need this file to be executable by the pushing user, so +x it:

 chmod +x ~/repos/my-app/hooks/post-receive

Now you just need to add it as a remote to your local repo. Hop off your box and go back to localhost. In your git app, add the repo as a remote:

 git remote add staging ssh://

The first time you push to this, you'll want to specify the ref:

 git push staging +master:refs/heads/master

Now on subsequent pushes, you can just

 git push staging

And your server will be updated.

Extra stuff:

I usually need something to be restarted when I update code, and it's likely you do too. In my case, I usually add a scripts folder to my repo with start & stop or restart scripts in it. They'll look something like this:

 PID=`pgrep -fl node server.js`
 if [ -n $PID ]
      echo "stopping services : ${PID}"
      echo $PID | awk '{ print $1 }' | xargs kill
      sleep 2
 echo "starting server…"
 nohup node server.js 2>&amp;1 &amp;
 exit 0

This script just looks for running process that were started with node server.js, then if it finds any, pipes the pgrep output to awk, which will split the arguments by spaces (they look something like 1181 nohup node server.js), and print the first one (the pid of the server). It then pipes those to kill, which shuts down the service.

It then waits for 2 seconds (an arbitrary shutdown time), then starts the server again, redirecting stderr to stdout, and using nohup, which will write all logs to nohup.out in the app's root directory.

This is obviously a very naive way of restarting a server, but is sufficient for boxes you don't care if are up or down and are prepared to manually troubleshoot (like testing servers).

Once you have a script like this, you can add to the servers' post-receive hook, and your apps will automatically restart when you push:

 echo "/bin/bash /home//apps/my-app/scripts/ &amp;&amp; exit 0" >> ~/repos/my-app/hooks/post-receive

This also has the advantage of allowing you to change your restart script with a deploy as well, so as long as your stop script works, you won't have to jump on your box very often.

Auto mounting EBS Volumes to an EC2 Instance

Yesterday I was looking through an EC2 setup that I had made a while ago. I needed to automatically mount an EBS volume at boot, so that the services running on the box could talk to a partition without mounting it, regardless of if I rebooted the drive or not. A quick google search got me here:

However, as mentioned in a comment, the absence or another failure of the EBS volume would render the ec2 instance un-bootable, as mentioned in an amazon thread here:

The dev mentioned adding an rc script - for my own reference, here's how I went about that:

First, if you haven't done so already, log in to, and create a new EBS volume (under your EC2 settings panel).

Then, attach the volume to the EC2 server you want to have it available in. You can do this by checking the box to the left of the drive, and selecting "Attach Volumes" under "Actions".

Take note of the attachment used. In my case, it was attached to /dev/sdf.

Next, ssh up to the EC2 server, and verify that the drive is attached.

$ ls -al /dev/

In my case, the drive wasn't actually called `sdf`, but instead was named `xvdf`. Turns out this is a known issue:

That works for me, I can just mount xvdf instead. To do that, edit the `/etc/rc.local` file:

$ sudo vi /etc/rc.local

Then, add the mount command. In my case, I wanted to mount the drive to `/images`:

`mount /dev/xvdf /images`

`exit 0`

If you want the drive without a restart, you can execute the command directly to get it mounted right now:

$ mount /dev/xvdf /images

Primavera 2013

This year I headed to Primavera festival in Barcelona. I managed to snap (bad) pictures of nearly all the bands I saw, I decided I'd put up my bad photos and some thoughts about the bands I saw. I doubt this will be entertaining for anyone but myself, but as I have a bad memory, it'll be good to have to look back on later.


We were still pretty jetlagged on Wednesday, but managed to make it down for a few acts. Only 1 stage was open, so only a few bands played. It was a nice warmup though for the weekend:


We managed to get to the venue to catch Guards close up their set, walking up to the beautiful stage for one song. I was a bit distracted by taking in the new venue, but we heard them play their single (Do it again), which was ok. I've seen them before and they are really into that thing where they try to get the crowd really riled up by being showmen. I find it really annoying, so I didn't love it. They sound like the album, and I couldn't ask for much more out of them though.

The Vaccines

The place was still pretty empty when Vaccines played, but they played an awesome hits-only set, and did a great job of porting their almost garage sounding music to the stage for a nearly brit-rock stadium show. Lots of foot-on-the monitor shouting hooks makes this show fantastic. Wetsuit was my favorite they played, a bit faster than the album and perfectly executed.


We just saw Delorean play in San Francisco, and I was excited to see them play an outdoor set to their home audience in Catalonia. However, the lead singer seemed to not really care, and only spoke a few words in english. The rest of the band seemed excited, but overall this was a bit of a disappointment - Delorean's lead singer is always a bit off key (in an endearing way, it's similar on the album), but in this case it almost seemed like it was because they weren't that into closing the first day. It was danceable, but I expected more.


This festival is unlike any other I've been to because it starts so late and runs so late. We slept in late and got some bikinis and cafe con leches, took the awesome transit down, and still showed up an hour or so early for the music. After walking around the huge venue to try to get a layout worked out, we sat down for some crappy venue food and some early beers and adjusted my planned lineup for the day for the size of the venue.

One of the main challenges at this festival was the fact that it took 5-15 minutes to get from one side to the other. This meant I had to make some concessions on split sets to adjust for the long walk times between sets, and we could split a little less than other festivals like Coachella or Outside Lands. The lack of sound bleed between stages makes the distance worth it though, not to mention the beautiful Parc Del Forum that would be worth walking around even if there were no bands playing in it.

Wild Nothing

This was the first band we saw at the biggest stage at Primavera, the Heinekin stage. We walked up pretty close and caught their whole set. Wild Nothing is pretty formulaic both in composition and performance, so song-to-song they don't have a ton to offer as far as live shows go. However, the lead singer actually sounded better live than on the album, and they managed to conjure the guitar band Radio Dept. sound live, which was all I wanted out of them. One of my favorite songs by this band is their Kate Bush cover of Cloudbusting. They didn't play it, (and I didn't expect them to), but played all their familiar songs well. Chinatown was probably the highlight.

White Fence

I've been hearing a ton about this band, but on recording haven't managed to fall in love with them yet. This live performance was definitely what I needed to turn around. They are an amazing combination of every rock and roll band you ever loved. The band members look & dress like they are representatives from each of the 60s, 70s, 80s, and 90s rock and roll scenes, which makes them almost look cobbled together, but they blend the history into a unique but familiar sound. It's an amazing live show, and was wonderfully executed.

Neko Case

I've seen Neko Case so many times that it's hard to be objective about her shows, and in retrospect should have skipped this set to go check out savages. However due to timing, we camped out at the back of the Primavera stage to catch her do her thing. It was as good as it always is, and was a nice reminder of how great she is live. Her voice is especially suited for an outdoor venue, it carries in a beautiful way in an outdoor venue, and was a great set to chill out and charge up for the rest of the day. The only thing special/weird about this specific performance was the constant banter from her backup singer (Kelly Hogan), which after a while got a bit annoying, especially because it was all in English, and included some sarcasm that completely went over the heads of the Catalonians at the show.

Tame Impala

This is probably the biggest buzz band we saw during the festival. It was a sunset slot, so we started to see the light show come out, which was OK, but I feel like this band still has a long way to go to make their live performance live up to the album. Instead of the more subtle prog indie music on their album, they played more guitar heavy rocked out versions of their songs, which just didn't translate that well. I still love this band on recording, but a song like Be Above It doesn't work well without all the nuance it had on the album. Overall I was disappointed, but I'll be sure to check them out at an upcoming event and hopefully with time they can figure out how to translate their awesome studio sounds to the stage.


This band is just amazing. I recently saw them at a tiny venue in San Francisco, and didn't expect an outdoor show (albeit, at a small stage) to work as well as it did. The openness of the space stopped this from being as heavy as it was in a small venue, but I think they made up for it by playing harder and louder than the last time I saw them. The singer almost certainly destroyed his vocal cords pushing out the loud, fast, and complex sound over their energetic performance. This was the first band we saw at the Pitchfork stage, and definitely near the top of the best performances I saw there.

Dinosaur Jr.

I've been going to Dinosaur Jr. shows since I was a kid, and although they don't usually differentiate from one another much, they manage to still be one of my favorite bands to see live. They played the Primavera stage to an adoring crowd, and J Mascis' sauron-like performance was perfectly complimented by the wind coming off the ocean to blow his hair around. This band is both mesmerizing and familiar - the new material fits perfectly into the old, and this was a seamless set. They wound down with Feel The Pain, and against the backdrop of the night finally closing in, it was a really perfect experience.

Jessie Ware

We stopped by this stage mostly because it was near and I've wanted to check out Jessie Ware without buying a ticket specifically to do so for a while. It was thinly populated at this stage, and Jessie had a huge band for her sound. The mix appeared to be trying to show us how amazing her voice was, but I didn't really enjoy it. She did versions of her songs re-worked to be played with a full band instead of the light electronics on her album, which was unfortunate as it felt like an Adelle performance at an awards show instead of an indie female fronted R&B act like I had hoped. We left before she played If You're Never Gonna Move, which is too bad because it's a great song, but I have no regrets, I expect that it was over-orchestrated and badly mixed like the rest of the set.

Postal Service

Seeing this band was completely surreal. A huge benefit to seeing them overseas was that we walked up to about 20 feet of the stage only minutes before they started, something that never would have been possible at a US showing. Credit to Primavera as well for not overselling the event at all. They played with the same stage lighting setup as Blur later used, which was what looked like a Bell Curve of strips of LEDs behind the performers, with a huge LED screen behind the entire stage. They played with Jimmy Tamborello on a giant pedestal at the back, and Benjamin Gibbard jumping between the the center-left of the stage and a drum set & mic setup on the far left. On stage right was Jenny Lewis backing them up, and another woman I didn't recognize behind her playing auxiliary backing instruments. I really would have liked to say I didn't like this performance. I haven't listened to this band since I was a kid, and Benjamin Gibbard trying to dance is a painful mess to watch. I also just don't like Jenny Lewis at all, and she was fully detestable during this entire set. But even through a disgusting grinding dance between Ben and Jenny, I was singing along and in full nostalgia mode. They played 3 songs not on Give Up (Turn around?, Tattered Line of String, and Be Still my Heart), and a Beat Happening cover (Our Secret) - all 4 of these songs were things I wouldn't pay money to go see (except maybe Tattered Line), but anyone of my generation would have had an amazing time at this show, they executed every song the way you dreamed it would be when you were 13, rocked out the electronic breakdown, and Jimmy even sang some hooks.

Hot Snakes

Hot Snakes is amazing. They were super fun, played an awesome energetic show, and just generally killed it. I don't think they're very popular in Spain, and neither is mosh pits, so this was the most laid back I've seen them audience wise, but they managed to keep the energy up and play a really solid set. We had to take off after they played LAX to catch Grizzly Bear, but I'll be sure to catch them any time I can in the future, they always kill it.

Grizzly Bear

When Yellow House came out, my friend bought it for me on vinyl, and I listened to it nonstop for a very long time. I've managed to catch them a bunch of times, and the last time I saw them was a few years ago right after Veckatimest came out. I was saddened by their shiny new instruments, accomplished stage performance, and although they were super tight, I missed their worn out old instruments and endearing stage presence, and haven't seen them since. Their performance at Primavera however was a perfect example of a band that started out small and learned how to become an amazing popular band. They had a stunning stage setup with moving glowing beehive orbs, a new band member, and they played immaculately. It was a sharp contrast to the entitlement of The Postal Service, as they were super humble even though they are gods, and every song was better than it was on the album. They even brought back The Knife at the end, and played a version that was updated but still wonderful. This performance converted me back to a current fan.

Death Grips

This band was pretty amazing live. It's a more noisy, less tight version (if possible) than the album, and if possible the live performance is even more terrifying than I imagined. The stage lighting was set up so that you couldn't see either member - the DJ had every light pointed at them, and the lead singer was completely in the dark. I'd love to see them again, but hopefully at a festival, as I'm hesitant to get any closer to them than I did.


Phoenix is another band that is hard to like because of their massive popularity right now - they play stadium sets that look like stadium sets, and it's hard to feel like a true fan at their shows instead of just another adoring radio listener. However, as Friday at Primavera was light on crowd, we got right up to the front and it really was an amazing set. They played across their discography without alienating new or old fans, and had awesome energy. The size of the Heinekin stage (relatively small for a festival) helped them get personal with the audience and they were super tight without acting so popular that they didn't need you to care. The highlight was absolutely when the lead singer stage dove & crowd surfed deep into the audience, and while you were looking away, the cast of Dinosaur Jr. appeared on stage and helped them close down their set with extra rock. It was a fantastic performance.

Fuck Buttons

We had little time to stop by Fuck Buttons and Four Tet as the Phoenix set ran pretty deep into them, so we just showed up for half a song and an extra long version of Swim Solar, which was amazingly entertaining and all around an awesome performance.

Four Tet

I was pretty exhausted at this point and don't really remember a whole lot outside of this set being way more housey than I expected. I love house music, but I expected more of an experimental set than this - I'll have to catch Four Tet another time at a venue to see if this is the norm, or if he plays a more unique set normally.

Animal Collective
(not pictured)
I've seen Animal Collective a few times, and it's always pretty poor live. This was no exception, outside of their amazing stage setup - they had giant spiky winding horn-like structures over the stage that were lit in fluorescent colors. It was a set of what sounds like improvised, over-worked versions of songs that once were great when they recorded them in the studio. But then Daft Punk showed up! Just kidding, they played My Girls and people danced, but that was about it. It was 4 in the morning so I left after 30 minutes or so and went home to get some sleep.


Kurt Vile & The Violators

I missed their SF show recently, so I was super stoked to see Kurt Vile play - I love his solo stuff, and he didn't disappoint live - it was laid back but super fun to watch. He's a master of folk rock, and just watching him try to sing around his crazy red curly hair was entertaining. The band is tight, and he manages to be a decidedly modern take on a genre that is not often well executed recently.

Daniel Johnston

This was the only planning mishap of the weekend - announced the day before, there was limited seating at Auditori Rockdelux and to get in you needed to buy a ticket that afternoon to get in. We obviously didn't get there early enough to snag one, so we had to miss his performance. I'm sure it was quirky and fun, but I wasn't there for it.

Nick Waterhouse

You'll recognize Nick Waterhouse from a Heinekin commercial (or at least I did), but he didn't play the Heinekin stage - it's pretty straight up Soul music, fairly uninspired, but respectable nonetheless. They did a good job playing their music, and I thought it was an entertaining enough show. I wouldn't go see him on purpose, but if you are a gigantic fan of the genre, he's one of a few new artists that can play it without butchering the genre. I feel like the entire band would be great session musicians, but it wasn't special.


I like Peace's recordings, although you can almost hear the doucheyness in recorded albums. The singer was a bit inflated, and they seemed to think they were a lot more important than they are. That said, they played well and put on what would be considered a good show if I were more of a fan. I like the recorded stuff better than the live performance, but I can't say they played poorly. The lead singer's voice is super unique, and translated well live, but they could have played tighter.

Django Django

This band is hard for me to listen to on recording because it feels so much like a Devo cover band - seeing them live though, that was probably their greatest asset. They put on an awesomely energetic show, moving all over the stage, and their somewhat non-palettable music was much better live than on the album. They were super coordinated including matching outfits with white shirts & geometric black shapes on them. I really enjoyed their set, and will definitely check them out next time they play near me.


I'm a huge Solange fan, so this was a great opportunity to get super close for one of her sets. She played superbly, running through most if not all of Trust, and busting out Would've Been The One, which was a special treat as I'm probably more a fan of the Sol-Angel stuff than the new stuff. She closed with a cover of Dirty Projectors' Stillness Is The Move (She's done this song before when I've seen her, but it took until this performance to actually place the song, as it's completely a unique version). It was an awesome dance party, the crowd loved it, and she busted out some very Solange dance moves. The only thing that would have made this set better would have been T.O.N.Y., but I lived.

The Jesus & Mary Chain

This band is absolutely fantastic live. They ran through a great selection across their discography with a preference for the hits, the stage set was a fantastic giant light up cross, and they jumped song to song reminding you why a reverence for their music is necessary. Watching them play reminds you how important they are as an inspiration to so many of your favorite bands. They wound up the set by bringing out Bilinda Butcher of MBV to sing the female part on Just like Honey.


This was an interesting experience, as at this point the festival had reached a critical mass of people (mostly annoying brits) in anticipation of Blur. This performance was the casualty of that mass, as it was at the smallest stage (Vice), and was completely packed with loud brits. Daughter performed wonderfully, and I was happy for her, as she was absolutely ecstatic about the size of the audience and mentioned a few times that this was the biggest crowd she had ever played for. It was bittersweet, as from where I was standing, the audience almost completely drowned out the music, but I was happy for her. Daughter is at the top of my list for bands I saw at Primavera to catch at another venue - as far as I could tell the performance was amazing, I just had a hard time enjoying it because of the crowd.

James Blake

This performance was also a bit overpopulated, but the stage size made it possible to put yourself somewhere that wasn't too noisy. James Blake is an amazing performance, and the live versions are expertly transformed from their almost ambient recordings to full on performances live. The sound guys had the bass way up, which made songs like I Never Leart To Share explode when they arrived at their climaxes in a way that was breathtaking. The importance of the vocal transformations James Blake uses is much more downplayed live, and you hear much more of the nuance in the composition, something that is almost absent on the studio tracks. I'll definitely be back to catch him again, but I think this venue was one of the best places I could have seen this performance.

How To Dress Well

This band was bizarre to watch live - it's two men, one that dresses like eminem and looks like a skinny jewish kid, and another goofy looking guy playing the violin and triggering electronic backing tracks. They were standing uncomfortably close together on an already small stage, with virtually no lighting help (just a projected video in the back). The lead singer uses two mics, one with very little reverb, and one with a super long hall echo/reverb on it. He grabs both and uses distance from each one to control volume and effect on his vocals. Seeing this, the first thought is that watching him karaoke an Usher song would be the greatest thing ever, partially because he has one of the most amazing voices I've ever heard, and partially because it would be so unexpected from his character. If you were to close your eyes, it would be an amazing performance. His control over a microphone is something I don't think I've ever seen before, and the result is a completely unique R&B sound. I liked his recorded stuff, and I have a ton more respect for him after seeing him live. However, it was such a bizarre performance I feel like half the audience walked away super confused, as the missing stage presence detracted from the performance. If they can get the stage presence figured out, I think he will be huge.

Glass Candy

We were on our way to Blur, and so only stopped by to catch their first song, which fortunately was Digital Versicolor. It was a shame they were against Blur, because it would have been a lot of fun to hang out there. I've seen them in the past, and it was a blast. However, because I haven't seen Blur, we had to move on.

(No decent pictures, but here's one of the audience enjoying it)

Blur opened with Girls & Boys, which was sadly the only song we missed. The rest of the set was a back-to-back hit parade, and even a non Blur fan like myself could appreciate how solid the setlist was. For me though, this just reiterated how little I care for this band in comparison to the rest of the world. Damon Albarn is an amazing frontman though, and no human could have avoided enjoying his performance. They absolutely defended their title of best Brit-Rock band ever, and this was likely the highlight of the festival for most of the people there. I'm glad I was there and had the experience, but I think I can confirm that this just isn't my genre.

The Knife

The Knife was probably the most amazing performance I've ever seen. I came in with very low expectations - listening to Shaking the Habitual was a nearly painful experience for me, I missed the re-imagined pop sound from their earlier albums, and when The Knife started, (opening with A Cherry on Top), I was ready to be disappointed. The stage started set up with an array of musical instruments assumedly invented specifically for this performance - a giant harp-like object that sounded like it reverberated in rusty mufflers (You can hear it in A Cherry on Top), a human-sized metal object that looked like playground equipment with giant bells on the top and round metal parts on the side, and a giant device that looked like a geometric cone laid on it's side that appeared to be able to spin around it's center to expose an array of different instruments. Along with these were a few other giant standing-performance setups around the stage. There were 7 or so performers on stage, all dressed in iridescent robes that looked much like Ghost B.C.'s Nameless Ghouls. The stage was lit eerily, and you couldn't tell where the vocals or backing tracks were coming from, but it was somehow still completely enchanting. As the performance went on, dancers started to join the performance, and some of the musicians would move to dancing or the dancers would jump on instruments. Each song had a definite end while still blending perfectly into the next. There were bizarre movements by the dancers during the songs, and you realized at a certain point that neither Olof Derijer or Karin Dreijer Andersson were on stage. The dancers had creepily painted faces under their hoods, and as the performance went on they started losing the robes. After one of the songs they cleared the center of the stage to reveal a completely empty stepped stage setup, while leaving some instruments on the periphery. The performance then became a completely choreographed modern dance performance that was so perfectly orchestrated with the music that you realized that there was a missing dimension on the album - they were meant to be performed together, one without the other was unfair to the performance. The performance was impossible to describe - it was impossible not to watch, you had no ability to analyze or think about anything else during it. It completely captured me. Karin did come out from the back during 3 songs, (Silent Shout, One Hit, and Bird) (I think - they played one song off of their Self-Titled, but I don't currently recall which…) - but her presence was almost unnecessary outside of to remind us that there was an artist behind this work. Watching this changed my feelings about the new album, it now sounds to me like a natural progression, the necessary next step to perform the way they intended. It's impossible to hear it without conjuring the live performance. I hope they can play enough to allow this experience to as many people as possible, it truly is something that has yet to and probably never will be recreated by another artist.


Pantha Du Prince & The Bell Laboratory

This was the first show we saw at Autidori Rockdelux, which was actually an auditorium somewhat outside the festival in a beautiful museum. The space was beautiful. I was expecting a Pantha Du Prince Live set, but this was something far more amazing. The Bell Laboratory is a group of musicians who all play different bell-based instruments, along with a pipe organ. Pantha Du Prince was in the center of the stage playing his live electronic setup, and there was a drummer stage right. Otherwise all the other instruments were acoustic bells, played with a range of instruments, including a giant bell piano at the back. There was no light show, just a single light casting a huge shadow of the drummer behind the band. This was some of the most beautiful electronic music I've ever seen - the highlight being when at the end of a song, all of the members (6 to 8) all picked up a single hand bell and slowly walked to the back of the audience. It was an amazing composition, and in the large auditorium the bells completely surrounded you - the audience was completely silent, and the experience was unlike anything I've experienced before. I don't know if it's possible to see this performance again, but I will be trying to.

Guadalupe Plata

We got some festival food and coffee, and briefly watched this band. I'm only familiar with one song, (Baby Me Vuelves Loco), it was fun spanish rockabilly.

Mount Eerie

I'm a huge Microphones fan, but never was able to get into Mount Eerie as much. This was a good show, they played super quietly and seemed super put off by the bit of sound bleed from Adam Green on the stage next door (I would be too). I was also getting a lot of attention for my disco suit, so had a hard time paying attention to the band. The songs I payed attention to were good, but this band is better suited for a venue. The lows were hard to listen to because of the venue, and the highs weren't quite epic enough to get you in the state you should be to listen to this band.


We went back to the Auditori Rockdelux to catch the Apparat set. Apparat played Krieg Und Frieden from start to finish with a live band - it was a wonderful diversion from a normal Apparat set - it was wonderful to watch, and had a live performance of sand & objects being manipulated over a light box as the light show. In the completely pitch black auditorium, it was impossible not to pay attention to only the music, and almost like watching a movie rather than a performance. You could hear Apparat more as an influence to the performance than the artist, truly enjoyable and a great intermission from the craziness of the festival.

Dead Can Dance

I've never been a huge fan of Dead Can Dance, but they have a few songs that I enjoy - thankfully they played Children Of The Sun early in the set, and I pretty much got what I came for. They weren't much to look at, but did a perfect job of playing super epic music and making you feel the goth in your bones.

Mac Demarco

I'm a big fan of Mac Demarco's recorded music, pretty pure lo-fi, complex as Ariel Pink without alienating any listeners. Unfortunately the gritty sound translated to a pretty thrashy live show - it lacked the preciseness of the record, and seemed more like a jam band than what I expected listening to the record. He did some classic rock covers that were entertaining if confusing, but overall this was a set I wasn't too impressed by.


Deerhunter played the day before and wasn't scheduled to play this set, but because of the tornados in the midwest US, the previous band in this slot, Band of Horses, had to cancel. This was fine with me, as I couldn't make it to the Deerhunter set the day before and was bummed about it. Bradford Cox had amazing audience banter, played a super rocked out set, and the whole band seemed like they were having a great time.

Thee Oh Sees

I've been hearing about Thee Oh Sees shows for a few years, and everyone seems to be super impressed by their live show. As someone who never really got into their horror-garage sound, I never made it to one of their shows when they were new, and was glad to be able to give them a trial run at a festival to see if they were as good as all my friends say. They really were fantastic, much less difficult to appreciate than their recorded stuff, they were more like a real band and less like they were trying to play inside of a genre. They had tons of energy and a really solid stage presence. I expected it to be King Kahn style thrashing, but they were more understated, and better than I expected.

Wu Tang Clan

I couldn't help but expect this to be a pretty epic show. I'm definitely more of a fan of the hip hop inspired by Wu Tang than Wu Tang themselves, but they did a fantastic job of reminding us why they're the OG. They were missing Method Man, but were able to put on a perfect show where you almost didn't notice he wasn't there. The highlight for me was definitely Shame On A Nigga, which they played after a solid 3 or 4 ODB songs that got the audience jumping. Other high parts were the Rza constantly asking for a mosh pit (which he almost got, but as I mentioned earlier, nobody could figure out how), and Mathematics showing off his skills by spinning his decks with his mouth, his, feet, and his back while showing off old school deck skills at the beckoning of Rza.

Dan Deacon

Dan Deacon played this show on the stage, a rare occurrence for him, so we got to see him in all his bouncy glory without shoving up to the front of a crowd, which made this even more fun. He spent a lot of the time attempting to co-ordinate different dance moves from the crowd, including a crowd-participated sports-style chant for the Moon, a follow-the-leader dance, human tunnel, and other stuff that took longer to get the multi-lingual audience to get ready to do than they could keep doing. Even while his best efforts to organize us were failing, his banter and epicly high energy performances made this set a highlight of the weekend - with Dan Deacon doing his best to remember his high school spanish between playing strobed-out versions of his awesome new and old material, backed by a full band, it was both a great performance and a reminder that I need to be at every show by this guy as I can.

Camera Obscura

We stayed late at the Dan Deacon show, which I was planning on splitting with Camera Obscura, so we only caught one song in passing by these guys. I've had a lot of fun at their shows in the past, and maybe coming off the high energy of Dan Deacon made their performance seem maudlin, but I'm pretty sure that's what they were going for in the first place. They sound exactly like themselves.

The Meat Puppets

I had high expectations, for The Meat Puppets, they're such an inspiration to a lot of bands I love and I assumed that seeing them live would be a somewhat epic, unique experience. Their live show left much to be wanted though, they honestly were just too old. They looked and sounded like tired old men jamming in a garage, which while entertaining, didn't live up to my (admittedly far too high) expectations. It was fun to see them, and they didn't seem to give a fuck, so I was glad to be there, but I wished they had proved that they were the rockers they started out as.

Nick Cave & The Bad Seeds

I've only had one Nick Cave phase, so I'm not as extensively familiar with his discography as most people I know, but it wasn't important to be familiar with the music to enjoy this. We stayed at the outskirts during this set and grabbed some much needed Patatas Bravas before settling in and watching Nick do his thing, which was a very well executed example of showmanship. He was as cocky as he needed to be to uphold his reputation, and played very well.


I've been looking forward to catching Phosphorescent live for a long time, so I had high expectations - this was way better than I even expected though - the new material and the old fit way better together than I expected, the full band played up the new material, and his voice held up to the record if not surpassing it. He played primarily off of Muchacho, busting out a few old ones (my favorite of the set was Los Angeles) to put together a set that was perfectly variated to keep the set completely capturing all the way through. This band would be my 2nd highest pick of the weekend to see again at a venue.

Nurse With Wound

I didn't have much interest in seeing Nurse With Wound live, but because of the way things were set up and the necessity to be at My Bloody Valentine early, we stuck around the ATP stage after Phosphorescent to catch a bit of Nurse With Wound before heading next door for MBV. It was as weird, creepy, and difficult to understand as I expected - it wasn't bad, although it was a bit less performance art than I expected. If you were a Nurse With Wound fan, I'm sure this wouldn't disappoint.

My Bloody Valentine

I've seen MBV once before, and it was nearly a religious experience. This time it was a little less so, both because of the new material - which while amazing, didn't seem to work onstage as well. There were also some sound issues, both Kevin's guitar going out once and just an overall loudness issue (this band is made to play LOUD, and this wasn't). I managed to overlook the issues and set list problems though, and this was one of the best performances of the festival, if only because I'm completely partial. This band is one of the least active, most engaging bands I've ever seen.

Hot Chip

This was a bit of a rough transition from MBV, but didn't take long to get into full swing with. I've been going to Hot Chip shows since The Warning had just come out, and it's been amazing to watch them grow from a fun, almost Dr. Dog like stage presence to full on rock divas. They command the stage better than most, especially because of their size, and they just do a great job of always giving you something to look at. They played songs from every album, and didn't diverge from the hits for even one song - it was a full-on dance party sing-along for everyone in the audience. I've never seen them put on a bad show, but this was probably the best, if only because the set list was immaculate.

The Suicide of Western Culture

We stopped by this on the way to DJ Koze, it was noisy electronic. I enjoyed the album much much more than the DJ set, and we were bored pretty quickly and headed over to Koze.

DJ Koze

I had mixed feelings about this set. It was the last band I was going to see at the festival, so I don't think it was entirely Koze's fault that I was feeling conflicted though. DJ Koze's latest album was one of the better albums to come out so far this year, and I was expecting a bit less of a straight up house set and more of the new material & sound mixed in. That didn't happen, this was pretty much exclusively a minimal house set. However, it was one of the best house sets I've seen. Koze does an amazing job of not using any cheesy drops or transitions that you expect out of a house DJ, but instead uses slower transitions and builds, which is difficult to do without getting boring. He was fantastic, and even though it wasn't what I was expecting, it was a wonderful way to wind down and get ready to say goodbye to Primavera.

We walked to the top of the Parc Del Forum and saw dawn over the Mediterranean - it was almost 6am, so we stumbled back to the city.