Managing a Spotify Library with Go and AWS Lambda

Sun Sep 30, 2018
go aws spotify lambda

Introduction

Spotify exposes a robust API that can be used to manage your (or someone elses) music library and do all sorts of fun things with it. Once OAuth 2 authentication is set up and ready to go, an application can continually interact with the Spotify API on behalf of a user to do anything from getting basic information about a user to seeking to a position in a currently playing song.

In this post, we’re going to walk through setting up a Go application hosted on AWS Lambda that does some simple library management involving the Discover Weekly playlist. Specifically, our lambda will run on a weekly basis and add any songs that were “liked” during the week from the Discover Weekly playlist and add them to another playlist. I’ve been doing this manually for some time because I like to keep track of all the songs that I’ve thumbed up from Discover Weekly without keeping replicas of the entire playlist. It’s a good introduction to the Spotify API because it’s simple enough to get going quickly, but does use a few different Spotify endpoints and do a bit of logic to get things properly situated.

All of the source code for this example is hosted on GitHub and is available under the Apache-2.0 license.

  1. Setting up a Spotify application
  2. Environment Variables
  3. Generating an OAuth 2 token for Spotify
  4. Configuring AWS infrastructure
  5. The AWS Lambda function
  6. Putting Everything Together

Setting up a Spotify application

The first thing to do is login to the Spotify Developer Console and register a new application by choosing “Create an App”: Spotify Developer Console Follow their instructions to register a new application and once you do, you’ll land on the application homepage where you’ll see your Client ID and Client Secret: Spotify Client IDs We’ll use those later on in the application. One last step - authentication with a Spotify application requires the application to know where the user is supposed to be redirected to after logging in. As we’ll see in a minute, we’re just going to run this locally to generate a token. So, from your application page, go to Edit Settings and then add http://localhost:8080/callback as a Redirect URI: Spotify URIs And that’s it for setting up the application!

A Few Environment Variables

The first thing you’ll need to do if you are following along with the repository code is set up a .env file at the root of the repo (or just export these however you want later on):

SPOTIFY_ID=
SPOTIFY_SECRET=
BUCKET=
TARGET_PLAYLIST=
TOKEN_FILE=
REGION=

We’ll fill these values in as we go along, but there are a few we can fill out now:

Generating an OAuth 2 token for Spotify

Now we need to generate an OAuth2 token that our app will use to authenticate itself to Spotify. Spotify has several different methods of authentication, detailed in their authorization guide. Since this application is going to access and modify actual user data (and not just accessing generally accessible data, like an artist’s information), we need to actually sign in and grant our app access to our own Spotify account. To do that, we’ll use the Authorization Code Flow from Spotify’s documentation. In this process, a user is directed to a webpage where they can sign into their Spotify account and grant access to an application. Then, we can use the Client ID and Client Secret created in step 1 to request a token from Spotify using the authorization code that is generated when a user signs in. Once we have a token, the Go OAuth 2 library will automatically handle refreshing the token when it expires (Spotify tokens expire after an hour).

Let’s take a look at some code to see what this looks like in reality. Some of the token generation code is modified from this awesome Go Spotify client that is used throughout this project to interact with the Spotify API.

These code samples are truncated for brevity - check out the repo for the full source code.

In cmd/get-token/main.go of the repo:

  1. We first set up an auth variable from the Go Spotify client library that will give us a URL to direct the user to sign in.
  2. Then we start up an HTTP server that will handle the redirect after a user signs into their account.
  3. Finally we print out the URL for the user to sign in on and then wait for them to sign in.

The auth variable deserves an extra bit of explanation: the various “scopes” that are passed into NewAuthenticator designate what our application will be able to access. These are also used to alert the user about what our application is trying to access before they grant access. The names are fairly self-explanatory and each Spotify API endpoint has certain scope requirements. Their API Reference has more information about other endpoints. This NewAuthenticator uses the environment variables SPOTIFY_ID and SPOTIFY_SECRET to authenticate, so make sure you have those set before running this code.

ch     = make(chan *spotify.Client)
auth   = spotify.NewAuthenticator(redirectURI, spotify.ScopeUserLibraryRead, spotify.ScopePlaylistModifyPrivate, spotify.ScopePlaylistReadPrivate)

// Start server with a callback handler to complete the authentication.
http.HandleFunc("/callback", completeAuth)
go http.ListenAndServe(":8080", nil)

url := auth.AuthURL(state)
log.Println("Please log in to Spotify by visiting the following page in your browser:", url)

// Wait for the authentication.
client := <-ch

Once the user (us in this case) signs in, they will be redirected to our little HTTP server and invoke the handler to process the authorization:

func completeAuth(w http.ResponseWriter, r *http.Request) {
	// Exchange the authorization code for a token.
	tok, err := auth.Token(state, r)
	if err != nil {
		http.Error(w, "Couldn't get token", http.StatusForbidden)
		log.Fatal(err)
	}

	if st := r.FormValue("state"); st != state {
		http.NotFound(w, r)
		log.Fatalf("State mismatch: %s != %s\n", st, state)
	}

	// Use the token to get an authenticated client.
	client := auth.NewClient(tok)
	fmt.Fprintf(w, "Login Completed!")
	ch <- &client

	btys, err := json.Marshal(tok)
	if err != nil {
		log.Fatalf("could not marshal token: %v", err)
	}

	// Save the token to S3 for later use.
	if _, err := s3.Upload(&s3manager.UploadInput{
		Bucket: aws.String(config.Bucket),
		Key:    aws.String(config.TokenFile),
		Body:   bytes.NewReader(btys),
	}); err != nil {
		log.Fatalf("could not write token to s3: %v", err)
	}
}

This handler exchanges the authorization code in the request body for a valid token. Then it saves the token off to S3 so we can use it later on in our Lambda function, and sends the client back out on the channel so it can be used to notify the user that they have been logged in:

// Wait for the authentication.
client := <-ch

// Get the user.
user, err := client.CurrentUser()
if err != nil {
	log.Fatal(err)
}

log.Println("You are logged in as:", user.ID)
log.Println("Token has been saved to s3")

Sweet! Now we have an OAuth2 token generated for our own Spotify account that can be used by our application to modify anything it has access to in our account.

Configuring AWS infrastructure

If you don’t already have an AWS account, you can sign up for one for free. There is a 12 month free tier associated with new accounts, but there is also an ongoing free tier. The resources used in this example are extremely minimal and are within the regular free tier, so if you have an account already this shouldn’t cost you anything to run.

Create an S3 Bucket

The first thing we’ll create is an S3 Bucket to house our OAuth2 token.

Create an IAM role for the Lambda function

Next we’ll create an IAM role that our Lambda will use to access S3.

Add your AWS credentials locally

If you have already set up your machine to interact with AWS, you may be able to skip this step. Since we’re generating a token locally and saving it to S3 we have to be able to authenticate to our AWS account. This is done with an access key. If you are using Root to login to your AWS account, I would highly recommend creating a separate user for yourself and using that for all account activity. That user will need access to write objects to the S3 bucket that you created.

[default]
aws_access_key_id = 
aws_secret_access_key =

and create a ~/.aws/config file to handle the defaults:

[default]
output = json
region = us-east-1

AWS Lambda function

Alright now that we got all this authentication and infrastructure stuff done with, we can move onto the fun part!

Our Lambda function is going to do the following:

Let’s walk through each of these with some code samples.

Download our token file from S3

First thing to do is grab the token from S3. Here we use the same TokenFile from when we first created the token to grab the same token from our S3 bucket and unmarshal it into a new OAuth2 token.

// Download the token file from S3.
buff := &aws.WriteAtBuffer{}
if _, err = s3dl.Download(buff, &s3.GetObjectInput{
	Bucket: aws.String(config.Bucket),
	Key:    aws.String(config.TokenFile),
}); err != nil {
	log.Fatalf("failed to download token file from S3: %v", err)
}

tok := new(oauth2.Token)
if err := json.Unmarshal(buff.Bytes(), tok); err != nil {
	log.Fatalf("could not unmarshal token: %v", err)
}

Create a new Spotify client with the token and check if it’s been refreshed

We first pass the token into a blank Authenticator to create a new client. Then we grab the token on that client and check if it is the same accesss token that we have stored on S3. Since the Spotify tokens expire after an hour, it most likely will be a different token, so we’ll save that off to S3 and overwrite the token that was there.

// Create a Spotify authenticator with the oauth2 token.
// If the token is expired, the oauth2 package will automatically refresh
// so the new token is checked against the old one to see if it should be updated.
client := spotify.NewAuthenticator("").NewClient(tok)

newToken, err := client.Token()
if err != nil {
	log.Fatalf("could not retrieve token from client: %v", err)
}

if newToken.AccessToken != tok.AccessToken {
	log.Println("got refreshed token, saving it")

	btys, err := json.Marshal(newToken)
	if err != nil {
		log.Fatalf("could not marshal token: %v", err)
	}

	if _, err := s3ul.Upload(&s3manager.UploadInput{
		Bucket: aws.String(config.Bucket),
		Key:    aws.String(config.TokenFile),
		Body:   bytes.NewReader(btys),
	}); err != nil {
		log.Fatalf("could not write token to s3: %v", err)
	}
}

Get a list of all the playlists for the user

Next we grab a list of all the playlists we have in our library and then loop through those to find the ID of the Discover Weekly playlist and our target playlist.

user, err := client.CurrentUser()
if err != nil {
	log.Fatalf("could not get user: %v", err)
}

// Retrieve the first 50 playlists for the user.
playlists, err := client.GetPlaylistsForUser(user.ID)
if err != nil {
	log.Fatalf("could not get playlists: %v", err)
}

// Vars used to designate the Discover Weekly playlist and the target playlist.
var (
	discoverID spotify.ID
	targetID   spotify.ID
)

// Get the ID of the Discover Weekly playlist and the target playlist.
for _, p := range playlists.Playlists {
	if p.Name == "Discover Weekly" {
		discoverID = p.ID
	}

	if p.Name == config.TargetPlaylist {
		targetID = p.ID
	}
}

// Bail out if one of the playlists wasn't found.
if discoverID == "" || targetID == "" {
	log.Fatal("did not get playlist IDs")
}

Grab all the songs from the Discover Weekly playlist

The array of playlists we retrieved earlier doesn’t actually contain the songs in each playlist, so we need to retrieve those for the Discover Weekly playlist with the ID of the playlist.

// Get songs from the Discover Weekly playlist.
// Don't need to worry about API limits here, since it always has 30 songs in it.
discoverPlaylist, err := client.GetPlaylist(user.ID, discoverID)
if err != nil {
	log.Fatalf("could not get Discover Weekly playlist: %v", err)
}

Check if any of the songs in Discover Weekly are saved in the users library

Now that we’ve got all the songs in the Discover Weekly playlist, we can grab the track IDs for each one of those and then use a Spotify endpoint that tells us which ones are saved in the users library. The response from that endpoint is a true/false for each track ID that was passed in, in the same order that the track IDs were passed.

// For each song in Discover Weekly, check if it is saved in the library.
trackIDs := make([]spotify.ID, 0, len(discoverPlaylist.Tracks.Tracks))

// Extract the Track IDs for each song.
for _, t := range discoverPlaylist.Tracks.Tracks {
	trackIDs = append(trackIDs, t.Track.SimpleTrack.ID)
}

// Check if they are in the library.
hasTracks, err := client.UserHasTracks(trackIDs...)
if err != nil {
	log.Fatalf("could not check if tracks exist in library: %v", err)
}

Save any of those songs into another playlist specified by an environment variable

Lastly we loop through that list of bools telling us whether each song is in the user library and create a final array of the track IDs that are in the Discover Weekly playlist and also saved in the users library. Then we save those songs to the target playlist and that’s it!

// Check which tracks came back in the library and mark them as songs to be added.
addTracks := make([]spotify.ID, 0, len(discoverPlaylist.Tracks.Tracks))
for i, b := range hasTracks {
	if b {
		addTracks = append(addTracks, trackIDs[i])
	}
}

// Add each song that needs to be added to the taret playlist.
_, err = client.AddTracksToPlaylist(user.ID, targetID, addTracks...)
if err != nil {
	log.Fatalf("could not add tracks to playlist: %v", err)
}

log.Printf("successfully added %d tracks to playlist %s\n", len(addTracks), config.TargetPlaylist)

Putting Everything Together

We’ve got all the pieces and now we’ve just got to put it all together. We’ll be using a Makefile to do a couple of these steps, which is partially shown down below. The full source code also includes an update and an update-env command to update the Lambda function.

include .env

token:
	go run cmd/get-token/main.go

upload:
	export `cat .env | xargs`
	env GOOS=linux go build ./cmd/sync-discover
	zip sync-discover.zip ./sync-discover
	aws lambda create-function \
	  	--region us-east-1 \
		--function-name sync-discover \
	  	--memory 128 \
	  	--role arn:aws:iam::526123814436:role/lambda_execution \
	  	--runtime go1.x \
	  	--zip-file fileb://sync-discover.zip \
	  	--handler sync-discover \
		--environment Variables="{SPOTIFY_ID=${SPOTIFY_ID},SPOTIFY_SECRET=${SPOTIFY_SECRET},TARGET_PLAYLIST=${TARGET_PLAYLIST},BUCKET=${BUCKET},TOKEN_FILE=${TOKEN_FILE},REGION=${REGION}}"
	rm -f sync-discover sync-discover.zip

First thing to do is generate the token using the HTTP server.

Then, deploy the Lambda function to AWS:

The last thing to do is set up a CloudWatch event to automatically invoke our Lambda function each week.

I hope this tutorial was helpful in getting started with the Spotify API. I have a bunch more ideas of things I want to manage within my own Spotify library and would love to hear what other ideas you have too!

Feel free to follow me on Medium to get updates when I have a new post.

back · blog · about · main

Zach Johnson - Software Engineer