Change (B)log

Week 22 Update - The SoundCloud divorce

June 07, 2019

I decided to not host my interviews in SoundCloud any more. I’ve used SoundCloud as the host of all my audio files since day one. When I made PAPIO, I used a library to help make SoundCloud the backend for my audio data on the site. However, it has recently gotten annoying managing two sources of truth for title, description, and image info. Since I just finished an admin portal last week to help me manage one source of truth (DynamoDB items in my AWS account), I thought about writing some code to replicate that to SoundCloud but very quickly into that decided to stop using SoundCloud.

This decision presents multiple challenges, but they all center around one thing… cost. Using my solution will be cheaper as long as there is low traffic to the site. But if I’m not careful, people using the site will download a lot of data they don’t need, and that will cost me more money unnecessarily. Luckily, the browser does a pretty good job of downloading an audio file progressively — only as it requires more data. But, if I’m not careful, it could be very easy to abuse.

To solve these problems, I decided to host the audio files in S3 (cheaper but slightly slower than using CloudFront) and require a signed URL. Signed URLs help prevent basic abuse.

When someone comes to the site, the audio file URLs are referenced in the source code of the website. These URLs point to objects in S3, but the objects are not accessible to the general public; just having the URL is not enough to retrieve the object. That URL has to be “signed” by my API before it can be used. The signing process attaches extra data to the URL that makes the link work, but only for a certain amount of time. Also, to get a URL signed, you have to be a visitor from https://papio.io.

While it’s certainly not incapable of being abused, it’s harder to do. This is good enough for now.

Because of their size, SoundCloud and many other streaming services have a more sophisticated approach that involves chunking the actual file into many smaller files. These files have random names and don’t arrive in perfect sequence. When they arrive in your browser just in time, a client library most likely stitches everything back in order. This makes it a real pain to abuse because even if you automated the downloading of all these files, you have to figure out how to stitch them back together.

My dad always said that “thieves are lazy” and so doing anything to slow them down will likely get them to stop altogether. In my case, the more roadblocks I put up in front of people trying to access my data outside of the ways I intend them to come in, the less I have to pay.

Sizable

We had our first mixing session at Wil’s studio last night. It went pretty well.

Brad recorded some drums. Then we mixed 2-3 songs afterward… all in 3 hours. Unfortunately, though, Wil is pretty booked for the next month. We only have one session in the next month; we’ll have to finish this album at the beginning of July. Since everyone’s schedules are so crazy though, we went ahead and booked all the sessions we need for the next album 3-4 months away.

New Instruments

Also, I bought a banjo a few weeks ago and have been trying to record banjo on the song “It’s You” for this upcoming Sizable record. Banjo is a lot harder to play than I imagined! I want that Scrugg’s sound for this particular song but that means my fingers have to be very coordinated: they aren’t. Hopefully, I’ll be able to get something decent down in the next week… because that’s all I have.

I’m most likely headed to St. Louis tomorrow morning to buy some more instruments. I’ve always wanted a Martin D-35. I may pick one of those up. I also want to pick up a beginning Dobro.

I want to start collecting bluegrass instruments. These musical instruments are ones I grew up hearing in Vienna, MO. And while much of that music is very virtuosic (and I’m not virtuosic on any instrument), I probably won’t be using them to make that kind of music in the long run, but instead, music that blends bluegrass (and country) with my formalized Art Music training.

I’ve been thinking about this for a while but recently decided that I’d take this more seriously now. There’s no point in waiting until I’ve done all these projects and then buy the things that I could’ve used all along.

Carfax

Carfax is going well at the moment. I’m helping with this Carfax Graph API Proof-of-Concept. I’m mostly just watching Conner code but contributing ideas here and there. This POC should wrap up in the next week or two. Not sure I’ll be working on next but I hope it’s cool

Also, I’m going to Las Vegas at the end of the year to the AWS Re:Invent conference. I’m pretty excited about it. I’ve wanted to go to that one in years past but now the time is finally right for me.

It’s a long way off, and while I’m now a bit hesitant to commit to things like this at Carfax, considering I recently tried to get a different job, I figured I couldn’t thrive by living in that state of paralysis.


Joseph Weidinger

Written by Joseph Weidinger.