This all started because the YouTube app on the iPad is terrible 😅. Let me explain, if you’re looking at your feed and see a couple of videos that you might be interested in, you had better save them to a playlist. The next time you open the app it’ll refresh and they’ll likely be gone from your feed. Don’t even get me started on trying to search for videos either.
At first, I would save videos into their own playlists but I found this to be a bit tedious. I changed my workflow to dump all new videos that I’m interested into a playlist called myWatchLater (see below why I can’t use the built in “Watch Later” playlist) and then I’d search for them (think how email is thrown into one big archive in Gmail).
I fired up rails new and built a way to import playlists and playlist video data using the YouTube API. A couple of things to note about using the YouTube API; your “Watch Later” playlist isn’t available through the API and there is a restrictive quota of API calls that you’re allowed within a 24 hour period. The latter is the reason that I added etag support to the yt Ruby gem. This allows me to store the last etag for a resource in the database to use for comparison. If the etag value that is returned by the API is the same as what’s stored in the database then I don’t need to update it further. That saves me on total API calls that I can make.
When I start an import process not only does it get new data to add to the database but I can also set videos to be removed from a playlist by simply checking a box in the web interface. The entire import process is offloaded to Sidekiq. I’ve been using this as I’ve been building this and so far Sidekiq has already processed 50,000 jobs 😲 (that’s because I use a fanout strategy to smaller jobs instead). As an aside, I had to fork the Sidekiq Status Ruby gem to work with Sidekiq 8 since I wanted to track the status of jobs. Core functionality to support Sidekiq 8 is there but there are a few things that really need to be updated.
Right now the application lives in local Docker containers but I might eventually deploy this somewhere.
