Sketch First, Code Later: A Hand Pose Adventure 🖐️
Before diving headfirst into C++ like a headless chicken in a keyboard factory, I decided to start with some good old-fashioned sketching. My lightbulb moment came when I stumbled upon The Coding Train’s Hand Pose Detection video on YouTube. It felt like the perfect jumping-off point for my project.
A Quick Detour: The Coding Train 🚂💻
If you’re not familiar with The Coding Train, let me fill you in. It’s this fantastic creative coding community led by Daniel Shiffman, who is, honestly, a national treasure. (I mean, I’m not sure which nation, but let’s just roll with it.) His videos are equal parts educational, inspiring, and downright delightful. I found his channel about a year ago when I was dipping my toes into creative coding, and let me tell you—this guy’s a magician when it comes to explaining complex stuff.
I was so hooked that I bought his book The Nature of Code a few months later. (Yes, that’s a shameless plug. No, I don’t get commission. Just trust me—buy the book. You’ll thank me later.) If you haven’t already, go check out his YouTube channel or website. Seriously, why are you still here?
Back to Business
So, what did I do after discovering that Hand Pose Detection video? Naturally, I watched it, tinkered with the code, and dove into my own version. You can try my take on it here and poke around the code on my GitHub repo.
But, full disclosure: I am not a JavaScript wizard. I don’t even pretend to be one. This is all about the sketching phase, so bear with me while I explain the tools.
The Tools 🛠️
1. p5.js: The Creative Coding MVP
I’m using p5.js, a lightweight JavaScript library designed for creative coding. It’s open-source, free, and comes with a web editor that’s perfect for experimenting. Of course, you can use VS Code if that’s your thing, but honestly, the p5.js editor gets the job done.
When you fire up a new p5.js project, it comes preloaded with:
- index.html (the backbone),
- style.css (for making things pretty), and
- sketch.js (where the magic happens).
Changes to the canvas are done by adding code to the sketch.js file. Every sketch file begins with two main functions: setup() and draw(). Setup runs once, while draw executes 60 times per second (or whatever frame rate you set) to keep the visuals alive.
2. ml5.js: Thank you for Existing
For the hand recognition part, I’m using ml5.js, an open-source library built to make machine learning accessible. It’s built on top of TensorFlow.js but has a much simpler API. Bonus: it’s integrated with p5.js! To get it working, I just imported the ml5.js library in my index.html file and loaded the model in sketch.js. So far, so good.
What Else Did I Build?
I created an oscillator (super easy with p5.js’s audio tools) and linked it to the hand pose detection. The left hand controls the frequency, and the right hand adjusts the amplitude. And—it works! Well, sort of.
It’s painfully slow. My first reaction? “F****ng JavaScript, why are you like this?” But hey, challenges make us stronger, right? Let’s see how I can speed this thing up.
Leave a Reply