Do You Know Exactly Where Your Fingers Are??

Winter Term is one of my very favorite things about Oberlin! I won't give you my personal take on why it's awesome and fun and rejuvenating, as I'm pretty sure our last 15 posts have been about that, but suffice to say, winter term is the best.

I will however, tell you about my project this year: Making a pair of Sensor Gloves!

This year's project ended up going in a very different direction than I had initially intended. Truth be told, I had forgotten to set up an off-campus opportunity for myself, and realized the project proposals were due only 12 minutes before the deadline. Eeek! So in a mad rush I spewed some stuff I thought I might want to do onto the page and sent it off to my favorite professor, hoping she'd do me a solid and approve it. Thankfully she did, but that's neither hide nor hair because I actually did nothing I said I would!

What I did end up doing when Winter Term rolled around, was to build a pair of sensor gloves that would track the spatial position of my hands and fingers. My inspiration for this project sparked from two fronts. The first was a live performance I saw two Timara students give late last semester. They had built an elaborate contraption involving harnesses, cigar boxes and 40 feet of piano wire, and they got up in front of a whole gaggle of people crammed into the Petal gallery in the basement of Bibbins to show it off. Their instrument consisted of climbing-style waist harnesses, affixed to heavily modified cigar boxes full of electronics, with mounting points for 6 metal strings connecting the two performers. They backed apart to the length of the wire, then leaned back against each other, counter balancing and using their weight to tune the strings. One performer used a bow on the strings stretched between them, while the other perfectly mirrored their partner's movements with a fingerless glove fitted with an accelerometer, the motion of which controlled digital effects applied to the amplified strings.

It was a pretty amazing performance! And caused me to want my own pair of accelerometer gloves, immediately.

So, after some online poking, I stumbled across the second source of inspiration: Imogen Heap's Mimu gloves. I linked to the website so you can see for yourself what they can do, but in short they're a suite of sensors on each hand that can learn gestures, track complex motion, and are great for live performance. They're an awesome piece of kit and used by very very famous musicians, but unfortunately they're about $4,000 and currently unavailable to mere mortals such as myself. But again, me want!

So I decided I was just gonna make my own! The Mimu gloves started life a decade ago as janky, hacked-together garden gloves, how hard could it be?

Turns out, quite hard! And expensive. I do not, in fact, have nearly the electronics and design experience as Grammy-award winning multi-media artist Imogen Heap, but after quite a bit of research and scrapped design attempts, I stumbled across a whole community of people trying to build open-source sensor gloves as cheaply as possible. They were just approaching it from a totally different angle than I was: Virtual Reality controllers. In the folks behind the collaborative LucidVR project, I found a slate of home-brew hardware and software that worked to track the position of each finger. And in the newer versions, use motors to physically stop your fingers from closing so that when you grab something in VR, it feels like you're actually holding an object! I also found, critically, a whole group of people who had done all the work of experimenting and prototyping these designs and settling on something usable and not terribly expensive to produce at home if you have the right tools. All you need is ~$50 worth of electronics, access to a 3D printer, and some WD-40.

Needless to say, I was overjoyed. I decided to scrap my own original glove designs and build myself a pair of these VR hand-tracking gloves in order to eventually misuse them, redirecting the data not into a VR game but into a format I could use to control all sorts of live audio-visual work.

I wasn't entirely sure if it was going to work, as I really hadn't figured out how to deal with the software side of things when I jumped into building the gloves, but I just went for it. It took many many hours of printing time, one jammed 3D printer and one that suffered a disastrous PLA explosion, two dozen retractable badge reels cracked open and gutted for their spiral springs, two nights spent winding them back into custom housings with the smallest pliers I could find, and many a long evening of cutting, stripping, twisting, soldering and shrink-wrapping wire, but it worked!

Each of those round things is holding a potentiometer with a spring-loaded rotating spool keeping tension on it as a string attached to each fingertip pulls the spool and rotates the potentiometer, giving me a numerical value of exactly how bent my finger is. Multiply that by 5 and you've got a stream of data for each finger!

Thankfully the software side went smoother than I feared, and I was able to pick up the data coming in over both USB cable and Bluetooth and process it with my favorite jack-of-all-trades live visual performance software TouchDesigner. I do a little processing on the incoming data to split it into individual values and to check if any of the gestures the glove recognizes (at the moment only a pinch, a fist, and a finger-gun) are currently happening, and bam! I've got a brand new way to control my live visuals.

Now of course, I actually have to do something with it! I must admit, I finished the gloves a couple days before I left for a superbly relaxing trip to visit some friends who had just walked from the Caribbean side of Costa Rica all the way to the Pacific (more on that in a later post), and my motivation to make a whole live performance with my new toys sort of evaporated, but I'm looking forward to using them in both the studio art class I'm taking this semester and hopefully some secondary lessons. And maybe my illegal junior recital!

All in all, this was a very fun project! It definitely pushed my knowledge of small electronics work and gave me a crash course in 3D printing, while at the same time letting me explore interacting with the very digital art I make in a more tangible way. If anyone out there is thinking of making a pair, I'd definitely recommend.