rhythmanalysis

I co-taught a Public School class this week — part of the series...

Fri, 25 May 2012 18:41:05 -0400

I co-taught a Public School class this week — part of the series On Time put together by Jonathan Basile. The imitable Jackson Moore discussed how music may have a biological basis as a faculty for synchronizing actions within groups of people, and he broke down the theories of James Tenney and Fred Lerdahl. I went from there to discuss some of Lefebvre’s structural approaches to examining rhythm in everyday life. We had a great discussion — my favorite points from the group included the idea that musical drones refocus attention on the body as a source of meter, and the suspicion that the CIA assassinated Coltrane because he was spreading subversive rhythms with his tunes.

Multitudes @ Cameo

Fri, 25 May 2012 18:28:47 -0400

Multitudes @ Cameo

I’m working on a new rhythmanalysis project for Eyebeam —...

Thu, 24 May 2012 11:08:01 -0400

I’m working on a new rhythmanalysis project for Eyebeam — working title is Forty-eight by Sixteen, it will be a video that is algorithmically edited with data that I generate while biking to work. Cadence, heartrate, respiration, GPS … this graph is looking at cadence (blue) vs heartrate (red) over the course of 30 minutes and 7.5 miles.

Familiarity is … enriched by an appreciation of seasonal...

Sun, 13 May 2012 14:34:32 -0400

Familiarity is … enriched by an appreciation of seasonal changes, such as the changing light, foliage and wildlife, inducing an apprehension of our enfolding within the larger temporal rhythms of a space of the changing same and the momentarily surprising.

— Tim Edensor

The end of ‘Tiger’ from our set last Saturday at a...

Thu, 10 May 2012 23:33:33 -0400

[Flash 10 is required to watch video.]

The end of ‘Tiger’ from our set last Saturday at a basement show. Killer crowd.

More from Multitudes.

Cutting records, literally, with favorite new collaborator and...

Sat, 05 May 2012 02:20:48 -0400

Cutting records, literally, with favorite new collaborator and artist Ted Riederer. We’re working on my new sonification piece, coming to Eyebeam in June.

HISTORICAL DATA FROM PACHUBE [code] Python (2.6), GPL Pulling...

Fri, 04 May 2012 17:17:09 -0400

HISTORICAL DATA FROM PACHUBE
[code] Python (2.6), GPL

Pulling historical data from Pachube is not entirely straightforward. I appreciate the design of their API, which is designed primarily to ingest live feeds, and otherwise encourages sampling data feeds at larger, regular intervals. However, for signal processing exercises, you really do want all the samples within a time range, and that necessitates a lot of requests (every 6 hours, paginated).

This script will pull the last week of data given a datastream identifier.

For the Sound Research Summit at Eyebeam last Saturday, I wanted...

Fri, 04 May 2012 17:16:00 -0400

[Flash 10 is required to watch video.]

For the Sound Research Summit at Eyebeam last Saturday, I wanted to develop a concept of sonification. Or musicalization. Or something having to do with some manner of legible sound that has a structural relationship to data. It’s something I’ve been resistant to exploring in the past, because it is so easy to lose any perception of the source (ie, what the data is actually about) in the audible result. One-to-one mappings threaten to be too naive (stock market goes up, pitch goes up), and complex mappings might as well be driven by a random number generator. Nonetheless, sonification is a potentially important rhythmanalysis strategy, so I wanted to try.

I am interested in primarily time-domain signals, so I think the key point of departure is to use sources that have inherent periodicity and to tease that out. I cant believe I havent done a project with pachube.com before — it’s a clearinghouse of open data for Internet-of-things enthusiasts, with DIY feeds of everything from the temperature of some guy’s fish tank to radiation level at Fukushima. Through the Pachube API, I downloaded historical samples of arbitrary feeds (code here). I wanted the sound to reveal any periodicity and highlight variations from it — to check for periodicity, I applied autocorrelation to the signal (getting comfortable with this technique and assembling a set of python functions was a super useful part of this exercise). What I discovered, unsurprisingly, is that many of these data streams cycle diurnally, and so for the purposes of this event I decided to just apply a 24-hour cycle across the board.

I downloaded a weeks worth of data for each feed, and chunked each one into 24-hour segments — each segment essentially becomes a measure. After smoothing and normalizing the signal, I resampled it to a musical pulse of 12hz. So the result is basically a single parameter that changes in musical time with a high and low value in each measure.

To give that substance, I opted for hardware. I am enamored of the Meeblip synth for its inherent limitations, hackability, and lo-fi sound (and price). Running it through my pedal board offers some nice possibilities, and hews closer to a more punk/noise sound which is more my thing rather than clean electronic aesthetic that screams futurism. I’m running everything in python and sending MIDI to the synth. I mapped the changing parameter to filter width to give a sense of pitch variation without actual tonal change. Each data source drones on a random fundamental.

For the purposes of the event, I fell upon the concept of using an old CRT display to show a visualization of what was being sonified. In building a display for Mary Mattingly’s Flock House (which I haven’t yet posted about), I was a little dismayed at having made a shiny touchable thing on a nice flat monitor, as it points too much toward screen-based interaction — this time, I wanted to invoke an old radar display, and I made a spiral which arrays the data with a turn for each day. Again, animated in python.

Artistically, this isn’t satisfying as a complete piece. I’ve created this sculptural display, which makes me somewhat uncomfortable, as I’d like it to have a greater relational component. In the context of the summit, that came for free — the piece became a focus of roving improvisers, and established a nice counterpoint with Ben Houge’s sonification work 10-ft over (he tuned his work to fit harmonically with mine).

But the next iteration of this idea will have to feature some sort of comparison between multiple sources to make it work. It’s about the inherent relative musicality of phenomena and the participation of the viewer in that relationship. I’m brainstorming ideas.

I regret that my documentation for this sucks, though it’s motivation to produce a followup.

I had the privilege of participating in an advisory summit for...

Mon, 30 Apr 2012 21:19:16 -0400

I had the privilege of participating in an advisory summit for StoryCorps today — it was a fantastic conversation about the possibilities an innovative oral history archive might fulfill in the shifting media and research landscapes. You may have heard some of StoryCorps inspiring content in edited form on NPR. What people may not realize is that each interview is 40 minutes long, and to date there are 41,000+ recorded stories in the database. That makes StoryCorps an audio repository of singular importance, and not just from a content perspective. It’s both a corpus of speech data spanning geography and demographics of immeasurable interest to linguists and computer scientists, and it’s a source of reference material that I think warrants mention in the same breath as NYT or Wikipedia. The trick, of course, is making that content accessible — there is currently no externally exposed search capabilities, let alone a programmatic interface that would make mash-ups with NYT or NYPL or innovative apps readily possible (my personal priority was to push deep linking of audio segments with canonical urls, to enable StoryCorps references in things like Wikipedia articles). All that is going to change as their amazing team plans the future of the platform. They are looking for ideas and collaborators, so get in touch.

image credit WXXI.org

Last weekend I participated in the Sound Research Summit @...

Tue, 24 Apr 2012 00:44:32 -0400

Last weekend I participated in the Sound Research Summit @ Eyebeam, a bazaar of sound-works in progress and experiments in listening curated by Jackson Moore. I thought the event was successful in juxtaposing a diversity of vocabularies through which we might approach sound, from Kyle Kessler directly manipulating a plate reverb (shown) to Ben Houge’s abstracted sonification of data from sensors at MIT. The result both embodied the spirit of research by challenging any particular framework as well as served as a collective performance in which the audible and conceptual boundaries between the works were continually renegotiated.

Particularly rewarding for me was the chance to work with Christine Sun Kim. Christine has been deaf from birth, but works with sound as her primary medium. She employs various strategies (such as transducers, piano wire, and feedback at the summit) to emphasize the tactile nature of sound and make it perceptible to her. However, I think that to suggest that she is simply transposing between senses is incorrect — it’s also the semantic and cultural components of sound, how and why we encode it, that comes across in what she does, and which for her I imagine is in many ways more readily approachable. Assisting her with her piece brought this into relief as we debated the qualities of sound it produced and I stumbled over my vocabulary.

More on my work for the summit next post.

56k RINGTONE I end up talking about the aesthetics of obsolete...

Mon, 23 Apr 2012 21:19:00 -0400

56k RINGTONE

I end up talking about the aesthetics of obsolete technology more than I’m comfortable with as it is, but somehow the other day it suddenly seemed really imperative to have a 56k modem handshake as a ringtone. @akamediasystem made me do it. This is already everywhere online, but I’m posting my version here anyway.

modem.m4r.zip

New Multitudes video by David Feinberg for “Horse”,...

Sun, 01 Apr 2012 11:01:29 -0400

New Multitudes video by David Feinberg for “Horse”, off our album Twelve Branches (download the album $free).

I am incredibly excited to announce that I’ve joined...

Sun, 11 Mar 2012 14:24:18 -0400

I am incredibly excited to announce that I’ve joined Eyebeam Art and Technology Center as a resident artist for the spring / summer cycle. it’s been a blast meeting everyone, and I’m intimidated and inspired by my new colleagues. The residency will complement my work in personal data and sensor networks at the Times — I’ll be exploring similar themes, but in more of a sound-art medium and context.

Specifically, Im going to try an execute the first iteration of a concept I first presented at Conflux in October (and named this blog after), The Rhythmanalysis Lab. Taking the essays of Henri Lefebvre as a starting point, I want to develop a methodology for investigating rhythm as a primary object in personal, architectural, and urban spheres.

Ideas, collaborations, talk to me. More to come.

This weekend at AAG (Association of American Geographers) Annual...

Sun, 26 Feb 2012 00:23:36 -0500

This weekend at AAG (Association of American Geographers) Annual Meeting. I presented the following paper in a session led by Seth Spielman (who does some amazing work identifying “personal cities” through geopaths). And I got to hang out with the imitable Sarah Williams.

OpenPaths: A new approach to aggregating personal geographic data

The collection of personal geographic data from mobile devices is a ubiquitous practice of service providers and application developers. These data are being stored, analyzed, and monetized primarily by corporate interests; there is limited agency for individuals over their own data. Awareness among the public regarding the value of their personal data is nascent. OpenPaths, created by the Research and Development Lab at the New York Times Company, is a platform and a model and a platform that demonstrates the collective value of personal data sovereignty. It was developed in response to widespread media coverage of the obfuscated but accessible location record generated by all Apple iOS devices. OpenPaths participants store their encrypted geographic data in a cloud infrastructure while maintaining ownership and programatic control. Projects of many kinds, from mobility research to expressive artwork, petition individuals for access to their data in exchange for a stake in the outcome of the project. Ultimately, we would like to activate the practice of “participatory sensing” on a large scale in a way that self-regulates the creation of ad-hoc geographic datasets. Furthermore, within a theoretical context, OpenPaths moves beyond locative media’s primary concerns with connectivity, the coupling of data to place, and spatial representation to address the components of an ethical implementation of crowd-sourced geographic systems in the age of “big data”. How can we seat the individual in a mode of control over personal geographic narratives in a society in which locative media has become banal?

Here’s me at the Internet of Things Meetup on Thursday. I...

Sun, 26 Feb 2012 00:11:59 -0500

Here’s me at the Internet of Things Meetup on Thursday. I presented OpenPaths while Jake evangelized Data Without Borders, and @edborden tried to get us drunk. Good times.

MULTITUDES w/ Erostratus, Cathy, Sun Ladders we’re playing...

Fri, 10 Feb 2012 17:53:00 -0500

MULTITUDES w/ Erostratus, Cathy, Sun Ladders

we’re playing tonight at Lulu’s, in Greenpoint, $free, starts at 9

PUCK: PUCK of Ubiquitous Contextual Knowledge™Internet-of-things...

Thu, 09 Feb 2012 11:44:00 -0500

PUCK: PUCK of Ubiquitous Contextual Knowledge™

Internet-of-things / ubiquitous computing … these concepts have been central to NYTLabs thinking over the last year (and related to much of my past work 1, 2). They are old ideas, but we are finally seeing their fruition in interesting consumer products on the market.

I’m into two concepts in particular: Twine and GreenGoose. Both are looking at dirt simple, generalized hardware (particularly the mighty accelerometer) that wirelessly spit data up to the cloud. Server-side software can then contextualize that data to serve any number of social purposes, like alerting you when the mail comes (Twine) or keeping tabs on whether the dog is getting fed (GreenGoose). It hits that middle range of Greenfield’s notion of scale — the level of the room (as opposed to the individual, building, or street) — which I think is a particularly ripe area.

So in tribute I wanted to create our own version at NYTLabs (after acronym brainstorming w/ @mboggie). The idea is a small, generalized piece of hardware that you can attach to a physical thing in your world which can be programmatically assigned to monitor interesting events. It should have long battery life and report data from the accelerometer and other sensors wirelessly. Its logic should be on the server somewhere, not burned into the device, and it should have HTTP and UDP interfaces. And finally, in my twist on the concept, I want it to respond to touch, so that if you pick it up, it turns into a high-resolution controller, like a Nintendo Wii. The idea is to then have hundreds of these all over everything. I’ve been picking at this the last few months, and finally have a proof of concept running.

The core component of PUCK is the XBee. XBees rule. A network of lightweight components is in itself aesthetically appealing, and in essence, all I want to do here is hook a sensor up to one. The XBee is smart enough that this can be done without the help of an Arduino or other microcontroller, which is essential to keeping down the cost.

The path I went down and should not have is trying to figure out how to program them directly in python over the serial port. Instead, Andrew Rapp has an awesome java library and a wealth of documentation that is the way to go. I got hung up in that I didn’t just want to collect data from the XBee, I wanted to reprogram its behavior on the fly, using ‘API’ mode — this library has all that hard work in place.

In any case, the rest of the design here comprises an accelerometer, tilt switch, and LED. The tilt switch is able to wake up an XBee — it’s quite sensitive, so moving the device at all can take it from a mode where its just periodically reporting data and put it into high-frequency reporting mode. Finally, the LED just indicates when the puck is awake. When we get a MakerBot I’ll design a housing.

On the server side, I’m pushing the data into Redis and rebroadcasting it via OSC. The next step is to figure out how to architect the signal processing and the nature of the interface.

But I’m far from a hardware ninja, so I’m pleased to have something together that I can experiment with. Updates to come.

Some OpenPaths news: in the last couple of weeks we’ve...

Fri, 03 Feb 2012 13:31:00 -0500

Some OpenPaths news: in the last couple of weeks we’ve revamped the service and launched iPhone and Android applications — it’s been exciting to hear about people using the platform again (OP on Twitter).

Flowing data posts:
“There are a lot of ways to collect your location, whether it’s for journaling and personal reflection or for sharing with others, but it can be tricky making use of your data once it’s stored behind company servers.”

OpenPaths is different.

Need some color on this blog. Heres some detritus from...

Fri, 13 Jan 2012 11:02:00 -0500

Need some color on this blog. Heres some detritus from data-sketching yesterday.

CLUSTERING DATA IN PYTHON [code] Python (2.6), GPLPreviously, I...

Tue, 10 Jan 2012 15:07:00 -0500

CLUSTERING DATA IN PYTHON
[code] Python (2.6), GPL

Previously, I demonstrated clustering approximate geographic points from OpenPaths to identify places of interest. Heres the code I used to accomplish that, which can be applied to any dataset, it doesnt have to be limited to two dimensions. The algorithm is known as agglomerative hierarchical clustering because it works by repeatedly grouping the closest two nodes together, starting with each data point as a node, and ending with the entire set in a single monster node. (“Closest” is defined by any distance metric; for many applications, it will be euclidean distance, but for geographic data, Im using the haversine distance formula.) Along the way, youve constructed a binary tree which represents the hierarchical relationships between the vectors in the set. The result is a kind of ad-hoc taxonomy, and is frequently used to hypothesize relatedness between images, documents, proteins, users, etc. (heres a nice diagram courtesy Razvan Musaloiu-E.)

To use agglomerative clustering as a classifier, however, what we really want is just a flat set of clusters. It’s akin to choosing the branches of the tree that best represent the natural divisions in the data. Cut too close to the trunk and the clusters will be too general; cut by the leaves and youve got too much noise. With geographic information from OpenPaths, at least, we have a heuristic to decide what is appropriate — the platform is only going to be accurate to a quarter-mile or so, so we can look for the largest branches that do not exceed that limit. The code provides a method, get_pruned, that takes such a parameter and returns a resulting set of clusters.

Clustering packages for python are out there, but I prefer touching the code and this is a dirt simple implementation that nonetheless should prove effective for most creative coding purposes.