Wiimote Experiments

In the interest of time and money I’m giving the Wiimote a spin as I think about hardware for gourdo v.3. The Wiimote has 11 buttons and a 3D accelerometer transmitted wirelessly over Bluetooth for $40 US. They’re available for a similar price nearly everywhere – I got mine at Target in Seekonk, MA. SparkFun.com sells the accelerometer in the Wii (the Memsic ADXL330) for $34.95 and a variety of Bluetooth transmitters starting at around $50 US. They also sell a package they call the WiTilt with accelerometer data over Bluetooth for $109.95. In addition to cost, I’m exploring the Wiimote in hopes of an increase in data stability over gourdo v.2. My current solution and my preferred homemade Bluetooth solutions depend on Max’s serial object – something I’ve never had great luck with. In gourdo v.2 I learned that serial communication on the PC is de-prioritized in such a way that I receive data drops under heavy processor usage (in Mac tests the data was more reliable but audio was severely sacrificed). In the past I’d considered flaws with serial object implementations a product of my own limitations, but recent conversations with Emmanuel Fléty convinced me otherwise. Emmanuel abandoned serial in Max years ago (preferring instead the imperfections of MIDI), so I feel justified in so doing. With the Wiimote, I can use third-party Max externals which see the data as a human interface device (HID).

There is currently an excess of Wiimote "hacking" information of the web. Wikipedia.com’s page on the Wiimote is a good overview of the device and there is an excellent technical specification page at Wiili.org. SparkFun.com did an excellent Wiimote deconstruction with pictures and commentary. Hackawii.com has a page dedicated to Wiimote hacks showing some popular applications of the device.

The WIili.org driver page is the best place to start for getting the Wiimote connected a computer. I’m using an Acer 8204WLMi which happens to contain a Wiimote-compatable Broadcom 2045 Bluetooth device. I’m able to connect to the Wiimote using XP’s system software and successfully view the Wiimote data in Max/MSP using the tk.wii object by Takeru Kobayashi (linked from the forums at Cycling ’74). On Mac the preferred solution appears to be the aka.wiiremote object by Masayuki Akamatsu. I’ve not yet tested the object for Mac, but in a peek at the help patch it appears that the data formatting is different such that the objects cannot be used interchangeably. To maintain cross-platform compatibility I’ll likely make the data collection patch an outside entity which sends incoming data into the primary patch. The correct data collector can then be launched on the side depending on the current platform.

Connection to the Wiimote was simple and data was immediately available in Max so I have begun to test the Wiimote’s sample resolution and rate. While the Bluetooth supports audio transmission, the Wiimote inherits any limitations inherent to the HID protocol. An article on ddj.com reports that HID implemented “mice are polled for data every 8 ms, and respond with 32 bits of data” and that “a keyboard transmits 64 bits of data over the same interval.” Because greater sensitivity is unlikely necessary for such applications these restrictions help keep power usage down, an essential feature in wireless applications. While such information about the capacity of the protocols is interesting, real-world (read: Max) performance may vary. As such, I have spent relatively little time investigating these two technologies opting instead for max-based testing with my existing gourd patches.

Data from with tk.wii object is 8-bit data reported every 5.02 milliseconds on average (with a 4.90 median and 5.86 in one test). The data is represented as a 0.0-1.0 value the tilt of the controller occupying 21% of that range – tilt values for the X-and Y-axis range from 0.396 to 0.608 with sharp accelerations using the entire range. Substituting Wiimote data in my hit-detection patch where taps of the device are reported as triggers proved successful. The data rate doesn’t appear to be high enough, however, to support directional tapping detection reliably enough. With tapping, the difference between subsequent sensor values is sufficiently large to distinguish those actions from tilting. I use a rolling buffer to hold the last 20 sensor values reported and, when a hit is detected, search for the maximum and minimum values in that set. If the maximum value is larger than the absolute value of the minimum, the hit came from the right, if not, it came from the left. Because the sample rate is too low, the peak value is not always captured. When this occurs the “recoil” value is usually recorded and is the highest value in the set, causing the incorrect direction to be reported. The data rate is more than sufficient for tilting applications.

The next step is to try the Tilt, Shift patch with the Wiimote and see if the data resolution over the tilt region is sufficient for performance with the granulator. While gourdo v.2 featured 10-bit data resolution, I believe that 8-bit data over the tilt range would be sufficient. The data from the Wiimote, however, represents the tilt range with only 56 values (the full 256 are for the entire sensor range). My next testing step will be to connect the Wiimote to the Tilt, Shift granulator to see if I’m satisfied with its performance. This will be my first opportunity to physically test the latency of the tilt and see if it’s low enough.

Short, Preliminary Dissertation Project Description

A Music of Vocal Technologies and Angolan Technologies of Voice in Literature

I will spend one year in Angola, beginning December 2007, to conduct the creative research for the dissertation project required for the doctoral degree in Computer Music and Multimedia Composition at Brown University. The final form of this project will be the live performance of a newly-designed electronic instrument which interfaces with a computer to play, process, manipulate, generate, and synthesize audio and video. The source material for this work will be created primarily in Angola and will feature texts by Angolan writers and the voices of Angolan writers reading their work.

This work examines themes of voice, writing, and technology in the context of cultural change. Considering oral cultural practices as technologies, the project will use voice and text to explore how Angolan writers draw from a rich set of oral traditions. The focus will be on the possibilities provided by an encounter between contemporary digital technologies and an Angolan literature successful at adapting narrative technologies to make important contributions to Lusophone and African literatures. Angola is an ideal place to explore these themes of technology, change, and adaptation: it has the fastest growing economy on the African continent, is modernizing at an historically unparalleled rate, and sustains a vibrant literary community. Of particular interest are the successes of Angolan authors in maintaining a uniquely Angolan voice through difficult periods of colonialism and civil war. I seek to trace this tradition through to the current generation of Angolan authors who maintain this voice and forge new ones in the present context of development and social change.

The portions of the project to be completed in Angola will be carried out in three stages. Stage one is focused on Portuguese study with particular attention to the language as is unique to Angola, including local vocabulary, accents, and speech rhythms. This stage includes initial contact with the literary community in Luanda facilitated by introductions from renowned writer Manuel Rui and through sponsorship by the União dos Escritores Angolanos (UEA, The Angolan Writers’ Union).

In stage two I will work with a variety of Angolan authors, their texts, and audio and video recordings of them reading their work to create short musical sketches. This composition allows writers exposure to the computer systems I have designed and allows them to evaluate their interest in further collaboration. The sketch production also allows me to tailor the system to the developing needs of the project and to change the system in response to input from the writers. These sketches will be performed for audiences in Angola as they are completed.

Stage three is dedicated to making audio and video recordings of authors reading the primary text or texts of the final project and to designing the overall compositional form of the work. Working with captured footage and with texts written or selected for the project by the collaborating authors, I will begin initial editing of the media and design of the performance system with on-going feedback from the collaborators.

The work will be finished after my return to the United States, incorporating final feedback from my academic advisors, and will receive an initial performance at Brown University. Recordings of the performance will be made available for permanent storage at the UEA, at the National Library and the National Archives in Luanda, and elsewhere as directed by the Director of the UEA and the Ministry of Culture. Every effort will be made for a performance of the final work in Angola at a subsequent date and for primary collaborators to receive invitation to and sponsorship for travel to the premiere.

The project is funded by a Dissertation Fellowship from Brown University and overseen by my dissertation committee chairman and by the Department of Music at Brown University.

So long PVD...

Many, many, many moons and nearly as many changes of plan later, the show has moved on from PVD and hit the road for an extended US tour before heading abroad. For work-related reasons our international departure was slightly delayed, and because it does little or no good to arrive in Luanda during the extended holiday period (not to mention the cost of holiday flights), we decided to remain in the US until the end of January. We’re currently in Atlanta, visiting (read: taking advantage of) friends and we hope to make a swing out west before we depart. Things are still up in the air, however, and no plane tickets for other US destinations exist.

Things that have been accomplished include packing our apartment and moving it to storage in Maine, delivering the lovely if irascible Ms. Egypt to her foster cat parents in DC, and passing my preliminary examinations to become a doctoral candidate and ABD. Technology experiments and dissertation proposal writing are officially underway with positive results in the early stages. I’ll be posting soon with results from max/msp experiments and hardware testing. I’m also going to post the short dissertation description I’ve been circulating and will be blogging bits and pieces as they come.