(transcript of a talk at On Borders and Edges)

my name is Kristina Anderson, and i am going to talk about touchable interfaces. i am a little more interested in ppl than technology, so that should become clearer from my work. today i am going to talk about a particular project called 'ensemble', which is really an investigation into sensors, i wanted to look at electronic sensors from a naive point of view. the things i am interested involve the idea of finding ways to get a way from box like machines and finding other ways to interact with computational matter. in this particular project I'm working with sound, basically this project is built on the notion of naive physics, the kind of physics we all know.

there is the kind of physics that people at universities know about, how light refracts thru water, and there is the kind of physics in which you know that if you let go of a glass of water it will fall to the ground and break. its an everyday, applied physics that we use in order to cross the street, or guess how heavy something is before lifting it, so you would use this basic, or naive idea of the physicality of the world. its a very practical physics, and i wanted to look at the same thing for electronics. i wanted to see if it was possible to make a naive understanding of how electronics works.

“naive physics investigates peoples knowledge of the everyday physical world. it is a common sense theory of how the world behaves.” p.Hayes

this is really the basis of the project, another aspect of the project is based on dressing up. as you can see form these embarrassing pictures of me as a child, this has been a lifelong obsession.

The system consists of 7 pieces of clothing; there are a pair of suspenders 0or braces, which react to pull, there are stretch sensors in them. there is a dress, with an accelerometer in it, which detects it movement. there is an umbrella, which has a pressure sensor in the tip so it makes noises as you push it toward the ground. there is a hat, which reacts to tilting movements. there is a bag which reacts to light. there are two jackets, there is a sonar in 2 of the sleeves which measures the distance between the 2 of them. all the costumes are wireless, with the exception of the suits.

the actual hardware I'm using, is built form game pads, which is an interesting way of making interfaces, they are really good tangible interfaces, they are touchable, you don't need to look at a screen to use them. so all we do, is rip them open and take out the parts we are interested in, and wire in new inputs.

the software we use is junXion (usb to midi) and Lisa (software sampler), available from steim.

so what am i doing with this? i am giving these things to children, around the age of 4 and leaving them to play as they want for about 45 minutes, and I'm interested to see what they do. what actually happens is that they arrive, we get rid of their parents, and then they tend to go thru a flow of experience, there are no computers, which i hide before the event, and since the output is sound, there are no screens. the only thing they see is dressing up clothes, which they understand how to use, they don't need to have it explained, so they just get going. then, after a while they begin to realise that these clothes are making sound and that they might be making those sounds. so they move into a second stage of trying figure out “if i make that sound, then how can i control it?”. of course the first thing to try to understand is how much noise can conceivable be made with these things? this stage can tend to get almost infernal. the next thing is of course, that if one thing makes a sound, then what else makes sound, does everything make sounds? I'm also including paper and pens in the room so they can can draw, and I'm interested to see what they draw, because children at that age using drawing to make sense of things. they are in a situation where they think that the world is a whacked out, strange place, where weird unpredictable things are happening, so they need time to try to come to terms with it, to understand how its working. so drawing really helps to work out how things work. The other thing of course is that if some children are drawing, they are not making noise, so its a chance for some relative silence, since if there a 7 children in the room all try to make noise it can get quite noisy and somewhat confusing and not everyone can make out what they are doing.

(shows video)

so, here are some children who have just discovered the sounds, in this bag are 2 light sensors, so the bag reacts with particular sounds as they walk towards the light. they open or close the bag, or they begin to do other things which change the amount of light entering the bag.

one of the things that happens in this particular clip is that this boy is wearing the dress as well as playing with the bag, the dress is making a whooshing sound, which he doesn't pick up on until later, he is only paying attention to the sound of the bag.

(shows video of jacket)

these are the 2 jackets which have sonars in the sleeves, they are measuring the distance between the jackets. here are some kids discovering how the jackets work. the jackets are very difficult to understand, since they are measuring something invisible, in this case they scratch thru a sound.

later he is trying got get the jacket working again, he understands the movement, but because he only has one jacket it doesn't work. he knows it somehow involves the other jacket, but he still cant quite figure it out, he is walking around, stepping on the other jacket. over the course of the workshop he came back to the 2nd jacket a few times, and eventually worked out a way of pointing the sleeves at each other. so in such a case, i am happy to leave them to discover things for themselves. i tend not to help them out unless they are frustrated.

(show video of hat)

here is the hat, which has tilt switches in it. this one boy only wanted to play with the hat, he wasn't interested in anything else. after a while, he gives the hat to his friend and walks over to where he left his shoes, he picks up one shoe and tilts it, picks up the other and starts to shake it the way he shook the hat. now this is a very interesting point, since the hat made sound, he was wondering “is this something i learnt how to do, does that mean i can make anything make sound, or is it just the hat?”. when the shoes didn't make sound, he tried a few other things in the room which didn't make sound, so he quite happily went back to playing with the hat. so this is an interesting testing of the borders, whenever you present a new ability or a new tool there come a point where it is natural to ask the question “what can i do, what cant i do, what is the extent of my new ability?”

(shows drawings)

This is an example of one of the drawings, this is by a girl called Mira, normally i don't interview the children, mainly because its just so difficult to interview four 7yr olds at the same time. so instead, they draw, and sometimes they will explain their drawings to me. So Mira was very concerned that i understood what she had drawn, she came over and said “this is a picture of the bag” and she explained “stuff comes into the bag and clown sparkles come out, thats how it works”. for me thats interesting because what happens is that i mix all the sound and its played over speakers sitting in the corners of the room, but I'm interested in the interaction. so we start to see that the interaction connects the sound to the object, even when the sound is obviously coming form somewhere else.

(shows video)

Here is one last video, near the end of a workshop, its a boy playing with the dress. what I'm interested in seeing in something like this, is that he is at the point where he has played for 45 minutes and he is almost treating this thing as an instrument, he doesn't care who is watching, he knows its making a sound, and he wants to see what it can do and how he can play with it.

The sounds that I'm using change all the way thru the workshop, so there are really simple cartoony sounds, than later in the workshop we introduce much more complicated things.

(demonstration of the hat and the bag)

Here you can see the bag, with the light sensors. now light sensors are really interesting, cheap sensors, they are the kind of sensors that turn on an outdoor light. every body i talked to about them thought it was a bit silly, but i thought ill make it anyway. my initial idea was that the bag could be played by opening and closing it. which is a bit silly, something like opening and closing a fridge for a light show. but what really happens, and something that i found out, is that light sensors are really interesting because of something the children did, they would turn the bag around, away from the light and play it by scooping up light and making shadows. it can be quite precise to use, something we didn't expect when we began, so the kids are really my researchers, trying to figure out what these things can do.