Posts Tagged ‘surface interface’

Proposal for Museum Lobby TUI

October 30th, 2009

Problem

The Davis Museum lobby has no method of informing visitors about their museum experience to come or contextualizing it afterwards. Without maps, interactive guides, and customizable tours, visitors are ill-informed of what the museum has to offer and why the works are important. Furthermore, this dearth of information and welcoming in the lobby prevents potential visitors from feeling as if the museum is a resource freely at their disposal.

Users

The users of the system will be visitors to the museum, seeking information about the exhibits and to customize the museum experience for themselves. We want to reach as many demographics as possible, addressing the needs of student visitors and providing the ability to collaboratively structure a tour. We believe a tangible approach is optimal because it is uniquely inviting within the lobby space, it is collaborative, and it offers a very fluid basis for our own application development.

Proposed Solution

We believe that developing a Surface application to be used in the lobby of the Davis Museum would be an optimal solution. For the aforementioned reasons, it will draw in new visitors and increase the welcoming aura of the space. Furthermore, the lobby is a unique space that visitors occupy before and after their viewing experiences. A Surface application in this space has the potential to work as a learning tool, extending the visitors’ knowledge, experience, and associations between works.

Untitled1

The user can choose from the surface an exhibition or piece to learn more about. Through gestural interactions with the surface, he or she can gain contextual knowledge associated with each piece (e.g. a cultural context for the piece that must be observed or experienced) that cannot be displayed in the physical exhibitions. For many, the surface application can replace the traditional brochure, providing interactive content and media about the museum history, maps, featured exhibitions, upcoming events, and contributors. It can further be used to collect, assemble, and share a mosaic of experiences from previous museum visitors.

Untitled2

Extended components of this project can incorporate mobile or other handheld technology.

Related Work

The ARCHIE project supports cooperative gaming in museum environments through handheld devices. It presents a framework for creating mobile guides that potentially enhance social relationships between visitors and stimulate youth to visit museums. Their goals are similar to ours in that we both aim to improve and enhance museum learning.

http://doclib.uhasselt.be/dspace/bitstream/1942/9128/1/Training%20Social%20Learning%20Skills%20by%20Collaborative%20Mobile%20Gaming%20in%20Museums.pdf

MobiTags allow museum curators to gather and analyze semantic, social, and spatial navigation in museums by tracking the navigation of visitors within the venue. This information can be used to evaluate the physical placement of art and resources within a venue in order to improve the visitor’s museum experience.

http://www.jonbax.com/static/mobitags.pdf

Collaborative/”Scaffolding” museum learning games and tools

The MUSHI TUI supports cooperative learning in informal environments such as museums. Museum visitors interact with each other through handheld devices and observe the results on a tabletop monitor. While this TUI supports cooperation between multiple users, the experiences are individual and physically separate. Their goals are similar to ours in that we both aim to improve and enhance museum learning.

http://tigger.uic.edu/~llyons/papers/chi_07_dc.pdf

No comments »

Posted in General

Tags:

Analyzing reacTable Through Reality-Based Interaction

October 8th, 2009

In 2008, a group of HCI researchers from Tufts Department of Computer Science and MIT Media Lab proposed the notion of Reality-Based Interaction (RBI) as a concept that ties together emerging human-computer interaction styles. Based on this concept, they provided a framework that can be used to analyze and understand new interfaces and their interaction techniques. Here I’ve used their framework to analyze the reacTable TUI.

Reality-Based Interaction (RBI) Themes

The reacTable’s basic interaction techniques build directly on the user’s knowledge of naive physics (NP) and physical space (EAS). To add a controller to the system, the user simply picks up a puck and places it on the surface. The user can move the puck across the surface or rotate it. Doing so will result in digital visual feedback displayed upon the physical surface and audio feedback from the speakers placed in the physical environment (EAS). Because the reacTable surface is round, the user can move his or her body around the TUI to change viewpoints, which leverages the user’s ability to move his or her own body to different positions in the environment (EAS). Because there are no privileged points-of-view or points-of-control, reacTable also encourages collaboration and competition between multiple users, drawing on their social interaction skills (SAS).

Tradeoffs

Designed to provide an immersive experience for the user and onlooking audiences, the reacTable utilizes a large, table-top surface with a camera, projector, speakers, visual and audio synthesizers, and many connection tools. They also defined the interaction environment, typically a large room, where there are no other lights or noises to distract the user interacting with reacTable. By making such decisions, the designers, by trading practicality for realism, incurred some space, size, and power consumption costs.

The reacTable TUI strikes a nice balance between reality and other qualities including expressive power, efficiency, versatility, ergonomics, and accessibility.

No comments »

Posted in General

Tags:

Analyzing reacTable through TAC Paradigm

October 8th, 2009

Token and Constraints (TAC) Paradigm

The TAC Paradigm describes a TUI as a set of TAC relationships, defined by tokens and constraints. In the example of the reacTable TUI, the tokens are those graspable pyfos – the tangible music controllers to be manipulated above the table-top surface. There are 6 groups of tangibles, including Generators, Audio filters, Controllers, Control filters, Audio mixers, and Global. The constraints are the bounds of the table-top surface, serving as a reference frame for the user’s interactions with the tangible music controllers, as well as the tangible music controllers themselves, as they relate to each other atop the surface. The variables – the digital information coupled with the tangible music controller tokens – are the behaviors of each of the tokens and the connections the tokens can make with each other.

Properties of the TAC Paradigm

There are five properties of the TAC Paradigm: Couple, Relative Definition, Association, Computational Interpretation, and Manipulation. We’ve already described Coupling as the relation between the variables and the tokens of the reacTable. We’ve shown Relative Definition exists when the tangible music controllers serve as both tokens and constraints depending on the table-top context. We’ll proceed by showing the latter three properties.

The Association property says that a new TAC is created when a tangible music controller A is placed upon the table-top surface. Because a tangible music controller is both a token and and constraint, the TAC consists of a token – the tangible music controller A – and a constraint list including the other tangible music controllers already on the table. By removing one of the neighboring controllers B, you remove not only one constraint from the TAC of controller A, but you also destroy the TAC associated with controller B, the token associated with that TAC. Likewise, when you remove the controller A from its associated TAC, you destroy the TAC and remove one constraint from each of the other TACs associated with the table-top controllers.

A token has Computational Interpretation when it is associated with its constraints. For example, when the user places a token atop the reacTable surface, it plays sounds and displays visuals (from the specific behavior of the specific tangible music controller). When the user surrounds the token by yet more constraints, any neighboring tangible music controller constraints, the token has more levels of computational interpretation (from the connections between the tangible music controllers). When the token is removed from the table (its constraint), the sounds and visuals associated with that token stop.

The Manipulation of reacTable is continuous. The manipulation of tokens and constraints atop the surface result in continuous feedback.

TAC Relationships in reacTable TUI

We can now describe the TAC relationships in the reacTable TUI:

reacTable TAC table updated

No comments »

Posted in General

Tags:

reacTable: Live Music and Tabletop TUI

September 28th, 2009

Design Concept

This TUI, presented as a tabletop music instrument controlled by pucks placed on the surface, revolutionizes the way we synthesize and perform sounds. The pucks used in the interface are easy to maneuver — they can be spun, flipped onto the other side, removed, or replaced. Depending on, for example, the orientation and relative position on the surface, the interface provides audio feedback and beautiful visual representations of the synthesized sounds.

Use Scenarios

It was designed for a diverse demographic of users, including visitors at a museum installation, casual users, and professionals in concert. Since there are no privileged points of view or points of control, it provides for a highly social and collaborate user experience. Unlike the TUI video editor or TERN, the possibilities and combinations on the reacTable are infinite, and the user’s continuous actions will result in realtime response.

Interaction Techniques

Each object represents a modular synthesizer component and has a specific function. Objects generate, modify, or control sound. A set of rules connects objects depending on type, affinity, and proximity to their neighbors.

The visual feedback projected around an object conveys information about the behavior of the object, as well as draws the connections between the object and its neighbors. The visual feedback can also convey the real sound waveforms. Users interact with the interface given these visual and audio cues.

Implementation

Components of the reacTable InterfaceUnder the tabletop surface, the TUI uses computer vision (camera) and a projector. The camera loads video into reacTIVision, which analyzes the markers on the pucks and their placement on the tabletop surface. The algorithms spit out a TUIO into the connection manager, which then transmits the connection info into an audio synthesizer and visual synthesizer. The visual synthesizer determines the video to be displayed through the projector. Meanwhile, the audio synthesizer not only plays the audio but also produces for the visual synthesizer the waveform data to be displayed.

Example of fiducials to be placed on the sides of pucks

Above the surface, markers and pucks tagged with fiducials provide the information needed for the detection algorithms written for reacTIVision. These markers and pucks are ideal for this interface because they allow for infinite combinations of patterns for information representation, and they are cheap to product. The latter reason is a particularly compelling for use scenarios such as a children’s museum, where markers and pucks can easily be damaged or missing.

Interface-related Skills and Knowledge

There are no skills required of the user, not even any related to the application domain, in order to perform on the reacTable. A user could certainly benefit from having knowledge in the physics of audio, since each synthesized sound is a function of a certain sound wave. The interface relies on a specific syntax comprised of 6 functional groups, including:
- Audio generators
- Audio filters
- Controllers
- Control filters
- Audio mixers
- Global objects
The pucks from each of the 6 groups have vary in their physical shapes and associated audio and visual feedback. They are designed to affect one another so that the final synthesized sounds can be very nuanced and complex. For example, the sound resulting from an audio generator can be further manipulated by N neighboring controllers. Thus, the reacTable is a venue for a variety of complex relationships between each type of puck. In order for an audio mixer to perform its function, it must be associated with 2 audio generators each with only 1 audio output. The audio mixer itself takes 2 audios inputs and returns 1 audio output.

Even without knowledge of these complex relationships, however, a casual user can still interact with the surface and enjoy an experimental experience.

To conclude, the reacTable’s continuous and multidimensional interaction combines “precision with freedom” and “expression and creativity with entertainment.” Most importantly, it successfully avoids the files, folders, and hyperlinks metaphors still dominant in HCI.

No comments »

Posted in General

Tags: