I am delighted to announce that the Cafegirl.org website has been revamped in Flash! This was a project that I took on for my ARTS 255: Dynamic Interface Design class. Drawing heavily from my practice with ActionScript 3 and my new understanding of XML, I designed and implemented a sleek and attractive interface supporting portfolios and image galleries. Check it out under the “Art Portfolio” section. Enjoy!
DavisTUI: Progress Report 5
December 12th, 2009 No comments »While running the newest version of our prototype, I ran into some unexpected difficulties. Whenever I touched a Word object to see the associated artwork thumbnails, the application would halt with an Operation Exception. After some investigation, I realized that the thumbnail images in each grid were previously assigned as logical children of the grid generated from previous contact with the same Word object.
I had designed the Word class to generate, remove, and regenerate a grid for each contact up and contact down event on a Word object. I couldn’t find a simple way to logically disconnect the thumbnail images from each grid after each Contact Up.
I later realized that the getImage() function in our Art class simply returned the image instance variable generated with the Art object. So, I revised the getImage() function to return a new Image object generated from the thumbnail image file.
The new method:
public Image getImage()
{
Image image = new Image();
String path = "Resources\\" + imgFile;
image.Source = new BitmapImage(new Uri(path, UriKind.Relative));
return image;
}
Afterwards, I:
- Continued to finesse the look and feel of the grids populated form the Word objects,
- Removed canned data and got the surface to display working results for my parsing of the Artworks.xml file,
- Wrote event handlers that would display the associated for the “Movement” and “Medium” tags,
- Got screen to display working results for tag event handlers from my parsing of my Collections.xml file.
I still had some problems to solve regarding each token’s tag event handlers. Previously, when a “Movement” or “Medium” token was placed on the surface, all of the items were cleared from the screen, including any Art and Panel objects that were generated. In the intended version of our application, only the Word objects populated on the native screen would be replaced by the Word objects associated with “Movement” or “Medium”.
The solution was a RemoveWord() function that removed each Word object’s ScatterViewItem from the main ScatterView, which was kept track of in a List structure of ScatterViewItems.
private void RemoveWords()
{
foreach (ScatterViewItem svi in sviList) {scatter.Items.Remove(svi);}
}
Given the high volume of programming that had to be done in 3 weeks, Lia and I can both agree that this project provided a valuable opportunity to practice our programming logic and modularize the problems at hand. Although we often worked on separate tasks, we found it necessary to communicate with each other about our programming problems and put our minds together to solve them.
Our desire to enhance the way people see and interact with the world, aided by our tenacity to make it happen, has allowed us to produce this final functional prototype of DavisTUI:
DavisTUI: Progress Report 4
December 8th, 2009 No comments »Towards improved information architecture by redefining the hierarchy of our classes, I moved the generation of each Word object’s ScatterViewItem and associated event handlers from the Surface Window1 class to the Word class. Further, for our input and output functions, I created two XML files containing input data for dynamically generating Word and Art objects on the Scatter View:
- Collections.xml, which contains the words to be displayed on the screen as well as the ID’s of the artwork associated with each word, and
- Artwork.xml, which contains the ID and tombstone information (i.e. metadata) for each artwork.
I wrote methods to parse the XML files and got the application to display dynamic results from Collections.xml.
At the cost of a GUI-like appearance, we are temporarily using TextFields and Buttons to test our I/O functions.
After this work session, I met up with Lia to touch base and resolve some other issues we had with the event handlers for Panel objects, which receives and displays user responses to the artwork. Comparing her event handlers for populating ScatterViewItems to mine, we found no syntax errors and no difference in our logic. Still, her event handlers were not functioning as expected.
We later discovered that the solution was simply to open the Surface Simulator application before debugging on Microsoft Visual C#. This was an unfortunate oversight that cost us about an hour of time. It is our firm belief that this frustration could have been avoided had Microsoft designed Visual C# with better application of user experience principles.
After the fact, Lia and I worked briefly to create the tag event handlers for the “Movements” (previous referred to as “Time Periods”) token.
DavisTUI: Progress Report 3
November 30th, 2009 No comments »
Native screen of our DavisTUI application
With regards to the Word objects, which are essentially ScatterViewItems displayed on the native screen, I finally decided to display each word (e.g. “Baroque”, “Renaissance”) using a String datatype, rather than a SurfaceButton or Label data structure. This design choice was made primarily because String datatype was the simplest to implement and sufficient for our purposes. Although conveniently equipped with contact event handlers, the priorly used SurfaceButton data structure had a toggle-like appearance on contact and consequently could not be moved across the ScatterView. Labels were also more complex than Strings than required for the purposes of our Word object.
Thumbnails appears when the user touches a Word object.
From there, I implemented the Contact Down event handler for the Word object, which gives the appearance of generating a selection of artworks about which to learn more. Rather, the event handler replaces the String in the ScatterView Item with a Grid object. The grid is populated with the word originally presented in the form of a String and, additionally, with the thumbnails of related artwork surrounding the word. For the purposes of the prototype demonstration to take place tomorrow in class, I used canned images for each of the thumbnails.
Towards enhancing the look and feel of our application, I also found a great background — a beautiful blue and black gradient that is darker along the edges. I am happy to report that, at this point, we have achieved high fidelity in the of the look and feel of our native screen!
DavisTUI: Progress Report 2
November 24th, 2009 No comments »As I set back to work on the SurfaceWindow1 file, I debated whether I should use Labels or SurfaceButtons to display the content of each Word object. I decided to use SurfaceButtons, because I knew I could rely on there being built-in the Contact Down, Changed, and Up events.
I also contemplated what the best way to populate thumbnails around each Word would be. Playing around with each of the built-in Microsoft Surface applications, I finally discovered Element Menus, which had a sleek and sexy look and feel. Unfortunately, I wasn’t able to figure out how customize each menu in a way that was helpful to our project, so I instead decided to use Grid objects to display the appropriate artwork thumbnails surrounding each word.
I had some difficulty making the Grid object appear. I ran many tests on the Grid generating function, as well as the event handler for Contact Down. At some point, I could make the cell dividers from inside the Grids appear independently. But I eventually realized that the issue was with updating the width and height of the ScatterViewItem containing the Grid to those of the Grid itself.
Reflections on Visions of Invisible Computing
November 23rd, 2009 No comments »A truly remarkable computing interface is one that is invisible. Even today, computer demand a focus of attention from users. Mark Weiser presents in his paper “The Computer for the 21st Century” a paradigm that pushes computers into the background of our daily physical environment. In the 90′s his team at the Palo Alto Research Center (Xerox) implemented a variety of computational devices including Tabs, Pads, and Boards and the infrastructure that allows these technologies to communicate with each other. These inch-scale tabs and paper-sized pads could be spread across the surface of a physical desktop, but they had the ability to store many more times the information than traditional Post-its and 8×11 paper. He envisioned dozens or hundreds of these interfaces embedded in our homes, cars, and offices, made invisible by our dependency on them to perform specific tasks and organize and present information in our daily routine.
In their paper “Tangible Bits: Towards Seamless Interactions Between People, Bits, and Atoms,” Hiroshi Ishii and Brygg Ullmer also strive to move beyond the dominant model of GUI strapped to computing via flat rectangular display, keyboard, and mouse. The main tenet of their research is that only tangible user interfaces (TUI) can make computing truly ubiquitous and invisible. Unlike Weiser’s model to embed new digital technology in our lives, this paper strives to enhance the physical objects in our current world (both analog and digital) through digital technology. For example, the Media Lab at MIT has enhanced the map-viewing experience through graspable map scaling and rotation devices.
I am part of a team that is using the Microsoft Surface to enhance museum learning and visitor experience at Wellesley College’s Davis Museum and Cultural Center. Through a natural set of gestural and token-based interactions with the surface, visitors can explore and generate interest in key pieces in the museum’s art collection and begin to think critically about the art and their awaited museum experience. At the end of their experience, they are invited to become part of a community of art explorers and interpreters by contributing their thoughts and interpretations to a dynamic “guestbook.” Our DavisTUI interface, to be placed at the center of seating clusters across the Davis Museum lobby, serves as a physical and metaphorical center for a naturally formed community of visitors. Seated in comfortable chairs around this enhanced “coffee table”, the visitors’ interactions with the surface will be candid and inspiring like the museum experience. We believe that DavisTUI has the potential to be a great contribution to the world of ubiquitous computing and novel HCI.
DavisTUI: Progress Report 1
November 20th, 2009 No comments »Today we began the first phase of the implementation of our DavisTUI application. I defined the basic information architecture and implemented several functions of the Word class, which defines Word objects with which users interact to see the associated artwork.
I also defined the basic informational architecture of the Art class, which defines Art objects that will display artworks chosen by the museum curator. These are the primary objects with which users interact to enhance their museum learning.
In the SurfaceWindow1.cs file, I implemented functions that populate the Word objects. I also generally implemented the look and feel of the native screen by generating ScatterViewItems on the ScatterView.
DavisTUI: Use Scenario & Visual Design
November 16th, 2009 No comments »USE SCENARIOS
WHO
The system may be used by any visitor to the Davis Museum. This incorporates a wide range of demographics, including children, students, faculty/staff, and visitors to the college. As such, there is a wide range of ages and potential occupations to account for. Additionally, we will want the user interface to be as intuitive and relatable to real life as possible, because there may be a wide gap in computer skills and comfort with technology.
WHAT
The system has four main functions: the user can choose to search the collection by different criteria. They can choose an individual work to look at, and can respond to prompts about the piece or see responses by other visitors to the piece. Notably, they can move pieces across the screen to share them with other users.
HOW
- Choose certain criteria by which to search the collection
In its native state, there is a “word cloud” in the center of the Surface which displays the themes by which the works are sorted in the collection. To change the way in which these works are being sorted, the user will pick up a token and place it on the surface, at which point the word cloud will change to reflect the search and its results. The tokens will represent groupings such as time periods, place of origin, medium, etc. - Choose artwork to look at
From the word cloud, the user will drag the word of interest to their “space.” After holding their finger still for a moment, the related images will populate around it, and using their other finger they can “drag” the image of interest out of the cloud, at which point it will act as an independent object on the Surface. After dragging out as many works as the user desires, the user will release the word and it will return to its position in the cloud.Alternatively, if you have already visited a piece and picked up its corresponding postcard, placing the postcard on the Surface is tantamount to having pulled out its digital version.
- Respond to prompts associated with the artwork
When the visitor touches an isolated piece which has previously been chosen from the cloud, a “speech bubble” will populate from the work with a prompt for the user to respond to. Depending on the prompt, the user will be invited to either “draw” on the painting itself, or on the speech bubble, below where the text appears. Using the eraser tool, they can correct their response at any time during the interaction. In order to invite others to respond to the same prompt, the user can either flick the piece in their direction or create duplicate speech bubbles which others can interact with.If the user is interacting with a postcard version of a piece, pressing down on the postcard will cause the speech bubble to appear to the right of the postcard, and all of the other interactions will take place as above. Notably, you are limited from painting on the postcard, so the system will determine whether or not it is a postcard and will adjust the prompts accordingly.
- See responses of other visitors to the artwork
On the speech bubbles, there will also be an icon in the corner depicting three people in a group. Upon pressing that icon, other peoples’ responses will branch out from the speech bubble, which the user can enlarge individually.
WHERE
The Surfaces will be positioned in the lobby by the entrance to the gallery, so that users are invited to interact with them while preparing to enter the gallery as well as preparing to leave.
WHY
Currently the museum has problems with (1) attracting visitors, (2) providing information about the work’s location and its context, (3) engaging visitors in critical thinking over the course of their museum visit, and (4) encouraging contemplation and sharing about their experience after their visit. By placing Surfaces invitingly in the lobby, where users shape their first impressions about the museum and its collections, as well as where they must pass through to enter and exit, the visitors will be engaged by the Surfaces even before they view the galleries, and will be encouraged to reflect on them afterwards.
VISUAL DESIGN
Spatial Layout
Tokens
STORYBOARD (and VISUAL DESIGN cont’d)
- Choose certain criteria by which to search the collection:
In its native state, there is a “word cloud” in the center of the Surface which displays the themes by which the artworks are sorted in the collection. To change the way in which these works are being sorted, the user will pick up a token and place it on the surface, at which point the word cloud will change to reflect the search and its results. The tokens will represent groupings such as time periods, place of origin, medium, etc. - Choose artwork to look at:
From the word cloud, the user will drag the word of interest to their “space.” After holding their finger still for a moment, the related images will populate around it, and using their other finger they can “drag” the image of interest out of the cloud, at which point it will act as an independent object on the Surface. After dragging out as many works as the user desires, the user will release the word and it will return to its position in the cloud.
- Respond to prompts associated with the artwork:
When the visitor touches an isolated artwork which has previously been chosen from the cloud, a “speech bubble” will populate from the work with a prompt for the user to respond to. Depending on the prompt, the user will be invited to either “draw” on the painting itself, or on the speech bubble, below where the text appears. Using the eraser tool, they can correct their response at any time during the interaction. In order to invite others to respond to the same prompt, the user can either flick the piece in their direction or create duplicate speech bubbles which others can interact with.
- See responses of other visitors to the artwork:
On the speech bubbles, there will also be an icon in the corner depicting three people in a group. Upon pressing that icon, other peoples’ responses will branch out from the speech bubble, which the user can enlarge individually.
Proposal for Museum Lobby TUI
October 30th, 2009 No comments »Problem
The Davis Museum lobby has no method of informing visitors about their museum experience to come or contextualizing it afterwards. Without maps, interactive guides, and customizable tours, visitors are ill-informed of what the museum has to offer and why the works are important. Furthermore, this dearth of information and welcoming in the lobby prevents potential visitors from feeling as if the museum is a resource freely at their disposal.
Users
The users of the system will be visitors to the museum, seeking information about the exhibits and to customize the museum experience for themselves. We want to reach as many demographics as possible, addressing the needs of student visitors and providing the ability to collaboratively structure a tour. We believe a tangible approach is optimal because it is uniquely inviting within the lobby space, it is collaborative, and it offers a very fluid basis for our own application development.
Proposed Solution
We believe that developing a Surface application to be used in the lobby of the Davis Museum would be an optimal solution. For the aforementioned reasons, it will draw in new visitors and increase the welcoming aura of the space. Furthermore, the lobby is a unique space that visitors occupy before and after their viewing experiences. A Surface application in this space has the potential to work as a learning tool, extending the visitors’ knowledge, experience, and associations between works.
The user can choose from the surface an exhibition or piece to learn more about. Through gestural interactions with the surface, he or she can gain contextual knowledge associated with each piece (e.g. a cultural context for the piece that must be observed or experienced) that cannot be displayed in the physical exhibitions. For many, the surface application can replace the traditional brochure, providing interactive content and media about the museum history, maps, featured exhibitions, upcoming events, and contributors. It can further be used to collect, assemble, and share a mosaic of experiences from previous museum visitors.
Extended components of this project can incorporate mobile or other handheld technology.
Related Work
The ARCHIE project supports cooperative gaming in museum environments through handheld devices. It presents a framework for creating mobile guides that potentially enhance social relationships between visitors and stimulate youth to visit museums. Their goals are similar to ours in that we both aim to improve and enhance museum learning.
MobiTags allow museum curators to gather and analyze semantic, social, and spatial navigation in museums by tracking the navigation of visitors within the venue. This information can be used to evaluate the physical placement of art and resources within a venue in order to improve the visitor’s museum experience.
http://www.jonbax.com/static/mobitags.pdf
Collaborative/”Scaffolding” museum learning games and tools
The MUSHI TUI supports cooperative learning in informal environments such as museums. Museum visitors interact with each other through handheld devices and observe the results on a tabletop monitor. While this TUI supports cooperation between multiple users, the experiences are individual and physically separate. Their goals are similar to ours in that we both aim to improve and enhance museum learning.
Analyzing reacTable Through Reality-Based Interaction
October 8th, 2009 No comments »In 2008, a group of HCI researchers from Tufts Department of Computer Science and MIT Media Lab proposed the notion of Reality-Based Interaction (RBI) as a concept that ties together emerging human-computer interaction styles. Based on this concept, they provided a framework that can be used to analyze and understand new interfaces and their interaction techniques. Here I’ve used their framework to analyze the reacTable TUI.
Reality-Based Interaction (RBI) Themes
The reacTable’s basic interaction techniques build directly on the user’s knowledge of naive physics (NP) and physical space (EAS). To add a controller to the system, the user simply picks up a puck and places it on the surface. The user can move the puck across the surface or rotate it. Doing so will result in digital visual feedback displayed upon the physical surface and audio feedback from the speakers placed in the physical environment (EAS). Because the reacTable surface is round, the user can move his or her body around the TUI to change viewpoints, which leverages the user’s ability to move his or her own body to different positions in the environment (EAS). Because there are no privileged points-of-view or points-of-control, reacTable also encourages collaboration and competition between multiple users, drawing on their social interaction skills (SAS).
Tradeoffs
Designed to provide an immersive experience for the user and onlooking audiences, the reacTable utilizes a large, table-top surface with a camera, projector, speakers, visual and audio synthesizers, and many connection tools. They also defined the interaction environment, typically a large room, where there are no other lights or noises to distract the user interacting with reacTable. By making such decisions, the designers, by trading practicality for realism, incurred some space, size, and power consumption costs.
The reacTable TUI strikes a nice balance between reality and other qualities including expressive power, efficiency, versatility, ergonomics, and accessibility.
Student of Computer Science and Economics at Wellesley College, B.A. May 2011. Cross-registered at MIT. Web designer. Curious problem solver. Always an aspiring artist. Family in Shanghai. Home in Sillicon Valley. Interested in HCI, user experience, interface design, product design and development, and all things progress and innovation. 
