A basic premise of collaborative mixed reality systems is that they enable mixed presence (collocated and remote) collaboration by effectively blending physical and virtual spaces. In this project we consider what it means to 'effectively blend' real and virtual for collaboration. We take a multi-pronged approach, combining infrastructure development, physical-virtual co-design, iterative prototyping and controlled experimentation.
TwinSpace is our infrastructure for collaborative mixed reality [2]. It provides a robust platform for rapid prototyping of applications that span physical and virtual spaces, allowing for comparative evaluation of designs. The system is applicable to a wide range of physical settings, virtual environments, and collaborative activities. TwinSpace provides the following four key features:
TwinSpace has been used in several research projects to date; most are outlined below.
The inSpace Lab is an experimental environment modelled after the project room, a room that provides a physical home for a project (its members, activities, tools, and products) over its lifespan [1]. The lab is designed to utilize the spatial layout and ‘collaboration affordances’ of the physical space in order to connect with a complementary virtual space [2].
The inSpace lab is composed of three main regions (see figure). Region A supports brainstorming and generating new content on a whiteboard, region B supports extended group discussions and sharing content on wall displays, and region C supports composing, modifying, and discussing content over an interactive tabletop display.
inSpace serves as our primary testing ground for mixed reality interaction techniques, physical-virtual co-design [1], policies for privacy, sharing mechanisms, and physical-virtual mappings.
Or de l’Acadie is a prototype collaborative game built using TwinSpace [2,3]. Set in an abandoned 18th Century French colonial fort, two pairs (French and British sides) compete for valuables distributed throughout the town, with the goal to collect the most valuables within a time limit.
The game emphasizes asymmetry in both game controls and team dynamics. Each British player controls a single soldier using keyboard controls. Players are not collocated, and can communicate with each other only when their soldiers are within earshot. Each soldier is equipped with a static top-down map of the fort, and can collect a limited number of riches before needing to return to a location at the edge of the fort to stash them.
The French side consists of a Mi’kmaq mystic who is able to communicate with spirits, and a deceased French Gendarme. The French players are physically collocated, and use a range of physical controls to compete for the valuables (see figure).
This project implemented a mixed reality solution to the design contest held at the 3DUI conference in 2010 [4]. Items were selected from virtual supermarket shelves by moving a physical trolley around a carpet that was mapped to the supermarket floor. The touchscreen on the trolley presented the first-person perspective of the shopper.
Each selected item was dynamically associated with a physical token. When it was time to 'check out', the physical tokens were carried to a physical table/display. The display presented the surface of a corresponding virtual table. When tokens were placed on the physical table, the associated items appear on the virtual table at teh same position and orientation.