Current Projects List

Digitally Merged Environments
Project Overview

Detailed Description
Principal Researchers
Press Coverage
Publications, Reports & Presentations

Canada-California Research Consortium in Digitally Merged Environments for Collaboration, Interaction, Simulation and Teleoperation

The main objective is to create digitally merged environments where an individual in one environment has the overwhelming sensory experience of being in a completely different environment. This requires technology sophisticated enough to suspend disbelief so that the user is free to concentrate on a primary activity without the distraction that results from awareness of the technology being used. This project will lead to the formation of a research and development consortium of highly skilled individuals that can create the next-generation software and hardware capable of producing such an experience. It will provide major improvements in distance collaboration, interaction, simulation, and teleoperation involving a wide range of applications advancing arts, entertainment, science, medicine, engineering, education, and management. The new research and development consortium can then proceed to the next stage of forming alliances with industry partners and developing a business plan that can attract major funding.

The R&D objective of creating the effective experience of being in another environment will be applied to both actual environments that exist somewhere else at the same moment in time and to virtual environments. Virtual environments can be simulations of real environments, imaginary environments created from data, or a combination of both. One can see, hear, and feel as if actually in another environment and interact there with others and/or real and imaginary objects and equipment. The potential applications of this technology are infinite and extend across every aspect of human life and experience. Its development could eventually rival that of modern transportation, motion pictures, telephony, television and the Internet. As with these previous technologies, there will be broad applications that impact everyone’s daily life and specialized applications that impact particular professions and areas of endeavour. This research will transform distance collaboration through the application and invention of advanced visual, audio, and haptic digital media strategies to create an experience that is potentially better than being there, as it facilitates coincident interaction with complex data in multiple locations. New types of collaboration will become possible with many of them reducing costs and environmental impact.

It is always easier to design systems that will cost millions of dollars to the ultimate user and create a niche market limited to the very few who can afford them. An objective of this project is to keep the cost of the systems being produced to a level that is within the budget of most university researchers and medium-sized enterprises. Mass market commercialization down the road will eventually bring the systems within the reach of ordinary consumers, but in the meantime it is important that the initial user groups being targeted can afford the systems being developed for them.

Another important objective of the collaborative event will be to focus on next-generation technologies that will have a major impact on both the arts and the sciences. R&D funding deliberations are often quick to identify the improvements in human life provided by the sciences and slower to recognize the existential benefits provided by the arts that make life worth living. What is extraordinary about the participants in this collaborative event is that as artists, scientists and engineers, and as combinations of all three categories, they wish to create together important new tools that can be used by both the arts and the sciences. A number of the participating researchers in both Canada and California who are working on the development of collaboration tools with very broad potential uses have chosen initial applications in the arts – music, motion picture production – because they are the most technically demanding. Bill Buxton, the noted human-computer interaction pioneer, is only half-joking when he notes that, “There are three degrees of tolerance: normal tolerance, military tolerance and musical tolerance. If you can satisfy the highly demanding eyes, ears and bodies of the musicians, the rest is a piece of cake.”

This multidisciplinary research brings together technologically informed artistic and scientific teams to enhance the semantic as well as the purely technical aspects of novel synthetic experience, creating a true next generation of digital media innovation. Just as the ubiquitous spread of video production technology led to the creation of video art, the new digitally merged environments technology will lead to new forms of environmental art that do not require expensive installations in art galleries and can be fully experienced by large numbers of people throughout the world. In the sciences, the ability to create virtual environments will be particularly important and will lead to new techniques for observation and the testing of hypotheses. It will be possible to experience environments into which the body cannot venture—abstract higher-dimensional information spaces, the worlds of the very small or very large, and the realms of the very fast or very slow. Many applications will become possible in remote and hostile environments especially with the capability for teleoperation of sophisticated equipment. Scientists will be able to experience being on the seafloor when an earthquake generates a tsunami. Instruments and cameras now being deployed on undersea internet networks will generate the data needed to create that experience using the technologies developed by this project. In medicine, there will be major improvements in the simulation systems used by university medical simulation centres for medical training and the development of new surgical techniques. Some of the most effective applications are expected to be in fields that combine the arts and sciences like architecture and environmental design where a far more palpable experience of what it will be like to live in the environment being designed can be achieved.

The collaborative event will also have as an objective the mapping of the possible development phases required to get from the current state of the art to our vision of what we think can be accomplished over a reasonable period of time. The plan is definitely not to obtain major funding, disappear into labs for many years and finally emerge with a revolutionary complete system that transforms human experience overnight. It is rather to define a basic system that could be developed collaboratively in two or three years and then to identify the incremental steps necessary to develop increasingly sophisticated systems that can be sequentially introduced, tested widely and commercially marketed for applications in both the arts and sciences. At each stage, the results of testing and marketing will be fed back into the development of the next enhancements to the system. Defining a basic system means choosing to focus on certain limited capabilities that will have significant usefulness in several fields of endeavour. Given the limitless possible applications of digitally merged environments, there must be a focus in the collaborative event on narrowing down the most promising starting points and identifying the resources that will be required in the first development phase.

Human resources are the most important ingredient in any project and an essential objective of the collaborative event will be to determine the required roles for system development – those that can be filled by the event participants and those that require additional players. A plan will evolve for how the participants will interact with each other to develop meaningful collaborative working relationships. There will be a partnership between technology developers and initial users of the technology to ensure that the tools created will be used immediately and can be easily adapted to the requirements of both the artist and scientist participants. The software and hardware technology research and development teams at each university must determine how their particular expertise and interests can best serve the larger enterprise and mesh with the capabilities of the other teams. The objective is to avoid duplication of effort and maximize the human resources working toward a common goal. This includes identifying the areas, such as hardware research and development, where industry partners may have both the human resources and the infrastructure needed to drive progress. Many of the participants are already working with industry partners and their expertise in managing these relationships will be used to plan the next steps.

The collaborative event will also identify valuable links to other Canada-California projects and research teams whose work will provide substantial benefits toward realizing this project’s objectives. Such efforts in supercomputing and next-generation internet network infrastructure will make it easier to link widely separated locations and handle the very large amounts of data that are necessary for digitally merging environments. Links to green IT projects will also be important to fully realize the benefits of using digitally merged environments to reduce travel and limit carbon footprints. The participants already have close working relationships with the organizations operating the next-generation high bandwidth research internet networks such as CANARIE in Canada, CENIC and Internet2 in California.

There are seven universities in Canada and five in California involved in the project:

University of Manitoba
University of Victoria
École de technologie supérieure
Université de Montréal
University of Alberta
Ontario College of Art and Design

UC San Diego
UC Berkeley
UC Davis
UC Santa Barbara