Digital display technologies for museum applications
Entry compiled by: Jeffrey A. Haines
As digital technology and gadgetry have permeated nearly every area of information distribution and presentation, audiences have come to expect information dissemination industries of every kind to utilize the latest storytelling mediums. In the historic information industry, museums have attempted to replace their traditional artifacts-behind-glass and description board displays to effectively entertain and educate their audiences."During the past few years, a shift has taken place in the museums’ identity, from simple holders of cultural objects to educational and at the same time entertaining institutions. Hands-on exhibitions allow their visitors, usually children, to interact with exhibits and to learn while they play with them." (Lepouras)
Because of their typically small operating budgets, museums have struggled to identify and implement the most advanced display technologies for their purposes. "Museums host exhibitions in order to convey their message to their visitors. In the museums’ context, this is achieved by combining exhibits and information in a carefully designed order and presentation style. New technologies such as multimedia, 3D graphics and virtual reality can be used to enhance the presentation, offering a more vivid and enjoyable experience." (Lepouras) Although there have been some forays into the deeper, virtual reality possibilities of digital exhibits, most displays are currently characterized by short “History Channel” style documentaries or recreations, which simply augment the experience of looking at objects in a glass case.
Currently, most digital museum displays consist of wall mounted video screens or interactive kiosks. “Modem computer technology has made possible 3D interactive public kiosks that provide the user with a multi-media rich environment that may include text, graphics, images, sound-clips, video, and animations. Often these environments allow the user to interactively select content and navigate through the 3D space to retrieve information, however, the navigation task may distract the user from this information. Ideally, the user should enjoy the benefits of these kiosks without sacrificing the ability to acquire the information they contain.” (Sullivan) Although current kiosk systems provide some interactivity, they do not approach the recreation possibilities of virtual environment exhibits made possible by recent advances in technology.
Complete virtual reality immersion systems are, for the most part, not physically or fiscally practical for most museums. "So far, only a small number of museums have tried and exploited virtual reality technologies…The development of a virtual environment is a difficult and time-consuming process, especially when viewed in comparison to traditional information systems, since it requires the cooperation of a group of people with a variety of expertise in different areas…The execution of virtual environments is also an issue…systems require a high end system with a graphics accelerator card and special input and display devices…such a state-of-the-art system is costly to build and maintain. In addition, peripherals such as 3D input devices, haptic devices, etc. are sometimes experimental and most often sensitive to frequent and/or hard usage, rendering them inappropriate for use in a museum. For virtual environments to be successfully exploited in the context of edutainment in museums, these impediments have to be removed or at least minimized." (Lepouras)
One solution to breaching the gap between sit-and-watch video programs and full out immersion systems is the customization of pre-existing consumer videogame systems for kiosk displays, either by developing software from scratch, or by utilizing existing game engines to build interactive environments. Creating an interactive experience using preexisting 3D game technologies "would allow users to view and navigate in a virtual world, even in the location of their home, without noticeable compromise in the overall quality. Furthermore, the interaction style of game applications in nongame domains can be compelling...This solution to the need for affordable virtual reality is offered by the game software industry, which, during the past few years…has come to offer colorful, crisp and life-like 3D graphics. Game technology has managed to develop systems that, on the one hand, exploit computers’ potential to the maximum and, on the other, offer the user a sophisticated, interactive environment with 3D graphics and, in some cases, immersion capabilities that can be enjoyed." (Lepouras)
The Smithsonian Institution’s National Museum of American History, along with other partnering groups, conducted a series of trials around a test exhibit that they constructed in a preexisting game engine. Their focus was to examine if a virtual museum experience could be significantly interactive and informative to a point where it would be tantamount to a real world museum environment. The game featured a walkthrough of a museum exhibition, and allowed users to explore and examine objects. "It has to be noted that, from the group of participants, those who had played 3D games before commented that the environment was familiar, easy to use and stimulating." (Lepouras) Criticisms that came from the study’s participants included that it was difficult to gauge the real-world size of objects in the game, and that their representations were not significantly realistic to be used for serious research purposes. (Lepouras)
One example of a successful virtual reality museum display was a user controlled undersea “flythrough” experience created for the New Hampshire Seacoast Science Center. The presentation allowed visitors to steer a virtual submersible through the undersea environment while audio clips, triggered by user exploration, described undersea life and environment. The exhibit was presented through a kiosk that displayed rendered 3D in a stereoscopic format, in conjunction with assorted informational video clips. It was designed around the idea that “presenting information using more that one sensory modality will result in better learning. The challenge was to develop a technique enabling users to, on their own, make audio-visual connections easily, quickly, and naturally, without hindering their ability to navigate around the virtual environment." (Sullivan)
In addition to representational limitations, exhibits must fulfill usability requirements. “Developing these types of interactive environments is a complex task due to the specific requirements of kiosks. They should be exceptionally easy to use, as they must be proficiently operated within a few minutes; they should be self-explanatory, as there are no human helpers to interact with; and they should engage users with interesting content, so their experience will be a memorable one." (Sullivan) Depth of learning experience and simulation navigation time must be carefully balanced in order to build a virtual recreation that can produce a valuable learning session, yet leave users with a feeling of completion and urge them to move on to the next exhibit once the experience has concluded.
Aside from purely visual applications of display technology, haptic interfaces offer users a chance to virtually “feel” the contours and textures of an object and personally experience history in a way not possible with most actual artifacts. "In a virtual environment, exhibits can be observed from different viewpoints or even be manipulated. Technologies such as haptic feedback may enable visitors to touch and feel valuable objects, or it may allow people with vision problems to sense an exhibit." (Lepouras) Aside from integrating usability for the disabled, displays with a haptic element create a physical connection between image and audience. This connection deepens the learning experience, and leaves the user feeling as if they have actually made contact with a cultural remnant from the past.
Stereoscopic projection offers another possibility for museum-based storytelling and information presentation. The Canadian Museum of Civilization features an exhibit where 3D scans of Paleolithic “ice age figurines” can be manipulated on a large view screen with a trackball. With the aid of passive stereo glasses, the figurines can be seen in 3D. The system is optimized to display a very high level of detail, although reduced from the extreme detail of the scanned models. The exhibit’s original data is also implemented in an online system, so that virtual “visitors” can manipulate the figurine scans from home. (Deitz) The 3D experience allows visitors to interact with prehistoric artifacts that cannot be easily displayed because of their age and rarity.
Novel applications of the latest display technologies are opening new frontiers of information presentation and historical storytelling for museums. The largest obstacles to implementation of these mediums are cost, and lack of standardization across the industry; in many cases, exhibitions are experimental and exploratory. Currently, kiosks and projection screens are the main delivery technologies for these exhibits, but as innovations are made, immersive virtual reality applications may become feasible. As museum management organizations experience success with these innovative exhibits, and technology becomes more standardized and accessible, traditional, stand-and-stare exhibits will gradually disappear, and visitors will be able to learn through engaging technology-facilitated experiences.
Bibliography:
- Deitz, Dan. Virtual reality for the Ice Age. Mechanical Engineering. New York: Sep 1995. Vol. 117, Iss. 9
- Lepouras, George and Vassilakis, Costas. Virtual museums for all: employing game technology for edutainment. Virtual Reality. Godalming, Surrey: Jun 2004. Vol. 8, Iss. 2
- Sullivan, Briana, Ware, Colin, and Plumlee, Matthew. Linking Audio and Visual Information While Navigating in a Virtual Reality Kiosk Display. Journal of Educational Multimedia and Hypermedia. Norfolk: 2006. Vol. 15, Iss. 2
The future of data display and written narrative distribution
Entry compiled by: Steve Nathanson
In the modern world, where data is measured in gigabytes one day and terabytes the next, it becomes impossible to physically archive the sheer volume of information, and impractical to print this information for use on the go. Because of advances in display technology, with companies such as E Ink creating the Sony Reader, a portable device using electronic display paper, the idea of owning multiple books and libraries of printed documents may soon become obsolete. Electronic ink will change the way people obtain and distribute information on a daily basis for generations to come.
E Ink’s electronic ink works by using millions of tiny black and white capsules, which are suspended in a clear fluid. White capsules have a positive charge while black capsules have a negative charge. When a charge is applied to the material, the black and white capsules will float to the top to reveal their color. By charging different areas of the field, one can create the symbols needed to form words and images. In order to create an electronic display, the ink gets pressed into a plastic film that is laminated to a layer of circuitry that can be controlled by a display driver. (E Ink) What sets electronic ink apart from a conventional screen is that it only uses power to set the capsules. Once they are positioned, the device shuts off, holding them in place. With a traditional computer monitor or a cell phone screen, constant power is needed to display the screen image, and once the power is lost, so are the images and text. The device is not backlit; it relies on reflective lighting, thus conserving more power. Another feature that sets the electric ink display apart from actual paper is that images and text can be cleared and refreshed, so once the user is finished reading page one, he can press a navigation button and page two will appear on the screen in its place. Other companies, such as NTERA, are working on displays that contain “electrochromic” dyes, which display color when they absorb electric charge.
In Japan, Sony has teamed up with E Ink to release the first “e-book” device known as the LIBRIe. The device cost around $375 when it was first released in 2004. It featured 10MB built in memory, a memory stick slot for expansion, a keyboard, and a USB interface for downloads. The LIBRIe was powered by AAA batteries, which were said to last around 10,000 page turns. The newest version released in the US costs $299, has 20MB of hard drive space and a rechargeable battery, but no keyboard. In Japan this product has been used not only for books and manga. In the USA, Sony has launched a service called CONNECT, where Sony Reader users can pay $1.99 for classic books and the suggested seller price for new releases to download to their device. The USA model can also display Adobe PDFs, Word documents, RSS newsfeeds, JPEGs, and Sony’s proprietary BBeB format, as well as MP3s and AAC audio files. (Sony)
Using electronic paper instead of an LCD screen has many benefits. Electronic ink has been reported to be easier on the eyes than an LCD screen, and electronic paper consumes much less power during use than LCD screen devices. With many companies trying to “go green” and cut down on energy use, implementing electronic ink wherever possible would save energy, money, and in the long run, our planet.
The primary application of electronic ink is for occasionally changing and updatable displays. The technology can be engineered to work with any product that features a display. Some of the latest applications of this technology have been in computer displays, billboards, newspapers, wristwatches, cell phone keys, portable devices, service schedules such as those used for train times, and even for price tags on shelves at a store. Anything that constantly changes and needs to be displayed can be a possible application for electronic ink. John A. Rogers of Lucent Technologies states that, hypothetically, “You could imagine making an electronic version of a newspaper that would consist of a single sheet of this electronic paper connected to the wireless Internet. You could download information content, view it, interact with it, and roll it up or fold it in the same way you can a conventional newspaper, but you don't have all the waste associated with a newspaper.” (Gorman)
Another practical use of electronic ink could be on a computer keyboard where the key configurations and labeling change depending on what application the user is working with. Instead of memorizing different hot keys for each and every program, the user would be able to see the corresponding icons on their respective hot keys as needed. While these features are all possible, it should be noted that any display media that has moving graphics or needs constant refreshing, such as video, is not feasible to use with electronic ink. Games that might work could be Sudoku, crosswords, or puzzles, but nothing time intensive or fast paced due to the lack of refresh rate.
Electronic paper could be life changing for college and high school students. Instead of carrying around massive textbooks for each class, the student could load their device with every textbook and resource they would need from 9th grade through graduate school. With features such as a highlighter and a touch screen notepad, students could even forgo buying pens and paper and just use this device as their all-in-one information center. Professors post assigned readings that could automatically be downloaded onto the students device through Wi-Fi internet access. All handouts, quizzes, and tests could be administered using these devices as well. Because of its ability to be used for display of information from any study area, electronic ink devices could become a necessity for education, and could be a blessing for students who struggle to stay organized with multiple class data materials.
Another practical application would be to create a program with the iPod/iTunes relationship, somewhat like Sony CONNECT, where a user could download books, magazine subscriptions, newspapers, textbooks, maps, even comics, directly to their electronic ink device. Every morning, the user’s newspapers of choice could be downloaded before he or she wakes up so the information is ready to go. Whenever the newest issue of a user’s favorite magazine came out it could be updated onto the device immediately, without having to wait for the magazine to arrive via snail mail.
With the use of electronic ink, the way people obtain and distribute information would drastically change the world for the better. There would be a substantial drop in the amount of paper being used and wasted. Steve Ditlea, a tech analyst, has claimed that “The economics look good too: E-books require no printing, binding, inventory or shipping costs, allowing these savings to be passed on to the author in the form of higher royalties.” (Easton 287)
The use of electronic ink has many benefits over the use of paper, including ease of data distribution, information modification, and its reusability. As electronic ink is further developed and becomes more cost and technologically accessible, it could be come the ubiquitous information display medium.
Bibliography:
- E ink technology. (2005). Retrieved 11/06, 2007, from http://www.eink.com/technology/howitworks.html
- Easton, T. A. (2006). The real E-books. Taking sides: Clashing views in science, technology, and society (Seventh ed., pp. 287). USA: McGraw-Hill Contemporary Learning Series.
- Gorman, J. (2001, New device opens next chapter on E-paper. [Electronic version]. Science News, 159(17) 262. Retrieved 11/06/2007, from http://proquest...
- Sony reader product page. (2005). Retrieved 11/06, 2007, from http://www.learningcenter.sony.us/assets/itpd/reader/
Virtual Reality display technology
Entry compiled by: Steve Benjamin
Multiple Head Mounted Displays in Virtual and Augmented Reality Applications: This article documents a new hardware setup using multiple HMDs connected to 1 PC and a collaborative virtual environment.
Projected Reality – Content Delivery Right onto Objects of Daily Life: Projectors are used to display an interactive virtual environment over real-life objects.
A huge multi-projector is being combined with a massive, curved VR screen for use with flight simulators for military training.
Another stereoscopic projection room similar to CAVE being developed. It was recently upgraded with new Nvidia GPUs and has projectors on all 6 sides. The room's technology and inner workings are explained in this article in detail, including its 8.1 surround sound system and pneumatically actuated wall for entering/exiting the room.
Drawing on air is an interesting application of VR technology that I had not considered. It would be of special interest to 3D modelers. Here is the complete .pdf research paper.
This MIT project takes the drawing concept to the augmented reality level. Instead of using a HMD, the developer decided to create a screen that shows a video feed of what is directly behind the screen, and then using a stylus, a 3D object can be drawn in the air and manipulated. The screen can be carried as a user walks around the object, and it will stay in "place" virtually, relative to where it was drawn in the depiction of real-world space.
Haptic Interfaces
Entry compiled by: Jeffrey A. Haines
The study focuses on the melding of senses: How can you create a concrete link between sight and touch? This interface proposal was interesting, as instead of using all CG assets, a camera extracts live video of the user's hand using a chroma key, and superimposes it in the LCD video screen, in relation to the virtual object that the user is interacting with through the haptic system. The whole camera/screen, setup rotates as the user tilts their head, creating another level of interaction and virtual mobility. The combination of carefully calculated live real world footage and virtual environment theoretically creates a new level of interactivity that is not found in a typical synthesized reality experience.
The HapticMaster, a new high-performance haptic interface:
This article talks about a specific interface design, and talks abouit the two types of Haptic interaces:
"Impedance control - the user moves the haptic device, and the device will react with a force if a virtual object is met. So, viewed from the haptic device, the paradigm is: displacement in, and force out. The user will inevitably feel the mass and friction of the actual device, but these can be made very small by careful mechanical design.
Admittance control - the inverse of impedance control. In admittance control the paradigm is this: the user exerts a force on the haptic device, and the device will react with the proper displacement. So, viewed from the haptic device, the paradigm is: force in, and displacement out. Admittance control allows considerable freedom in the mechanical design of the device, because backlash and tip inertia can be eliminated. As a result, the mechanism can be quite robust, capable of displaying high stiffnesses and high forces. Admittance control has been used for control sticks in the flight simulator industry for many years."
The specified HapticMaster device is basically a Admittance control setup that consists of a robotic arm and hardware box. The robotic arm simulates a "volumetric workspace." This type of device is used for many small scale simulations, but is impractical for world creation.
Haptic Feedback for Virtual Reality:
This survey of haptics summarizes the information surrounding the field of study and its relation to virtual reality. "Haptics means both force feedback (simulating object hardness, weight, and inertia) and tactile feedback (simulating surface contact geometry, smoothness, slippage, and temperature)."
The Haptic Display of Complex Graphical Environments:
This article, using a lot of technical language and formulas, talks about representing 3D objects in a haptic system. Instead of simulating rough feedback information about an object, the article discusses ways to accurately create feedback relating to highly detailed models, based on geometry, surface normals, and even creating feedback models per textural shading information. It is a bit hard to wrap your head around, but I think even my limited comprehension of some of the principles helped me to better understand some of the issues involved in these complex systems.
Advanced display technologies
Entry compiled by: Steve Nathanson
The way humans interact with computers will change drastically in the upcoming decade. With display technologies, such as the multi-touch technology, advancing as fast as they are, the user interfaces for these devices will reinvent the way we think and interact with computers.
The multi-touch technology has been fantasized about in the science fiction film, Minority Report, staring Tom Cruise. In the movie he flips through pictures and video using his fingers on an interface that looks identical to what can be seen today from a company called Perceptive Pixel and features the use multi-touch technology in many applications. This technology also allows many users to interact with it at once, which opens up new and intuitive ways to co-create projects. A future example of this could be when one user is drawing a character, another user can be coloring a backdrop, while another user is animating, all on the same computer at the same time. Musicians could have jam sessions all on the same computer, with on-screen keyboards and even on-screen drum pads. Even video games could be made better, using gestures to give commands to your armies in Real Time Strategy games and being able to issue multiple commands at once, building units while sending others to attack.
Photosynth is another new technology that is being developed by Microsoft Live Labs. This technology can take photos of a place or an object and will display them in 3D space based on where they are in actual space relevant to each other. The way this technology works is the program will take all of the images and pick out distinct features in each picture based on vision algorithms and then photos that share features are linked together. When the program spots the same feature in multiple pictures it can determine its location in 3D space and it will map all the points. The program has a feature where you can chose a spot in 3D space and then view the object you are looking at from that exact place by showing you all the photos from that angle. Microsoft suggests that the future of this program will allow the user to take a picture of any object and have it search the web for that object and be able to give you information about it (Microsoft). Another application could be if someone is lost all they would need to do is take a couple pictures of their surroundings with their cell phone and then send them to a server which would compare those pictures to the pictures other users have uploaded and then generate a map of the location.
A new technology called BumpTop is a new user interface to replace the desktop on Windows or Mac machines. The concept is to make it feel like it is an actual desktop and the icons become paper or books on the desktop which the user can throw around, organize, or even crumble up. It features a 3D box interior complete with a physics engine, where the user can add a certain weight to more important files and less weight to not as important files so the user can find what he needs to find right away. There are also mouse gestures to put groups around objects and then stack them, fan them out, or flip through them like an actual book (Bumptop). Check out the Bumptop demo video.
This technology might seem like a messy waste of time for a desktop organizer, but this could be a a great application for looking through a photo library.
With the invention of these new display technologies and new intuitive interfaces, the way humans and computers interact will soon become so seamless that anyone will be able to walk up to a computer and be able to do anything they want with it.
Bibliography:
- Microsoft Live Labs: Photosynth. (2006). Retrieved 12/09, 2007, from http://labs.live.com/photosynth/default.htm
- Bumptop. (2007). Retrieved 12/09, 2007 from http://bumptop.com/