The Cooltown User Experience

John J. Barton ( and
Tim Kindberg (
Internet and Mobile Systems Laboratory
Hewlett Packard Laboratories
Palo Alto, CA 94304

The Cooltown project at HP Labs applies Web technology to develop systems that support the users of wireless, handheld devices interacting with their environment, anywhere they may be. The basic elements of Web technology useful for ubiquitous computing are described in several papers and on-line resources from the Cooltown project [1-5]. Here we will give a flavor of the Cooltown user experience, discuss ways that context can be used in web systems, and explain why we think a web-based system for ubiquitous computing can achieve network effects. In our conclusion we will mention some of the many remaining challenges and our initial user studies platform.

User Experience

The Cooltown user experiences a mixture of web browsing and the manipulation of a handheld device. The "browsing" experience may not resemble the desktop web experience. Indeed the device may load web pages and submit web forms without user intervention. The web client embedded in the handheld device is necessarily compact and utilitarian; the handheld device is enhanced with sensor technology triggered by the user and integrated with the web functions. Each digital device, being specialized for certain purposes, will differ from others in the way it uses web content. The detailed experience depends upon the device and its application by the user. The user develops the sense that the handheld device is their control for services depending upon and integrated with their physical surroundings.

For example, a user at a conference with a PDA or cellphone device equipped with infrared sensors will approach an infrared "beacon" at the entrance to the conference hall. Holding the device like a Star Trek "tricorder" in front of the beacon, the user will see a web page describing the room and its facilities. This page acts like a directory of services in the room; navigating the directory uses the familiar web hypertext model. Here the infrared receiver is a sensor enhancement for the web client in the PDA, the beacon emits a URL pointing to a "PlaceManager" server configured to return web pages for the the corresponding room.

The user scheduled to speak at the conference would point their web client—a PDA or even something smaller like a watch—at the room's projector and transfer to the projector the URL for their presentation. The user experiences a mobile clipboard for web bookmarks: they copied a URL off their desktop PC earlier and now they paste it in to the projector. Here the handheld device has minimal web client function: it only needs to hold URLs. The projector on the other hand is a sensor-enhanced web browser: it senses URLs from users in its environment and displays the corresponding pages.

A user with a Cooltown camera would also have a mixed web/real world experience. They would first present a "badge" to the camera, some token that identifies their web page of camera services. The camera would respond by presenting the user with the web page listing image services suitable for the user. These services could correspond to waypoints, storage, or destinations for image data, all combined (within the infrastructure) with format adaptation services[4]. The user can select one of these services, say "Send to Home Picture Frame". and begin to take pictures. The selection invokes a hypertext link that loads a web form. The camera continues to act as a web client, using HTTP POST to send pictures. Thus the camera is a web client enhanced with "badge" reading and imaging sensors. The user however perceives a camera with a smart "upload" button.

Our first user in the Cooltown conference hall consumed information with their PDA: they browsed the pages about the conference room after picking up a URL from a beacon. A different kind of user is the person in charge of the hall. They must create the information in those pages. In particular, they add links to the pages for all the items in the conference center that are relevant to attendees. That includes equipment such as telephones, projectors and printers, and non-electronic items that have a web counterpart -- for example, posters and the conference's (physical) guest book. The daily configuration of the conference hall must be accurate if it is to be useful.

Here too a web-based approach can be applied. Items in the conference center can be given identifiers that map to URLs. These identifiers could be bar-codes or RFID tags placed directly on the items and sensed by handheld web clients. The user in charge, the "registrar" then simply picks up the URL for the hall from a beacon, sets it as the target of subsequent operations, and senses the items in the room to load the URLs of the hall contents into the hall's directory. The registry of items belonging to the conference is thus compiled by a registrar who understands what belongs. For example, they know not to include the printers at the front desk that should only be used by conference staff. That printer may be the same network as printers available to conference guests but the registrar simply omits it from the directory if guests should not use it. Again the key element is a sensor enhanced web client, a PDA with a barcode reader in this case.


A web-based approach allows a wide spectrum of context-dependent applications for nomadic users of handheld devices. As web-clients, the handheld devices need not be bound to particular applications or particular environments. As users interact with their environment, their devices contact servers and download pages. The particular server that they contact depends upon their physical context and the pages they download represent a context for their next interactions.

Consider the conference hall scenario once again. The beacon in the conference hall contains a URL that points to a server configured with the state of the conference. The user's act of picking up the beacon selects the "conference room directory service". The beacon has become part of the room's user interface. It has the effect of a physical icon representing the room's directory service. The user's act has changed the device from a generic PDA with a web-browser to a conference room service selector. This is a simple form of context-dependent computing where the context is physical location directly sensed by the user.

Similarly the Cooltown camera changes from a simple digital camera into an endpoint for digital communication by sensing the user's token and responding to the user's destination selection. Of course in principle we could have had the user attach a keyboard to the camera and type in login identifiers, commands, and addresses to achieve the same result. Since the keyboard solution does not rely on context--it does not require a user token or pre-configured image-destination menus--it is more versatile and general purpose. However it is also more complex and error-prone. By selecting a limited input mechanism, a token plus a hypertext link selection, we create a much simpler user interface for the image-sending service, one that creates a context for the camera and then operates within it.

The Cooltown registrar both uses context and modifies it for others. First the registrar picks up the conference hall beacon. This creates a context for sensor operations in the handheld device. When the registrar selects an item by sensing its barcode, the barcode value can be directed to the corresponding conference hall database. Exactly the same activity in a different room preceded by picking up a different beacon results in updates to a different room's database. The end result in either case is the modification of the directory for these rooms, directories that other users will download on their next visit. Context, stored as a URL in a room beacon, allows the registrar to complete their task with gestures of a handheld device.

The web model for ubiquitous computing provides other ways to incorporate more elaborate context. The web-client request use can include environmentally sensed data. Contextual parameters such as the date and the user's activity feed into the provision of different web content under those circumstances.

Extending Web Technology for Ubiquitous Computing

A web-based system for ubiquitous computing adapts web technology for nomadic users with sensor-enhanced, networked, handheld web clients. We can adapt the web user interface approach and its system model, then integrate it with physical artifacts using sensing technologies.

The web provides us with a model for presenting a service's user interface on a service-independent platform. As a mobile user moves into a new area they can carry a device they are comfortable using. In that new area they can download content that corresponds to that area. They can invoke services they did not know existed until just before they used the service. They still use their familar device. Other users with different devices can use the same services. There are other systems with this property, but none have the combination of simplicity and proven effectiveness of web technology. The breadth of web sites and the proliferation of web browsers testifies to the effectiveness of this technology in achieving network effects on the desktop and similar potential exists for mobile users.

The web content transfer model allows cameras, handheld scanners, PDAs, and laptops to send content to printers, projectors, kiosk screens, picture frames, and to Internet services. Of course lower level protocols such as IRDA or Bluetooth can already do some of this, but these solutions tend to be transport-dependent. By adopting a web-compatible content transfer model, device communications scales from simple device-to-device transfer up to Internet scale client server transfer. The key addition of the Cooltown approach here is the embodiment of URLs in local device-to-device transfer. Consider a wireless web-enhanced camera and a wireline web-enhanced printer. By sensing the printer's URL, by infrared beacon or barcode scanning, we can print images by direct transfer without elaborate service discovery systems. We can also upload images to web sites and print online photo albums. Furthermore, we can also use a scanner to upload images and we can put photos in to online picture frames. Thus we have a scalable, device-independent communications system.

Our model integrates the physical world with the virtual world of the Web. That integration is achieved by identifying one or more web resources with objects in the physical world, then placing a "link" from the object to the resources. The link can be an IR beacon for the conference hall place or a barcode tag for a printer in the hall, for examples. The handheld device senses the link and activates the corresponding service. For beacons, the link can be direct: the URL for the web resource is emitted from the beacon. For barcodes or similar identifiers that are too compact to hold a URL, we add a"resolver" that turns sensed values into URLs. With a resolver in place, a wide variety of objects can be associatied with web pages. Then the deployment of physical/virtual links should proceed much like web-page construction.


The Reality of Cooltown

Our overview of the Cooltown user experience, use of context, and potential for achieving network effects belies a great deal of uncharted and interesting research and development territory. In general the concepts we have outlined are all feasible as tested in laboratory demonstrations. Some aspects of our description are even deployable with today's wireless technology. Among the many challenges, we focus here on ones related to user interface.

We are engaged in a study of museum users to measure  the efficacy of several types of handheld device in augmenting the museum exhibits with various types of web content. By observing the users' behavior and logging their activities, we aim to understand the influence of physical factors such as the device's size and input modes, and the interaction styles involved in viewing both physical exhibits and virtual resources. For example, we include study of how the synchronous exhibit-browsing experience compares with an asynchronous one, in which users pick up links for later perusal on a PC at home or within the museum. By the time of the CHI 2001 workshop we should have the first study system in place.

Our user study recognizes that a critically important ingredient in moving a web-based model forward will be effective work on its computer/human interface. For this reason we hope to join the CHI 2001 workshop on building the ubiquitous computing user experience.


This paper describes work with contributions from the entire Cooltown team at HP Labs ( The discussion of infrastructure services has benefited from discussions with Armando Fox, Ben Ling, and Andrew Huang at Stanford University.


[1] D. Caswell and P. Debaty. "Creating Web Representations for Places". Proc. 2nd Int'l Symp. on Handheld and Ubiquitous Computing 2000, p 114-126.

[2] T. Kindberg and J. Barton. "A Web-based Nomadic Computing System". To appear in Computer Networks, Special Edition on Pervasive Computing, Elsevier. HP Labs Technical Report #2000-110.

[3] T. Kindberg, J. Barton, J. Morgan, G. Becker, D. Caswell, P. Debaty, G. Gopal, M. Frid, V. Krishnan, H. Morris, J. Schettino, B. Serra, and M. Spasojevic. "People, Places, Things: Web Presence for the Real World". Proc. 3rd Annual Wireless and Mobile Computer Systems and Applications, Monterey CA, USA, Dec. 2000. p 19.

[23] V. Krishnan, and G. Chang, "Customized Internet radio". Proc. Ninth International World Wide Web Conference, Amsterdam, 2000.

[4]"Running the Web Backwards: Appliance Data Services" Andrew C. Huang, Benjamin C. Ling, John J. Barton*, Armando Fox Stanford University and *Hewlett-Packard Laboratories, WWW9 Proceedings.