RSS Feed
Human-Computer Interfaces: The Mouse Is Dead
"The Mouse Is Dead" is the title of this post by Mike Elgan at the Datamation site.
Despite the sensationalistic title, the post is well thought out.
Let me start by saying that, in my observation, any proclamation of any technology being dead is bound to be exaggerated and premature, especially for a technology that has been firmly entrenched. When I worked in the Paging Division at Motorola, the growth of the cellular phone market resulted in regular proclamations that the pager market was dead. Immediately. Like, sometime around close of business that same day. Despite that, we somehow managed to continue with years of growth before the market started dying out. But products and technologies mostly do tend to come and go.
For many people, the thought of computing without a mouse is inconceivable. It's a little easier if you happened to be there before the mouse was in common use. Yeah, I hate to date myself, but I remember having conversations about what input devices would catch on, other than the keyboard. There were alternatives, like the "puck", trackballs, stylus pad (familiar to at least some graphic designers) and even the joystick. (By the way, does anyone really use that nipply thing in the middle of laptop keyboards?)
But you don't need to look back in the archives or into the future to find alternatives. As Mike suggests in point #1, an alternative has been sitting there for some times as the trackpad or glidepad on laptops. Many people chose to override it with a mouse, or consider it a form of mouse, so it's easy to dismiss. But it's not a mouse, and the addition of gesture capabiity makes the trackpad a strong contender.
Then there's the iPhone (and iTouch). I currently still prefer the QWERTY keypad on my BlackJack to the iPhone touchscreen if I have to choose. But that's for text entry, because I have always been frustrated by text entry on touchscreens. For Web browsing, however, the iPhone touchscreen is a breath of fresh air on a mobile device (except maybe for one-handed use). In any case, the iPhone is driving home the point that other ways of interfacing are not only possible but useful.
I'm not sure I want a computer to read my brainwaves, and I'm quite sure I don't want to dictate out loud to my computer for extensive periods (or be anywhere near someone else doing it). But I could definitely see the utility of having it track my eye movements and pick up on my gestures (yes, that gesture too) in front of the screen.
In other words, in the same natural way in which people interact with each other.
rk
Bluetooth Sex: Intimate Human Computer Interfaces
Exploration of ubiquitous computing networks and human computer interfaces would not be complete without the consideration of sexual applications of the technology. I debated on doing this post, not because I had any moral issues but because I didn't think it had emerged yet or warranted more than its 15 minute worth of popularity. It turns out that I was actually behind the curve.
I was motivated to do a bit more research when I noticed the traffic stats for this blog, which I had left dormant for a period while I worked on another project (picobuzz.com). Yet, despite the inactivity, I am still getting traffic for a particular post:
iPod Bikini: 2nd Sexiest Bluetooth App EverNot that I was surprised to get some attention with that sort of title but the post wasn't so provocative that I expected to keep getting traffic a year later. So I thought I should follow up. In that post, I added a closing link to a product named "The Toy". I checked up on the website and the product is still alive and well.
The Toy is a personal vibrator worn internally. The "unit" is linked via Bluetooth to a mobile phone and controlled by text messages sent to the phone, which are transformed into different vibrations by The Toy. As an example scenario for its use, here is an excerpt from their website:
Monday morning - she leaves with The Toy inside...
She's given you the power - You alone control The Toy
I thought web browsing at work was a distraction. This just totally killed the productivity of two people. Check out the website for further reading. You know you want to.
The Toy is not the only toy of its kind (literally)--there is also something known as the Wiibrator. As you would guess from the name, the Wiibrator provides similar functionality for the Nintendo Wii. Yes, the kid-oriented system, which my grade school-age son uses to play games with excessively cute characters, has been co-opted for adult recreation.
As you can see from the link, the Wiibrator is not a product as such but a program that interfaces the Wii’s Wiimote and the PS2's Trancevibrator. That's right--before the Wiibrator there was a PS2 vibrator named the Trancevibrator, still available for purchase (page may be NSFW).
I think it is particularly relevant to note the date of the Wiibrator post (December 17, 2006) relative to the release date of the Wii as reported by Wikipedia. Specifically, when the Wii was launched in key markets on December 8, 2006, about one week later there was a hack to use it as a sex toy.
There is actually a field of research dedicated to this sort of application of technology named "teledildonics". Furthermore, you know something has arrived technologically when there is an open source movement for it--in this case appropriately named "opendildonics" which can found at wiki.opendildonics.org.
So what is the point of all this, other than"sex sells"? Should we be focused on developing sex/adult infrastructure and applications? Actually, the point is that we don't need to. People will use the technology for their own desires and applications--be they erotic or something else. We aren't compelled to encourage adult applications, but they will happen, and to expect otherwise--or worse try to resist--would be futile and unproductive. And as is often the case, they may drive (will) adoption of technologies.
rk
New Futuristic Series: Charlie Jade
SCI FI Channel acquired (because let's face it--that works out much better than their original stuff) the U.S. broadcast rights to the science-fiction series "Charlie Jade" from Park Entertainment and it's first episode aired this past weekend. The series has already aired in various other countries and apparently gained a cult following. The following is the synopsis from the series' SciFi Channel webpage:
While tracking her suspected killer, Charlie discovers a secret desert facility. A massive explosion propels him into a parallel universe that seduces him with its similarities and baffles him with its differences.
Charlie is soon drawn into a conflict that involves his home universe, the one he now inhabits as well as a third universe. The third world is pristine and pacifist … with unsuspected terror at its heart.
Based on the opening episode, the multiverse theme of the show is guaranteed to support a nicely tangled weave of plotlines as well as provide a steady stream of techogadgetry and futuristic 'scapes.
Rethinking The Interface: Minimalism
Ryan Tomayko has an interesting post on his blog where he discusses a minimalistic approach to web interface design. He apparently had an epiphany after watching Edward Tufte’s critique of the iPhone. A central concept to this approach relates to "computer administrative debris":
The idea is that the content is the interface, the information is the interface – not computer administrative debris.
More precisely, anything that exists outside of the content, or that does not fit naturally with the information being presented, qualifies as administrative debris and is therefore a candidate for elimination from the design. While this definition is not exactly synonymous with minimalism, the end result in practice may be very minimalistic indeed. As an example, we need look no further than Ryan's blog, a snapshot of which is shown in the image above. Ryan took the concept to heart, and overhauled his blog. As you can see, the result is quite low in debris.
This sort of rethinking of interface design will be essential to the evolution of ubicomp. In my previous post: What Can We Do Differently With Ubicomp? , principle #2 states:
We will layer the interfaces onto our environment and blur the line between information and things.
Clearly, we will not be able to blur these lines until we remove the "debris" we have tolerated in computer graphical user interfaces, and make them more compatible with design principles which have evolved in the non-digital world.
rk
Seth Godin On The "Clowd" (Cloud Computing)
Seth Godin has a post on his blog talking about the "Clowd"--a term he coined which combines "crowd" with "cloud" (as in cloud computing). In the post he briefly discusses some scenarios on how cloud computing and /or (depending on who you as) ubiquitous computing would work in real urban environments. His examples are a refreshing improvement over the stale examples (e.g. the "coupon") often used.
He points out the obvious privacy concerns but I think also shows the utility of the ubiquitous cloud. I especially liked his comment: "..it can do Zagats to the ten thousandth power by..".
With no further ado, I refer you to Seth's thoughts in Seth's own words: link
Comments On Gartner's Top Ten Disruptive Technologies
Gartner recently released a list of the Top Ten Disruptive Technologies for 2008 to 2012. This has been commented on in a number of blogs around the Web, but since it has some technologies related to Ubicomp, I figured I'd throw in my two cents worth.
Here is the list:
- Multicore and hybrid processors
- Virtualisation and fabric computing
- Social networks and social software
- Cloud computing and cloud/Web platforms
- Web mashups
- User Interface
- Ubiquitous computing
- Contextual computing
- Augmented reality
- Semantics
On Disruption
It seems like a worthy enough list of technologies to watch and which will have some impact in the near future. However, the first observation I have is that few of these seem immediately disruptive. Disruptive to who or what? Some of these have been around for some time under other names: multicore processors (parallel processors), virtualization (virtual computing), mesh computing (distributed processing). I took courses on these concepts many years ago (more than I care to reveal). They are just the latest in an ongoing evolution. Gartner sees these items from their list as threats to IT organizations, which will result in reduction or elimination of IT staffing and management jobs. To this I respond: I wish that part of my job would go away, but I will have to see it to believe it. A concept like virtualization just moves the focus somewhere else--it doesn't eliminate much.
Other items on the list like semantics and contextual computing operate too far under the radar to be distruptive. These are being discussed by providers and infrastructure people--the same people who championed things like XML and SOA. They ultimately provide great utility but don't experience a viral growth or disruptive singularity. Social networking is different in the aspect of being very much on the surface, but by definition it has the harmonious goal of connecting people in more and different ways. Social networking may in fact disrupt marketeers and advertisers, but even on social nets, the old rules of conversation still apply (even if we are just recently writing down some of the rules).
Mashups could be genuinely disruptive in some ways, since it shifts the architecture and the attention. As this grows as a business model, it raises questions such as:
- Who owns the information? What is the value of data feed vs. mashup?
- What are the revenue streams? It is better to be the "invisible" data provider or the easily cloned mashup?
- As Gartner questions: do they lend themselves to investment dollars?
If I were to pick one item from the list as most potentially disruptive, I would pick augmented reality. First, the name itself implies you are changing reality in some way which makes it a candidate for disruption. More importantly, If or when display and other sensory I/O technology becomes seamless and unobtrusive enough, I believe the level of fantasy, escapism, and withdrawal possible could have a huge impact on relationships and socialization, and not all good. I am still trying to figure out the appeal of FurryMUCK and yiff. The suspension of belief and judgement possible with good 3D Virtual Reality will make Grand Theft Auto look seem like Ms. Pacman.
Ubicomp
In addition to ubiquitous computing (ubicomp) being listed explicitly in the 10 technologies, other items on the list also figure into ubicomp, like User Interface (non-traditional interfaces), contextual computing (your computing environment changing on your location/context) and to some degree, cloud computing (being surrounded by a distributed cloud of computers). For a description of ubicomp, see my post What Is Ubiquitous Computing?, and also What Can We Do Differently With Ubicomp?
According to the Gartner article:
"...within the next five years, information will be presented via new user interfaces such as organic light-emitting displays, digital paper and billboards, holographic and 3D imaging and smart fabric."
I think this is a safe bet--display and other interface technologies have consistently driven change in computers, home entertainment and mobile phone applications. As far as ubicomp, I will likely be blogging specifically on this in the near future. For a taste, see my post on Human-Computer Interfaces In The Iron Man Movie. In general one of the recurring themes of ubicomp is to move displays and interfaces away from the traditional desktop display (or even laptop display) and blend them into the environment. As OLED displays and digital paper become cheap and easy to mass produce, they will be applied to the surfaces of all kinds of objects. This will not only open up possibilities for information to be displayed in more convenient places (such as the kitchen, mirrors) but also for applications for nontraditional objects (such as the fridge) to display information and interact with us. And while I have a hard time imagining an actual, working holographic display, when it does happen it can't help but change the way we display and visually process information. And form-factor wise, it is as likely to appear on your coffee table as your desktop.
Relative to ubicomp, contextual computing and cloud computing will serve to enable interaction of the local environment with the user, as well as provide mobility of the user's profiles and applications as she moves from location to location. More ubiquitous and pervasive computers in the environment, as well as deeper and more comprehensive interaction with users, is at the heart of the ubicomp movement.
rk
Ubicomp Watch--Cisco: Visual Networking
Cisco has been working hard lately to change their image to be more consumer-focused in contrast to their traditional network infrastructure, business-oriented focus. Their branding efforts come under the umbrella of "The Human Network" in which they are showing how Cisco products enable and touch end users. They even offer Cisco ringtones for download. You can find general information on The Human Network at this site:
One of their television commercials which caught my eye recently is one entitled "Visual Networking" (another Cisco branding term). This commercial takes place on a city street and shows a number of ways in which video (as delivered by Cisco in some way) is impacting people's activities, for example telenavigation and providing audio-visual instructions to help a guy fix a scooter.
Mixed in this are a couple of Ubicomp application examples, specifically: proximity-aware applications. In one case, a woman is walking her dog, and as she approaches a bus stop shelter, the poster-like display on the side of the shelter changes to an advertisement for gourmet dog food--an advertisement targeted for an "audience of one". Presumably, the display system has detected and identified the woman (based for example on her mobile phone ID), performed a network access to a database somewhere, identified her as a dog owner/lover and selected this particular advert for display. I am assuming this is the scenario as opposed to the display sensing the presence of her dog (although technologically this could be done as well).
In a second example, a guy is passing a large flatscreen monitor in a glass storefront window, and the screen displays a woman--presumably a significant other--who is also apparently passing a similar display on a street somewhere else on the planet. This causes them to recognize each other, stop and wave, and probably start a conversation. The logistics of this can make you think a bit--like coordinating the timing and synchronization of these two people and displays. But this is where things could get interesting in an application like this. Maybe a number of people in a person's personal network could be tracked. Or maybe this is an acquaintance who happens to be traveling in the same city, in which case the service offers a chance for the two to hook up.
The commercial is careful in portraying positive reactions by the actors in the video. The woman walking the dog stops in her stride and shows interest in the dog food advertisement. And the man and woman in the "telepresense" link are happy to see each other. An alternative reaction to these applications could easily be concern about the implications of the underlying surveillance and personal privacy issues involved in the tracking and profiling of people. My personal opinion is that this stuff is going to happen sooner rather than later, so it is good to get these ideas out in the open so they can be debated and managed accordingly.
The video is on this page (you will need to scroll down to the Visual Networking video):
Cisco video: Visual Networking
rk
What Can We Do Differently With Ubicomp?
In my recent ponderings on how to move ubicomp from theory and philosophy into practice, I found it necessary to ask (re-ask, really--it's not like this is the first time) what exactly we might want to accomplish with ubicomp implementations. I will spare you my journey of convoluted mental ramblings, which scare even me sometimes, and present my current conclusions in the form of some questions which I believe address some real human-computer interaction challenges:
- We don't want to give up our online access to information and community, but how can we avoid becoming a slave to our computer screens?
- How can we deliver and present information differently and more effectively?
- How can we filter the ever-increasing flood of information we are trying to process on a daily basis?
- How do we use technology--including but not limited to computers--to better interact with our environment?
To answer each of these questions could fill a volume of writing on its own. However, I will summarize my answers to the questions in the form of 4 principles for ubicomp research and development. I propose these principles as a work in progress and as a basis for discussion.
Four Working Principles Of Ubicomp
- We will make the network and information come to us and follow us arround, not the opposite.
- We will layer the interfaces onto our environment and blur the line between information and things.
- We will use location-awareness and techniques like collaborative filtering to make information personally, physically and temporally relevant.
- We will use a variety of technologies to make things and our environment more aware and able to sense and communicate with us.
Although these principles do not exactly coincide with any particular list of known principles in the field of ubicomp, they are based on literature from known experts and not just naively springing from my head. I expect that someday we will have a relatively-agreed-upon set of working principles. Until then, I reserve the right to modify the content and number of the principles. Finally, I will also point out that the modifer "working" in "working principles has two meanings: a work in progress, but also that I am more interesting in the practical aspects of rolling up our sleeves and getting ubicomp systems constructed.
rk
Human-Computer Interfaces In The Iron Man Movie
I just saw the Iron Movie which, as expected, was replete with high-tech gadgetry and special effects. I also noticed that it specifically contained many examples of technology that embodied Human Computer Interfaces (HCI) concepts and themes, and I wanted to capture my observations and thoughts while the movie is (sort of) still fresh in my mind.
I myself have only seen the movie once, and if/when you see it you will notice that it has action pretty much from start to finish. So I apologize in advance if I don't get the details of the human-computer bits exactly right, or even if I saw one thing and imagined something different in my head. The items I will discuss were rather fleeting and subdued compared to the shots of high-tech armor and explosions they were wedged between. But they were cool in their own right and worthy of consideration.
Spoiler Disclaimer: I won't be discussing anything that is particularly revealing and certainly won't give away any significant parts of the plot, but I will include this disclaimer anyway.
Computers Are Ubiquitous
It should come as no surprise, in a movie featuring the comic character of Iron Man, that computers occur everywhere from bunkers and planes to his labs, home and of course his high-tech armor. HCI is not synonymous with ubiquitous computing, nor does it require it. However, they go tend to go hand-in-hand, since, the more computers become ubiquitous or pervasive in a person's environment, the more important is the requirement for intuitive, natural and effective interfaces.
Interfaces are overlaid on the environment
There were a few examples of this in Iron Man:
- The security access keypad to Tony Stark's personal lab appears on demand on the pane of glass next to the door.
- At one point in his lab Tony invokes a control keypad which appears on a black glass surface.
- Probably the best example is in his bedroom. When the (ahem) guest awakes in his bedroom, displays again materialize on the glass of the windows overlooking his beach view. It is hard to recall what they displayed, but they contained information relevant to the home, daily routine and environment.
- Last but not least, heads-up displays appear inside the helmet of the armor.
Icon Interfaces
In at least one case, the virtual overlaid control interface consists of icons rather than having the standard qwerty keyboard. There is a reason that fast food places have keyboards with specialized, iconic keys. It's not because the workers are not intelligent enough to think about the corresponding words. Rather it's not particularly efficient to have to think about them or worse to have to type them in. If an icon can simply, naturally, and effectively communicate an idea or command, then why require anything more?
Smart Home Control
Tony's home is controlled by an intelligent software assistant in the form of a virtual butler which accepts voice commands. Although it appears a bit too intelligent at times, this form of HCI has been developed and produced for years for smart home applications and will continue to be refined over time.
Anticipating user needs
Tony's software assistant also anticipates his needs and offers suggested courses of action it can take on Tony's behalf. It would desirable for any worthy software assistant to reduce the burden on it user by anticipating user actions and offering options, or even automatically performing actions which need no intervention or approval.
Mobile Agents
Rather than develop new programming for the armor, Tony downloads the home/lab butler program into the armor. Presumably it actually creates a copy since it makes little sense to leave the home unattended. But for Tony, the effect is that the software agent he is familiar with, and his probably extensive customized preferences and usage profile, seamlessly travel with him.
Ambient Information
In addition to whatever else the displays in the bedroom were conveying, they were also communicating information about the ambient environment including weather and surf conditions.
Intelligent Robotic Assistants
Tony Stark uses robotic assistants which are intelligent in the sense that they respond to his verbal commands and nonverbal cues. However, these robots are not android in their physical design, something which would not only require more suspension of belief but would also not be necessary or desirable for their dedicated purpose.
In the movie, Tony's robo-assistants mostly serve to provide comic relief. However, to casually dismiss them with this superficial analysis would be to ignore a couple of more subtle points they illustrate about human-computer interactions:
- The comic element of the robots stem from their failure in some cases to follow Tony's commands. This is precisely what we should expect in an intelligent artificial assistant. A software agent with a level of programming sophistication sufficient for exhibiting useful assistive behavior would necessarily need to work with incomplete and fuzzy information, and is further interacting with an imprecise and fuzzy (by computer standards) human being. So it is occasionally going to make mistakes in the same way that a human assistant would sometimes (or frequently, depending on the human) make mistakes.
- We see an example where the human being is probably exhibiting anthropomorphism (the attribution of human characteristics to nonhuman entities) toward the robot. We hear Tony delivering fairly extended and human-like communication to a robot. Yet we have no other indication that the robot is comprehending more than the basic elements of the speech, or that it's behavior is really anything more naive and imperfect responses to the situation. Humans are well known to be capable of attributing intelligence and even feelings to inanimate objects, even those which lack intelligence by any standards. On one hand, this says more about the human than the agent, but on the other hand it serves as a sort of metric by which to judge the behavior of the agent. It has achieved some level of success if it invokes anthropomorphism in the human.
As you can see, the Iron Movie movie demonstrated a variety of human-computer interface concepts. It is not the first movie to include such concepts (Minority Report comes to mind), but I believe it does a fair job and in closing I will leave you with a thought: was this a conscious effort on the filmmakers part, or are these ideas starting to entrench themselves as memes in our subconscious?
rk
Photo Disclaimer: The photos are most just blog candy as it was not possible to to find actual shots of the displays I discussed.
Xerox Plans For The Future
With their history of computer innovations at PARC, Xerox can certainly be credited in some ways with creating ubiquitous computing, at least the ubiquitous part. This new article Xerox plans the future of today (courtesy of BBC news) talks about research they are doing in different areas these days.
While I was hoping to see more ubicomp stuff, the work is meaningful nonetheless. The erasable paper idea might be co-opted for interesting applications. For example, I imagine a laser beam which could update posters in a room with fresh new information.
I still like the idea, though, of digital ink where a display is thin like paper but can be updated like an LCD screen.
rk