Hello, my name is Jack Sparrow. I'm a 50 year old self-employed Pirate from the Caribbean. 
Envisioning how will Apple use its new touchless gestures.
It was recently reported that Apple is working on a screen that senses your finger without even touching it. The idea isn’t entirely new, but Apple does have a remarkable knack for bringing novel technology to the mass market. So, we must ask: What should a touchless gesture do?
We don’t know for sure that Apple will work this technology into any actual products—but if they do, they’ll be adding to a crowded lineup: Tap, long tap, double tap, tap and hold, push, and four directions of swipe are all already gestures. This creates a challenge not only for users, who need to learn what yet another thing does, but for developers, who need to decide what yet another thing does. Apple will have to figure out a semantic meaning for this new gesture that feels intuitive and desirable, while avoiding annoying accidental triggers.
To do so, instead of bringing mobile paradigms to its desktop platform, it may need to go the other direction. That’s because hovering already has an important, but easily overlooked, function in the desktop computing environment: It’s a request for more information. When a button has only an icon, hovering displays a text name. When a button already has a text name, hovering displays a more in-depth explanation. When text is cut off, hovering displays the full string. And when a link’s destination is hidden, hovering shows it. Hovering means “I need more information before deciding whether to click.” It’s a response to hesitation that gets you unstuck. It’s nonessential, perhaps, but it sure is useful.
The need for this sort of progressive disclosure is, in some ways, more urgent on mobile devices: With smaller screens, there’s less room to show details by default. And yet, we’ve somehow gotten by for over a decade without tooltips on our phones. Implementing such a garden-variety UI element now would be uninspired at best. But there’s more we can do — much more. That’s because our devices have an ambient intelligence nearly unimaginable back when the tooltip was invented: They know far more about the world, and about us, than ever before. There’s no way to know for sure what Apple will do with this research — if anything — until the WWDC rumor mill starts up a few years now. But we can certainly guess…
Siri Insights
Apple has been integrating Siri’s intelligence deeper and deeper into the iOS platform, and hovering may provide the surface where that intelligence finally fulfills its potential. This post envisions Siri Insights as a layer of iOS that augments whatever’s on screen with the most relevant possible details of interest to the user, pulled from the current context, the user’s interests, and the web at large, to help with making satisfying choices. And unlike 3D Touch, whose implementation is largely up to software developers, Siri Insights is out of reach to third parties: It’s Apple’s exclusive surface for extending the iOS experience with information deeply relevant to the user’s life — while keeping that information private from the app on screen.
Siri Insights are triggered by a moment of hesitation or curiosity — A finger held above the screen, not yet ready to tap. At a distance of perhaps an inch, a hovering finger excites a small, colorful, undulating waveform, indicating Siri has details to share. A closer approach of the finger triggers a full blossom, along with a simple overlaid message:
So, how does this look in everyday life?
Choosing music
Scrolling through a (long) list of recent releases in Spotify, a cover intrigues me. Siri knows I’m an avid Pitchfork reader, so the Siri Insight gives me that review score — and my mind is made: Clearly I need to listen.
Choosing TV
Selecting a late-night talk show to watch, I pause on a thumbnail with an actress I don’t immediately recognize. Siri knows I love 30 Rock, and points out that the actress is Tiffany Haddish, who stars alongside Tracy Morgan in a new comedy show. That’s enough for me — I want to hear about The Last O.G.
Choosing news
Opening the the Apple News feed, I’m met with an article about Windows Phone (yes, in 2018). Hovering, Siri points out that it talks about Microsoft’s and Google’s competing design languages. Being an avid follower of mobile UI trends, I’m sold.
Choosing an event
Checking out what bands are coming to town, I come across a band I’ve been wanting to see. Hovering, Siri confirms that I’m free that evening — so I can go ahead and grab a ticket.
Choosing a dessert shop
And, finally, looking for a little something sweet, I search for ice cream places nearby. I’m a fan of Molly Moon’s, and on hover Siri tells me they have their seasonal Cardamom flavor — literally my favorite of all time. Siri Insights has provided me exactly the info I want to make the choice I’ll enjoy the most. I know where I’m going.
Upward & onward
We’ve used up, down, left, right, and into the screen. We have one remaining direction, and we’d better use it well. The space above the screen gives us the opportunity to react to uncertainty — and resolve it. Cognitive computing and touchless interaction are being realized; can we use them to turn hesitation into decision?
Siri insights: hesitate, decide was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.
No comments:
Post a Comment