Pranav Mistry 談「第六感官」技術的驚異潛力

=========================
講者：Pranav Mistry
2009年11月演講，2009年11月在TED上線
MyOOPS開放式課程
TED繁體原譯：Lucy Chang
TED繁體編輯：Coco Shen
TED原譯網頁
編輯：洪曉慧
後制：洪曉慧
字幕影片後制：謝旻均
------------------
Pranav Mistry 談「第六感官」技術的驚異潛力
我們生長在和周遭物體互動的環境裡，有很多很多物體是我們每天都要用到的。和大部分電腦設備相比，這些物體有趣多了！當我們提到物體，就自然而然出現與這東西有關的另一件事－就是姿勢，也就是：我們怎麼使用這些物體？我們在日常生活中如何使用這些物體？我們不只用不同姿勢來使用物體，也用來和別人溝通。合十的姿勢，可能表示尊敬某人，或者是－在印度，不必教，連小孩都知道。這手勢是指板球中「4次跑壘」的意思，這成了我們的生活常識。
所以，我從一開始就很有興趣，想瞭解我們對於日常用品和姿勢有何認知，以及如何將這些物品運用在我們和數位世界的互動上。為什麼沒有鍵盤和滑鼠，我就沒辦法使用電腦?也不能像與實體世界溝通那樣使用電腦?
我從八年前開始進行這場探索，從我桌上的滑鼠開始，我不是將它用於電腦，而是拆開它。你們或許都知道，在那個年代，滑鼠裡面有個球，還有兩個滾輪，所以當滑鼠移動時，滾輪會告訴電腦，球滾動的方向。我對這兩個滾輪很有興趣，但還需要多幾個，所以就跟朋友借他的滑鼠，一直沒還他－這樣就有了四個滾輪。有趣的是，我怎麼使用這些滾輪？基本上，我把它們從滑鼠裡拿出來，放在一條線上，有線、滑帶、和一些彈簧。我做出的，基本上是一個感應姿勢的介面裝置，像個動作感應裝置，成本只要兩美元。所以，不論我在實體世界做什麼，就會複製到數位世界裡。只要用這個我8年前做的小東西就可以，那是在2000年。
我對於整合實體和數位世界有興趣，我想到便利貼。我想,「何不讓實體便利貼的介面，整合到數位世界中呢?」在便利貼上，寫給我媽媽的留言紙條，變成簡訊傳遞；或是寫在紙上的會議通知，自動和我的數位行事曆同步－待辦事項會自動在電腦上同步。你也可以在數位世界裡搜尋，或者是,你可以寫下問題, 如 「Smith教授的地址?」然後這個小系統就會把地址列印出來，這就像是個紙做的輸出入系統，只用紙就可以製作。
另一項探索是－我想製造一枝可以畫3D效果的筆，所以就設計出這支筆，能夠幫助設計師和建築師，不只以三度空間思考，也可以實際畫3D圖形，這樣用起來就更直覺多了。
我又想,「何不做一個實體世界的Google地圖?」不必輸入關鍵字找東西，只需將物體放在地圖上。如果我把登機證放上，它會顯示登機口的位置；放咖啡杯，它會顯示咖啡廳位置，或垃圾筒位置。
這是我早期所做的一些探索，因為我的目標是緊密銜接這兩個世界。在這些實驗中有一個共通點:我嘗試把實體世界一部份帶進數位世界中。我把某些物體，或現實生活中直覺式的東西帶進數位世界裡，目的是讓電腦介面更趨直覺式。
但是後來我瞭解到，人們並不真的對電腦感興趣，人們感興趣的是資訊。我們想知道更多事情，我們想知道周圍的各種動態。
所以，大約在去年，去年年初的時候我開始想, 「我何不顛倒研究方向呢?」或許,「不如把數位世界的數位資訊描繪在實體世界裡?」因為這些影像，事實上都被限制在這些方形的裝置裡，再放進你的口袋。我何不把這個形體打破，把資訊放進日常生活中?這樣我也不必學新的語言來和這些像素溝通?
為了實現這個夢想，我真的想過把一台大型投影機放在頭上，我想，這就是為何它叫做「投(頭)影機」，對吧?我照著字面意思做了，我把自行車頭盔稍微割掉一點,讓投影機可以放得進去，這樣，我就可以用數位訊息將環繞我的真實世界擴大。
但是後來，我發現我也想和這些數位像素互動，所以我加了一個小攝影機，當作數位眼睛。之後, 我們更進一步,做成使用者導向的頸掛型式，就是大多數人所知的「第六感官裝置」。
這技術最有趣的地方是－你可以把數位世界帶著走，到任何地方都可以！你可以使用周遭任何表面或牆壁當成介面，這台攝影機追蹤你所有的動作，不論雙手在做什麼，它都瞭解姿勢的含意。還有，你看那些色筆，那是我們最初使用的型式，可以在任何牆壁上畫畫；可以停在牆壁前面，開始在牆上作畫。而且我們不只追蹤一根手指，我們讓你可以自由使用雙手全部手指，所以你可以實際用雙手去放大、縮小地圖，就是捏畫面所呈現的圖形。事實上，攝影機是用來收集所有影像，執行圖形邊緣辨識和顏色辨識。有很多小程式在裡面跑，技術上有點複雜，但在某種意義上來說，給了你一個更能直覺使用的輸出結果。
我更興奮的是，可以帶到戶外去用，不必掏出口袋裡的照相機，只要擺出一個照相的姿勢，它就替你照相了。
(掌聲)
謝謝！
拍完之後，隨便找一面牆，就可以開始瀏覽這些照片，或是, 「我想修改一下這些照片，然後用電子郵件寄給朋友」。我們所尋覓的世代，是資訊處理能確實與實體世界融合在一起。當然，如果沒有任何平面可用，就用手掌作簡單的操作吧！看這個，我只用我的手，就能撥電話號碼。攝影機不只瞭解你手部的動作，而且很有趣的，它還知道你手裡拿的是什麼東西。
我們現在正在做的是－舉個例, 在這裡，這本書的封面，正在和線上數以百萬計的書封面比對，找尋這是哪一本書。一旦找到資料，它會找到更多相關書評。或者，可能紐約時報有個有聲的簡介，你就可以在實體書上，聽到有聲的書評。(「在哈佛大學的著名演講…」)。
這是歐巴馬總統上周來MIT的演講。(「…我特別要感謝2位傑出的MIT…」)。我在戶外，從報紙上看到他演講的實況錄影。你的報紙會給你看即時天氣報導，不必去更新資料－這原本要上電腦才找得到,是嗎?
(掌聲)
我回印度時,只要拿出我的登機證，就可以知道我的班機延誤多久。因為在這種時候，我不想打開iPhone點選某個功能圖案。我相信此技術不只改變這個－是的…(笑聲)也會改變我們和別人交流的方式，不只在實體世界裡。好玩的是，我在波士頓的地鐵上玩踢乒乓球遊戲，在地上玩,不錯吧?(笑聲)我相信當這種技術和實體生活融合在一起時，想像力是唯一能限制我們的東西。
但很多人會說，我們的工作不只和實體物品有關。事實上，我們做的大多是會計、編輯這類的事，那該怎麼辦呢？很多人很期待新一代的平板電腦上市，與其等待，我乾脆自己做一個﹐只用了一張紙。我把攝影機上的－所有網路攝影機都內建有麥克風，我把麥克風拿下來，把它夾在－就像是做了個夾式麥克風－夾在紙上, 隨便的一張紙，每當我碰到紙的時候，這在紙上接觸的聲音就會通知電腦，攝影機追蹤我手指的移動。
當然，你也可以看電影 (「午安，我叫Russell」) (「…我是第54團的荒野探險者」)。
當然也可以玩遊戲 (汽車引擎聲)攝影機知道你拿紙的方向，就可以玩賽車遊戲 (掌聲)。
很多人一定早想到了，當然，還可以瀏覽。對，當然可以瀏覽任何網站，甚至在紙上做各種運算，在任何你有需要的地方。更有趣的是，我對如何把這個功能變得更強大點很感興趣。當我回到桌上，我可以捏住那個資訊，然後放到我的電腦螢幕上，這樣我就可以用桌上電腦了。
(掌聲)
幹嘛只用電腦?用紙也很好玩。紙的世界充滿樂趣，我把文件的一部分放在這裡,第二部份來自另一份文件－然後調整這裡的資訊。好，然後我說,「OK, 看起來不錯，我要列印出來」，就有了一份列印文件。這樣－工作流程比我們一般所用的方式更趨直覺式了，像20年前的工作方式，而不是像現在，要在兩個世界間切換。
最後，我相信把資訊和所有物件整合，不僅能幫助我們消除數位落差，消除兩個世界間的鴻溝，還可以以某種方式幫助我們保有人性，加強我們與實體世界的連結。最終會幫助我們，不要變成坐在另一部機器前面的機器。
就這樣。謝謝。
(掌聲)
謝謝。
(掌聲)
Chris Anderson：嗯﹐Pranav，首先，你是個天才！這太不可思議了,真的！你預備怎麼做? 想開個公司嗎﹖還是繼續研究下去? 或有其他的打算?
Pranav Mistry：很多公司－其實就是媒體實驗室的贊助者，有興趣以其他方法繼續研究。像行動通訊公司和印度非營利組織的應用方法便不同。這些機構想「為什麼只有『第六感官』?我們應該為殘障人士，設置『第五感官』」。就像是啞巴，這種技術可以讓他們用另一種方式「說」出來，像是配個喇叭之類的。
CA：你自己的打算呢? 要留在MIT嗎？還是你也要參與這些計畫?
PM： 我正試著讓這技術更廣泛的被人們使用，讓大家都能開發自己的「第六感官」裝置。因為硬體其實不難製造，自己做也不難，我們會提供所有開放源碼軟體給大家，可能從下個月開始。
CA：開放源碼？哇！
(掌聲)
CA：你會回到印度做這些計畫嗎?
PM：是，是的，當然。
CA：你的計畫呢? 關於MIT的?還有印度的?未來怎樣分配你的時間?
PM：這裡有很大的能量。很多東西可以學。你剛剛看到的，都與我在印度所學有關。如果以成本效益考慮，這個系統只要300美元；和2萬美元的Surface Table，或其他類似產品相比，或甚至2美元的滑鼠手勢系統，當年可能要價 5,000 美元?當我在一場研討會上，把這東西展示給印度總統Abdul Kalam看時，他說: 「我們應該在Bhabha原子研究中心裡，用這個技術做些研究」。我很期待可以把這些技術帶給普羅大眾，而不是把這些技術留在實驗室裡。
(掌聲)
CA：目前來過 TED 演講的人之中，你可以稱的上是少數2、3個世界上最頂尖的發明家之一，TED很榮幸能邀你來這裡演講，非常感謝，真是太棒了！
(掌聲)
============================
以下為系統擷取之英文原文
About this talk
At TEDIndia, Pranav Mistry demos several tools that help the physical world interact with the world of data -- including a deep look at his SixthSense device and a new, paradigm-shifting paper "laptop." In an onstage Q&A, Mistry says he'll open-source the software behind SixthSense, to open its possibilities to all.

About Pranav Mistry

Pranav Mistry is the inventor of SixthSense, a wearable device that enables new interactions between the real world and the world of data. Full bio and more links

Transcript

We grew up interacting with the physical objects around us. There is an enormous number of them that we use every day. Unlike most of our computing devices, these objects are much more fun to use. When you talk about objects, one other thing automatically comes attached to that thing, and that is gestures: how we manipulate these objects, how we use these objects in everyday life. We use gestures not only to interact with these objects, but we also use them to interact with each other. A gesture of "Namaste!", maybe, to respect someone, or maybe -- in India I don't need to teach a kid that this means "four runs" in cricket. It comes as a part of our everyday learning.

So, I am very interested, from the beginning, that how -- How our knowledge about everyday objects and gestures, and how we use these objects, can be leveraged to our interactions with the digital world. Rather than using a keyboard and mouse, why can I not use my computer in the same way that I interact in the physical world?

So, I started this exploration about eight years back, and it literally started with a mouse on my desk. Rather than using it for my computer, I actually opened it. Most of you might be aware that, in those days, the mouse used to come with a ball inside, and there were two rollers that actually guide the computer where the ball is moving, and, accordingly, where the mouse is moving. So, I was interested in these two rollers, and I actually wanted more, so I borrowed another mouse from a friend -- never returned to him -- and I now had four rollers. Interestingly, what I did with these rollers is, basically, I took them off of these mouses and then put them in one line. It had some strings and pulleys and some springs. What I got is basically a gesture interface device that actually acts as a motion-sensing device made for two dollars. So, here, whatever movement I do in my physical world is actually replicated inside the digital world just using this small device that I made, around eight years back, in 2000.

Because I was interested in integrating these two worlds, I thought of sticky notes. I thought, "Why can I not connect the normal interface of a physical sticky note to the digital world?" A message written on a sticky note to my mom on paper can come to an SMS, or maybe a meeting reminder automatically syncs with my digital calendar -- a to-do list that automatically syncs with you. But you can also search in the digital world, or maybe you can write a query, saying, "What is Dr. Smith's address?" and this small system actually prints it out -- so it actually acts like a paper input-output system, just made out of paper.

In another exploration, I thought of making a pen that can draw in three dimensions. So, I implemented this pen that can help designers and architects not only think in three dimensions, but they can actually draw so that it's more intuitive to use that way.

Then I thought, "Why not make a Google Map, but in the physical world?" Rather than typing a keyword to find something, I put my objects on top of it. If I put a boarding pass, it will show me where the flight gate is. A coffee cup will show where you can find more coffee, or where you can trash the cup.

So, these were some of the earlier explorations I did because the goal was to connect these two worlds seamlessly. Among all these experiments, there was one thing in common: I was trying to bring a part of the physical world to the digital world. I was taking some part of the objects, or any of the intuitiveness of real life, and bringing them to the digital world, because the goal was to make our computing interfaces more intuitive.

But then I realized that we humans are not actually interested in computing. What we are interested in is information. We want to know about things. We want to know about dynamic things going around.

So I thought, around last year -- in the beginning of the last year -- I started thinking, "Why can I not take this approach in the reverse way?" Maybe, "How about I take my digital world and paint the physical world with that digital information?" Because pixels are actually, right now, confined in these rectangular devices that fit in our pockets. Why can I not remove this confine and take that to my everyday objects, everyday life so that I don't need to learn the new language for interacting with those pixels?

So, in order to realize this dream, I actually thought of putting a big-size projector on my head. I think that's why this is called a head-mounted projector, isn't it? I took it very literally, and took my bike helmet, put a little cut over there so that the projector actually fits nicely. So now, what I can do -- I can augment the world around me with this digital information.

But later, I realized I wanted to interact with those digital pixels, also. So I put a small camera over there, that acts as a digital eye. Later, we moved to a much better, consumer-oriented pendant version of that, that many of you now know as the SixthSense device.

But the most interesting thing about this particular technology is that you can carry your digital world with you wherever you go. You can start using any surface, any wall around you, as an interface. The camera is actually tracking all your gestures. Whatever you're doing with your hands, it's understanding that gesture. And, actually, if you see, there are some color markers that in the beginning version we are using with it. You can start painting on any wall. You stop by a wall, and start painting on that wall. But we are not only tracking one finger, here. We are giving you the freedom of using all of both of your hands, so you can actually use both of your hands to zoom into or zoom out of a map just by pinching all present. The camera is actually doing -- just, getting all the images -- is doing the edge recognition and also the color recognition and so many other small algorithms are going on inside. So, technically, it's a little bit complex, but it gives you an output which is more intuitive to use, in some sense.

But I'm more excited that you can actually take it outside. Rather than getting your camera out of your pocket, you can just do the gesture of taking a photo and it takes a photo for you.

(Applause)

Thank you.

And later I can find a wall, anywhere, and start browsing those photos or maybe, "OK, I want to modify this photo a little bit and send it as an email to a friend." So, we are looking for an era where computing will actually merge with the physical world. And, of course, if you don't have any surface, you can start using your palm for simple operations. Here, I'm dialing a phone number just using my hand. The camera is actually not only understanding your hand movements, but, interestingly, is also able to understand what objects you are holding in your hand.

What we're doing here is actually -- for example, in this case, the book cover is matched with so many thousands, or maybe millions of books online, and checking out which book it is. Once it has that information, it finds out more reviews about that, or maybe New York Times has a sound overview on that, so you can actually hear, on a physical book, a review as sound. ("famous talk at Harvard University ...")

This was Obama's visit last week to MIT. ("... and particularly I want to thank two outstanding MIT ...") So, I was seeing the live [video] of his talk, outside, on just a newspaper. Your newspaper will show you live weather information rather than having it updated -- like, you have to check your computer in order to do that, right?

(Applause)

When I'm going back, I can just use my boarding pass to check how much my flight has been delayed, because at that particular time, I'm not feeling like opening my iPhone, and checking out a particular icon. And I think this technology will not only change the way -- Yes. (Laughter) It will change the way we interact with people, also, not only the physical world. The fun part is, I'm going to the Boston metro, and playing a pong game inside the train on the ground, right? (Laughter) And I think the imagination is the only limit of what you can think of when this kind of technology merges with real life.

But many of you argue, actually, that all of our work is not only about physical objects. We actually do lots of accounting and paper editing and all those kinds of things; what about that? And many of you are excited about the next generation tablet computers to come out in the market. So, rather than waiting for that, I actually made my own, just using a piece of paper. So, what I did here is remove the camera -- All the webcam cameras have a microphone inside the camera. I removed the microphone from that, and then just pinched that -- like I just made a clip out of the microphone -- and clipped that to a piece of paper, any paper that you found around. So now the sound of the touch is getting me when exactly I'm touching the paper. But the camera is actually tracking where my fingers are moving.

You can of course watch movies. ("Good afternoon. My name is Russell ...") ("... and I am a Wilderness Explorer in Tribe 54.")

And you can of course play games. (Car engine) Here, the camera is actually understanding how you're holding the paper and playing a car-racing game. (Applause)

Many of you already must have thought, OK, you can browse. Yeah. Of course you can browse to any websites or you can do all sorts of computing on a piece of paper wherever you need it. So, more interestingly, I'm interested in how we can take that in a more dynamic way. When I come back to my desk I can just pinch that information back to my desktop so I can use my full-size computer.

(Applause)

And why only computers? We can just play with papers. Paper world is interesting to play with. Here, I'm taking a part of a document and putting over here a second part from a second place -- and I'm actually modifying the information that I have over there. Yeah. And I say, "OK, this looks nice, let me print it out, that thing." So I now have a print-out of that thing, and now -- The workflow is more intuitive the way we used to do it maybe 20 years back, rather than now switching between these two worlds.

So, as a last thought, I think that integrating information to everyday objects will not only help us to get rid of the digital divide, the gap between these two worlds, but will also help us, in some way, to stay human, to be more connected to our physical world. And it will help us, actually, not be machines sitting in front of other machines.

That's all. Thank you.

(Applause)

Thank you.

(Applause)

Chris Anderson: So, Pranav, first of all, you're a genius. This is incredible, really. What are you doing with this? Is there a company being planned? Or is this research forever, or what?

Pranav Mistry: So, there are lots of companies -- actually sponsor companies of Media Lab -- interested in taking this ahead in one or another way. Companies like mobile phone operators want to take this in a different way than the NGOs in India, [who] are thinking, "Why can we only have 'Sixth Sense'? We should have a 'Fifth Sense' for missing-sense people who cannot speak. This technology can be used for them to speak out in a different way with maybe a speaker system."

CA: What are your own plans? Are you staying at MIT, or are you going to do something with this?

PM: I'm trying to make this more available to people so that anyone can develop their own SixthSense device because the hardware is actually not that hard to manufacture, or hard to make your own. We will provide all the open source software for them, maybe starting next month.

CA: Open source? Wow.

(Applause)

CA: Are you going to come back to India with some of this, at some point?

PM: Yeah. Yes, yes, of course.

CA: What are your plans? MIT? India? How are you going to split your time going forward?

PM: There is a lot of energy here. Lots of learning. All of this work that you have seen is all about my learning in India. And now, if you see more about the cost-effectiveness: this system costs you 300 dollars compared to the 20,000 dollar surface tables, or anything like that. Or maybe even the two-dollar mouse gesture system at that time was costing around 5,000 dollars? So, we actually -- I showed that, at a conference, to President Abdul Kalam, at that time, and then he said, "OK, we should use this in Bhabha Atomic Research Centre for some use of that." So I'm excited about how I can bring the technology to the masses rather than just keeping that technology in the lab environment.
(Applause)
CA: Based on the people we've seen at TED, I would say you're truly one of the two or three best inventors in the world right now. It's an honor to have you at TED. Thank you so much. That's fantastic.
(Applause)
--------------------------------------------------------------------------------