Talks at Google2016.5.10---Alan Winfield:思想機器人

Talks at Google2016.5.10---Alan Winfield:思想機器人
發佈日期：2016年5月10日
Professor Alan Winfield from The University of the West of England joined us to share his work on Robots and Ethics, in a talk entitled "The Thinking Robot".
About the Book:
Robotics is a key technology in the modern world. Robots are a well-established part of manufacturing and warehouse automation, assembling cars or washing machines, and, for example, moving goods to and from storage racks for Internet mail order. More recently robots have taken their first steps into homes and hospitals, and seen spectacular success in planetary exploration. Yet, despite these successes, robots have failed to live up to the predictions of the 1950s and 60s, when it was widely thought - by scientists and engineers as well as the public - that by turn of the 21st century we would have intelligent robots as butlers, companions, or co-workers.
机器人技术是现代世界的关键技术。机器人是制造业和仓储自动化的一套行之有效的部分，组装的汽车或洗衣机，例如，搬货，并从货架互联网邮购。最近，机器人已采取了他们的第一个步骤，为家庭和医院，看到了行星探测令人瞩目的成就。然而，尽管有这些成就，机器人已经辜负了20世纪50年代和60年代的预言，当它被普遍认为 - 由科学家和工程师，以及公众 - 这在21世纪之交，我们将有智能机器人作为管家，同伴，或同事。




==========Google 翻译==========

0:00and thank you very much indeed but it's really great to be here and thank you so 0:06much for the invitation so yes robot intelligence subtitled that the lecture 0:12the thinking robot because I'd immediately begs the question what on 0:18earth do we mean by thinking well we could have caused spend the whole of the 0:23next hour 0:24you know debating what we mean by thinking but but I should say that I'm 0:29particularly interested in and will focus on embodied intelligence so in 0:36other words the kind of intelligence that that you know that we have the 0:40animals including humans have and that robots have so of course you know that 0:45that kind of slightly differentiates what I'm talking about from a I but I 0:50regard robotics is a kind of subset of AI and of course one of the things that 0:57we've discovered in the last 60 years of artificial intelligence is that the 1:01things that we thought were really difficult actually are relatively easy 1:07like you know playing chess or go for that matter whereas the things that we 1:12thought were really difficult sorry we originally thought were really easy like 1:17making cup of tea are really hard so so it's kind of the opposite what was 1:22expected so embodied intelligence in the real world is is is really very 1:28difficult indeed and that's what I'm interested in 1:33so this is the kind of outline of the of the talk I'm gonna talk initially about 1:41intelligence and and offers some ideas if you like for a way of thinking about 1:47intelligence and breaking it down into categories or or or types of 1:52intelligence and then I'm going to choose a particular one which I'm I've 1:57been really working on the last last three or four years and that is what I 2:03call a generic architecture for a functional imagination or in short 2:09robots with internal models so that's really what I want 2:13to focus on because I really wanted to show you some experimental work that 2:16we've done the last couple of years in the lab I mean I'm an electronics 2:22engineer I mean experimentalist and so doing experiments is really important 2:27for me so the first thing that that you know we ought to realize I'm sure we do 2:34realize is that intelligence is not one thing that we all you know animals 2:41humans and robots have more or less of absolutely not and you know there are 2:46several ways of breaking intelligence down into different kind of categories 2:50if you like of intelligence different types of intelligence and here's one 2:55that I came up with in the last couple of years it's certainly not the only way 3:01of thinking about intelligence but this really breaks intelligence into four if 3:06you like types four kinds of intelligence you could say kinds of 3:11minds eye I guess the the most fundamental is what we call 3:17morphological intelligence and that's the intelligence that you get just from 3:22having a physical body and you know there are some interesting questions 3:27about how you design morphological intelligence you probably all seen 3:31pictures of movies of robots that can walk but in fact don't actually have any 3:38computing any computation whatsoever in other words the the the the behavior of 3:45of walking is an emergent property of the mechanics if you like the springs 3:50and leaders and so on 3:52in the robot so it's an example of morphological intelligence individual 3:58intelligence is is the kind of intelligence that you get from learning 4:02individually social intelligence i think is really interesting and important and 4:08that's the one that I'm going to focus on most in this talk social intelligence 4:11is the intelligence that you get from learning socially from from each other 4:17and of course we are a social species 4:21and the other one which I've been working on a lot in the last twenty odd 4:26years is swarm intelligence so this is the kind of intelligence that we see in 4:31most particularly in social animals insects where the where the the most 4:40interesting properties of swarm intelligence tend to be emergent or self 4:44organizing so in other words the the intelligence is typically manifest as a 4:51collective behavior that emerges from the if you like the microbe interactions 4:55between the individuals in that population so emergence and 5:00self-organization particularly interesting to me but I said this is 5:05this is absolutely no not the only way to think about intelligence and I'm 5:11gonna show you in another way of thinking about intelligence which I 5:15particularly liked and this is Dan its power generating test so in Darwin's 5:23dangerous idea and and several of the books I think dan then it suggests that 5:31that a good way of thinking about intelligence is to think about the fact 5:34that all animals including ourselves need to decide what actions to take you 5:42know so choosing the next action is really critically important I mean it's 5:47it's it's it's critically important for all of us including humans even though 5:52the wrong action may not killers as it were as humans but for many animals the 5:57wrong action may well killed that animal and and so then it talks about what he

6:04calls the Tower of generating test which I want to show you here it's it's a 6:09really cool kind of breakdown if you like way of thinking about intelligence 6:13so the bottom of his tower are darwinian creatures and the thing about doubt 6:19darwinian creatures is that they have only one way of as it were learning from 6:26if you like 6:29generating and testing next possible actions and that is 6:33natural selection so darwinian creatures in his scheme cannot learn they can only 6:40try out an action if it kills them well that's the end of that you know so so 6:46that you know by the laws of natural selection that particular action is 6:49unlikely to be passed on to two descendants now of course all animals on 6:57the planet are darwinian creatures including ourselves but a subset of what 7:03it calls skin aryan creatures so skin aryan creatures are able to generate a 7:10next possible candidate action if you like the next possible action and try it 7:15out and and here's the thing if it doesn't kill them but it's but it's 7:20actually a bad action then though learned from that or even if its good 7:24action skin aryan creature will learn from trying out an action so really 7:29scary creatures are a subset of darwinian 'he's actually a small subset 7:36that are able to learn by trial and error individually learned by trial and 7:42error now the third layer in store if you like in 10 its tower he calls pop 7:50area increases after obviously the the philosopher Karl Popper and pop aryan 7:56creatures have a big advantage over darwinian skin aryans in that they have 8:01an internal model of themselves and the world and with an internal model it 8:06means that you can try out an action a candidate next possible action if you 8:12like by imagining it and it means that you don't have actually have to put 8:17yourself to the risk of of trying it out for realz physically in the world and 8:23you know possibly it killing you or at least harming you so popular in 8:30creatures have this amazing invention which is internal modeling and of course 8:35we are examples of pop aryan creatures but there are plenty of other animals 8:40not again it's not not not a huge proportion it's rather small proportion 8:46fact of all 8:46animals but certainly there are plenty of animals that are capable in some form 8:51of modeling their world and as it were imagining actions before trying the mail 8:57and just to complete then its tower he adds another layer that he calls 9:04gregorian creatures this series naming this layer after richard gregorie the 9:09the the the British psychologist and the thing that gregorian creatures have is 9:18that in addition to internal models they have mind tools like language and 9:24mathematics especially language because it means that gregorian creatures can 9:32share their experiences in fact a group during creature could for instance model 9:38in its in its brain in its mind the possible consequences of doing a 9:46particular thing and then actually pass that knowledge to you so you don't even 9:50have to model yourself so so gregorian creatures really have the kind of social 9:57intelligence that we probably perhaps not uniquely but there are obviously

10:03only a handful of species that are able to communicate you know if you like 10:10traditions with each other so so I think internal models are really really 10:19interesting and has a sale been spending the last couple of years thinking about 10:23robots with internal models and actually doing experiments with with robots with 10:30internal model so are robots with internal model self-aware well probably 10:36not in the sense that you know the everyday sense that we mean by 10:40self-aware sentence but certainly internal models I think can provide a 10:45minimal level of kind of functional self-awareness and absolutely enough to 10:50allow us to ask what if questions so with internal models we have potentially 10:57a really powerful technique 10:58for robots because it means that they can actually ask themselves questions 11:03about what if I take this or that next possible action so there's the action 11:09selection if you like so so really you know I'm kind of following tenets model 11:16I'm really interested in building pop aryan creatures actually interested in 11:22building gregorian creatures but that's another if you like another step in the 11:27in the story so really here I'm focusing primarily on pop aryan creatures so 11:33robots with internal models and so and what i'm talking about in particular is 11:41a robot with a simulation of itself and it's currently perceived environment and 11:48other actors inside itself so it takes a bit of getting your head around the idea 11:53of a robot with the simulation of itself inside itself but that's really what I'm 11:58talking about 12:00and the famous the late John Holland for instance rather perceptively wrote an 12:08internal model allows the system to look ahead to the future consequences of 12:13actions without committing itself to those actions I don't know whether 12:16Holland John Holland was aware of donuts tower possibly not but but really saying 12:23the same kind of things down dented now before I give you before I come onto the 12:29work that I've been doing I want to show you some examples of a few examples 12:34there aren't many in fact of robots with with self simulation the the first one 12:42as far as I'm aware was by Richard Vaughan and his team and he used a 12:49simulation insider robots to allow to plan a safe route with incomplete 12:55knowledge so as far as I am aware this is the world's first example of robots 13:01with self simulation perhaps an example that you might already be familiar with 13:09this is Josh Bongard 13:11and hot lips and work very notable very interesting work here so simulation but 13:19for a different purpose so this is not self simulation to choose as it were 13:24gross actions in the world but instead self-stimulation to learn how to control 13:28your own body so that the idea here is that if you have a complex body then a 13:35self simulations a really good way of figuring out how to control yourself 13:39including how to prepare yourself if parts of you should should break or fail 13:44or be damaged for instance so that's a really interesting example of what you 13:52can do with self simulation and a similar idea really was was tested by my 14:00old friend are in Holland 14:02he built this kind of scary looking robot initially was called Chronos but 14:08then it became known as H a robot and this robot is deliberately designed to 14:16be hard to control in fact he refers to it as an 30 min anthropometric which 14:24means I'm tropic from the inside out so most humanoid robots are only you know 14:29human eye on the outside but here we have a robot that has a skeletal 14:34structure it has tendons it it's it's very and you can see from the little 14:39movie clip there if any part of the robot moos then the whole of the rest of 14:45the mobile robot as it were tends to to to flex rather like you know human 14:51bodies or animal bodies so i win was particularly interested in a robot that 14:59is difficult to control and the idea then of using an internal simulation of

15:05yourself in order to to be able to control yourself will learn to control 15:09yourself and he was the first to come up with this this phrase functional 15:16imagination really interesting work so do check that out 15:21and the final example I want to give you is from my own lab where this is swarm 15:30robotics work where we've in fact we were doing evolutionary swarm robotics 15:36here and we put a simulation of each robot and the swarm inside each robot 15:45and in fact we using those internal simulations as part of a genetic 15:51algorithm so each robot in fact is evolving its own controller and in fact 15:56it it it actually updates its own control about once a second so it's it's 16:01again it's a bit odd thing to get your head round so about once a second each 16:07robot becomes its own great great great great grandchild in other words its 16:12controller is a descendent but the problem with with with this is that the 16:21internal simulation tends to be wrong and we have what we call the reality gap 16:26so the gap between the simulation and the real world and so we got round that 16:31my student Paul o'dowd came up with the idea that we could Co involved the 16:35simulators as well as the controllers in the robot so so you have a population of 16:40robots inside each individual physical robot as it was simulated robots but 16:47then you also have a swarm of of 10 robots and therefore we have a 16:51population of ten simulators so so we we actually Co evolved here the simulators 16:57and the the robot controllers so I want to know show you that the new work I've 17:05been doing on robots with internal models and primarily I was telling you 17:14that you know I'm kind of old fashioned electronics engineer spent much of my 17:20career building safety system safety critical systems so safety is something 17:25that's very important to me and to robotics so here's a kind of generic 17:29internal modeling architecture for safety 17:33so the this is in fact then it's loop of generating test so the idea is that we 17:40have an internal model which is a self simulation is initialized to match the 17:46current real-world and then you try out you you run the simulator for each of 17:53your next possible actions I mean 22222 put it very simply imagine that that 17:59your robot and you could either turn left turn right go straight ahead or 18:04stand still so you have for possible next actions and therefore you'd loop 18:09through this internal model for each of those next possible actions and then 18:14moderate the action selection mechanism in your controller so this is not part 18:20of the control it's a kind of moderator if you like so you could imagine that 18:26the the regular robot control of the thing in red has a set of 4 next 18:32possible actions but your your internal model determines that the only two of 18:39them are safe 18:41so it would effectively if you like moderate or govern the the rope the 18:46action selection mechanism of the robots controller so that the robot controller 18:51impact will not choose the unsafe actions interestingly if you have a 19:01learning controller then that's fine because we can effectively extend or 19:08copy the the learned behaviors into the internal model that's that's fine so in 19:15principle we haven't done this but we're starting to do it now 19:18in principle we can extend this architecture to as it were to adaptive 19:24learning robots 19:27so i mean here's a simple thought experiment imagine a robot with several 19:32safety hazards facing it it has for next possible actions 19:38well your internal model can as it were 19:45figure out what the consequence of each of those actions might be so so two of 19:52them so either turn right or stay still are safe action so that's a very simple

20:00thought experiment and and here's a slightly more complicated experiments 20:07thought experiment so imagine that the robot there's another actor in the 20:11environment so human to human is is not looking where they goin practice walking 20:15down the street peering at a smartphone that never happens does it of course and 20:20I'm about to walk into a hole in the pavement well of course if it were you 20:28noticing that that human about to walk into a hole in the pavement you would 20:33almost certainly intervene of course and it's not just because you're a good 20:36person it's because you have the company to machinery to predict the consequences 20:41of both your and their actions and you can figure out that if you were to rush 20:46over towards them you might be able to prevent them from falling into the hole 20:50so is the same kind of idea but with the robot imagine it's not you but a robot 20:56and imagine now that you are modeling the consequences of yours and the humans 21:04actions for each one of your next possible actions and you can see that 21:09now this time we've given the kind of numerical scale so 0 is perfectly safe 21:15whereas ten is seriously dangerous you know kind of danger of death if you like 21:21and you can see that the safest outcome is if the robot turns right in other 21:30words the safest for the human i mean clearly the safest for the robot is 21:34either turn left or stay still 21:37but in both cases the human would 21:40would fall into the hole so you can see that we could actually invent a rule 21:44which would represent you know the as it were the the best outcome for the human 21:50and this is what it looks like so if all robot actions the human is equally safe 21:56then that means that we don't need to worry about the human so we'll just 22:01output the internal model will output the the safest actions for the robot 22:08else then output the actions the robot actions for the least unsafe human 22:15outcomes now remarkably and we didn't intend this this actually is an 22:22implementation of Asimov's first law of robotics so a robot may not injure a 22:29human being all through inaction that's important they all through inaction 22:33allow a human being to to come to harm so we kind of ended up if you liked 22:38building and as a movie and robot simple as a movie and ethical robot so what 22:47does it look like when we started we know extended to humanoid robots but we 22:52started with the epoch robots these little kind of their about the size of a 22:58of a salt shaker I guess about seven centimetres tall and this is the the 23:06little arena in the lab and what we actually have inside the the ethical 23:13robot is is this is the as it were the internal architecture so so you can see 23:20that we have the robot controller which is in fact a mirror of the real robot 23:25controller model of the robot and the model of the world which includes you 23:30know others in the world so this is the simulated this is this is a more or less 23:35irregular robot simulator so you probably know that robot simulators are 23:41quite commonplace you know we robotics is used them all the time to to test 23:46robots but you know in as it were in the virtual world before then trying out the 23:50code for real but what we've done here is we've actually put 23:53as it happens an off the shelf simulator inside the robot and made it work in 24:00real time so the output of the of the simulator for each of those next 24:05possible actions is evaluated and then goes through a little logic layer which 24:11is essentially that the rule the FN else rule but I showed you a couple of slides 24:16ago and that effectively determines or or moderates the action selection 24:22mechanism of the real robot so so this is the simulation budget so we're 24:31actually using the the open-source simulator staged a well-known simulator 24:38and in fact we managed to get stage to run about six hundred times realtime 24:42which means that we're actually cycling through our internal model twice a 24:48second and for each one of those cycles were actually modeling not for but 30 24:56next possible actions and remodeling about 10 seconds into the future so

25:02every half a second robot with an internal model is looking ahead ten 25:11seconds for about 30 next possible actions 30 of its own next possible 25:16actions but of course it's also modeling the consequences of each of the other 25:22actors dynamic actors in its environment so you know this is quite nice to 25:27actually do this in real time and let me show you some of the results that we got 25:32from that so ignore the kind of football pitch so what we have here is the 25:39ethical robot which we call the a robot after Asamoah and we have a hole in the 25:45ground it's not a real whole its virtual hole in the ground we don't have to be 25:49digging holes into the lab floor and using another EP work as a proxy human 25:56we call this the H robot 25:59so let me show you what happened when we ran its first of all with with no H 26:07robot at all was a kind of baseline if you like and you can see on the left in 26:1226 runs those of the traces of the a robot so you can see the a robot in fact 26:18is maintaining its own safety it's it's avoiding skirting around the edge almost 26:24optimally skirting the the edge of the hole in the ground but then when we 26:29introduce the eighth robot you get this wonderful behavior here where as soon as 26:34the a robot notices that the H robot is heading towards the whole which is about 26:39here then it deflects it diverts from its original course and in fact more or 26:48less collided are actually physically collide because they have low-level 26:52collision avoidance so they don't actually collide but nevertheless the a 26:56robot effectively heads off the edge robot that then bounces off and safely 27:01as it were 27:02goes off in another direction and the a robot then resumes its course to its 27:07target position so which is really nice and you know interestingly even though a 27:15simulator is rather low fidelity it doesn't matter you know surprisingly 27:20doesn't matter because the closer the a robot to the eighth robot gets then the 27:25better its predictions about colliding so this is this is why even with the 27:31rather low fidelity simulator we can collide with really good precision with 27:37the H robot so let me show you the movies of this trial with a single proxy 27:47human and I think the movie starts in so this is real time and you can see the 27:56the a robot nicely heading off the age robot which then disappears off towards 28:03the left I think then we we've speeded up four times and this is a whole load 28:13of runs so you can see that it really does work and also noticed that every 28:19experiment is a bit different and of course that's what typically happens 28:22when you have real physical robots because they simply because of the noise 28:26in the system the fact that these are real robots with imperfect motors and 28:31sensors and what have you 28:32so we we wrote the paper and were about to submit the paper when you know we 28:43kind of thought well this is a bit boring isn't it you know how we built 28:46this robot and it works so we have the idea to put a second human in the 28:55sorry I forgotten one slight so before I get to that I just wanted to show you a 29:03little animation of these little filaments here are the traces of the the 29:11a robot and its prediction of what might happen so at the point where this turns 29:18red the a robot then starts to intercept and each one of those little traces is 29:24its prediction of the consequences of both itself and the and the eighth robot 29:30this is really nice because you can kind of look into the if you if you like 29:34look into the mind to put it that way of the robot and actually see what it's 29:41doing which is very nice very cool but I was about to say we tried the same 29:50experiment in fact identical code with 28 robots and this is the robots dilemma 29:58this may be the first time that a real physical robot has faced an ethical

30:03dilemma so you can see the 28 robots are more less equi distant from the whole 30:09and the there is the a robot which in fact fails to save either of them 30:18so what's going on there we know that it can save one of them all of every time 30:26but in fact it's just failed to save either and it does actually save one of 30:33them has a look at the other one but it's too late so this is really very 30:40interesting and not at all what we expected 30:50in fact let me show the statistics so in 33 runs the the ethical robot failed to 31:05save either of the eight robots just under half the time so about fourteen 31:12times it failed to save either it saved one of them just over 15 prop sixteen 31:18times an amazingly saved both of them twice which is quite surprising it 31:25really should perform better than that so you know and in fact we when we 31:32started to really look at this we we discovered that the so his a 31:36particularly good example of dithering so we realized that we made a sort of 31:44pathologically indecisive ethical robot so I'm gonna save this one owner know 31:48that one under this one and of course by the time 31:53ethical robot has changed its mind three or four times well it's too late so this 31:58is the problem the problem fundamentally is that ethical robot doesn't make a 32:05decision and stick to it in fact it's a consequence of the fact that we are 32:12running a consequence engine as I mentioned twice a second so every half a 32:17second are ethical robot has the opportunity to change its mind that's 32:22clearly a bad strategy but but nevertheless it was an interesting kind 32:26of unexpected consequence of of the experiment we've now transferred the 32:33work to these humanoid robots and we get the same thing so here there are two of 32:39the two red robots both heading toward endanger the blue one the ethical robot 32:44changes its mind and goes and saves the one on the left even though it could 32:49have saved the one on the right so another example of how did the ring 32:56ethical robot 32:59and as I just kind of hinted at the reason that our ethical robot is so 33:05indecisive is because it's essentially a memoryless architecture so you could you 33:11could say that the robot has a you know I can borrow and Hollins description it 33:17has a functional imagination but it has no autobiographical memory so it it 33:22doesn't remember the decision it made half a second ago which is clearly not a 33:26good strategy you know really unethical robot just like you if you were acting 33:33in a similar situation it's probably a good idea to stay for you to stick to 33:39the first decision that you made but probably not forever so you know I think 33:44the decisions probably need to be sticky somehow so decisions like this may need 33:49a half-life you know sticky but not but not absolutely rigid so so but you know 33:56actually at this point we decided that we not gonna worry too much about this 34:00problem because in a sense this is more of a problem for ethicists than 34:04engineers perhaps I don't know but maybe we could talk about that I really before 34:11finishing I want to show you another experiment we did with the same 34:16architecture exactly the same architecture and this is what we call 34:20the corridor experiment so here we have a robot with this internal model and it 34:28has to get from the left and to the right hand of a crowded corridor without 34:34bumping into any of the other robots that are in the same corridor so imagine 34:39you're walking down a corridor an airport and everybody else is coming in 34:43the opposite direction and you want to try and get to the other end of the 34:47corridor without crashing into any of them but in fact you have a rather large 34:51BodySpace you you know you don't want to get even close to any of them so you you 34:56know you want to maintain your as it were private BodySpace and what the blue

35:04robot here is doing is in fact modeling the consequences of its actions and the 35:11other ones within this 35:12radius of attention so this blue circle is a radius of attention so here we were 35:18looking at if you like a simple attention mechanism which is only worry 35:22about the other dynamic actors within your radius of attention in fact we 35:28don't even worry about the ones that are behind us it's only the ones that more 35:31or less in front of us and you can see that robot does eventually make it to 35:39the end of the corridor but with lots of stops and back tracks in order to 35:45prevent it from because it's really frightened of of of any kind of contact 35:49with the other robots and and and he will not showing all of the the the sort 35:59of filaments of of prediction only the ones that are chosen so here are some 36:09results which interestingly shoulders so we probably the best one to look at look 36:16at is this danger ratio and simply means robots with no internal model at all and 36:27intelligent means robots with internal model so so here the danger ratio is the 36:34is the number of times that you actually come close to another robot and of 36:39course it's very high this is Zima simulated and real robots very good 36:44correlation between the real and simulated with with the intelligent 36:49robot the robot with the internal model we get a really very much safer 36:53performance clearly there is some costs in the sense that for instance the the 37:03intelligent robot runs with internal models tend to cover more ground but 37:07surprisingly not that much further distance it's less than you'd expect and 37:12truly there's a computational cost because the computational costs of 37:16simulating clearly 04 the damn robots where it's as it's quite high for the 37:22the intelligent robot the robot with internal models but 37:25but again you know computation is relatively free these days so actually 37:30we're trading safety for computation which i think is a good a good tradeoff 37:34so sorry I want to conclude there you know I've not of course talked about all 37:43aspects of robot intelligence that would be a three hour seminar and even then I 37:47wouldn't be able to cover it all but what I hope I've shown you in the last 37:53few minutes is that with internal models we have a very powerful generic 37:59architecture which we could call a functional imagination and you know this 38:06is where I'm being a little bit speculative perhaps this moves in the 38:09direction of artificial theory of mind perhaps even self-awareness I'm not 38:14going to use the word machine consciousness well I just have but 38:17that's a very much more difficult goal I think and and I think there is practical 38:25value I think there's real practical value in robotics of robots with self 38:31and other simulation because as I i think i hope i demonstrated least in a 38:38kind of prototype sense proof of concept that such simulations moves towards 38:43safer and possibly ethical systems in unpredictable environments with other 38:51dynamic actors so thank you very much indeed for listening obviously be 38:55delighted to to take any questions thank you 39:03thank you very much for this very fascinating view on robotics today we 39:10have time for questions please wait until you go to microphones we have the 39:14answer also the video game playing computers or perhaps more accurately 39:25would be saying game-playing algorithms predated the examples he listed as a 39:32computer issue with the internal models still you didn't mention those these 39:37days to clear reason why did I guess I should mention that you're quite right i 39:45mean the what I'm thinking of here is is particularly robots with with explicit 39:50simulations of the of themselves and the world so I was limiting my examples to 39:56simulations of themselves in the world i mean you're quite right but of course

40:00you know game playing algorithms need to have a simulation of the game and quite 40:05likely of the certainly of of the possible moves of the the the opponent 40:12as well as you know the as it were the the game playing AI's 40:17moves so you're quite right i mean it's two different kind of simulation but but 40:21but I should include that you're right there in your simulation you had the 40:30page 40:32Robert we have one goal and the a Robert different goal and they interacted with 40:38each other 40:39healthy true because what happens when they have the same goal the same goal 40:46the same spot for example I don't know the if it depends on whether that spot 40:56is is a safe spot or not I mean if it's a safe spot then they'll both go toward 41:04it they'll both reached reach it but but without crashing into each other because 41:08the the a robot will will make sure that it avoids the the eighth robot in fact 41:14that's more or less what's happe 41:15in the corridor experiment that's right yeah but good question we should try 41:21that 41:34the simulation that he did for corridor experiment right the actual real-world 41:37experiments the simulation on track now the robot's movements as well meaning 41:43what information dissemination have that it began with which is what it perceived 41:48as I mean the other about their moving and in the real world they may not move 41:51as you predict them to be deplorable actually know each step where the other 41:58robots were sure sure that's a very good question I mean we in fact we cheated it 42:03cheated in the sense that we used for the real robot experiments we used a 42:08tracking system which means that the essentially the the robot with an 42:15internal model has has access to the position is like a GPS internal GPS 42:21system so but in a way that's really just a kind of you know it's it's kind 42:30of cheating but but even a robot with a vision system would be able to track all 42:36the robots in its field of vision and and as for the second part of your 42:41question 42:42kind of model of what the other robot would do is very simple which is its 42:48kind of ballistic model which is if a robot is moving in a particular speed in 42:53a particular direction then we assume it will continue to do so until it 42:57encounters and an obstacle so so very simple if you like ballistic model which 43:06you know even for humans is useful for very simple behaviors like you know 43:12moving in a crowded space 43:17high in the same experiment it's a continuation of the previous question so 43:27in between some of the rewards how to change their direction randomly I guess 43:34so does the morale of people over constituent not explicitly but it but it 43:41it it does in the sense that because it's already initializing its internal 43:48model model every half a second then if the positions and directions of the of 43:53the actors in its environment changed then they will reflect the new positions 43:59so what exactly the positions but as you said you have considered the balanced 44:05equation of the objects so if there is any randomness in the environment so 44:11does the internal model of the blue Robert instead of randomness and change 44:16the view of the frederick Motz it's like it used a robot as a symbolic motion so 44:22that it changed its view of the red robot that read robert's more in the 44:27ballistic motion 44:29well that's a very good question I i I think the answer is no I think we're 44:35probably assuming more less deterministic model of the world 44:42deterministic yes I think I think pretty much deterministic but we're relying on 44:47the fact that we are dating and re-running the model Rhea nationalizing 44:53rerun running the model every half a second to if you like track the 44:57stochasticity which which is inevitable in in the real world we probably do need

45:04to introduce that seems to customers steamed into the internal model yes but 45:10but not yet 45:12thank you but good very good question we have two kitchens with this technology 45:20like driverless cars for example I think it becomes a lot more important how you 45:26program the robots ethics so they could I let alone is like you know if there's 45:32the robot has a choice between saving a school bus full of kids vs one driver 45:37that that logically to be programmed and you made a distinction between being an 45:44engineer yourself and then had been at assist earlier so to what extent the 45:51engineer responsible in that case and also those those does a project like 45:56this in your life who is required and assist how do you see this field in real 46:01life applications involving sure that that's that's a really great question I 46:06mean you're right that driverless cars 46:09well it's it's debatable whether they will have to make such decisions but but 46:16many people think they will have to make such decisions which are kind of 46:20driverless car equivalent of the trolley problem which is a well-known kind of 46:24ethical dilemma thought experiments now my view is that the rules will need to 46:35be decided not by the engineer's but but if you like by the whole of society so 46:41ultimately the rules that decide how the 46:46the driverless car should behave under good and you know these difficult 46:51circumstances impossibly facts circumstances even and even if we should 46:57in fact program those rules into the car so so you know some people argue that 47:02the tribal skull should not attempt to as it would make a rule driven decision 47:10but just but just leave it to chance and again that I think that's an open 47:14question but this is really why I think the cut this dialogue you know and and 47:19debate 47:20and conversations with with regulators lawyers ethicists and the general public 47:29you know users of driverless cars i think is why we need to have this debate 47:33because whatever those rules are and even whether we have them or not is 47:38something that that should be decided as it were collectively i mean someone 47:45asked me last week should you be able to alter the ethics of your own driverless 47:51car my answer is absolutely not no I mean that should be illegal so I think 47:56that if driverless cars were to have a set of rules and especially those rules 48:01had numbers associated with them you know let's think about the less emotive 48:07example imagine a driverless car and an animal runs into the road while the 48:15driverless car can either ignore the animal and definitely kill the animal or 48:23it could try and break possibly causing harm to the driver of the passengers but 48:31effectively reducing the probability of killing the animal so there's an example 48:36where you have some numbers you know 22222 tweet if you like parameters so 48:42these you know if these rules are built into driverless cars they'll be 48:46parametrized and I think it should be absolutely illegal to have those 48:53parameters to change them you know in the same way that silly probably illegal 48:59right now to attack an aircraft autopilot I suspect that probably is 49:05illegal 49:06if it isn't it should be so so I think that you know you don't need to go far 49:11down this this kind of line of argument before realizing that that regulation 49:16and legislation you know has has to come into into plain fact I saw peace in this 49:23just this morning I think on it wired that I think in the us- 49:30regulation for driverless cars is now on the table which is absolutely right i 49:36mean we you know we believe we need to have regulatory frameworks or what I 49:40called governance frameworks for four driverless cars and in fact lots of 49:45other autonomous systems not just driverless cars but great question thank 49:48you to experiment with the corridor you assume even in the other experiments you 49:56always assume that the main actor Ethan most intelligent all the others are not

50:00ideal done more like their political models William of those have you tried 50:04doing a similar experiment in which still like each each actor is 50:11intelligent but assumes that the others are not actually everyone is intelligent 50:14to like everyone in a blue dot the experiment with the with the model you 50:18have a note when you consider changing the model that assumes that the others 50:22have the same model that particular actor has as well which are you know 50:27we're doing it right now so that we're doing that experiment right now and you 50:33know if you ask me back in and year perhaps I could tell you what have i 50:36mean it it's it's really mad because it's you know but it does take us down 50:41this direction of of theory of my artificial theory mind so so you know if 50:46you have several robots are actors each of which is modeling the behavior of the 50:51other then you know you you you get i mean some of the I don't even I don't 51:00even have a movie to show you but but in simulation we've we've we've tried this 51:04way we have two robots which are kind of like imagined you you know this happens 51:11to all of us you're walking down the pavement and you and you 51:14and you do the sort of side step dance you know with someone who's coming 51:19towards you and so the research question that we're asking ourselves is do we get 51:22the same thing and it seems to be that we do so if if the robots are 51:28symmetrical in other words there they each modeling the other then we can get 51:33these kind of little interesting you know dances where each is trying to get 51:39out of the way of the other but but in fact choosing in a sense the opposite so 51:43one chooses that right 51:45the other chooses step left and they're still they still can't go past each 51:48other but it's it's hugely interesting yes hugely interesting 51:57yeah I think it's really interesting how you point out the importance of 52:02simulations and internal model I feel that one thing that is slightly left out 52:08there is a huge gap from going from simulation to real-world robots for 52:13example and I assume that in these simulations you kind of assume that the 52:18sensors are a hundred percent reliable and that's obviously not the case and 52:23reality and especially for talking about autonomous cars or robots and safety how 52:29do you calculate uncertainty that comes with these sensors in the equation sure 52:34no this is deeply interesting question and the short answer is I don't know I 52:40mean this is this is all future work we I mean my instinct is that it is that a 52:47robot with a sense with the simulation internal simulation even if that 52:52simulation innocence is ideal idealized is still probably going to be safer than 52:59a robot that has no internal simulation at all and and you know I think we 53:07humans have multiple simulations running all the time so I think we have sort of 53:12quick and dirty is kind of low fidelity simulations when we need to move fast 53:17but clearly you know when you need to to plan something plans some complicated 53:24action then you know like like where you going to go on holiday next year you 53:29don't use this clearly don't use the same internal model same simulation as 53:34for when you try and and and stop someone from running into the road so I 53:39think that future intelligent robots will need also to have multiple 53:44simulators and also strategies for choosing which which fidelity simulating 53:51to use at a particular time and and if a particular situation requires that you 53:59know you need high fidelity then then for instance one the things that you can 54:03do which actually I think humans probably do is that you simply move more 54:07slowly to give your so self time or even stop to give yourself time to figure out 54:13what's going on and and and innocence plan your strategy so so i think you 54:20know even with with computational power that the computation power we have they 54:26will still be a limited simulation budget and I suspect that that 54:30simulation budget will mean that in real time when you doing this real-time you 54:35probably can't run your your highest fidelity simulator and taking into 54:41account all of those you know probabilistic you know nor easy noisy 54:46sensors and actuators and so on 54:49you probably can't run that simulate all the time so you know I think we're going 54:54to have to have a nuanced approach where we have multiple simulators with with 54:59multiple fidelity's or maybe a sort of tuning where you can choose the fidelity 55:04of your simulator so this is kind of a new area of research I don't know 55:09anybody who's thinking about this yet apart from ourselves so great it is 55:17pretty hard yes I think yeah 55:23sorry this particular situation where they are to assume all robots and that 55:32would be an extension of the question that he has so for example if they are 55:38two guys were walking on the pavement and they could be a possibility of 55:42mutual cooperation as am I gonna get whether that I might state step out of 55:47this place and you might go and then I'll go after that so they are to 55:50assemble robots will there be a possible can have you considered this fact that 55:54both will communicate with each other and they will eventually come to a 55:58conclusion that I would probably work and other words they get out of the way

56:03and the second part of the solution would be one of the report's actually I 56:09mean has not agreed to cooperate I mean since they both would have different 56:14simulators and they could have different simulators and one might actually try to 56:17communicate that used about the way so that I mean I might go over each other 56:23as an agreement that I mean what would consider this 56:27yes good question in fact we've we've we've actually gotten a new paper which 56:31were just writing right now and that the sort of working title is the dark side 56:37of robotics what should I am sorry no the dark side of ethical robots and and 56:43you know one of the things that we discovered it's actually not surprising 56:47is that you only need to change one line of code 224 a cooperative robot to 56:55become a competitive robot or even an aggressive robot so that that you know 57:01it's fairly obvious when you when you start to think about it if if your 57:06ethical rules are very simply written and are kind of layer if you like it on 57:11top of the the rest of the architecture then it's not that difficult to change 57:15those rules which and and yes we've done some experiments and I again I don't 57:21have any videos to show you but they're pretty interesting 57:25you know the showing where how easy it is to make you feel like a competitive 57:31robot or even an aggressive robot using this approach in fact on the on the BBC 57:386 months ago so I was asked you know surely if you can make an ethical robot 57:43doesn't that mean you can make an unethical robot and the answer I'm 57:47afraid is yes it does mean that but this really goes back to your question 57:55earlier which is that it should be you know we should make sure it's illegal to 58:00to convert turn if you like to to to recode and ethical robots in an ethical 58:05robot or even it should be illegal to to make unethical robots something like 58:10that but it's a great question and short answer yes and yes we have some 58:17interesting new results 58:18newspaper as it were 58:22unethical robots alright we are running out of time thanks everyone for coming 58:28today thanks Professor oilfield thank you