I am Elizabeth Pilgrim, and I'm really not sure if this is the right audience for this, but I have a confession(忏悔) to make. I really don't trust(信任) computers. I don't trust the computers in my bank. I don't trust the computers in my car. I don't trust the ones I use in my job. And I also don't trust the ones in my health care.
And this is all starting to sound a little bit tinfoil hat. So let me tell( 告诉) you some stories to justify(证明合法) this. This blurry photo is Therac 25. It was a machine that delivered radiation(发散) therapy(治疗) to cancer(癌症) patients in the 1980s. Due to a software bug(小虫) called a race condition, where two threads(线) of software race to access the same bit of memory at the same time, this machine delivered over a hundred times the dose(剂量) it was supposed to to some of the patients. Three people died.
And if we fast forward to the 1990s, this is the Ariana 5 rocket(火箭) launch(发射). It is widely(大大地) considered to be one of the most expensive software failures(失败) ever. So there was a variable([数] 变量) that stored information about the orientation(方向) of the rocket(火箭), and this was stored in a data type that was too small for it. So we got what's called an integer overflow(溢出), where the number got too big for the data type. It overflowed, the software crashed(碰撞), the rocket veered drastically(激烈地) off course, and it exploded. That was pretty expensive.
And so some of you weren't even born(承担) in the '90s. So you're probably thinking, oh, but this is all ancient history, isn't it? Well, okay(好), so I hope you've been looking at the news recently. You've probably seen the Horizon(地平线) Post Office scandal(丑行). I'm not going to pretend I know what the bug is here, because I don't think that's public. But the upshot is the accounting software issued(发行) by the Post Office widely(大大地) reported completely incorrect(不正确的) totals.
Postmasters(邮局局长) were falsely(虚伪地) accused(控告) of stealing from the Post Office and falsely(虚伪地) imprisoned. This absolutely devastated(毁坏) lives. And this is just a small sample of the software disasters I could have told you about. I could have told you about how Delta(三角州) Airlines software bug(小虫) grounded all of their planes worldwide. I could have told you about the Lime(石灰) Scooter(小轮摩托车) software bug(小虫), that means as you're riding, it will just randomly(随机地) decide to break and throw you off onto the floor. And who knows what the next news story is going to be that we're all going to see?
So the question is why are we so bad at computers? Well, firstly, software is getting really, really big. It's getting really difficult for humans to reason about systems this big, even if they were the ones that wrote it. And this system, this problem is getting much, much worse. Because now we're letting AI write its own code. And this article boldly claims that's a good thing.
I really don't think it is. The robots look cute(可爱的), but they are demonstrated(证明) to introduce security vulnerabilities(弱点) and they are reliant on software engineers checking their output. And we've already shown that humans are bad at that. We couldn't get this right when we were manually programming 8-bit microcontrollers. How do we expect to get this right now? What I'm saying is join me as leader of the Luddites.
Let's smash(粉碎) up all of the computers. Let's get rid(使摆脱) of the tech companies. Let's live a life that is free of technology and it will be glorious(光荣的). But I didn't tell you what my job is. So I'm a lecturer(讲师) in computer science for the University of Edinburgh. I also have a research fellowship(团体) from the Royal Academy(学院) of Engineering.
And none of this is very compatible(谐调的) with being leader of the Luddites. So, instead, I'm going to tell you about how Sherlock Holmes has helped me work through some of my software trust issues. Why Sherlock? He's tall, he's good looking, he's got dubious(可疑的) standards of hygiene(卫生), but most importantly, he has impeccable(无瑕疵的) logical(合逻辑的) reasoning(推理) skills. When Sherlock first met Watson in the BBC TV series, Sherlock invites Watson to share a flat with him. And Watson says, "But we don't know anything about each other."
And Sherlock says, "I know that you're an army doctor who's been invalid(无效的) at home from Afghanistan(阿富汗(西南亚国家)). I know that you've got a brother who cares about you, but you won't go to him for help because you don't approve(赞成) of him, possibly because he's an alcoholic, more likely because he's just walked out on his wife. And I think that's enough to be going on with, don't you?" And so, Watson is surprised.