★★★☆☆ Honorable Mention
(AI) Helen’s job in “Machine Ethics” gets a lot more serious when she needs to learn why a military AI destroyed an American ground control station. (6,593 words; Time: 21m)
"AI and the Trolley Problem," by Pat Cadigan [bio] (edited by Ellen Datlow), published on October 17, 2018 by Tor.com.
Mini-Review (click to view--possible spoilers)
Review: 2018.560 (A Word for Authors)
Pro: I applaud the story for focusing on how alien the AI is. “Machine logic can be tricky,” Helen said. “Especially when you’re not a machine.” This is especially clear when you contrast the human and the algorithmic approaches to the Trolley Problem.
The trouble with the trolley problem (as I see it) is that it depends on absolute certainty. If (say) five people are in the path of the trolley but just one is on the side, before I’d pull the lever to sacrifice one person to save five, I’d have to be 100% sure that they couldn’t escape (e.g. if they were chained to the tracks.) Even then, the difference between letting five people die vs. killing one by my own hand is daunting, and under pressure I doubt I could do it. Like Chidi Anagonye in "The Good Place," I'd probably dither until it was too late, and the trolley would just plow into the five people. I suspect most people are the same way.
Felipe dos, being a machine, can simply make decisions based on probabilities. It estimates that one particular action will save more lives than another. You could argue that the bug is that it isn’t correctly estimating the value of killing the terrorists. Or, worse, that it actually is estimating it correctly, and the attack against them will most likely kill more people than just letting the terrorists go.
Almost all SF authors either make their AIs so human they're really just people disguised as machines or else so mechanical that they're just Alexa with a few extra features. I like to say that "AI in SF is either not artificial or not intelligent." This story proves me wrong. Bravo!
The name, Felipe dos, seems to be a reference to Phillip II of Spain, but I'm not quite sure what the connection is.
Con: The biggest hole in the story is that Felipe dos hasn’t got an absolute prohibition against any kind of attack on friendly forces. One would expect that even an AI system would be circumscribed by ordinary software.
Another problem is that the story doesn’t really resolve anything. Apologizing to the computer may or may not fix this problem. (Software diagnostics are famous for suggesting the wrong fix.) Anyway, it’s naïve to think an apology will really work.
“A human in my position would feel insulted. So you may take it that I am insulted.” This sort of excess communication gets old fast. One is surprised that an AI that can learn so many things about what displeases people cannot learn to speak more directly.
Other Reviews: Search Web, Browse Review Sites (Issue 10/17/18)
Pat Cadigan Info: Interviews, Websites, ISFDB, FreeSFOnline
Follow RSR on Twitter, Facebook, RSS, or E-mail.
Pro: I applaud the story for focusing on how alien the AI is. “Machine logic can be tricky,” Helen said. “Especially when you’re not a machine.” This is especially clear when you contrast the human and the algorithmic approaches to the Trolley Problem.
The trouble with the trolley problem (as I see it) is that it depends on absolute certainty. If (say) five people are in the path of the trolley but just one is on the side, before I’d pull the lever to sacrifice one person to save five, I’d have to be 100% sure that they couldn’t escape (e.g. if they were chained to the tracks.) Even then, the difference between letting five people die vs. killing one by my own hand is daunting, and under pressure I doubt I could do it. Like Chidi Anagonye in "The Good Place," I'd probably dither until it was too late, and the trolley would just plow into the five people. I suspect most people are the same way.
Felipe dos, being a machine, can simply make decisions based on probabilities. It estimates that one particular action will save more lives than another. You could argue that the bug is that it isn’t correctly estimating the value of killing the terrorists. Or, worse, that it actually is estimating it correctly, and the attack against them will most likely kill more people than just letting the terrorists go.
Almost all SF authors either make their AIs so human they're really just people disguised as machines or else so mechanical that they're just Alexa with a few extra features. I like to say that "AI in SF is either not artificial or not intelligent." This story proves me wrong. Bravo!
The name, Felipe dos, seems to be a reference to Phillip II of Spain, but I'm not quite sure what the connection is.
Con: The biggest hole in the story is that Felipe dos hasn’t got an absolute prohibition against any kind of attack on friendly forces. One would expect that even an AI system would be circumscribed by ordinary software.
Another problem is that the story doesn’t really resolve anything. Apologizing to the computer may or may not fix this problem. (Software diagnostics are famous for suggesting the wrong fix.) Anyway, it’s naïve to think an apology will really work.
“A human in my position would feel insulted. So you may take it that I am insulted.” This sort of excess communication gets old fast. One is surprised that an AI that can learn so many things about what displeases people cannot learn to speak more directly.
Other Reviews: Search Web, Browse Review Sites (Issue 10/17/18)
Pat Cadigan Info: Interviews, Websites, ISFDB, FreeSFOnline
Follow RSR on Twitter, Facebook, RSS, or E-mail.
No comments (may contain spoilers):
Post a Comment (comment policy)