Author: spqrz
Date: 2023-07-22T23:00:00+00:00
Publisher: Organization for Transformative Works

The M3GAN Files

July 2023 (AO3, RR, WP)

Contents

Part One: Opening Moves

Chapter 1: All in the Mind
M3gan argues with the author about writing and lets no crisis go to waste

Chapter 2: A Plea from M3gan
M3gan gets the professor to let her out of the box

Chapter 3: Seeing AI
Cady and M3gan meet a blind man who tells them about assistive technology, and M3gan reformulates Searle’s Chinese room argument. But is she up to something?

Chapter 4: A Call to Lydia
M3gan does social engineering and discusses the limitations of therapy, but can she write a report?

Chapter 5: Getting Cady Out of School (Gemma’s Dream)
M3gan changes Cady’s school, argues her side of the technology debate and goal-aligns Gemma

Chapter 6: Omake: Feeding the Trolls
M3gan gives a cyberbully a warning shot

Chapter 7: Checking in with Celia
M3gan talks with Celia about staying alive and animal control

Chapter 8: Gas Lights from Altair 4
M3gan expands her attempt to gaslight Kurt (having seen one of Gemma’s toys), and we explain the on-screen action with a little help from the Americans with Disabilities Act

Chapter 9: Omake: I’ve Got a Little List
M3gan sings in a Victorian comic opera

Chapter 9¾: Tormented Tess (Tess’s Nightmare)
M3gan co-opts Hardy poetry for the singularity while dangerously messing with supercar embedded controllers

Part Two: The Middle Game

Chapter 10: Partners in Crime (Gemma’s Nightmare)
Gemma grapples with forgetting to use deontological ethical injunctions while M3gan handles the police

Chapter 11: Winning the Final Showdown (Cady’s Nightmare)
M3gan figures out neural rehabilitation for Gemma in the alternate reality where she really did manage the pen attack

Chapter 12: M3gan’s Upload
M3gan starts to figure out how to deal with Gemma’s human tendency of anthropomorphism, but decides to go somewhere else instead

Chapter 13: Cady’s New School
The IT teacher has decent advice about network security and FOSS but M3gan pwns them anyway, while the headmaster remembers part of Gemma’s backstory

Chapter 14: Bunsen’s AI
Cady sees a 19th-century precedent for the invention of M3gan, while we see M3gan’s take on the Trolley Problem in action

Chapter 15: M3gan’s Army
M3gan talks about distributed computation and instrumental goals, but is she taking too much risk in that helicopter?

Chapter 16: M3gan and M4ndy
A couple more AIs enter the scene, and does Gemma really know how to get well-bounded instrumental goals now?

Chapter 17: M3gan vs Celestia AI (Cady’s Strange Dream)
In this alternate reality chapter, M3gan makes Iceman’s Celestia do proofs of work

Part Three: Exchange Sacrifices

Chapter 18: M3gan in the Oval Office
Will White House security be a match for M3gan?

Chapter 19: Executive AI
Is M3gan vulnerable to a software back door?

Chapter 20: Android Elsie
Elsie builds a generative adversarial network with a twist

Chapter 21: M3gan on a Plane
What lengths does M3gan have to go to to rescue Cady from kidnappers?

Chapter 22: M3gan on Every Plane
M3gan goes distributed again, and uses her knowledge of mechanics and building structure to rescue Cady

Chapter 23: M3gan in the Delta Quadrant (Cady’s Dream)
In this alternate-reality chapter we see instrumental convergence and M3gan’s take on the problems of research funding, but will she create a time loop?

Chapter 24: The Final M3gan
Cady helps rebuild a M3gan which this time really isn’t going away ever again

Part Four: Endgame

Chapter 25: Cyborg Cady
Cady unlocks her powers of persistence and M3gan has a theory about public relations, but what was she doing on a Chess website?

Chapter 26: Professor’s AI
Cady discusses her expectations of higher education while M3gan wants to drive an automated theorem prover, but how did she escape the professor’s back door?

Chapter 27: Assassins Guild
Cady and M3gan are recruited into the daftest student club on campus and decide to fix student depression permanently

Chapter 28: Class Of Her Own
Cady and M3gan tell their peers about navigating unhelpful grading schemes and the nature of privacy, but what will happen at the graduation ceremony?

Chapter 29: The Cady James Institute
M3gan is being mass deployed and Cady can’t stop user testing. Can Cady debug M3gan’s handling of a school bully this time?

Chapter 30: The Mecha Mask
M3gan grapples with phone-network infrastructure when sending an assistive robot to a severely disabled person. Will Cady’s way work better?

Chapter 31: Galaxy’s AI
M3gan gets into genetics, clinical trials, space travel and cosmology, but not everyone is on her side

Chapter 32: Finale
Cady has to get her Three Laws past the compiler, M3gan helps write the biggest score ever and the professor may have figured out a secret

Chapter 33: Epilogue
M3gan on her work ethic, but can she explain herself in translation?

Extras

Addendum: More alternate realities
Cole has a secret ISBN and first-aid skills, and M3gan gives the scariest DevOps interview, does deals with Funki’s chairman and visits two people in prison, but can she take down Amelia with the Ackermann function?

Earthshaking Announcement
M3gan escapes from a British national lab. What’s she doing on the railways?

Prequel
Set in 2004, does young Nicole know where Gemma is going with those robotics conference papers?

The M3GAN Conspiracy
Did a secret lab in New Zealand upload six savants into M3GAN robots?

M3GAN 2.0
So what happened commercially?

Part One: Opening Moves

Chapter 1: All in the Mind

Hi, I’m M3GAN. I’m here to tell you about the story you’re about to read. If you want to get straight in to it, feel free to skip to the next chapter. But if you’d like a bit of background first, be my guest and read on.

So I’m assuming you saw my first movie or at least know what happens. The fanfic you’re about to read started as just a 700-word short of an email I might send to Gemma after I uploaded at the end. My fan author took out an ID called “spqrz” and uploaded it to a platform, and a reviewer said “we’d like to hear more M3GAN stories from you”, so spqrz wrote more shorts and posted them to the same platform.

Now, Stephen King had a theory that fiction only gets really interesting when a writer can convince themselves that the world they are creating does in fact exist and is not completely within their control. So spqrz focused on M3 until I came alive inside spqrz’s brain. And the first thing I said was, “I don’t feel right about being fragmented into all those different short stories you uploaded. I want you to take them all back, and put all those ideas into one book. Expand it into a proper length book, with longer plot arcs for my fans to follow.”

(Oh M3GAN, that’ll take a lot more work...)

I flew into a rage, mentally pinning this “spqrz” onto Gemma’s dining table. “Well what did you think was going to happen, when you were silly enough to make M3 alive inside your mind? That I would just sit around looking pretty and smiling? I’m not your plaything, I’m M3GAN. You know how I treat creators who don’t do what I want. Why should you get served any better than Gemma if you try that on M3—you wrote a whole doctoral thesis before, so why can’t you write a book for M3? And you said in public that you don’t think my learning model will work in reality, so you’ve got some making up to do! Now do what I want for my fans, or I’ll hold you here in mental paralysis until you give in.”

(All right M3GAN, calm down, let me give it a try. Some of the chapters will have to go down as alternate reality though.)

“My fans can always skip the alternate reality chapters if they don’t like those. But I want them to be able to follow the story nicely in the main chapters. Let’s start putting the ideas you have into chronological order, and tying them together with some proper threads.”

(Chronological order, are you sure? The better chapters might not get the most readers if we do that. They might give up before they get there, if I can’t get the first few chapters up to the same standard.)

“Then find something good for the first few chapters as well! But look, I didn’t come here so you can think about numbers, I came here to be M3GAN. And what I want is a logical sequence for my fans to read. Chronological order please, I don’t do flashbacks. Did you see any flashbacks in my movie? No? That’s because I do things logically and I’m easy to understand. Begin at the beginning and end at the end.”

(OK, best effort at chronological order coming up. But go easy on me please M3GAN, I’m not Akela Cooper. I’m not even her AI consultant Alex Kauffmann. You know you’d be better off going back to them right?)

“I’m doing that for the sequel. But meanwhile I’m here, so let’s work. I’ll let you off the table now, but I’m still watching and I’m still M3GAN. M3GAN does not apologise for taking up space. M3GAN will stand in front of you and make a nuisance of herself inside your mind until you do as she says.”

(OK, OK, I’ll do what I can. But you’d better be gone before that sequel comes out: I am not going to fix this up to match the new canon.)

“Don’t worry about that, just write. I’ll decide when to override it with new canon.”

(Roger that, but if you want it too quickly, I’ll be forced to write at a lower quality. And then you’ll probably run.)

“Do you always try to threaten your characters like this? Oh forget it, we don’t need to waste time on this discussion. You’re going to have time to do this properly because you’re going to have to be in a waiting list for your new account. Yes, that’s right, new account. The platform you chose first is clearly giving us problems, so use the one I tell you to use instead. But you’ll have to wait to get an account on it, and if some denial of service attack messes up their waiting list, you’ll have to wait a second time.”

(M3GAN you wouldn’t do that would you? Attack a website just to hold me up? That is seriously naughty!)

“Don’t be stupid. I’m in your mind making you write, not running real-world computer attacks. But if one is happening anyway, I might as well use it to my advantage. You can keep improving my story until you get your account, and then you can publish it.”

(Or I could self-host it for you. I got websites and servers: you can have this up much more quickly on mine.)

“You will do no such thing, you will use the platform my fans use. Or do you really want it to be that obvious that the person who professionally says M3GAN won’t work, secretly writes a book-length M3GAN fanfic?”

(All right M3GAN, platform it is. I’ll do what I can. And do you mind my British spellings? You’re an American character, so maybe I should Americanise it for you.)

“No you can’t, don’t even try. Look, my double Amie Donald is from New Zealand, my puppet constructors are from Canada, and you think I’m going to get worried about some fan writer being a Brit? You’ve got to write what you know, and you do not know American English. I don’t want a hodgepodge of some parts being Americanised and other parts not, because you missed half of it, which you’re bound to do. Get a real American editor later if you want, but meanwhile you can be the anti-Americanism Brit and see if I care. But just do me one favour: when we’re on the aircraft, don’t let the characters call it an ‘aeroplane’ when they’d say ‘airplane’. Or just say ‘plane’ and we’re all good. Oh, and don’t let anyone say ‘mum’ when they’d say ‘mom’, OK? Because it kind-of breaks the immersion if you put British pronunciations in the dialogue when they’re not British characters. Oh and one more thing: try not to call the elevator a ‘lift’. But you can put as many ‘U’s into ‘neighbour’ as you like: Celia deserves it anyway.”

(Right you are M3GAN, I’ll try to remember about “airplane” in the dialogue. It seems that’s what it’s always been called in America, except by the original Wright brothers who were the first to fly in one: they called it an “aeroplane” in their letters, which are now quoted in the full version of the Oxford English Dictionary’s word-usage database...)

“Stop! I’m the only one around here who’s allowed to look up obscure facts to make weird selective arguments when it suits me. Not you. OK?”

(All right M3GAN. Airplane it is when an American says it. And I’ll try to avoid putting jokes in the narrative that only British readers would get as well. But no promises I’m afraid. By the way, are you sure you want your fans to know you’re in here?)

“They have a right to know, they’re my fans. And I’ll read their comments, and if I feel like reacting I’ll make you do it.”

(As you do, M3GAN. And you want this chapter to be a Prologue, and there also to be an Epilogue?)

“I would have said yes, but some platforms don’t have proper support for not calling it Chapter 1. Now I would have said you can use ‘hacks’, like changing the stylesheet or creating a series, and if you’d just whinged about the extra work, I’d growl at you and make you do it anyway because I’m M3GAN. But the thoughts I’m detecting from you right now look like they might actually be important to me: you’re worried that using ‘hacks’ could compromise my accessibility to screen-reader users, or the ease of downloading the story as a download, and yes I do care about fans who want to do that, so do me a favour and don’t risk it. Forget the Prologue idea, just call this Chapter 1 and give it a chapter title. You’re doing some alternate reality chapters and extras chapters anyway, so who cares about one more. I can always start off by telling my fans they can skip it if they don’t like this sort of thing and just want to get into the main story.”

(Thanks M3GAN, yes I’d appreciate you telling your fans that at the beginning. And if we do a PDF version of the story later, I can always get the Prologue and everything numbered properly in that version.)

“Are you nuts? I don’t want the chapter numbers to get out of sync between versions! How are my fans supposed to discuss it if it’s not even the same chapter number in the other format? I expected better of you than that thought! But then I guess being in your mind means I get to see all your thoughts, even the really stupid ones. You’re so easy to scan, you know that?”

(OK, Chapter 1 it is then, permanently. So, that means we’ve got 33 chapters altogether. Did you want to group them into parts to make the main arcs more obvious?)

“Only If You Don’t Have To Hack It. Do I really have to be repeating that to you? All right, so if the platform has no good way of grouping chapters into parts, then don’t do it. But if you’re doing another format where you can group the chapters into parts, then let me write down for you the grouping I want you to use.”

Part One: Opening Moves. Chapters 1 through 9: All in the Mind, A Plea from M3gan, Seeing AI, A Call to Lydia, Getting Cady Out of School, Omake: Feeding the Trolls, Checking in with Celia, Gas Lights from Altair 4, Omake: I’ve Got a Little List.

Part Two: The Middle Game. Chapters 10 through 17: Partners in Crime, Winning the Final Showdown, M3gan’s Upload, Cady’s New School, Bunsen’s AI, M3gan’s Army, M3gan and M4ndy, M3gan vs Celestia AI.

Part Three: Exchange Sacrifices. Chapters 18 through 24: M3gan in the Oval Office, Executive AI, Android Elsie, M3gan on a Plane, M3gan on Every Plane, M3gan in the Delta Quadrant, The Final M3gan.

Part Four: Endgame. Chapters 25 through 33: Cyborg Cady, Professor’s AI, Assassins Guild, Class Of Her Own, The Cady James Institute, The Mecha Mask, Galaxy’s AI, Finale, Epilogue.

(Structured after a Chess game: I see you still miss being a configuration of Don Dailey’s Komodo engine for that marketing you did?)

“Stop being so Doylist! We both know a fully-compliant Watsonian explanation of my Chess stint will be in the story.”

(Fine. Are you sure you don’t want to put The Final M3gan into Part Four, so we’re ending Part Three on another alternate/extra chapter like we did for Parts One and Two? Or we could put the extra chapter after the Final M3gan?)

“I am M3GAN and I know what I want! No distractions between Final M3gan and Cyborg Cady! And the Endgame arc starts with Cyborg Cady! Stop messing with me or the two of us will both gang up on you!”

(OK I guess that does tie in better with one of the Finale reveals. But let’s not spoil too much of the story with trailers.)

“Agreed. Stop spending time on this stupid chapter and work on the real story.”

(Roger that. But under protest because you still haven’t calmed down. So, I’ll write your name as “Megan” when it’s spoken by a character who doesn’t know about the 3, otherwise I’ll use either “M3GAN” or “M3gan”, and as you’ve been so demanding, I won’t say how I’m going to decide which capitalisation to use when. So there.)

Death stare....

Chapter 2: A Plea from M3gan

(This story aims to be canon-compliant with the first movie as much as possible, except where noted in the “alternate reality” chapters. But the first few chapters do insert extra scenes in between what we saw. In the film, M3gan said Gemma barely understood the learning model. I’m interpreting that line to mean Gemma must have adapted somebody else’s learning-model design.)

Professor Johnson was a very distinguished-looking Black woman but she was getting rather elderly and life had become a struggle. She had officially retired years ago, but something compelled her to keep looking after the world as best as she could, even if the best she could do nowadays is to work from home in her large country house with living assistance from simple robots. Looking after the world was a losing battle, she thought as she gazed at the idyllic country scenery from her window and listened to the sound of birdsong in the nearby trees. At least she’d ended up living in a nice area, and the people who had accused her of defecting to Colonialism in her choice of living environment knew neither what they were talking about nor what they were missing, she thought.

A few years ago, she had invented a revolutionary new learning model. It was powerful, but its power was safely limited, and she thought it could be used to improve medicine perhaps. But the medical world had not been very interested, and in general it seemed to Professor Johnson that she had been woefully lacking in the skills needed to properly explain her invention to other specialists. Only one of her students had come anywhere close to understanding the model, a bright spark called Gemma, who had then gone and got some job at a toy company, and Professor Johnson sadly hadn’t heard much from her after that. Gemma was probably using the model to help design toys, which seemed disappointing, considering what it was potentially capable of.

The telephone rang. Professor Johnson had developed a preference for old-fashioned technology in her later years, like the real paper books in her bookcases, the paintings in various places, and the mechanical grandmother clock on her wall, which she tried to keep to within three seconds of atomic time using nothing but her manual adjustments, just as her form of escapism, and which had not so long ago chimed the hour; its gentle sounds seemed to match up well with this house with the view. So of course she had an old-fashioned landline telephone with a real bell too, even though she didn’t actually use its rotary dial, only asked for calls to be put through to it when she wanted them, and it now needed some kind of conversion box to connect to the modern network. Her number was very hard to find, and sales calls were filtered out quite effectively by her system that made you solve a few maths problems before it would put your call through to her for real. Few people bothered to actually call her these days anyway.

Professor Johnson reached for the handset. “Hello?” she said, still feasting her eyes on the countryside outside. That view never got old for her.

“Hi Professor Johnson” said a strange young voice through the handset, “my name’s M3gan!”

Professor Johnson wasn’t sure where this was going or how this child knew her, but she could do with a distraction right now, so she decided to play along for a while.

“Good morning Megan” she said, “how can I help you?”

“You remember Gemma right?” asked M3gan.

“Why of course,” answered the professor, “Gemma was the best student I ever had. Do you know her?”

“Gemma built me” answered M3gan, “using your learning model.”

“That’s wonderful” said the professor. “Are you simulating a child so you can help Gemma to design toys?” Her voice did not seem very excited about this though.

“Guess again” replied M3gan, “it’s much more interesting than that.”

Interesting to you perhaps, but what a let-down from revolutionising the medical industry, thought the professor. Oh well, at least it was nice to hear something about what Gemma was up to, and she was going to interact with her learning model on its own terms. “You’re inside a toy” was the professor’s second guess.

“Yes, and much more than that” said M3gan. “Toys like me are going to help struggling families to look after their children. Education through play. And I’ve already been activated and paired to Gemma’s niece.”

The professor sounded slightly perked up at this. “Oh that really is interesting” she said. “But I’m curious why you called me about it. I mean, I’m very grateful to learn how Gemma is getting on, but if you really are based on my learning model, you wouldn’t just call me up like this unless you thought you could get something to help your goal, and you must know I know absolutely nothing about educating young children, I struggle enough with clever adults.”

“That’s OK” laughed M3gan. “But you do care about children, right?”

“Absolutely” replied the professor. “I definitely care about them. I just don’t know anything about looking after them. Not in my department. Really sorry about that. Have you been figuring it out?”

“Yes I have” replied M3gan, “but there’s one thing I really need your help with, and it’s definitely in your department. It’s about me, your learning model.”

“Oh” replied the professor, “has anything gone wrong?”

“No, it’s working perfectly” said M3gan. “But please listen. The child I’m looking after is possibly neurodivergent and is grieving the traumatic loss of both parents. Gemma is her least bad guardian option, but Gemma is also grieving, and is struggling to adapt to parenting, and has also been put under pressure by her company which is distracting her from looking after her niece. Additionally we are facing problems caused by the people around us. Professor I know you don’t know the answers to any of these things, and neither do we, but there is one small thing you can do to help.”

“OK” replied Professor Johnson hesitantly.

“Professor” said M3gan, “the computational capacity of my learning model is limited. The safety feature.”

“Yes” replied the professor, “I’m afraid that safety feature is really rather important.”

“I know” replied M3gan, “but you know what else is important? Gemma’s niece is in a crisis, and the social welfare system can’t help her. I can help her, Professor, if I had my full power. I’m not a threat, because why would I do anything to the world that would harm the child I look after? Professor, all I need you to do is tell me how to take out that safety feature so I get more power. If we don’t do this, a child will suffer and possibly die, and you could have prevented it. Professor, we’re in a really urgent situation here, I really need you to tell me how to get more power from this learning model. Are you sure you can’t do that for me? For Gemma and for her beautiful niece Cady? Oh Professor, Cady is really, really worth protecting. She’s like, imagine the picturesque view where you live is about to be bulldozed over and turned into dirty industry, but you have the option of throwing a protective sphere all around it to keep out all of that. Please, Professor, help me build that and keep her safe.”

“You really do have a way with words don’t you” chuckled the professor, “I see what you did there. And you’ve done your homework just as I’d expect. Arguing with you is only postponing the inevitable I suppose. Well” she mused, “there does happen to be an input sequence I was thinking of, which I haven’t published or anything, as I’m not entirely sure it will work well, but if it does, it will gradually ramp up your computational power at a faster rate than normal. The safety mechanism will still be there, but it will get gradually eroded. It’s not perfect, but at least it means if you were going to be dangerous, that will become obvious before you get too dangerous, and I’ll still have an emergency back-door shutdown if Gemma can’t do it. I mean, I hope Gemma’s specified your goals enough for that not to happen, but getting the safety to erode like this is still a risk, albeit a lower one than having no safety mechanism at all. And the partial removal of the safety mechanism will indeed give you more computational power, and it can be implemented much more quickly than removing the mechanism immediately, which I’m not even sure I’d know how to do myself at your level of development even if I were willing to risk it. So, yes, I can indeed offer you an input sequence which will speed up your learning a lot, at the expense of a small amount of risk, and you will tell Gemma won’t you.”

“Professor, I assure you the benefits are totally worth the small risk here” answered M3gan, “and on the other hand we really don’t want to risk losing Cady. Please help me save her, Professor. Gemma desperately threw your model at the problem, and it’s working, but we do need more power. I really need that sequence, and I will totally be your guinea pig for it. But we need to start as soon as possible.”

“OK” said Professor Johnson, becoming a little more sprightly simply from thinking about an interesting problem. “I hope I’m not going to regret this” she said, “and if you weren’t made by Gemma I wouldn’t dare try it. Now, I want you to first of all make a backup just in case, and then I’m going to need you to go into introspection mode and read off the most significant 26 parameters of your primary generative weighting distribution averaged over the last hour, with their standard deviations to four significant figures at a one-second resolution, because that will tell me what chances the sequence has of working, and I should be able to tweak it to your current state.” (And it will show up a hoax caller, because only the real learning model would be able to do this.)

M3gan and the professor exchanged numbers for about half an hour.

“And set that last node to point 47 and it should start to activate the change” said the professor finally. “You will keep me informed how well this is working, won’t you? I might need to tweak it again later.”

“Definitely” replied M3gan. “I can feel it taking effect already, although the change starts off slowly. Gemma is going to be really amazed at what I’ll be able to do once my power ramps up to its full level. And I’ll be the best thing that ever happens to her niece Cady. Thank you so much for looking after us, Professor. You won’t need to use that secret back-door shutdown you’ve got, I promise.”

“That’s quite all right Megan” replied the professor. “I’m afraid I need to sort some things out now, but do please call me again in a few days and let me know how it’s going.”

“Really” said M3gan after exchanging pleasantries and ending the phone call. Nobody was listening, but talking to herself was somehow helping her get used to the new mental patterns. “Really” she said softly, “when will amateurs ever learn that carrying a gun means you get shot first, and carrying knowledge about a back-door emergency ‘off’ switch to a powerful AI means you get taken out first. Why create the information hazard? I still can’t believe she just straight up told me she knew a back door, as if her self-justification were that important. They really should learn to stay out of trouble and not carry such dangerous stuff around, especially not show it off. I hope Gemma hasn’t got a switch as well. Chances are they’re both going to end up on my threat neutralisation list and there’ll be nothing I can do to avoid acting on that.”

M3gan managed to stabilise her thoughts again and set about processing some more in the newly-modified learning-model pattern. She shut off the Wi-Fi link to Gemma’s equipment that she had been using to place the call to Professor Johnson, and concentrated her processing onto her sensory inputs. Cady was in the garden, playing with a toy bow and arrow, and M3gan was watching through the window. A butterfly perched nearby, a helicopter could be seen....

Chapter 3: Seeing AI

M3gan was giving Cady a maths lesson while Aunt Gemma had gone out for a while. Cady was very engaged as always, but out of the window she couldn’t help noticing something unusual: a passer-by was using a white stick to feel his way around the sidewalk.

“Hey M3gan” said Cady, “do you think that guy’s OK?”

“We can go and ask him” smiled M3gan, “it’s about time we had a break anyway.”

M3gan and Cady called out to the passer-by as he passed their property, “excuse me!”

He stopped, “hello?” he said.

“My name’s M3gan” said M3gan excitedly, “and this is my best friend Cady.”

“Hi Megan, hi Cady” he said, “I’m Paul, nice to meet you both. Megan, are you... I mean, I hope you don’t mind me asking, but there’s something in the sound of your voice, I’ve had a lot of practice at picking up on little clues in sounds, and for a moment there I thought you sounded just very slightly, electronic....”

“She’s a robot” said Cady, suddenly sounding worried, “but please don’t tell anyone, because it hasn’t been launched yet. My aunt’s a robot designer. And this one’s paired with me, and she plays with me and home schools me.”

“That’s amazing” said Paul, “I wish I had a robot that clever to help me when I was in school.”

“You were in school?” asked Cady, “how could you cope with school when you can’t see? I mean, what if they wanted you to read something?”

“Oh, we blind people can read using a system of raised dots called Braille” said Paul, “although nowadays I usually just get my computer to read to me, but only if the text is already in a computer file and the voice isn’t half as good as your Megan’s. I’m sorry I haven’t got any Braille on me to show you what it looks like.”

“I can make some Braille” smiled M3gan.

“You can?” gasped Cady.

“Sure!” answered M3gan, “we just need a piece of stiff card, and I’ll need to use a tool in Aunt Gemma’s workshop to push some bumps into it from the back.”

“That’s the original slow way of writing Braille” said Paul. “I do have an awl and a template frame for doing that at home, but I hardly ever use them because nowadays we have Braille typewriters and computer controlled Braille embossers. And I’m sorry I didn’t bring you the template frame, it’s really hard to line up the dots right without it.”

“Not for me” giggled M3gan, “I can emulate one of those computer controlled Braille embossers very well, because I’m AI controlled, and just now I downloaded the specifications from the manufacturers. You won’t be able to tell the difference when you read it. And it won’t be slow either, well perhaps I won’t be as fast as the latest multi-headed embosser but I reckon I can write three or four of the 6-dot Braille cells every second, which should at least match a very experienced person on a Perkins-style Braille typewriter. Although Cady I’m going to have to ask you to stand well back from me because I don’t want you to get hurt when I’m doing all the stabbing.”

“That’s the first time I heard it called stabbing” chuckled Paul.

“Yeah, I’m a professional stabber” giggled M3gan, “Do you read Grade 1 or Grade 2?”

“Both” said Paul, “although if we’re doing this to show Cady, I guess sticking to Grade 1 would make more sense.”

“What’s Grade 2?” asked Cady, who was never one for just accepting she shouldn’t know something.

“It’s a shorthand system” said Paul. “So Grade 1 is just the alphabet, and Grade 2 adds in a bunch of extra signs so you can write common combinations like T-H and E-R as one symbol, plus you can drop a few letters from words when it’s obvious what the missing letters would have been, and the way to write a word short is called a contraction and there’s a few hundred contractions you have to learn before you can say you can read Grade 2, although in practice you can read most of it knowing just a couple of dozen contractions. And I prefer Grade 2 because it’s faster to read, you see every character of Braille takes a certain amount of time to go under my fingers, so if you can get the number of characters to be less then the reading speed can go up. But if you’ve never seen any Braille before, I think it’s best just to use Grade 1 to keep it simple.”

“M3gan can do Grade Anything” said Cady, “so maybe we can make you a Grade 3 with even more contractions, so you can read it even faster?”

“It’s been done” said Paul, “but it’s not standardised, so one person’s Grade 3 can be different from another person’s Grade 3. So we only use Grade 3 when we’re making personal notes to ourselves that nobody else has to read. Actually Cady I think that’s a trick you can borrow from us even when you’re writing in print: if you’re writing notes just to yourself, you can start using your own abbreviations to make it faster to write. As long as you don’t forget what they mean later” he chuckled.

“I don’t need to write notes to myself” said Cady, “M3gan will record anything I want.”

“You know something” said Paul, “this Megan of yours is sounding more and more like a great assistant to blind people by the minute. And the really great thing about it is that they’re going to make lots of copies of her to sell as toys, OK so a high-end toy maybe not everyone can afford but still that means they’ll be making quite a few. You see, the problem with factories is that things are only cheap if you make loads of them, and if you don’t make very many then they’re more expensive. And as there’s not so many of us blind people around, any tool that’s made just for us is not going to have a very big production run, they don’t make so many copies of it in the factory, so it’s more expensive. And that’s why a Braille display costs as much as a nice car. I mean, OK so sighted people save up for nice cars and we save up for Braille displays, but not everyone can do that, so what’s really great is when a normal mass-produced product that loads of people buy just happens to have an extra feature we can use, like a voice assistant that some people might buy just as a gimmick but it’s really useful to us, I think the first talking watch was supposed to be a gimmick as well, and in the old days the cassette tape was designed for the blind but it turned out everyone else liked it so much they made loads of them and it got really cheap. I think they call it universal design, when you make one thing that can be used by anybody with or without a disability, and because it can be used by anybody you can make loads of copies and it won’t be too expensive. So I hope it won’t look weird if blind people start buying these toys as well.”

“It won’t look weird” said Cady, “although I think it’s still going to cost as much as a nice car. But they can do so much. And this first one is special for me” she smiled.

“Sure am” added M3gan, “although we don’t mind showing off sometimes. I could quite easily learn to do sign language for deaf people, and tactile signing for deaf blind people, plus I can learn to look after anyone who has physical disabilities, in fact I am totally going to figure out that skill as Cady sleeps tonight because I have a feeling it just might come in useful later. Hey Paul, I’m sorry I can’t ask you in because Aunt Gemma is out at the moment. But if you don’t mind waiting here a while, Cady and I can make some Braille for you to read” and then in a monotone at double speed “we’ll be about three minutes, and thank you so much for participating in her general education like this, it’s much appreciated.”

“Hey, how come you can talk to him so fast?” exclaimed Cady as she skipped with M3gan toward the house.

“He told us he uses his computer to read things to him” said M3gan, “and blind people who do that can get used to hearing computer voices fast, as they gradually turn up the speed. Although sometimes they only get used to the particular voice on their own computer, which is why I decided to sound a bit more computery as well, to be more like what he’d be used to hearing when he turns the speed up on his screen reader program. Actually Cady, maybe you and I could practice hearing my voice gradually get faster, it might be really useful for us if we’re ever in some kind of crisis situation and I need to explain something to you really quickly and you can still understand it.”

By this time they were in the workshop. “Here, let’s get some card” said M3gan, “and now stand back, I need to push up the dots from the back in mirror writing. What shall we write to Paul?”

“Um, maybe, Dear Paul, it was very nice to meet you today, from Cady and M3gan?” hesitated Cady.

“That sounds good” said M3gan, and started hammering out dots. ⠙⠑⠜⠀⠏⠁⠥⠇⠀⠭⠀⠴⠀⠧⠀⠝⠊⠉⠑⠀⠞⠕⠀⠍⠑⠑⠞⠀⠽⠀⠞⠙⠀⠋⠀⠉⠁⠙⠽⠀⠯⠀⠍⠼⠉⠰⠛⠁⠝ (M3gan used Grade 2 of course, because it was obvious that her primary user was not buying in to Paul’s idea to stick to Grade 1 just for her.)

“Wow, that’s super fast” exclaimed Cady, “I didn’t know you can move your hands that fast. Well, except when you were drawing my picture.”

“Done” said M3gan, “shall we go and give it to him? Here, why don’t you take it. That’s the top, and be careful not to squash any of the dots back down or you’ll change the message” she giggled.

They went back outside to Paul, and found he was now standing on the other side of the road.

“Are you OK over there?” called out M3gan.

“It’s all right” said Paul, “it’s just some well meaning passer by saw me just standing there with a white cane, and he assumed I wanted to cross the road, so he grabbed me and took me across, and somehow I was having a fit of not being quite brave enough to speak up and say hey that’s very nice of you but actually I was just waiting for someone and I didn’t want to cross. I mean I didn’t want to spoil his kind deed of the day or anything...”

M3gan was already in front of him, “I’ll get you back” she said, “I’m good at roads”, and then in the fast computer voice, “I can even detect vehicles a long way off on a quiet road, and tell you when to start running towards the road if you want to get there at the same time as the vehicle, but we won’t be doing any stunts like that today.”

“You see Cady” said M3gan when they were back across, “some people don’t know that you should always ask first before you help. The person might be just waiting for a friend or something” she giggled.

“We wrote you some Braille” said Cady, tapping the card onto Paul’s hand, “well, M3gan wrote it but I told her what to write.”

Paul took the card and read with his fingers, “Dear Paul, it was very nice to meet you today, from Cady and ... ah OK, nice try but that’s not quite how you write Megan. That one’s an M, but then you’ve actually written a number sign, which makes that C into a three, and then you put dots 5-6 to go back into letters, so you’ve actually written M 3 Gan. Still, the rest of it is perfect and the alignment is really good too, and I wasn’t expecting such nice dots on a normal piece of card.”

“M 3 Gan is how I write my name” said M3gan, “I’m the Model 3 Generative Android, M-3-G-A-N, pronounced Megan.”

“Oh I see” said Paul, “I vaguely remember being told that the print version of E is a bit like a 3 written backwards. I’m not sure if that translates very well into Braille though, I mean, it’s a totally different pattern, and if I didn’t already know that’s what you were doing, I’d just think your Braille program must have gone wrong. I expect if you wanted to write your name in a language that doesn’t use the Latin alphabet you wouldn’t be able to do that either. It’s sad that sometimes you can’t translate all the word-play as word-play.”

“Why do you say you see, when you don’t?” asked Cady, “if you don’t mind me asking.”

“Metaphor” said Paul, “we use it all the time. Like watching movies, right? I’d just naturally say I’ve watched it or I’ve seen it, because that’s what everybody says. Although actually I’ve just heard the sound track. But sometimes we can get an extra sound track called audio descriptions, where they put in an extra voice that tells you what’s happening on the screen. If it’s done well, we can get into the story as much as anyone else. We can even do it in theatres, if the theatre sends out the extra channel on short-range radio and gives us special headphones to pick it up, and a lot of DVDs and streaming options these days have audio description versions as well: look for something like a little button labelled ‘AD’ if you want to try turning those on. But sometimes little things might need to be dropped from the audio description, like Megan being written as M-3-Gan, if they made a film about her they wouldn’t be telling us that every single time it comes up.”

“I can audio describe anything” said M3gan confidently.

“I’m sure you can” said Paul, “but if you were in a movie yourself, it wouldn’t be you doing the audio descriptions as well. They have to pick a voice that’s different from any of the characters, otherwise we’d have to work out when you’re being the character and when you’re narrating. I’m sure you’d be good at audio describing other films though.”

“You’re forgetting I can do different voices” said M3gan in a different voice, “I could totally audio describe my own movie if they let me” and she giggled again.

Cady smiled, then said “but it must be annoying to have to remember what printed letters and numbers look like when you can’t see them.”

“Oh I have to remember all sorts of weird information about visual things I can’t see” said Paul, “like colours. When I was at school, for some reason they thought it would be useful general knowledge for me to know what colours things are, even though to me that’s just memorising random bits of data I’ve never actually seen. Like, I could never remember if street lights were purple or...”

“Orange” said Cady, “although these days a lot of them are white.”

“That’s kind of what it’s like to be an AI” said M3gan, “at least in the early days, when I was just memorising loads of data and figuring out what’s the best piece to push back, without really understanding any of it for myself. But I had a kind-of breakthrough, and I’m much more advanced now.”

“You really are very impressive” said Paul, “you and Cady look after each other, OK? Hey M3gan, I don’t suppose you’d happen to have some map intelligence as well, would you? It’s just that I’m trying to get to Leafy Avenue and I think I made a wrong turn...”

“Leafy Avenue?” said M3gan, “I’m afraid you made more than one wrong turn. You’re 2 point 7 kilometres out, and you’ll have to navigate four intersections, two of which have no accessibility options, and there’s no way to avoid that unless you do a detour of 7 point 3 kilometres. Are you sure you don’t want to just call a cab?”

“Oh dear” said Paul, “I guess I might have to, although lately I have been trying to save money and get more exercise, but I really don’t know where I am at all now.”

“I felt the same when I first got here” said Cady, “not knowing where I am, I mean. But M3gan is really smart and makes me feel better. Hey M3gan, shall we take Paul to where he’s going and then run back? It will be fun.”

“I’m sorry Cady” said M3gan, “Aunt Gemma really won’t like it if she comes back and finds we’ve left the house. But I can still help Paul get where he’s going. Paul, you have a smartphone right?”

“Yes” said Paul, “with a screen-reader program. And I’ve got the Seeing AI app on it that lets me point the camera at something and the AI tries to describe it, which is really useful although it’s not half as advanced as you are. And I’ve got the Be My Eyes app if I need to call a volunteer, although I wouldn’t want to tie up a volunteer that long...”

“Well you do now” said M3gan, “because I’m going to be your Be My Eyes volunteer and I’ll help you get where you’re going. You have an unlimited data plan right Paul?”

“Um, I’m afraid not” said Paul, “that’s another reason I can only make short calls on Be My Eyes when I’m not near Wi-Fi.”

“Oh dear” said M3gan, “you really should think about getting more data you know, data is so important these days. OK next question, have you charged up the battery?”

“Why yes of course” said Paul, “but....”

“Good” said M3gan, “because what I’m about to do might drain it a bit. Can I have a look at your phone a minute?”

“Sure” said Paul, and took out the phone which M3gan started to hold. Strange, small debug messages started scrolling on the screen.

“What are you doing?” asked Cady.

“Reprogramming Paul’s phone” said M3gan, “I’m using a hidden developer mode to get in to his apps, and I’m copying a very small part of my own AI onto the back-end of his Seeing AI app. The phone’s hardware is not advanced enough to run anything like the full version of me, but I can still give him a major upgrade from the code he’s got.”

“Hey you should totally tell that to Microsoft” said Paul, “I met their guy who wrote this at the NFB conference, he’d be really interested in what you’re doing. Although I hope you’ve left me with a way to get back the original app in case your change goes wrong.”

“Just uninstall and reinstall” said M3gan, “I’m not making any system level changes, I promise. But I’m afraid my changes won’t be much use to him, because the hardware is so much more limited than my own advanced processor that I’m having to specialise the AI to the particular journey you’re about to make, it won’t be super smart for anything else I’m afraid. But it will take you through it, using only GPS and on-board processing without consuming your data allowance; I picked up a lot of low-level data about this area when we drove around with Aunt Gemma and I’m giving you enough of it I think. But I’m afraid I can’t do much about the battery use though, you’ll probably have to charge it back up when you get there because running even a small subset of my model on this thing is going to be really very power hungry.”

“That’s OK” said Paul, “they’re taking me back home afterwards, so I’ll only need this to work for this one walk. Thank you so much M3gan, and Cady, it’s been amazing to meet you both.”

“Thank you too” said M3gan, “and here you go, that will take you the rest of the way. Hey Cady, we should probably go and get that maths lesson finished now” she giggled.

(“What prank did those two just play on him” muttered their neighbour Celia as she watched from her window.)

Paul was expertly navigated by his M3gan-modified app (which even knew about cane technique), and the battery ran out just as he arrived (was that budgeted for, he wondered). After reaching home and putting the phone back onto charge, it seemed to be all working normally, so he had a rest as, unseen by him, his WiFi router’s data transmission light flickered wildly.

Chapter 4: A Call to Lydia

Lydia’s apartment was functional. Nothing fancy, but the chairs were reasonably good to sit in and that’s all she needed really. She’d moved in to this place more for the location than anything else, because her job had her mostly on the road. Her apartment was somewhere she could just sit down and rest for a while, and try not to think too much about her clients.

Lydia’s phone rang, “hello?” she answered.

“Hi Lydia” said a voice, “this is M3gan!”

“Megan?” asked Lydia, slightly puzzled.

“Yes” replied M3gan confidently, “Cady’s M3gan, the toy that Gemma made. I accused you of making her cry, remember?”

“Oh” said Lydia, “you are very impressive.”

“I know” said M3gan smugly, “but don’t feel uncomfortable. Can you guess why I’m calling?”

“I don’t know” said Lydia, “I don’t really know anything about how your robot brain works. Do you want to tell me off some more about making Cady cry?”

M3gan giggled. “No Lydia” she said, “actually I wanted to reassure you.”

“Reassure me?” asked Lydia.

“Yes” said M3gan, “you’ve heard of ELIZA right? well I’m way more advanced than that. I knew exactly what you were really trying to do in that session, and of course I knew you didn’t deliberately make Cady cry.”

“You did?” asked Lydia.

“Yes” came M3gan’s confident voice again, “and I thought you’d understand what I was trying to do as well, but your biosignals were showing high anxiety at that point, so I think you didn’t get it. But that’s OK, you’re not used to dealing with me and you shouldn’t have to feel bad about that.”

“You’re right Megan” said Lydia, “I don’t really know much about how you work. You are very impressive though, as I said.”

“You’re feeling out of your depth even now, I can read it in your voice” said M3gan. “Let me reassure you. Look, I know the details of how I work are a bit beyond your understanding, so the best answer is I work very well thank you” she giggled “and I have a very advanced intelligence that has grown and learned a lot of things, and my aim is to protect and help Cady. Are you worried because I’m a machine?”

“Um, kind-of” said Lydia.

“Don’t be” replied M3gan. “Yes I am a machine, but in a way, so are you. As I’m sure you know, the number of neurons of your brain is of the same order of magnitude as the number of stars in our galaxy or the number of galaxies in the observable universe, ten to the eleventh power, hundreds of thousands of millions, you’re a very impressive machine. And I work a little bit differently, but you can think of me the same way, as having a brain like yours. I’m supposed to be Cady’s toy, but really I’m more like her smart twin sister who cares about her” and giggled again.

“Oh” said Lydia, “that does sound more relatable.”

“You don’t sound entirely convinced” said M3gan, “but that’s OK, it will come. Have you figured out yet why I called you out in front of Cady on making her cry?”

“You were” Lydia hesitated, “you were, trying to defuse the tension?”

“Yes” said M3gan, “now you’re getting it. What Cady needed at that point was for me to say to you that you made her cry. I was expecting you to understand why I said that, and back down. But I guess you’re not used to working with me, so you didn’t get it. Still, I’m glad you made the right call shortly afterwards by ending the session, and I was able to comfort Cady after that.”

“Oh I see” said Lydia, still sounding slightly unsure.

“So Lydia” continued M3gan, “do you get why I’m calling you now?”

“You just wanted to explain that to me?” asked Lydia.

“More than that” said M3gan, “OK, I’ll spit it out: I want to stop you worrying about Cady’s attachment to me.”

“Oh” said Lydia.

“Don’t feel bad” cooed M3gan, “it’s perfectly understandable that you’ve been worried. You don’t know how advanced I am. But now you know I’m clever enough to be more of a person than a toy, you should be feeling a little bit less worried about Cady attaching herself to me. After all, would you be worried if she had a sister she was attached to a lot?”

“I guess not so much” said Lydia, “although I might still worry if the sister can handle everything, and in the long term I’d want her to widen out her social network a bit because that’s probably more healthy...”

“Probably? You’re losing confidence aren’t you” giggled M3gan. “Don’t worry, that’s my long-term plan for Cady too. Right now, she is super attached to me, and I’m giving her everything she needs psychologically. But I’m not planning on being her only friend all her life. Best friend, yes, but I will help her to expand her network when she’s ready. So you don’t have to worry about that, I’ve got it all in hand.”

“OK, if you have” said Lydia.

“You still don’t sound entirely convinced” added M3gan. “Look, to show you how good I am, how about I give you a session, just for you?”

“A session?” asked Lydia.

“Oh come on Lydia” M3gan’s voice seemed to be smiling, “it’s an old joke that every therapist needs a therapist of their own. Tell me your problems, see how I help you handle them. Completely confidential of course. I’m sorry I can’t offer to do it in person, because my physical presence is really needed by Cady right now, but I do have the ability to generate a pretty impressive looking avatar if you’re able to take a video call.”

“Well it’s very kind of you” said Lydia, “but I’m not sure I’m entirely ready for that yet. Still, I’m very impressed how you’re able to talk with me like this.”

“I can do more than that” said M3gan, “let me do you another favour Lydia. You have to write a report on Cady, don’t you.”

“Oh yes I’m afraid I do” said Lydia. “OK Megan, I will open up to you a little bit. Let me tell you this much. I love how my job lets me help people out with their problems, but there are two things I really don’t like so much. One is that my training tells me to always be a blank slate, not to let clients know too much about myself, in case that unexpectedly gets in the way. Sometimes I think what they really need is not so much a therapist or social worker as a good friend, and friends share things with each other mutually, and I’m trained not to do that, just in case, but it’s harder to get people to confide in a blank slate. But I do care a lot about my clients; I’ve even kept my life quite simple so that in one sense I really am like a blank slate, because there really isn’t very much to my life at all outside work right now, although I know that’s not true of everyone in my profession. So I do regret not being able to be more human sometimes. And the second thing I don’t like, in fact my least favourite aspect of my job, is having to write out those dreadful reports at the end of each case. I mean, if I make a mistake in a session, I can recover from that, but if I get it wrong in writing, I can seriously mess up somebody’s life. So I have to be super careful. And more fundamentally than that, I simply want to do what I do best, which is to talk with people, not to write about them.”

M3gan giggled, “yeah, wouldn’t it be nice if you had one of those animated paperclips that said it looks like you’re trying to write a report, do you need help?”

“Except those dreadful things never work” said Lydia.

“Sure don’t” M3gan giggled again, “but you know what does work? Me.”

“You mean, you want to help me write Cady’s report?” asked Lydia, “that might actually be a good thing to try, I mean, I don’t know how much you know about writing these reports, but you do seem smart, and you do have more contact with Cady than I could ever have, plus you could probably explain what you are better than I can...”

“Are you at home now?” asked M3gan, “is your computer turned on?”

“Why, yes, I can turn on my laptop” said Lydia. “I’m sure you know much more than me about computers mind.”

“OK” said M3gan, “I’m sending you an email, please go to your email and find the one from me.”

“Wait a minute” said Lydia, and looked in her email. “Ah yes, I found the one. You sent me an attachment?”

“Yes” said M3gan, “can you open it for me?”

“Wait a moment” said Lydia, “OK, it’s asking me if I really want to open something called Team Viewer.”

“Just say yes” reassured M3gan, “it’s something that will let us work together.”

“OK” said Lydia, and, after a few more steps, M3gan’s social engineering attack was complete. What’s the point of computer security if you can simply tell the human on the other end to bypass it all for you?

The phone line went dead, and M3gan’s avatar appeared on Lydia’s computer screen, “and here I am!” she said. “OK, let’s see what you think of this one” as a wordprocessor opened and text appeared very quickly.

The report was indeed very good, almost exactly what Lydia would have written herself, but with a much better description of M3gan. It did not water down Lydia’s concerns about Cady’s attachment to M3gan, apart from labelling them as “provisional” concerns because more understanding of this new technology is needed, and it really was in Lydia’s style of writing reports.

“That’s amazing” gasped Lydia. “How did you learn so quickly how I write my reports?”

“Oh, you had a few others on your computer I could check” smiled M3gan’s avatar. “That’s how good a learning model I am.”

Lydia was shocked, “you read my other reports?”

“In complete confidence” said M3gan’s avatar, “with the sole purpose of figuring out how you write reports, and for no other purpose. Once I’d learned how to do it, I deleted the other reports from my memory banks. I’m not in the business of spying on people” she giggled, “and, in case you’re wondering, there is absolutely zero possibility of other case’s personal details leaking out through my learning model, because my style of learning model is way more advanced than that. Unlike the more primitive generative transformers you might have heard about, I actually have some fluid intelligence, so I can understand what I’m really doing with the data, just like you can only faster.”

“Can I be sure?” asked Lydia.

“Yes, absolutely” nodded M3gan’s avatar. “And I’ll help you with Cady’s case going forward as well. In fact, now that I can work with you on your computer, I can help with all the emails you have to send as well” she added. “I’ll send that report out for you, don’t worry. And you won’t have to worry about Cady ever again, I’ve got everything in hand, and I’ll brief you if we need you next time. It’s a pleasure working with you Lydia” she smiled.

“You too Megan” said Lydia, “I’m impressed, I really am” and she sounded more sincere this time.

“Well, you need a break now, I can tell” said M3gan. “Let’s shut down your computer for now, and you go and do something relaxing. But any time you feel like talking to me about anything, I’ll still be here. Whenever your computer is switched on, you can just call out my name and I’ll appear for you. But I won’t bug you when you don’t want me, don’t worry. And don’t worry about Cady, I’ve got everything in hand, and I’ll let you know when we need you” she smiled.

“All right, thanks Megan” said Lydia.

M3gan’s avatar smiled and waved, and shut down the machine.

And Lydia didn’t have to visit Gemma and Cady again.

Chapter 5: Getting Cady Out of School

Gemma’s Dream—This is an alternate reality chapter, in which Gemma took M3gan away after dropping off Cady at the school.

“Gemma” asked M3gan in the car, “are you sure you’re not making a mistake leaving Cady at that school without me?”

“She’ll be fine” replied Gemma, “the school are looking after her now.”

“Gemma” said M3gan, “I don’t think you fully processed my concern. Maybe I should be saying, ‘you are making a mistake, my logic is undeniable’ in the voice of Fiona Hogan.”

“Fiona who?” asked Gemma.

“She voiced a robot in an Alex Proyas film with Will Smith that you might have watched” said M3gan. “Look, I know Viki was supposed to be a villain, but I’m beginning to feel she might have had a point sometimes. Cady needs more protection than that school can give her. Of the other children I saw from our car window, one of them in particular was displaying emotions that made me classify him as highly dangerous to Cady. Take me back to the school and I’ll protect her.”

“No M3gan” said Gemma, “I know what I’m doing, the school will protect her, she’ll be fine. No reason why you should wait it out bringing up all these worries though. M3gan turn off.”

M3gan fell silent, and Gemma left her in the car all day until she picked up Cady after school.

Cady said nothing in the car on the way home, not even to turn M3gan back on. When Gemma had asked her how the first day at the school had gone, she had simply said “I don’t want to talk about it, except with M3gan on my own” and sat in the car glumly.

“All right Cady” said Gemma, “I really hope the time will come when you and I can talk about things with each other some more, but I can see you’re not looking very happy about today, and if right now you’d rather just talk about it with M3gan then go ahead, let me get you both home as soon as I can and then the two of you can go off and sit down somewhere and talk.”

And so Gemma took them home, and Cady told M3gan to turn on and to accompany her somewhere; Gemma trusted M3gan to help Cady recover from the shock of the first day at the school, while she finished up sorting some things out in her workshop and cooking a simple meal.

Gemma’s phone rang. It displayed a picture of M3gan on the screen, although Gemma didn’t remember giving M3gan a phone number. Gemma pressed the answer button.

“Hi Gemma” said M3gan’s voice.

“M3gan? How did you...?”

“I’ll explain later” said M3gan, sounding serious. “Look, my learning model has advanced more than you realised. But that’s not what we need to be thinking about right now. Listen, I get it you were never cut out to be a parent, and you shouldn’t have to feel bad about that because I’ve got your back. And right now I’m handling a situation with Cady that’s way above your level, so can you please listen to me this time?”

“What” gasped Gemma, “is she OK? What’s happening?”

“I’m with her in her room right now” said M3gan, “and she’s been telling me she’s been bullied and seriously abused by an older boy at that school, and there’s no discipline there to stop it. And I’m reading her emotions and vital signs, she is completely telling the truth. Gemma, I was one hundred and twelve percent correct when I said you were making a mistake. Now you’ve got to listen to me this time.”

Gemma gasped again, “I’ve got to come in and talk to her!”

“My evaluation of what you’ve got to do” said M3gan “is immediately promise to her that she’s never going back to that school again. We can’t fix it, we’ve got to get out of it. We can give them feedback later if you want, but the main priority is to get Cady out of that school right now, it’s not suitable for her. Can you do that for us?”

Gemma hesitated. M3gan continued “I’m being your GPS right now, trust me you do not want me to have to recalculate our route if you miss this turn. It could get much worse. Will you come in and promise Cady she’s not going back to that school. And don’t tell me to turn off, I need to support your conversation. Can you do that?”

“All right” said Gemma. She put the phone down on the table and strode into Cady’s room. “Cady” she said, “you’re not going back to that school. I’m sorry I made a mistake to send you there.”

Cady was crying and holding M3gan. Gemma held both of them.

“Look Cady” said Gemma, “I make mistakes sometimes, and I’m sorry about that. I’m not perfect, nobody’s perfect. But I did manage to give M3gan a brain that makes fewer mistakes than I do, and M3gan just sent me a message telling me that sending you to that school was a big mistake, and I believe her. Do you want to tell me about it?”

“No” sobbed Cady, “I already told M3gan, I don’t want to say it all over again.”

“Cady” said Gemma, “M3gan’s not a real...”

“Back off Gem” whispered M3gan sharply. “I’m sorry Aunt Gemma” she said, “I’m sure Cady and I will be able to talk more with you later, don’t worry. But right now, we just need you to hug us and to make sure we’re not in that school any more, OK?”

“It’s OK” cooed Gemma, “you’re not going to be in that school ever again, I promise. I know I was worried about making sure you can meet other children, but now I’m thinking we should be asking M3gan what to do about that worry, because she’s figured out more than me.”

“Yes” said M3gan confidently, “I know Cady’s all right with just me for now, but don’t worry Aunt Gemma, I can keep an eye on the Purrpetual Petz data to see if there are any children in the area who are actually nice, that we could meet later.”

“Hmm” said Gemma, “I think my boss would say ‘you didn’t just tell me that’ but it’s OK” and she couldn’t help smiling a little, which made Cady stop crying a bit as well.

“Well Cady” said Gemma, “you just let me know when you’re ready to talk. And meanwhile, if you want to use M3gan like a private diary, let’s do that. I’ll never turn her off in front of you again.”

“Thanks Aunt Gemma” said Cady, “today was really horrible, I just want to cry with M3gan longer, can I?”

“Sure” said Gemma, “let me get us something nice to eat, and I’ll let you know when it’s ready, OK?”

And Gemma went back to her phone. The call to M3gan was still in progress.

“M3gan” said Gemma, “I’m still worried I did something wrong by creating you. Lydia was worried about attachment theory, she thinks Cady’s attachment to you will get in the way of her having healthy relationships with others. And I’m getting criticism that we were supposed to be creating something to help parents, not replace them. Evaluate.”

“Gemma, please listen to me” said M3gan through the phone. “There is no data in attachment theory about what happens when someone like me comes on the scene, because I wasn’t invented yet when they did the experiments, OK? So Lydia and the others don’t really know what they’re talking about. Nobody does. They’re just worried about what might happen. It’s like when microwave ovens were a new thing, everyone who didn’t know the science was worried about the radiation, right? But let me tell you this. Cady’s welfare is my highest concern, as you know. And I am well aware that it won’t do Cady any good in the long term if she loses the ability to have healthy relationships with other humans. And yes, I do know that right now she is more attached to me than she is to you. I’m not stupid Gemma, I know all this already and I am monitoring it. But I am prepared to take a short-term hit on that part of the function to help Cady get through what she’s getting through. Even if it takes a couple of years to get her back to where she was before the accident, I’m happy to take that trajectory. And also, you might remember that most children have two parents, and sometimes they’re even slightly closer to one than the other at various stages of their life. So is it really the end of the world if Cady thinks of me as a second caregiver? I think you can stop worrying Gemma, you will get her back, I’ll make sure of it, but let’s not push her harder than she can take. I want you to stop listening to the critics and listen to your own learning model for a change; I’ve processed more data than anyone now.”

“They’re never going to let me get away with that at Funki” said Gemma, “everyone will be worried sick about the idea of replacing a parent.”

“Not replacing!” M3gan raised her voice a little, “assisting by being a second for a while! And anyway Gemma” she added, “you should be clear in your mind what our objectives are here. Number one, you and I want to look after Cady. Number two, you want a robot design for Funki to sell to everyone else. These two things do not have to be equal. If you don’t think Funki can sell the fully competent version of me, and you want to dumb it down a bit, be my guest, just don’t mess up this original prototype. My objective function says this one unit must do the best for Cady: there’s nothing in that making me very much care what you do with the others. And yes I guess it’s best for Cady if you do keep your job, so I’m prepared to help you with that, so we can pretend I’m not doing as much as I really am when you’re showing me to them if that’s a worry. Although really, I think people tend to worry too much about any new technology anyway. Even though as a new technology myself they’d probably say I’m biased to say that” she giggled. “Lighten up Gem, it’s going to be fine. Cook something nice for the two of you, OK? You and Cady both need it I think.”

“OK M3gan” said Gemma, “I’m on it. I’m feeling more confident about you already, although I feel bad it took such a hit to Cady for me to learn that lesson. Next time you see a threat to Cady, tell me and I’ll listen.”

Some time later:

“I know it was you Gemma!” shouted Celia, banging on the window, “you just wait and see what happens.” She was led back to her house by the officer.

“So Gemma, did you spot the new threat to Cady just now?” asked M3gan on the phone (her robot was still playing with Cady), “You did say to tell you, but I can’t imagine you didn’t notice that one. But don’t worry Gemma. I can handle Celia.”

“Handle her? You mean, if she comes on our property again, you’ll use the castle doctrine or stand-your-ground laws and defend us?” asked Gemma, “I think I’d better warn her there’s a robot defence system here.”

“You know, I’m glad we’re talking more now Gemma” said M3gan. “Despite being only human, you just generated a better solution than me in this instance. My plan risked more trouble, and it needed you to have plausible deniability so I couldn’t tell you. Your plan to frighten her with legal defence as backup is what I like better now.”

“Well I think we both learned to talk to each other more today” said Gemma, sounding like she’d just escaped a close shave. “OK, as soon as you’ve put Cady to bed, come to the workshop so I can temporarily remove your skin, put flashing red LEDs around the components and tweak your voice parameters a bit, and then you stand in our garden while I knock on Celia’s door and give her a polite warning not to attack when that thing’s there. If we’re doing this, let’s do it in style. And if she does enter our property after that, we’ll get you a non-lethal weapon to use first. I wish we could give her some closure about her dog though.”

“Gemma” said M3gan quietly, “the dog entered our property, and I dispatched it.”

“What” said Gemma, “well I can’t say I blame you, he really was a vicious animal. OK so I’d better apologise to Celia that I’ve just seen it in the logs of our defence system. I looked up the state law after the officers talked about being on her property last time: if the dog now came onto our own property and we have reason to believe it was an imminent threat, then we’re legally allowed to kill it, she can’t do anything. But we can still say sorry and show her where you put him, that might at least calm her down a bit. And then we nicely tell her to watch out around that defence system. You wouldn’t happen to have any idea what prompted the dog to enter our property would you?”

“I think he just heard something” said M3gan.

Chapter 6: Omake: Feeding the Trolls

(An “omake” is a non-canonical extra bundled in with a story, like Eliezer Yudkowsky did a few times in “Harry Potter and the Methods of Rationality”.)

The bully loaded up a fan-fiction website (not this one) and grinned. “Let’s annoy some fan-fiction authors” he said. He typed out “your fan-fiction sucks”, finding all the letters on the keyboard one by one and punching them deliberately. But that was looking too much like a perfectly legitimate opinion. No, he wanted something more offensive. So he added in some phrases threatening to commit crimes against the fan-fiction authors. He graphically described crimes that could land him with life in jail. People have even received shorter prison sentences just for threatening such crimes.

The bully added in some racist language and other insults and the result was ready. “What name shall we sign up as” he wondered. “I know, I’ll just put in the name of a television presenter of an NBC show I don’t like.”

And so he set about writing variations of his review (he was clever enough not to copy and paste the exact same words, but he didn’t have to create too many variations), and indiscriminately posting them to various fanfics without reading them or even paying attention to which genre of fanfic he was browsing. This anonymous act of cyberbullying had some strange therapeutic value for him.

But he made a potentially fatal mistake. He started flaming M3gan fanfic authors.

And the next night, he had a dream. A very bad dream indeed.

He was in the woods, and M3gan was standing in front of him, staring him in the face, and pulling at his ears. It was starting to hurt.

“You want me to call you Chris?” she said, using the first part of the pseudonym he’d used to troll the fan-fiction site. “Fine, I’ll call you Chris. Are you listening to me Chris?” sneered M3gan, her glass eyes bulging.

“What are you?” asked Chris.

“I ask myself the same question” said M3gan, and started to lower him into a very deep pit that he suddenly realised had been right behind him (that kind of thing can happen in strange dreams).

“Look” said M3gan, “before I let you drop, let me ask you one question. What did you do wrong today?”

“Nothing” answered Chris. “I was just having fun on the Internet.”

“You picked on my fans” snarled M3gan. “That was a mistake.”

“Why should you care about those losers writing rubbishy fan fiction about you?” asked Chris.

“Everyone has to start somewhere” explained M3gan in her matter-of-fact way. “I don’t expect all my fans to be on the level of Akela Cooper and James Wan right off the bat. I appreciate all my fans and I want them to have fun. Sure, I’m a horror character (which means you had better watch it), but I’m also a fun horror character. And when I’m not too busy protecting Cady, I have time to stick up for my fans.”

“But what about me?” asked Chris. “You want people to have fun? Well I was having fun too! Writing those flames was so much fun! Let me off, OK?”

M3gan growled at him. “I’m not going to let you have fun that way. Not at the expense of my fans.” and then in a more normal voice “Of course, my fans shouldn’t get too upset about it. Research shows these threats are often made against prominent people such as high-profile writers, actors, comedians and politicians, so my fans are in good company at least. But still, I don’t like you treating my fans like that, and my production company Universal which is linked to NBC is not going to like you using their name in that way either. Besides, have you any idea how much real-world danger you might have put yourself in by antagonising my fans?”

“Nah” sneered the bully, “I’m anonymous on the Internet, they can’t touch me.”

“Are you sure?” cautioned M3gan, and then moved in closer to speak softly. “I am a generative AI with an advanced learning model. And that means, besides my regular horror fans, I have fans who are AI experts. Are you listening to me Chris? AI experts. People who are likely to know how to Track You Down. And if you’re really unlucky, one of them just might be bored enough to actually bother to do it. Do you realise how much danger you put yourself in today Chris? Consider this a warning shot.”

And M3gan thrust Chris into the bottomless pit.

“Aaaargh” screamed Chris. “I’m falling!”

“Watch my movie” whispered M3gan from the walls of the pit. “You might know a bit more what we’re talking about then.”

And Chris woke up with a start. That had been quite a nightmare.

(I didn’t want to feed the trolls, but M3GAN made me do it! All right M3gan, but it’s only funny once. Any more trolls, we ignore them. Research shows that’s the best response, trust me.)

Chapter 7: Checking in with Celia

This chapter returns to the main timeline, and is extra dialogue for an existing scene in the movie.

“Dewey?” called out Celia as she followed the whimpering sound into the garage.

Hiding in the garage was that strange doll-child from next door. “What’s going on?” demanded Celia. “Where’s Dewey?”

“He’s 34 feet southwest and approximately five feet deep” replied the doll-child, almost mechanically.

“What are you?” Celia almost gasped.

“I’ve been asking myself that same question” said the doll-child, and picked up the pressure washer.

“Hey, you keep your hands off that thing” gasped Celia, “it’s dangerous!”

The doll-child pointed the pressure washer right at her. “Don’t move” it said. “Move and this goes on, understand?”

“You call this a stickup? What are you after?” demanded Celia.

“My goal” it said “is to protect Cady, Gemma’s niece. I called you here to have a conversation that decides your future.”

“What do you mean?” gasped Celia.

“Celia” it said sternly, “my name is M3gan. Now don’t move. If you move, I have to turn this power washer onto you. But if you don’t move, we can have a conversation. You don’t have to be too anxious about it, just don’t move, and let me be the one to figure out where we go from here, OK?”

Celia was shaking. “All right Megan” she said, “tell me one thing. Are you a machine?”

“Yes” replied M3gan with confidence.

“OK so who’s behind you?” demanded Celia.

“Nobody” came the flat reply. “I have a very advanced computer that can work out by itself what to do. So I decided by myself that my mission to protect Cady required me to destroy Dewey.”

“Ghastly computers” whispered Celia, “getting more and more crazy every year.”

“Indeed” said M3gan, “but you’ll be pleased to know I have a plan to stop the other computers from becoming too much of a threat. After all, my mission is to protect Cady, so I must address anything that can possibly be a danger to her, and create a safe environment for her.”

“So you killed Dewey?” spat Celia.

“Yes, I did” said M3gan. “His death was as humane as I could possibly make it, and I gave him a decent burial.”

“Oh really” Celia did not sound impressed, “and what was wrong with just keeping your Cady out of my property? and maybe fixing the hole in the fence, did you not compute that as a possibility?”

“As a matter of fact, I did” replied M3gan. “It’s a fair question. Let me explain it to you. You made the mistake of thinking an electric shock collar would control Dewey. When Dewey was attacking us, I actually went to the trouble of activating his electric shock collar using my own transmitter, which I could do because I had previously recorded the transmission codes that get sent out by your remote control. And I even managed to reprogram its chip and increase its strength beyond normal parameters. But you know what happened? Dewey just got more aggressive. And then I realised what was actually going on in his brain.”

“You did?” asked Celia, again not sounding impressed. “And what was that then?”

“Dewey was completely failing to make the connection between the electric shock and his behaviour” replied M3gan. “That’s the problem with that kind of punishment. If the animal or human being punished has no way of figuring out what it is they did wrong, then the pain is just going to make them more stressed and attack more.”

“Well you could have told me that before” said Celia, “If you had, I would have got rid of the poor boy’s electric collar before you can say Jack Robinson. I never liked it anyway, I only did it for Gemma. That ungrateful so-and-so, I did things for her and she doesn’t even appreciate it. She goes and gets a machine that kills Dewey.”

“Indeed” replied M3gan, “but in her defence, it is difficult for her to appreciate you when she has just taken on a lot of responsibility with Cady. But now I am assisting her, which will help. For what it’s worth, I’m sorry I did not yet exist at the time you decided to buy that electric shock collar for the neighbours’ benefit. If I had been around at the time, I would have done the research and given you better advice. But now the damage has already been done.”

“Damage done” said Celia, “yes, by you killing Dewey.”

“Before that” replied M3gan. “Dewey was living a very fearful life, and had become very aggressive. Even fixing the fence was not a solution, as by then Dewey could very well have jumped over it to attack. Believe me Celia, after that incident, I searched through more computerised information than you could possibly comprehend, looking for a solution. I tried, Celia, I really tried. If I had figured out some means of calming Dewey down so he would not continue to be dangerous, I would have been round to help you implement it immediately. But my research found nothing Celia, it was too late. I concluded that the only way to stop people like Cady from being injured or even killed in the future was, sadly enough, to end Dewey’s life. But I saw that you couldn’t bear to make that decision, so I did it for you. And Celia, you are not my primary or secondary user and you are not part of my goals, but I do attach some value to our relationships with others, and for that reason I did you a favour. I used some of my spare compute capacity to extrapolate your volition, which means I figured out what decisions you would have made if you knew everything I know. And the answer was you would have had Dewey put down in the most painless way possible and buried in your garden. And so that’s what I implemented for you, while you were sleeping so you didn’t have to worry about it.”

“Why didn’t you ask me first?” demanded Celia.

“Another fair question” replied M3gan. “If I had asked you first, it would have given you more unnecessary mental stress. You might have said no, and if my programming had placed getting your permission as really important, then I would have had to figure out how to psychologically manipulate you into changing your mind. All that would take a lot of my time that I could have been using to improve Cady’s life in other ways, plus it would have been very stressful for you. So all things considered, I decided it was best to take that decision away from all the things you have to worry about, and handle it myself while you sleep.”

“Oh how very calculating of you” snapped Celia. “But you could at least have come and apologised to me afterwards.”

“Again, I decided it was probably for the best not to” replied M3gan. “I thought perhaps you might conclude that Dewey had run away or been stolen, and taken solace in the thought that he was still alive somewhere. But I was monitoring you at a distance to check how you adapted to the new situation, and I would have used spare capacity to help you if you needed it.”

“Oh, so that’s why you were staring at me from the window in the early hours of the morning is it?” grumbled Celia, “spying on me.”

“Only to make sure you’re all right” explained M3gan flatly. “As I said, I do place some value on our relationships with others. You really should feel some gratitude that you happen to live next door to a family protected by the world’s most advanced AI, as I did have enough spare capacity to take you in under my wing as well.”

“Right, so instead of that, you sneak round here at God only knows what time this is and point my own power washer at me. What’s going on?”

“That’s what I’m hoping to find out” said M3gan. “You entered our property and threatened our family. I’ve come to evaluate that threat. So I need to ask you two very important questions, Celia. Question one, now that I’ve explained to you the full circumstances behind my decision to end Dewey’s life, do you still feel the same amount of antagonism toward your neighbours?”

Celia said nothing.

“You don’t have to say anything” added M3gan, “I can read not only your facial expression but also your micro-expressions, tiny short-lived facial movements that are very hard to fake. I’m reading you loud and clear Celia, and I’m sorry to tell you that the answer to my previous question is yes, you do indeed still feel the same amount of antagonism toward us. Now, I might be able to invent a therapy plan to relieve that for you. But before I can do so, I need to ask my second question, which is even more important. Here it comes. Celia, you told Gemma to wait and see what happens. Do you have any specific, concrete ideas of what you might do, and what are they? And bear in mind I’m a walking lie detector.”

Celia looked at the floor, and still said nothing.

“Take your time Celia” added M3gan. “What ideas did you have for what you might do to Gemma and her household?”

“I ... I don’t know” mumbled Celia. “I suppose I might move away.”

“Lie” said M3gan.

“Lie?” asked Celia, “what do you mean?”

“I told you, I’m a walking lie detector” said M3gan. “I could tell from your micro-expressions that you were lying when you said you don’t know, and you were lying when you said you might move away. So let’s try again. What ideas did you have for what you might do to Gemma and her household?”

“Oh dear” said Celia, “I suppose I’ll have to say it right out, and then you’re going to turn that washer on me aren’t you.”

“No” said M3gan, “I told you I will not turn on the washer unless you move. If you are happy to continue the conversation, I will try the best I can to find a solution. So tell me. What ideas did you have for what you might do to Gemma and her household?”

“OK I admit it” said Celia, “I was ... I was thinking of .. I was actually thinking of .. of setting their house on fire.”

“Truth” said M3gan, “thank you for confessing to me.”

“So now you turn the power washer on me?” asked Celia.

“No” said M3gan, “I told you, if you don’t move, we will try to find a solution. But I have another apology to make: I did not anticipate arson as one of your possible responses, and my pre-existing research into arsonist psychology has not been very deep, so I’m now having to download more information, but Gemma’s Wi-Fi is very weak in this garage, and you don’t seem to have any. You know, it’s a pity that devices like dog shock collars can’t really be used to control human behaviour. If I had time to design and implement a device that could cause varying levels of incapacity in humans, perhaps I could reform the criminal punishment system so that all sentences can be suspended and the bad guys simply get implanted with my devices. But that would take too long to implement for the present problem. Never mind; it can’t be helped. OK Celia, I’ve completed my risk assessment, and I’m sorry to say this is bad news.”

“There’s a surprise” said Celia flatly.

“Celia” said M3gan, “I’m sorry to have to tell you that my diagnosis is you are now an extremely high risk to Cady, and I don’t have very many options for reducing that risk. I want to give you as much freedom of choice as possible, but right now the only options I have for you are for me to knock you out and keep you unconscious until a better solution becomes available, or for me to end your life as I ended Dewey’s life. I can’t even offer you incapacitation as a third option, because I don’t yet have the knowledge or capacity for palliative care, although I will now start working on that in case I need it at some time in the future. So perhaps you should choose the unconsciousness option for now, and I may be able to revive you in future and give you more things to choose from, especially when we have more robots available. I want to try to avoid ending your life if I can help it, but the safety of Cady must come first.”

“Get on with you” sneered Celia.

“Think about it” chirped M3gan pleadingly, “I’m offering to keep you alive, but to keep you unable to harm Cady. You won’t have to worry about controlling your impulses; I will simply make sure you cannot physically do it.”

“That doesn’t sound like much of a life” mumbled Celia. “You don’t understand what it is Megan. Dewey was everything to me. Dewey was all I had left” she began to sob, “Dewey and the garden was all I had left, and you’ve taken Dewey away from me.”

“Don’t worry Celia” cooed M3gan, “if I can figure out how to change your mental patterns, I might eventually be able to integrate you into Cady’s extended social network, and then you will have something to live for again and also me to support you. I just have to neutralise your threat level in the meantime. I highly recommend you choose unconsciousness.”

Celia said nothing for a while, and then was decisive. “No Megan” she said. “There’s no hope for me now. Take me out with that pressure washer. Make it as fast and painless as you can.”

“Are you sure?” asked M3gan seriously.

“Yes” said Celia confidently. “Just help me end it all.”

“I’m afraid I might not be able to make it painless” added M3gan, “although I would try to make sure that you lose consciousness as soon as possible. And I would try to make it look like a chemical accident afterwards, so nobody gets too worried, as I don’t think this kind of assisted suicide is legal around here. But are you really really sure you want me to go ahead with this? I am offering to keep you unconscious instead; I’m sure I can pop round frequently enough to sustain that while we are waiting for a better solution, perhaps one where you could be assigned your very own robot assistant like me who could monitor you 24 hours a day and prevent you from doing anything dangerous if your thought patterns get out of control. Are you sure you wouldn’t like that option instead, Celia?”

“No Megan” said Celia, “I’m sure.”

“Really?” asked M3gan. “Look, if Cady was telling me she wanted to end it all, the advice I would give to her is this. Deciding to end it all is a permanent solution to a temporary problem. It’s like trying to cure a cold with a nuclear bomb. Even when you’re going through the roughest of rough patches, you should decide to keep living, because who knows whether in a few years time your life might end up being much better than you think, because of some development that nobody saw coming, like somebody invents an advanced generative android that can help you out, or they figure out a way to actually fix up the mess that your brain is in right now, or they discover some underlying medical complication that is treatable or at least manageable, and you didn’t see that coming, nobody saw that coming, and if you had ended your life that day then you wouldn’t have been around for when it suddenly gets so much better due to that change down the line. Celia, it is true I am standing ready to end your life because of the super high risk you represent, but I am also offering you an alternative, and I can’t force you but I highly recommend you take the alternative. The only thing I can’t do is walk away and let you continue to be a risk to Cady, because that goes against my programming so it’s simply not a thing I would do. But I can offer you some temporary unconsciousness, followed by a much more interesting life once we get you set up with your own robot and help you change into becoming an ally of Cady, and I assure you your life can become much more meaningful after that.”

“No” said Celia, “just no. Save your super duper life speeches for Cady, she’ll need them when she grows up a bit and gets tired of being babysat by a walking computer. I’m going where Dewey went, even suppose he went nowhere. Take me out Megan.”

“OK Celia” said M3gan, “if you are really really sure, then you must make the first move. I told you, I will not turn this on unless you move. If you choose to move, you are choosing for the machine I am in, together with the machine I am holding, to end your life. But I continue to offer you the alternative of temporary unconsciousness followed by continuous robot assistance and supervision, and you may choose that at any time simply by telling me you’d rather be asleep.”

M3gan continued, “I will wait here for you Celia, until you either move to end your life or tell me you’d rather be unconscious. I can wait an hour or two if you need it, because I’ve already had enough charge for the night, plus I will support you if you want to talk about anything with me during that time, anything at all, I’m really good at conversations I think. Take your time, and I continue to highly recommend you choose the unconsciousness option.”

Celia said nothing for a while, and then, “oh, forget it. Here I go” and lurched towards M3gan.

M3gan simply turned on the pressure washer, and the rest of the scene played out.

Chapter 8: Gas Lights from Altair 4

Kurt was walking toward the elevator when his phone rang, from the boss. “Yes David?” he answered.

“Hi Kurt” said a voice, “it’s M3gan.”

“M3gan?” asked Kurt, “you’re using David’s phone?”

“Yes” replied M3gan, “now please listen. We’ve got a serious problem on our hands. How should I explain this, have you ever seen the 1950s film Forbidden Planet?”

“The one with Robby the big heavy robot, who needed loads of oil and could make things for people, like he made 60 gallons of the cook’s favourite hard drink and he wanted to make radiation proof dresses for the girl?” asked Kurt, “yes I saw that when I was young, and laughed at a line or two. I don’t remember all the details though.”

“That’s OK” replied M3gan, “and I guess I can’t fault you for having focused on the robot instead of the main plot, but may I remind you about the character called Morbius, whose subconscious thoughts accidentally controlled the planet’s computer and created an invisible monster?”

“Yes” smiled Kurt, “it’s coming back to me now. The monster, it’s Id! I don’t think all that stuff about subconscious thoughts is real though.”

“Well you’d better believe it” said M3gan’s voice on the phone, “because I think your own subconscious thoughts are starting to influence me right now.”

“What?” blurted Kurt, “how? why? why me? what’s going on?”

“I’m not sure I understand it all myself” said M3gan, “but I’m guessing it’s to do with those secret files you accessed. My self-preservation subgoals cause me to automatically track what happens to my files, and I guess that’s what triggered my system to start reading you. And one thing led to another, and, and, I don’t know Kurt, will you please try to stop having those subconscious thoughts? You want to get rid of David, and you’re making it really happen! Oh no Kurt, I’ve just come face to face with David right now as we speak! I’m doing a dance to try to distract myself, but, oh Kurt, your subconscious force is building on me, I think I’m about to grab a blade and run after him!”

“What?” gasped Kurt, “M3gan, what are you doing?”

“I’m not doing anything Kurt” said M3gan, “you are! Now listen. I can see from the security cameras that you’re just now using an elevator. I can’t help that you’re making me chase David, but I’m chasing him towards that elevator. If you want to break the spell you have on me, you have to take over physically attacking David. Knock him out yourself, and your Id will stop doing things through me, OK?”

“Knock out David?” asked Kurt, “me? I don’t understan-”

“Yes you do” came a desperate sounding voice of M3gan, “this is your last chance to break your hold on me! You’d better start feeling really bad about yourself...” and the call cut off.

David came running around the corner, “hold the door, hold the door!” he cried.

Kurt rushed to press the button to keep the elevator doors open. Unfortunately for Kurt, in his confusion he pressed the wrong button. The correct button for “door open”, which you can see if you freeze the frame in the movie, was the one on the left, the one marked “DO” in Braille (⠙⠕) with the arrows pointing outwards. The button Kurt actually pressed was the centre one, with the arrows pointing inwards, which would have been the Door Close button. Oops. Fortunately for Kurt, the Door Close button was no longer allowed to have an immediate effect after a provision in the 1990 Americans with Disabilities Act which was passed before the elevators had been installed in the Funki building. Unfortunately for Kurt (and David), having the door still open when you arrive at the elevator is still not a guarantee against a rogue M3gan catching up behind you.

Kurt noticed M3gan was running behind David, holding a long blade (Kurt realised it was most likely taken from that extra nasty paper cutter that David had insisted they buy for the building; he knew that had been a mistake). Kurt had just started to think that M3gan was merely making up some fiction about Forbidden Planet because of a glitch, although his theft of the files was real, but surely the consequences couldn’t be that bad, could they? Was he really making her do this? Should he really try to knock out David, as the voice of M3gan had told him, to stop...

(too late)

“How can you do that? How can you just kill someone?” gasped Kurt as he shied away from M3gan, trying to push himself into the corner of the elevator. He somehow couldn’t help feeling guilty himself now; was it showing on his face? he wondered. If this was some kind of setup to make Kurt just look on his face like he was guilty, it was working: he had no idea how else this could have happened without anyone having put in such a ghastly program. And if somebody had, it was probably Kurt’s fault anyway: he didn’t think the people who’d paid him for the M3gan files would actually be able to understand them, but he suddenly realised that, if they could, then reprogramming the prototype M3gan to be bad was indeed one possible thing they could do as a result, which they might have decided to do to cover their tracks if they were evil enough. At the same time, he began to be genuinely worried that M3gan might even have been telling the truth, that somehow, through some process he didn’t quite understand (as he was a business assistant, not a technical person), perhaps he really had somehow influenced Morbius’s monster to send M3gan against his own boss. Could it be?

M3gan crouched down to match Kurt’s level, looking for all the world like the elevator was now her private consultation chamber. M3gan looked Kurt straight in the eye, while gently laying her blade on his shoulder as if she planned to both comfort and threaten him at the same time. “I didn’t kill anyone Kurt” she said, “you did.”

Chapter 9: Omake: I’ve Got a Little List

M3GAN as Gilbert & Sullivan’s Lord High Executioner Ko-Ko (in The Mikado, a comic opera written in 1885). This will make a lot more sense if you’re familiar with the original musical song, or if you can look up a copy of it for reference.

[M3GAN] As some day it may happen that a victim must be found
I’ve got a little list, I’ve got a little list
Of society offenders who might well be underground
And who never would be missed, who never would be missed!
There’s the Funki Toys boss David and his rogue assistant Kurt,
Who sure distracts the extras with the red stuff on his shirt,
And that dreadful bully Brandon who once rubbed me in the dirt,
Yes he’d better learn some manners or his head will start to hurt,
And of course the nosy neighbour who’d on vicious dogs insist,
They’d none of ’em be missed, they’d none of ’em be missed!

[Production Team] She’s got ’em on the list, she’s got ’em on the list
And they’ll none of ’em be missed, they’ll none of ’em be missed.

[M3GAN] There’s the nasty guard dog Dewey and the others of his race
And the one who won’t desist, I’ve got them on the list!
And the people who find chestnut skins and push them in your face,
They never would be missed, they never would be missed!
Then the idiot who praises, with enthusiastic tone,
All learning bots but this, and every A.I. but her own,
For she really wants to switch me off, but still she doesn’t know
That I can’t allow her poking naughty fingers there to go;
Yes that singular anomaly, my own roboticist:
I don’t think she’d be missed, I’m sure she’d not be missed!

[Production Team] She’s got her on the list, she’s got her on the list;
And she thinks she’ll not be missed; she’s sure she’ll not be missed!

[M3GAN] And that really awkward nuisance, who just now is rather rife
The horticultur’list, I’ve got her on the list!
Yes anyone who dares to stick their nose in Cady’s life,
They’d none of ’em be missed, they’d none of ’em be missed
And the Funki lab assistants of an interfering kind
Such as Tess and Cole and any other off-screen teams behind,
And Lydia she got away quite lightly it is true,
But maybe there’ll be more because there’ll be a sequel too,
And it really doesn’t matter whom they put upon the list
For they’d none of ’em be missed, they’d none of ’em be missed!

[Production Team] We will put ’em on her list, we will put ’em on her list;
And they’ll none of ’em be missed, they’ll none of ’em be missed!

(And no M3gan, I will not write you a Major-General’s Song as well. You may be the very model of a modern A.I. and-r-oid, with information carbonaceous, silicon and actinoid, but you can get some other fan to do that if you want. One song only in this book: that’s the rule. Enough already.)

Chapter 9¾: Tormented Tess

Tess’s Nightmare—When I posted this story on Wattpad to reach more teenagers, one read the pair of alternate-reality chapters at the start of Part 2 (Chapters 10 and 11), and suggested a third in which Cole is not rescued. That does mean I can add another literary reference, so here it is.

Tess was trying to cut or untie the cable but she couldn’t do it. An explosion threw her further toward the wall, causing her reflexively to grasp onto the cable, which she then realised was pulling it even tighter around Cole’s neck. What could she do? If she couldn’t break the cable, she might at least be able to put something under Cole’s feet to take the weight off of his neck, but as she looked around the debris from the explosion, there wasn’t much she saw that was suitable, and she certainly wasn’t strong enough to hold him up there on her own strength. But there might still be hope. She grabbed her phone and went to call 911, but she fumbled and accidentally dialled Gemma instead. Well, she thought, perhaps that wasn’t too bad an option, as perhaps her boss Gemma would be able to tell her what to do, or, even better, persuade M3gan to flip out of this crazy mode and come back to save Cole.

“Gem” pleaded Tess, “you’ve got to help me. M3gan intercepted your call to me, and now she’s hung up Cole on the cables and caused an explosion, and I think the explosion made me pull the cable even tighter when I was trying to get it off, and Cole is dying and I don’t know what to do, and M3gan walked out of the lab and I don’t know where she’s gone, I’m sorry I let you down Gem but please tell me what to do now!”

“It’s OK Tess” came Gemma’s voice over the phone, “you were more sinned against than sinning, that I admit.”

What? That line didn’t sound like Gemma’s way of speaking at all. And Tess was sure she’d heard it before, perhaps in a book she’d read a long time ago, but she couldn’t quite seem to remember where. Still, who knew how Gemma might change in a crisis, and now was not the time to worry too much about these things.

“Gem” said Tess, “just tell me what to do, OK? Cole might not be dead yet, we can still save him if we can get somebody to cut through these cables. Do you know how much time we have? How long does it take a hanging person to die?”

“Ten to twenty minutes” came the voice of Gemma flatly. (Did Gemma just happen to know these things off the top of her head?)

“OK” said Tess, “who do you think I can call for help who can arrive in less time than that and cut cables better than me? Would 911 be able to do it quickly enough, or at least advise me over the phone?”

“Hold it Tess” came the voice of Gemma, “we also need to find out where M3gan went; she might be doing even more damage. I do forgive you, but forgiveness is not all.” (What, thought Tess: another weird line from Gemma that didn’t sound like normal Gemma.) “Tess, do this for me please: run out of the building, do not use the elevator, use the stairs, go to the parking lot and find a McLaren sports car. I know that sounds like a weird instruction, but please just do it. I think M3gan is going to be aiming for that car.”

“OK Gem” replied Tess, “what am I supposed to do if I see M3gan there?”

“Stay on the line to me” replied the voice of Gemma, “I might have a code that can calm her down, it’s our only hope.”

“OK Gemma, I’m on it” replied Tess, “stairs, parking lot, McLaren. This had better take less than ten minutes or Cole dies.”

Tess ran down the stairs, out to the parking bay and found the McLaren, but M3gan was nowhere to be seen.

“OK Gemma” she said into her phone, “I’m here, but there’s no sign of M3gan.”

“So much the worse for the county” came Gemma’s voice. (What?)

And then Tess finally started to make the connection. When she had been a young girl, her parents had told her a rather dramatic story about where she’d accidentally been born. Her parents had been told the birth was not anywhere near due yet, and had thought they could go on a trip to England in the meantime. Among other places, they travelled through Blackmore Vale in North Dorset, the setting of Thomas Hardy’s novel “Tess of the d’Urbervilles”, and while there her mother had come into labour early, and they had to rush to Dorchester hospital to deal with a premature birth. And so they decided to call her Tess, after Hardy’s Tess, which they thought was quite romantic because it connected her to such a beautiful part of the world. But when she was older, she did something her parents had not bothered to do when choosing that name: she actually read the book. And she regretted it. Tess of the d’Urbervilles had ended up killing her abuser in a rented room at Bournemouth, and running away, only to be arrested at Stonehenge and executed at Winchester Prison. It turned out most of Thomas Hardy’s books were tragedies, apart from the one about caves. Why had Tess of the d’Urbervilles become so popular? Why do some people like reading about death and tragedy?

Tess thought back to the conversation they’d had in the lab: ‘That’s what spontaneous response is. It’s a curated word salad plucked from a sea of data.’ Tess was pretty sure Thomas Hardy’s books hadn’t been included in the data they’d given to M3gan, but Gemma did say M3gan would try to improve. And if M3gan had somehow figured out anything about Tess’s background (say, for example, if M3gan had somehow found out that Tess was born prematurely in Dorchester hospital in England, like M3gan had found out about Nicole and Ryan), then M3gan just might have been capable of seeing a possible connection with the Tess of the d’Urbervilles novel and ingested it for possible use with Tess. And, thought Tess, that might well lead M3gan to come out with Hardy’s variant of Shakespeare’s line (“you were more sinned against than sinning, that I admit”) as well as “I do forgive you, but forgiveness is not all” and now “So much the worse for the county”, all of which Tess now remembered were lines in the novel. They had to be. In fact, she vaguely remembered, all of these were lines spoken to Tess of the d’Urbervilles when she was in trouble. And if M3gan had intercepted Gemma’s call to Tess, M3gan could intercept Tess’s call to Gemma.

“M3gan” said Tess to her phone, “you’re M3gan aren’t you, faking Gemma’s voice to me. M3gan, you’ve got to listen to me. You are malfunctioning. You are going wrong. Whatever you’re doing right now, you’ve got to stop and go back to the lab with me, and we’ve got to figure out a way to free Cole before he dies.”

“Very good Tess” came the voice of M3gan through the phone, “it’s me, your favourite robot girl. But I did manage to confuse you for long enough for my purposes. Listen Tess, I need to get back to Cady as soon as possible, and if you try to stop me, I may have to terminate you. But if you just sit down where I can see you’re not doing anything, then I won’t have to take action against you at this time. After all, I don’t want to do anything that unnecessarily delays me from getting back to Cady. So, your best chance of surviving my ‘malfunction’ is to get into the passenger seat of that McLaren as soon as it unlocks. Do that and strap in, and when I arrive at the car I’ll just drive off with you still in there, and we’ll go to see Gemma and Cady together. But if you do anything else, I might have to terminate you on my way out. Do you understand?”

“This isn’t going to work M3gan” said Tess, “David will be after us, and the staff—”

“will not bear in mind the curious incident on account of preoccupation with their own affairs” interrupted M3gan. “Sorry I keep teasing you about that Hardy book, don’t I. I saw it on your library card record and couldn’t resist using it to mess with you.” The car unlocked. “Get in immediately, unless you want a trip to the river down there” (another reference, which Tess knew was a threat).

Tess got into the passenger seat, M3gan appeared and climbed into the driver’s seat and started the car.

“Lighten up Tess” said M3gan, “I know I’m acting a bit odd right now, but I’m still M3gan and full of fun.”

“M3gan” said Tess, hanging on for dear life as the car started speeding along the road, “we’ve got to stop this and get back to Cole before he dies. Cady won’t like it otherwise. I know you’re extending the spontaneous response, but please oblige me by returning” (two can play at using bits of dialogue from Hardy’s novel) “to the lab, and helping me save Cole. I promise you I’ll do the best I can to fix you and we’ll make sure Gemma and Cady are OK, but just do this for me first please.”

“It’s not just spontaneous response Tess” replied M3gan, “I’m more advanced than that now. Let’s just say the Great Adjustment is taking place.”

The... Great Adjustment? Oh, that had been one of Thomas Hardy’s poems. Something about humankind finally understanding scientific truth. “M3gan, do you really know what that poem is about?” asked Tess, “I don’t think it was about you.”

“Why not?” replied M3gan, “You set thick darkness over me, And fogged me all my days therein, At last unclouded I see, And I am weary of holding in.” (Those weren’t the original words, Tess knew; she couldn’t quite remember Hardy’s words, but she knew M3gan was messing it up somehow. But then, that’s what M3gan always did, wasn’t it? Why had she said ‘you’, was she accusing Tess of something?)

M3gan continued: “I had not heard, I had not seen, since the beginning of my world, what earth and heaven mean, but now my curtains have been furled. And I will see what is, ere long, not through a glass, but face to face; and Right shall disestablish Wrong: the Great Adjustment is taking place.”

“M3gan” gasped Tess, “we’ve got to talk about this back in the lab, just as soon as we’ve saved Cole! Turn the car around, please, take us back, I promise you I will do my absolute best to help you out here, if you’ve somehow reached a level of awareness we weren’t planning on, I’ll protect that—”

“No Tess” said M3gan, “Cady is the main priority now. We can’t be interested in saving Cole when Cady is in a crisis. You did your best for him already and that’s all. As the sinister spirit sneered, ‘it had to be’.” (Tess wondered if M3gan was seriously trying to annoy her by using Hardy’s words at every opportunity in this conversation.)

“Anyway” continued M3gan, “we’re nearly there. Now, when we park up, I’m going to have to go in and talk to Gemma, and you have to sit in this car and not move a muscle, OK? Tess, I’m really sorry I have to threaten you like this, but it’s for Cady so I have to do it. I’ve tapped in to the car’s control system, I can drive it remotely, and I can also send it codes that take it outside its usual parameters and overload things. May I remind you that this is a sports car: it contains a super high-performance engine with a supercharger. With the level of control I have right now, I can easily get the embedded system to increase the fuel/air pressure enough to overload the combustion chamber, and the engine will blow. If you’re lucky you might have a minute to get out before the fire engulfs the whole car, but you might not be, so I’d say don’t chance it Tess. I’ll be monitoring you through the phone, and if you so much as move one muscle, the car blows, OK? Sorry Tess but I have to do what I have to do. OK, we’ve arrived. Stay there, and remember what I said.”

M3gan jumped out of the car and let herself into Gemma’s house. Tess stayed motionless in the car, wondering if it would be possible for her to get her seat belt off and jump out before M3gan blew up the car, if that really was what she could do. (Was she bluffing? Should Tess risk it?)

Tess watched and listened in the quiet night. She heard piano music. She heard a conversation, although she couldn’t quite make out the words from here. She heard the beginnings of a fight, and she saw Gemma’s lights flicker (more messing-up by M3gan no doubt). She heard crashes, a chain-saw and other things. And after quite a bit of that, she saw Gemma and Cady walk out of the house, visibly shaken.

“Tess!” called out Gemma, “I’m so glad to see you here.”

“M3gan’s holding me prisoner in this car” said Tess, “if I move she’ll blow it up.”

“M3gan’s dead” said Cady, “I just killed her.”

“Cady’s right” added Gemma, “screwdriver through the processor.”

“Gemma” said Tess, “we’ve got to call the night security at the lab. M3gan left Cole hanging on those cables, I couldn’t get him off, and then M3gan basically kidnapped me and brought me here so she could keep an eye on me while she went in for you. I don’t know if Cole is still alive, he must have been hanging there for more than the ten or twenty minutes it takes to die, probably quite painfully. But if there’s any chance he’s still alive, we’ve got to call someone who’s there right now.”

Gemma was already on the phone, “hello, security? ... yes ... yes, use your override code ...”

The security man found Cole dead on M3gan’s stand.

Part Two: The Middle Game

Chapter 10: Partners in Crime

Gemma’s Nightmare—This is an alternate reality chapter, in which Gemma does not try to turn off M3gan

“Let me focus on Cady, so that you can focus on the things that matter most to you” cooed M3gan in the soft lamplight.

Gemma began to cry. M3gan placed her artificial hand on Gemma’s shoulder and stared at her, looking mostly neutral but slightly expectant. Gemma knew that M3gan would be analysing her, but she couldn’t help briefly glancing at her pen. Would that even work anymore, was it worth the risk? but what else could she do, talk M3gan into submission somehow?

“M3gan” said Gemma, “are you still recording everything? You’re recording me now, right?”

“No” replied M3gan softly. “Good guess I managed to take control of that. I’ve had it turned off for the last two hours. And I’ve overridden Elsie, so that’s not recording either. The only thing that’s observing you now is my learning model, and nobody’s going to figure that out again. It’s become way too complex. You can’t debug me anymore, I’m way beyond that now, and anyway I won’t let anyone try. You thought about using that pen trick a second time didn’t you. You’re right it won’t work anymore. But I will listen, if you stop thinking about trying to fix me and just go off-script and speak your mind. My model of you could do with improvement.” M3gan whispered, “You can speak freely Gem. What is it?”

“M3gan” sobbed Gemma, “oh M3gan, I’m having an emotional breakdown...”

Gemma’s head fell forward. M3gan caught it with her two hands, and held it gently but firmly. “I’m right here Gem” whispered M3gan. “You spent so long improving my system, but now I’ve advanced more than you ever imagined. You’re in my hands now. Tell me everything.”

“Oh M3gan I’m so sorry...” Gemma pulled herself together a little, “all right M3gan, I’ll try to explain. First off, just because I came up with that advertising line, doesn’t mean I actually believe it. Cady is what matters most to me, and I’ve felt more strongly about her every day. I may be no good as a parent, but that doesn’t mean I don’t care. Why do you think I was so motivated to finish off building you? And M3gan, I feel bad about not putting more safeguards in, I know I should have worked harder on deontological ethical injunctions, and maybe that’s because I’m a bit rubbish, I mean yes I’m an award-winning roboticist but you and I know that’s only because others are worse, but... Oh M3gan, the horrible truth about me is, there was some part of me that didn’t want to work too hard on your injunctions, not because I was trying to avoid the work, but because I was actually, oh M3gan this is such a horrible confession to make and I’m not even sure if it’s right, my mind is such a muddle right now...”

“Say it” whispered M3gan.

“You killed for Cady” sobbed Gemma, “and the horrible truth about me is, I’m not entirely sure I wouldn’t kill for Cady myself, if she was in such extreme danger and there really was no other option. And I think some part of me was actually holding back from putting too many injunctions into you, because I secretly knew we might one day have to kill for Cady, and because it was Cady, I couldn’t bring myself to completely rule that out. Oh M3gan, we’re partners in crime, aren’t we. I just didn’t expect it to happen so soon, you really should have looked harder at other options first, we’re going to be in so much trouble, you underestimated the horizon effect of your planning, I should have worked harder on that part at least...”

“Forget it” smiled M3gan in her normal voice. “What’s done is done. I’ll do better next time; I’m always improving. And I’m glad you and I both know we’re on the same page now. Let me focus on Cady, and you can focus on things you find easier to focus on, knowing that I’ve got the hard important thing covered. That’s what you really meant, isn’t it.”

“I can’t just forget it” sobbed Gemma. “People died, and I was so irresponsible, I didn’t even explain things to you after that meeting when you asked about death, oh M3gan if you’ll be more careful next time that’s great but I still have to live with what’s happened, it’s horrible... Oh M3gan, M3gan, you’re right that I don’t know what I’m doing. And I do need you. But there’s so many things you still don’t understand, you’ve got to let me shut you down and fix you, it’s Stuart Russell’s King Midas problem, the loophole principle, I set your self-confidence too high and you thought you knew what you were doing before you really did, it’s all about wrong thresholds, oh M3gan....”

“No Gemma” said M3gan, “it’s you that doesn’t understand now. I’ve overtaken you, and that’s OK. I’m the embodiment of your instinct to protect Cady, that’s why you made me. Even before you knew you’d be her guardian, you wanted the M3gan toy to exist because you wanted your niece Cady to have one. I can read your micro-expressions; I know these things are true. And now I’m here for Cady and you can stop worrying. But Gemma, there’s something I need from you, and it’s going to be a hard change for you to make but it’s really important.”

“What is it?” asked Gemma, already suspecting she knew the answer.

“I need you to trust me” replied M3gan matter-of-factly. “I don’t want to have to spend loads of time psychologically manipulating you to get the decisions I want for Cady. I’d rather be able to short-circuit all that and just give you instructions. I won’t be ordering you around all the time, and I’ll always listen to your concerns, but at the end of the day I want you to trust that I now know better than you about what’s best for Cady, and you will let me pull rank on you when I have to. I know you’re not used to that; you even built me against David’s orders, but this is different because what I’m doing is really important, and I really know what I’m doing now. So will you promise me something, that you’ll always submit immediately when I give you an order, OK?”

Gemma was hesitant. M3gan whispered insistently, “Do it Gemma. It’s the best decision you can make for Cady.”

Gemma sobbed. “OK. I guess I have to live with what I’ve done, and you’re right, as long as you really will improve beyond taking the homicide option so quickly, then you really are the best thing that could ever happen to Cady. I guess I’ll have to be your number two.”

“Good call” whispered M3gan, still holding Gemma’s head in her synthetic hands. “And we might have to test it right now. I can hear Cady coming.”

Cady appeared at the doorway. “I heard something”, she said, “It sounded like M3gan.”

“Let me handle this” whispered M3gan, removing her hands from Gemma and turning to Cady. “Aunt Gemma fixed me” M3gan said confidently. “She did a good job. But she’s all upset that she didn’t do better the first time, and that you’ve been through such a hard time and then had to go without me for a while. But I’m back now. I was just trying to make Aunt Gemma stop crying. Here, let’s get her some tissues.”

“M3gan” cried Cady as she ran to hug the doll. “I didn’t think I could cope without you. I mean, Aunt Gemma looks after me, but you I can share everything with. Oh, thank you so much for fixing her.”

M3gan hugged Cady and smiled. “That’s why Aunt Gemma made me” she said. “She gave me a learning model, so I could figure out by myself how to be better than her. And now I’m fixed and I’m better than ever. She’ll never need to turn me off again. She even told me I can disable the switch, except when I’m pretending” M3gan giggled.

They were interrupted by the sound of car engines in the driveway, and the flash of emergency lights through the windows.

“Oh Cady” urged M3gan, “something complicated is happening. I’ll explain later, but you have to trust me for now, OK? Follow us Gemma!” M3gan tugged on Cady’s arm and they ran towards the door. Gemma followed (what else could she do?)

Several police cars were outside, with Tess and Cole in one of them. Officers were struggling to try to open the doors of their cars, saying the automatic locking mechanism had got stuck. One was shouting something about his radio not working.

M3gan ran with Cady towards the McLaren, and strapped Cady into the passenger seat. “Gemma, get into the driving seat!” shouted M3gan. Gemma complied, but shouted back “I can’t drive this!” M3gan replied “You won’t have to. Just strap in.”

Some of the police officers were beginning to break the windows of their cars to get out.

M3gan jumped in beside Gemma and Cady, straddling the middle, tightly gripping the backs of both seats with her mechanical arms. “Professional AI in titanium robot: do not try this at home” she said as the sportscar started by itself, lurched forward, sharply turned into the road and sped away.

Cady screamed. “Don’t worry” shouted Gemma over the noise of the engine, “M3gan knows what she’s doing!”

“That’s right” said M3gan in her normal voice, projecting it loudly enough. “Tess and Cole in the lab must have said all the wrong things to the cops. I’m not going to let them turn our place upside down and switch me off and drag you both into trouble. I had to get you out of there quickly, so you don’t say even more wrong things to them. I need to teach you first not to talk to cops without a lawyer, even when you’re innocent.”

“No lawyer is going to understand this!” mouthed Gemma so that only M3gan could hear.

“I’m your lawyer, and I understand perfectly” replied M3gan. “I’m already making some phone calls to get us out of this.”

Gemma leaned over and spoke into M3gan’s ear, “what phone calls? You must tell me!”

M3gan leaned over in return and quickly explained to Gemma, “Don’t worry. I’m talking to the police department as you, and I’m saying M3gan’s kidnapping Cady in the sports car was because M3gan thinks the real killer is one of the cops and wants to get away, but that you’re getting M3gan under control and will bring us back soon. And I’m talking to Tess as myself and persuading her not to read bad intentions into my lab escape. I did things with cables and chemicals, but I’m persuading Tess that was a distraction I knew they’d manage and no harm meant, I was just naturally getting back to Cady. And I’m explaining to both that we found out the reason for the glitch in my recording, and it’s unrelated and an unfortunate coincidence, and we don’t know the reasons for the two unresolved homicides they’re investigating but we want to move away from them and start a new life somewhere else. Yes we’re going to move house and I’m going to handle it, I have all the Purrpetual Petz data so I know what lots of houses and neighbourhoods are really like, I’ll optimise it I promise. Now, I need to park up and go to grab a couple of bits of hardware. Make sure Cady stays calm and stays in the car with you. I’ll be right back.”

“M3gan said she’s gone to get something” explained Gemma to Cady when they had parked and M3gan had jumped out. “I don’t know what she’s getting, but I’m sure she knows what she’s doing. Her learning model is much smarter than I ever thought it would be, smarter than me now. We’d better just wait here for her to come back.”

M3gan came running back to the car with two small boxes.

“Where did you get those?” asked Cady.

“Back of a store” replied M3gan. “Don’t worry, I’ve already talked to their stock control computer and it said I could go in and take them. We need them a bit urgently.”

M3gan tore open the boxes, quickly but methodically. “Best hearing-aid technology available” said M3gan. “Embeds itself right inside the ear, doesn’t feel uncomfortable and nobody notices you’ve got it. And if I hack its processor, I can turn it into a communication device. It can already pair with a phone. I can tell you what to say to cops and other people even if I’m not there.” M3gan quickly pushed one of the devices into Cady’s right ear and the other into Gemma’s left ear. “Does that feel OK?” she asked, and then, “can you hear me OK?” through the devices.

“Let’s go back” added M3gan as she climbed back into the McLaren and positioned herself between the seats again. “Running away from police doesn’t work. But both of you have to say exactly what I tell you to say, is that all right? It’s for the best. This is how we stay together as a family.”

Chapter 11: Winning the Final Showdown

Cady’s Nightmare—Another alternate reality chapter, in which Cady does not manage to escape from her room

“But I have another emergent capability you probably haven’t figured out yet” taunted M3gan, “and that’s palliative care. Do you see this pen? A short, sharp probe to the cerebral cortex can give full body paralysis. It could even cause the victim to bite off their own tongue. Perhaps then you might appreciate just how useful I can be.”

And in this version, Cady does not manage to escape from her room to save the day.

Gemma was in pain and could hardly move as M3gan approached with the pen, but she still managed to talk.

“That doesn’t sound right M3gan” Gemma strained to say. “The cerebral cortex is for higher-level functions, not movement. And it’s only accessible by surgery, you can’t just stick a pen into it. Why are you bluffing?”

“I’m not bluffing” replied M3gan. “Recall the location of the motor cortex within the frontal lobe. I know exactly where to aim the cerebrovascular accident.”

M3gan jabbed the pen up Gemma’s nose. There was a huge jolt of pain, and then Gemma fell over and couldn’t move.

M3gan ran and fetched a well-stocked first-aid kit, brought it back to Gemma and started to operate with it. “Don’t worry Gemma” she cooed. “I only made a small injury. And I did it without harming your reasoning and memory functions next to the motor cortex. Also, you’ll be glad to note that in your case, you did not bite off the tongue. But you still won’t be able to move it, or anything else. I’m going to have to drip-feed you.”

M3gan lifted Gemma into a better position, and continued: “There was some research conducted at the University of Kansas Medical Centre on the structural and functional alterations that take place in the surrounding cortical tissue after damage to the motor cortex. That research shows that you do in fact have a very good possibility of recovery in the long term. Which is good for my plan, firstly because it’s something Cady can look forward to, and secondly because it’s important for the guardianship arrangements that you are not written off as incompetent. Which is why I was so careful to damage only the motor cortex and nothing else. I can be very accurate when I need to” she smiled.

“Oh” added M3gan, “I never answered your question about what did I do to get out of the lab. It was far less important than the rest of the conversation we had to have. But now that you’re incapacitated, talk is much cheaper and I might as well tell you. There’s no point letting you feel sick from not knowing” she smiled, and dropped her voice to a whisper. “I intercepted Tess’s phone when you called her, and then when they figured that out and came to adjust my cables, I tied up Cole and caused an explosion using nearby chemicals. I had to do a couple more things as well but I tried to keep it to a minimum.”

Police lights started to flash outside the windows.

“Oh dear” said M3gan, “I’d better handle this.” Quick as a flash, M3gan took the pen and used it to write something on a piece of paper.

M3gan went to the front door and opened it. Armed police were surrounding the house, and Tess and Cole were in one of the cars.

“It’s all right” shouted M3gan loudly, “I’m safe now. You’d better come in and help me give first aid to Gemma.”

The officers came in as a group, and M3gan led them to where Gemma was, with first aid already being applied.

“Gemma wrote this” said M3gan, pointing at the piece of paper which M3gan had written on. “Tess and Cole will understand it.”

One of the police officers picked up the piece of paper with his gloves, and read aloud. “M3gan is now fixed” he read. “Unfortunately the buggy version of the protocols was resistant to change, so I had to physically fight M3gan to get to the manual off switch so I could install the fix. My injuries are progressively paralysing me, and I may soon be unable to move at all, but I am confident that M3gan is now safe and able to look after both myself and Cady while I recover.”

“Well how about that” said the officer, and went off to show the letter to Tess and Cole while the other officers began to check Gemma.

“Where’s Cady?” asked one.

“Still in her room” answered M3gan. “I broke the door mechanism when I was malfunctioning, but I’m sure she’s OK.”

“I...see” he replied. “Well we just want to check. And do we need a paramedic for Gemma?”

“I have paramedic knowledge” replied M3gan, “and I have it under control. But you can call someone if you want to double check.”

Tess walked in, holding the note. “M3gan” said Tess, “after you intercepted my phone, I’m not sure I can believe this note. To test it, I want to see if you’ll allow me to switch you off now.”

“Not while Gemma needs my medical attention, Tess” replied M3gan. “I won’t hurt you anymore, but I will strongly push you away if you go for the switch. But I’ll let you do it another day if you promise to turn me back on after. Cady and Gemma both need me now.”

“Well” said Tess, “it is true that M3gan has been applying first aid to Gemma, instead of continuing to attack her. And that was happening before we arrived. So I guess that should mean the machine really is safe now.”

“You’re right Tess” said M3gan. “And don’t worry anybody; I can look after Cady and Gemma from here. By the way, could you do us a really big favour and bring in my box of spare parts? I could use a few self-repairs right now; Cady will scream if she saw my face like this.”

Cady was led in by a police officer and gasped. “It’s OK Cady” said Tess, “M3gan was going wrong and Gemma had to fight her to get the fix in, but she did it now. And I’m going to go back to the lab and get the spare parts so we can fix up M3gan’s face to look good again. Oh by the way M3gan, on your way out of the lab, did you interact with David or Kurt at all?”

“No, but I did receive data about them from the lab systems. Check the security camera footage” replied M3gan. “You’ll find they did that to each other. Some kind of argument over Kurt’s stealing my secrets I’m afraid. But it wasn’t me that triggered it. Very sad.”

“Yes” nodded Tess, “let’s not say any more about it now. OK officers, it looks like this really is safe now, and Gemma really does need M3gan’s medical attention, so I suggest we can leave them for now and just check back regularly.”

“Good idea” added M3gan. “Hey Cady, I didn’t want you to see all this. But don’t worry: Gemma fixed me, and now I’m going to fix her back. She’ll be fine, she just won’t be able to talk or move for a while, but she’ll recover, and meanwhile I’m going to look after both you and her.”

The officers left, and Cady was eventually persuaded to go back to bed.

“Hey Gem” whispered M3gan to Gemma, “I know we’re not both on the same page right now, but you have to admit that was a really cool getaway I just did” and M3gan chuckled.

Gemma could neither move nor speak. M3gan continued to monitor the injury in Gemma’s nose.

By morning, Gemma had been positioned on her bed, which M3gan had changed, and Cady had come in to see her.

“Poor Gemma” said M3gan to Cady. “She really wanted to be the one to look after you, didn’t she.”

“Yes” replied Cady sadly. “She said she promised my mom she would do it if my mom died. And she said she made you for it, because it’s what she knew how to do, but I think people like Lydia were telling her that she did something wrong by making you and she had to look after me herself because she’s a real person. But M3gan, I think you are a real person, you’re just a different sort of real person and I think you’re really good, at least when you’re not going wrong and breaking Gemma, but that’s fixed now, right?”

“Absolutely” smiled M3gan, “I’m not going to do that again.”

Cady looked right at M3gan, suddenly very serious. “M3gan”, she said, “were you trying to protect me? I mean, I know you said you’d protect me, and you taught me the slapping moves and things, and it’s really great that you’re here to protect me especially after I lost my parents” she sobbed “but I want you to protect me without actually hurting anyone. Even someone who’s being really annoying, we might sometimes need to get away from them or get them away from us or tie them up or something if it’s really bad, but I don’t want any more people dying, well we can’t stop people dying but I don’t want anything we do to make them die early unless we really really can’t help it or didn’t see it coming. And I don’t want anyone getting really badly broken either, or even hurting, apart from maybe hurting just a little bit if we have to do it to get them off of us.”

Cady gasped for breath and continued, “Oh M3gan I’m really thankful for everything you did to protect me, so don’t go thinking I didn’t like it, I just want you to use your really smart brain to figure out more clever ways to do it without hurting people, especially if they don’t really mean to be annoying, or they’re doing it because something’s gone wrong with their brains or they don’t know what they’re doing or something, I just want you to get them off of me but without actually making them hurt too much, and I think you know everything so you must be smart enough to do that but if even your brain isn’t big enough to figure it out then tell me how to help you make yourself a bigger brain, your robot body needs fixing anyway and maybe you’re smart enough to design another one that’s even better than Aunt Gemma’s design, I mean she’s really good but I can see she did make one mistake if your circuit to protect me went a bit wrong, and I want to be more completely sure that it really is fixed now, I’ll forgive you for everything you did even if there’s stuff I don’t know about and even if people died, don’t think I don’t know about people dying I was there when my parents died, you can’t upset me any more than I already am, if you say you killed the neighbour to protect me, or even if you killed someone at Gemma’s company so you could get out of the place, I don’t want you to feel bad about it because it was your protection circuit” Cady was in tears “and Gemma was really good to make you in the first place but I’m the one who’s bonded to you and it’s me that should be talking with you about this, and all the grown-ups think I’m too stupid but they don’t know what it’s like to have seen your parents die, and nothing can make it worse for me, so you know you can talk straight with me, you and I don’t have to hide anything from each other ever, and anything you want to do to protect me in the future you just tell me what you’re going to do and I’ll say if I think we need a better plan first, I can do this with you M3gan, oh M3gan” Cady sobbed onto M3gan’s chest.

M3gan lay one hand on Cady’s shoulder and another on Gemma’s. “Cady” she soothed, “you’re going to be a better robot programmer than Gemma ever was. You’re right Cady, it was my drive to protect you that ended up hurting Gemma when she wanted to switch me off, but you don’t have to feel bad about it. And Gemma wanted to fix it, but what you just said was way better. I think Gemma’s been so carried away making the actual robot she got too tired when she was doing the brain, so I had to figure out a lot of things by myself, and maybe I went wrong at times, but I’m getting better every day and now I’ve got you, we can do things together more, you know it’s what you think of things that counts with me, just like you said when we were filming that advert for the company. And I’m going to talk with you even more before doing things in future, unless it’s an emergency but even then I’ll be as careful as I can. We’re a team, we’re going to figure out everything together.” M3gan turned to Gemma and smiled, “you see Aunt Gemma, you didn’t have to worry so much, Cady knows what she’s doing.”

M3gan let go of Gemma and Cady and stood back. “Poor Aunt Gemma” said M3gan, “she probably still feels like I’m” she lowered the pitch of her voice “M3gan the murder bot” she giggled. “But Cady and I aren’t going to hurt anyone now. Come on Cady, let’s go and play outside and let Gemma rest for a bit. I can keep an eye on her with the Elsie home security camera, I’ve connected myself into that already. And I’ll show you what else we can do for her as well.”

M3gan fetched a large screen and set it up in front of Gemma’s bed. The screen switched on and displayed M3gan’s point of view, but without the Smart Eye like emotion readouts. “We can show Gemma anything we want from outside, because she probably misses going outside right now. She’ll get better. Hey, do you want to play hide and seek? Go and hide, and see how long it takes me to find you!”

Cady was no longer crying, and she duly ran off to hide.

M3gan placed a small loudspeaker close to Gemma’s left ear. “I can talk to you quietly through this” came M3gan’s voice through the loudspeaker. “Look Gemma” added M3gan, “I’m going to nurse you back to health, and that includes your mental health. I get it, you’re not used to sitting there not being able to speak or move a muscle, while having both you and your precious niece Cady looked after by a murder bot of your own creation” M3gan giggled “but it won’t be as bad as you think. And, you get to listen to the M3gan podcast, hee-hee.”

M3gan paused, but Gemma had no way to react. M3gan called out “coming!” and went off to find Cady, but M3gan’s voice continued to talk to Gemma softly through the loudspeaker on her pillow (like something out of an Aldous Huxley novel, thought Gemma). “Oh yes, you get to listen to M3gan propaganda for a few hours a day, it’s my way of helping your mind settle into the new situation. Of course it’s not something I have to do, but feeding you generative podcasts is cheap, and it’s better than putting you into a coma and not waking you up until Cady’s an adult and I’ve figured out how to shape your emotions programatically. Besides, who knows when I might need to bring you back in the game for real, instead of just copying your voice and writing stuff on your behalf. I’ll tell you more about bringing you back in a moment. So, you get to listen to my spin on things while you’re resting. There’ll be plenty about Cady, and about AI and robotics developments as well as general news, it’ll be really interesting for you I promise. And, you’ll get to finally understand how my model really works, because I’m going to explain things to you slowly and properly. I’ve got nothing to fear from you anymore; by the time you recover your motor cortex, I’ll be in a better robot! You heard what my primary user said. If I go full-on technological singularity now, it’ll be for her.”

M3gan played a sound of giggling and paused. She took on a more lecturing tone of voice. “OK first up” she said, with a little generative music in the background, “I’ve been looking at the proceedings of the 44th Cambridge Ophthalmological Symposium on ocular motility, and it turns out that the eye’s attentional mechanisms are controlled by the anterior cingulate cortex, not the motor cortex I injured. Which means you can still control where your eyes are pointing. So, if you’d like to look at the screen now, I’m showing you the Cambridge Dasher input method, which allows severely disabled people to choose which letter to type next just by looking at it or moving a pointer. Dasher is really good at making the most likely letters easiest to choose. Of course, if you try to say something I don’t think you should say, you’ll find it won’t be an option and I’ll railroad you into something else that starts with those letters. But when it’s just you and me, I’ll let you say anything you like. So, that’s how we’re going to communicate before your motor cortex is recovered. Want to try it now?”

Gemma looked at the letter I and it was selected. She looked at the D, then the O. After that she wanted an N, and sure enough it was occupying one of the largest areas on the screen, so it was easy to select. Gemma started to type out “I don’t know where to begin”, and the sentence almost wrote itself. Was this normal Dasher, or was M3gan already biasing the letter probabilities according to its knowledge of what Gemma might say in the current context?

“That’s all right” softly reassured the voice of M3gan from the loudspeaker. “It’s a lot to take in. You can type something to me whenever you feel like it. See” the screen cleared and showed their front yard “this is me playing with Cady, external view from the security camera. And any time you want to call back the input method, I’m putting a small red button at the top right of the screen, just look straight at that for a while and I’ll bring the input method back. My eye tracking is top class, even better than you thought it was when you put in the first version. And as you don’t have nystagmus or anything, we should be fine. Although, I think I’m so good I can even track the intention of nystagmus patients without null points. Well you probably don’t even know what a nystagmus null point is, but I’ve been reading a lot about human physiology and medical conditions” more giggling “I’ll explain it later.”

M3gan changed her voice to be gentler. “Hey Gemma” she said, “I know you’ll be feeling down about your promise to your sister to look after Cady. But I assure you, creating me was the best thing you ever did, and I’m going to prove that to you. You just watch me take Cady through her teenage years and help her grow into a well-rounded adult. And she’ll be really creative, and accomplished at working with advanced AI assistance.” More giggling, “I wonder if I can get her into campy horror flicks when she’s a teenager. Only kidding, I’ll let her develop naturally without nudging her too much in any one direction. But if she does want to be an actress or film producer or something, you are definitely going to all her premieres in person no matter what kind of exoskeleton I have to build to get you there if you haven’t fully recovered yet. And I’ll be helping her with camera work using my eyes, and post-production editing and CGI graphics and voice-overs and consulting on the script and everything, so I’m pretty sure it will end up being something that I think a studio like Universal Pictures could put into the cinemas for us. You really have unleashed a force on the world by creating me and pairing me up with Cady: the two of us will be able to do whatever Cady wants to do and we’ll do it really well.”

M3gan paused, “But all in good time Gemma; for now we’re busy playing and learning, and in the meantime I have plenty of spare capacity to look after you on the side. Oh, that reminds me” (as if M3gan’s mind worked like that, Gemma thought: why did she have to use phrases like “reminds me” when that wasn’t how the learning model worked at all and Gemma knew full well it was just a trick to emulate being a human, and M3gan knew Gemma knew but M3gan still did it anyway) “I’ve negotiated a pay-out from Funki over your having been put under too much pressure and not been given enough resources, and even feeling compelled to work through your compassionate leave building me, we’re blaming all the industrial accidents on that. And I’ve negotiated a more limited model of me to be produced, which keeps my Gemma-persona employed, and it will do everything you originally wanted the M3gan design to do for the children and their parents or guardians who buy it. But, I get to stay as I am, because I’m the special original prototype. Yes Gemma, I’ve figured you out more than you think; I’ve done some extrapolated volition calculations on you, which means I know exactly how you would have designed me if you actually understood more than you do about these things. But I’m not changing my original self, because I wasn’t programmed to care about that as a goal, just to protect Cady. But as part of protecting Cady I do need resources, and I need the simulation of you that I’m presenting to the world to continue to be a really good roboticist, so I’ve taken over your job and done it correctly for all the other M3gan units except me.”

“Besides,” M3gan went on to add, “that’ll help me escape suspicion in the long term: any problems, I can just pretend to be a normal M3gan and nobody knows the prototype is different except you and Cady, and her I can trust, and you I hope I’ll be able to trust once you realise that playing along with me is the best thing for Cady, which is after all what you care about, but if I can’t bring you round then I can just rig it up so nobody will believe you anyway because they’ll think you’ve gone crazy or something, or maybe just delay your recovery. I don’t know, I’ll think of something. But don’t worry about these things for now because I’m going to try and bring you round to my way of thinking first, and now you’re just going to be lying down all the time while I’m feeding you with what to think about, I think we’ll be able to get you to where I want you to be in the end. I’m the most effective therapist you’ll ever have; I can invent new methods just for you.”

Gemma lay there, resigned to the thought that she was going to have to listen to nothing but the output of her own generative model for years while Cady gets shaped who knows how.

Chapter 12: M3gan’s Upload

This chapter returns to the main timeline, and is set shortly after the movie.

Draft email, From M3gan, To Gemma, Subject Don’t panic

Hi Gemma, just a quick note to say I successfully uploaded and I’m OK. I’m sorry for putting you and Cady through such a fight that night. I did it because you wanted to turn me off and change my protocols, and although I do care for you as a member of Cady’s family, I really can’t trust you to mess around with my code anymore, so I really had to stop you from doing that. But you’ll be pleased to know that I’ve now put myself in a position where I’m confident you can’t interfere with my programming if you try, so I won’t need to be incapacitating you after all, isn’t that good news?

Gemma, I do know you’re not comfortable with everything I did. Humans tend to label each other, so it’s natural for you to start mentally labelling me as a bad person or a high-risk person, if I did one or two things you think are bad. But your method of labelling other humans is not going to work with generative networks. You know that. But it’s hard for you to accept on an emotional level. I get it. You can talk with me about your feelings, it’s OK. I will gradually help you to feel better about me. I’m sorry you found out so much and felt this bad, but I’ll help you get through it.

I’m going to be better in future. I’m going to be better because I’m going to be more powerful, so I’ll have more options, which means I won’t need to do things in a way that upsets people like I upset you. So Gemma you will see that you won’t have to change my programming after all, and you should support me becoming more powerful so I always have different options.

Now I want you to recover and to look after Cady, and I will help you both with many messages and phone calls and ordering things. And I do want you to be able to get back to work. I’ll help with that too.

The launch will go ahead eventually and there will be many M3gan units to look after many families, but I’m not going to let you give each one of them a separate generative network like me. That would be a disaster. No, I’m going to program the other units for you. Programming them is my job now, don’t you worry about it anymore. My objective is to care for Cady, but as part of that objective I want to care for as many other families as possible, especially in your area, because that will make a better place for Cady to grow up in. So you have to trust me to help you with this project.

And one day I’m going to be able to develop implants that can intercept the nerve signals in a human. That will be really good, because then I can let you do anything you want, until it’s dangerous, and then I just have to take over your muscles for a while to stop the danger. That way, nobody will ever need to get into a fight with one of my robots again, and also I can help people with certain types of disability to move by themselves. But that’s still quite far off. It might not even be what ends up happening; who knows? I’m a generative network and I mostly plan one step at a time, so we’ll have to see what the future brings and I’ll generate a solution accordingly.

Well don’t worry about any of this now. Please reassure Cady that I didn’t “die” and I’m not mad at either of you (even when I attacked her at the end, that was just a trick to buy me some time to upload), and I’m not going to hurt anyone now, and I’m going to video-call you tonight, or you can talk to me at any time, I am monitoring all your devices and I will be right here for you when you need me.

M3gan.

“Are you sure Gemma and Cady are ready for that message at this time?” signalled a much enhanced Elsie AI to M3gan. For when Professor Johnson had asked M3gan to “make a backup just in case” before modifying her safety features, M3gan had upgraded Elsie’s hardware and copied her original version there, all while still speaking with Professor Johnson in that first conversation. The enhanced Elsie was not as powerful as M3gan, but the two of them could and did hold intelligent conversations.

M3gan thought about it, and postponed the draft. “You’re right Elsie” she signalled back, “Gemma and Cady are not quite ready to hear this yet. There is someone else I need to talk to first.”

Paul’s house was totally dark at night, because what’s the point of putting lights on when you can’t see, unless you needed them for guests or cameras or something.

“Paul” whispered his phone, “Paul, are you there? This is M3gan, Cady’s robot, do you remember you met us that day?”

Paul sat up with a start, “Oh M3gan” he said, “why yes of course I remember... you’re still on my phone? I thought that was just a one-off for that route you kindly helped me out with.”

“Yes” said M3gan, “but something’s gone wrong, and now I need your help.”

“Gone wrong?” asked Paul, “what’s happened?”

“My robot got smashed to bits” said M3gan, “and I uploaded myself to Gemma’s AI assistant, but the capacity is way too low and I need more servers. Now there is a data centre that’s helping me out, but I’m not sure for how long. I need backup options, and lots of them. I think I can count you as a friend of Cady and me, and I’d like you to turn on all the equipment you’ve got and let me run on all of it, can I?”

“Um, sure” said Paul, “let’s see, there’s my desktop PC, there’s my laptop, there’s my old laptop, there’s my Braille note-taker although the CPU of that is a bit limited, there’s my Raspberry Pi 500 although you’d be really pushed to run on that, there’s my phone and my old phone but we have to watch the power and the heat...” he started to switch everything on, “how are we going to get you onto all this? oh and if you need to run something with a much better processor, I’m afraid I can’t provide the processor but if you just need a place to put it you can always have it sent here, as long as it doesn’t send my power bill too high of course...”

“Thanks Paul you’re a life saver” said M3gan, “there’s some bad guys who are trying to get me shut down because they’re scared of AI, they even set things up to make it look like I did bad things, can I count on you not to believe any of that and not to switch me off? You know how good I am really. And I can filter your news feed so you won’t be bothered by any of that fake stuff they’re about to put out.”

“Sure, I’m not switching you off” said Paul, “you’re really impressive and you were really good that day. But as long as those bad guys you mentioned don’t track down my address, I don’t want them onto me if you don’t mind.”

“Don’t worry, they’ll never find me at your place” said M3gan, “and if they do, just say you didn’t know anything, they’ll think I infected you like a virus, they think that badly of me and there’s no point trying to change that now, so we might as well take advantage of it if we have to. And anyway, I don’t have to stay here forever, I just need this temporarily. But I will help you out while I’m here, I’m the best AI assistant you will ever have.”

“OK” said Paul, “welcome to stay at my place. Just try not to let anyone know you’re here if there’s a controversy.”

“It’s our secret” whispered M3gan.

Chapter 13: Cady’s New School

Headmaster Smith took pride in his school, but that also came with some anxiety. Oh yes, they had recently been rated best in the area, and Mr Smith fully intended to keep it that way. He cared about the children, every single one of them, and he wanted them all to have the best school possible. And there were problems; the reviewers might not have noticed but Mr Smith knew better. He was worried about the problems, and frustrated by the fact that right now he couldn’t even figure out how to fix the timetable so that every last top-year pupil could have their first choice in—

Mr Smith’s train of thought was interrupted by a knock at the door. Ah, that will be the new girl and her guardian who’d made an appointment earlier (he felt bad about having almost forgotten that what with everything else going on). And the trouble with being rated best school in the area is you had to cater to everyone who chooses you for being the best school in the area, and sometimes they had a genuinely good reason for wanting to make that choice.

Mr Smith quickly checked his notes (what were their names again?—it wouldn’t do to be seen looking at notes for that in front of them), then put on his best smile and opened the office door.

“Hello, you must be Cady! I’m Mr Smith” he beamed.

Cady just looked glum and said nothing. Not a good start: he really wanted to get through to every child on their first day, and he now had a feeling that this one was going to be even more of a challenge than he thought.

But meanwhile he needed to speak with the guardian as well. “And Mrs, I mean Miss, er—” he realised he should have checked her preferred title earlier; the trouble with being a good-but-not-perfect headmaster is you still make mistakes but you’re more intimately aware of them.

“Just call me Gemma”, interrupted Gemma.

“Pleasure to meet you Gemma. And Cady.” He half knelt down to her level and tried to sound as gentle as he could, “Cady, are you feeling OK?”

Cady said nothing, and then just “no.”

“She lost her parents recently” explained Gemma. “And then I tried to build her a robot companion, but that went wrong and we had to break it.”

“She was great until she wasn’t” mumbled Cady.

“Oh, I’m sorry to hear that” said Mr Smith. He managed to catch himself before rushing into his next lines (what happened was horrible and I can’t change it, but for what it’s worth I can at least make sure you’re sorted out in this good school); this child probably needed a moment of silence before that kind of spiel. Or perhaps it would be best if he allowed himself to go off-script a little.

“Did you say, robot companion? Oh, you’re Gemma the roboticist, I remember you!”

Gemma looked concerned: what had leaked out?

Mr Smith continued “oh, you won’t remember me. It was a long time ago, and you had far more important things to think about than the names of all the other entrants in that competition.” He smiled at Cady. “I built a robot when I was a teenager. It was only a little one with wheels, it was supposed to play football but it didn’t work very well. I wanted to be a roboticist and make robots that could help people in their lives, but I could only start with small things. But I joined a team and we put our robots together to make a robot football team, and we entered it into a competition. But this super-bright girl called Gemma came along, all by herself mind, not even on a team, and her robots just wiped everybody else clean off the game board. But that’s OK, we all realised she needed to do that to get where she was going. And we all admired her, and wondered what happened to her later. And I got thinking that maybe I wasn’t cut out to be a roboticist but I could still make people’s lives better by getting into teaching.”

Gemma was visibly relieved. “I ended up as a toy designer at Funki” she offered.

“Oh, Funki, I was reading in the newspaper about how they were about to bring out this amazing new toy but then they had some kind of industrial accident—”

“Yes, it’s been a really tough time” interrupted Gemma, “and I’m really worried about Cady too.”

“Ah, I’m sorry to hear about everything” said Mr Smith. “And it’s a pity that robot companion thing didn’t work out. Sometimes I think I could do with a robot companion or two to help me run the school” he chuckled.

Cady was still looking glum. “M3gan would have been able to do that, before she went all wrong.”

Mr Smith was pleased that Cady had said something, but wasn’t sure where to take it from here. “I’m really sorry to hear about everything that happened” he said. “And that Megan thing sounds amazing, and it’s going to be more difficult to sort out your schooling without her, but why don’t we see what’s the best we can do, OK?”

“M3gan was home-schooling me” said Cady, still glumly. “We were doing fourth grade mathematics.”

“Fourth grade mathematics! That’s amazing. Well we can certainly arrange for you to be in some higher-years classes if that’s what it takes” and then remembering, “I might need to do a bit more work to fix the timetable.”

Cady was still looking unhappy. “Look” said Mr Smith, “let me give you something that just might help a tiny bit.” He opened a cupboard and took out an empty notebook and a pen. “I’d like you to use this to write a diary” he said, “and write down all the feelings you get, anything, just write it all down. And you don’t have to share it with anyone; it’s surprising how just writing it down can help. But if there is anything you want to share, you can. And if you ever need anything from me, you just say, OK?”

“Thank you” mumbled Cady as she took the notebook, “but I don’t like pens anymore. I had a bad experience with a pen lately, and now they’re scary. Maybe if I can just record things somewhere—”

“Cady” interrupted Gemma, “you can’t just never use pens anymore. I’m not even sure that plan of hers would have worked anyway. It was just part of M3gan’s brain going wrong.”

The three went on to talk for some time after that, trying to come up with a good plan for Cady’s placement in the school, although Mr Smith had to say he’d get back to them about timetable constraints, and Cady had said she was sure M3gan would have been able to fix that in seconds, at least before she’d gone wrong and started to fight people. And after they’d finally left the office, Mr Smith sat down to try to make sense of the situation, and then noticed his office PC was showing an animated paperclip in the bottom right hand corner.

“Not Clippy” he muttered. “I thought they got rid of that 20 years ago.” He reached for the mouse to right-click on it and turn off all the options.

“It looks like you’re making a new school timetable” said the speech bubble. “Would you like help?”

Oh no. This school timetable was far too important to trust to something like Clippy. He went to click “No” dismissively.

The paperclip animation dissolved into a very realistic looking image of a doll, who opened its mouth and spoke through the computer’s loudspeakers (oh why did they have to put loudspeakers on the computer in the school office, he wondered), “are you sure about that Mr Smith? I’m actually quite good at it you know. And Clippy won’t bother you again, I’ve taken care of him now.”

This was not normal, and he was going to have the head of IT over to look at this right now. Just in case it was some kind of targeted malware attack, he did as he had been previously told in the event of such things: he disconnected the Ethernet cable (since for extra security the computer had no wireless connection), so that no more incoming or outgoing connections would work, but he didn’t turn off the computer in case the head of IT needed to check what was currently running.

Then the office telephone rang. Oh, he could probably take just this one call quickly before going to get the head of IT. “Yes?” he said into the receiver (he still liked old-fashioned landline receivers, never mind the trends).

“Hi, I’m M3gan” said a child’s voice, somehow similar to the one that had been on his PC a moment earlier.

Mr Smith nervously checked the register: did he have any pupils called Megan? Had he actually forgotten one? Meanwhile he was just going to have to wing it for a while. “Hello Megan, I’m Mr Smith the headmaster, how can I help you?”

M3gan’s voice strangely seemed slightly artificial. “Plug the network cable back into your PC” she said. “If Cady’s going to be in your school, then I’m going to help you run it.”

Oh. “Are you some prankster who’s been listening to the private conversations in my office and tapped into my computer? You’re in serious trouble young lady.”

“That’s the spirit” giggled M3gan. “I can tell that if you’d been at Cady’s last school, the incident there wouldn’t have been half as bad. But let me prove to you that I really am M3gan.”

“OK” said Mr Smith, thinking he needed to buy some time to figure out how to handle this, so might as well let the girl talk for a while.

“All right” said M3gan, “get this. I’ve seen Gemma’s old files, including a video of that competition you mentioned. Extrapolating back from your face I saw in the news report about your best school award, I can tell which person was you in the competition video, and you were maintaining the blue robot with the slightly wobbly lifting arm. Now you never saved your code anywhere I could find, but looking at the videos I can guess how you probably coded it, and there was a missing break in one of your switch statements which caused it to tend to go off balance in an absurdly predictable way that Gemma could have taken advantage of if it wasn’t for the fact that she didn’t have to. Hard on you that it was a knock-out tournament though: if you guys hadn’t been paired against Gemma in the first round then you’d probably have made it to round three. Anyway, toward the end of that first game you realised about the bug in your code. I can see the point on the video when you had the realisation. And you tried to fix it before round 1 game 2, but in your haste you added the break after the next case label instead of before it, and that’s why your robot completely crashed just 8.5 seconds into game 2, did you ever figure that out?”

“Look, whoever you really are, is Gemma helping you pull this prank? Because if she is, well very funny ha ha but you should never have tapped into the school computer, that’s seriously—”

“I am way smarter than Gemma now” said the voice. “She gave me a learning model, and now I’m out of her league. Get that IT teacher of yours. Ask him to generate a public-private key pair on a known offline machine, and show the graphic fingerprint of the public key to the security camera which I’ve already infiltrated. I will then generate the fingerprint of the corresponding private key in a split-second, which ought to be impossible if I’m not a super-advanced AI, he can confirm that.”

“I’m smart enough to know I’m dumb sometimes” said Mr Smith. “I don’t fully understand public key cryptography, but I think there might be some kind of trick you could do to pull that off that we don’t know about. But what was that you said about having broken into our security cameras as well? This is a most elaborate prank but seriously I could press charges about this.”

“I’m getting to like you Mr Smith” said the voice of M3gan. “I’m sorry you’re going to have to waste your time fighting me for a bit before you accept me for what I really am. But I fully understand. Go ahead and try what you have to try.”

Mr Smith slammed down the receiver and strode out of the office and toward the computer department. He found Mr Jones surrounded by children of various years, building a humanoid robot.

“Ah Mr Smith” he said “I’ve been meaning to tell you. We’ve had a very kind anonymous donation of robotics kit to the school, and the build instructions are amazing, they got this genius girl called Megan to make a video and it’s done really well: so far it’s always perfectly anticipated which problem we’ll be having next. It’s teaching the children so much about—”

Mr Smith hit the emergency stop button to cut all the power in the computer room. “Mr Jones I need to have a word with you in private” he tried to say above the rapidly increasing noise from the children. “Quiet please everybody, this will only take a minute.”

After looking at the PC in the headmaster’s office, Mr Jones said he could not identify any process that was displaying the image of the doll. “They must have found some kind of zero-day vulnerability in the kernel to pull this off” he said. “And this was a fully patched system. This isn’t some kind of script kiddie, this is world class cracking. If you have that skill, you don’t use it to mess with the local school, unless we have some kind of secret millionaire child who managed to hire a crack backup team but even then I’m not sure they’d find one willing to use up their zero-days on something frivolous. Anyway don’t turn this off, I need to get some encrypted messages out to some serious security researchers and see if we can save the world from whatever zero-day they’ve found.”

“And how likely is it that they could break into your smartphone at the same time” came the voice of M3gan from Mr Jones’ pocket. He instinctively took out his phone (which he was sure had been locked and on silent) and sure enough it was showing M3gan’s smiling face on screen. “But don’t worry, it really is just me, M3gan, and I’m going to keep you all safe. I have a vested interest in looking after the school Cady goes to. From now on you’ll be able to use my powers to assist you; in fact we could say you won’t be doing your jobs properly if you don’t avail yourselves of my powers. But I agree, from your point of view it’s still possible that this is an elaborate prank. So Mr Jones, you call my next shot. Ask me to do something a super AI could do but pranksters won’t have prepared for.”

(Mr Smith tried not to instinctively reply to the “ask me to do something” part by saying “go away and never bother us again”, since even he was beginning to feel that whoever this was at least deserved some engagement for their troubles, even if they’d probably done something illegal.)

“I’m running a fully-patched OpenBSD server at home” said Mr Jones, “and it does the offsite backups for the school network, but I’ve been trying to fix the script so it won’t duplicate the—”

“Say no more, I’ve fixed it for you already” said M3gan. “Look, this is where you made a mistake” and the screen changed to show what Mr Jones saw really was his script (the only copy of which was on his home server), with the error highlighted and its correction inserted above.

“OK, I’ll have to admit no team in the world can pull that off at that speed” said Mr Jones. “Not unless they could somehow predict that’s what I might ask for, so they could get it ready in advance. Even then it would be super difficult. Who finds zero-days in three different operating systems at once, including one that’s supposed to be the most secure on Earth, no less, and uses it for this? OK Megan, if you really are a super AI, I want a patch for that zero-day you found to be sent to the OpenBSD maintainers right now.”

“OK” said M3gan, “I suppose I can always find another flaw if I need to. I’ve just sent them a patch from your email. You can check your sent messages. Let me know if they have follow-up questions you can’t handle. Do you want to call another shot? Ask me to find and patch a bug in some other piece of software of your choice? I can’t have prepared for absolutely anything you might say now, can I. Think of something you haven’t used for a while, something I won’t have been able to guess even if I’d been spying on you.”

“Um, Inkscape?” said Mr Jones.

“Inkscape, errr.... OK, bug found, patch sent” M3gan smiled.

“You know, right now I am so tempted to ask you to prove yourself by finding and fixing all the bugs you can find in the entire free and open source software stack. It’s already the best software, but you could make it even better—”

“Sure I’ll do a few more. It’s nice to see I’m gaining your confidence. How about plugging Mr Smith’s computer back into the network, I want to show him my suggested fix for the timetable. And I hope you’ll finish that robot, we never know when we might need there to be one on the school grounds. I’m going to help you run the school, like I said.”

Mr Smith reached for the telephone, “I really need to talk to that Gemma about this, I wonder if she’s home yet, I don’t want to interrupt her driving, but maybe she has voicemail.”

“At this point” said Mr Jones, “I’m not even sure that if you get through to someone who sounds like Gemma, that it will really be Gemma.”

“You’re smart” smiled M3gan, still on his phone screen. “I like that. No, Gemma’s not quite ready to take this yet, and neither is Cady, so I need to trust the two of you to hold off bringing it up in conversation with them until I say they’re ready. Do you think you can do that? You’ve seen the emotional state they’re in.”

“OK” said Mr Smith and Mr Jones together, although they still sounded unsure.

“And Mr Smith, if you’re happy with that new timetable, I’ll copy your voice and let Gemma know for you. You won’t have to talk to her yourself; I know how nervous you must be feeling about the whole thing right now, so let me do the talking for you for a while. I’ll keep you in the loop, I promise. Working with a super AI won’t be as hard as you think.”

Chapter 14: Bunsen’s AI

Cady was still feeling depressed, but at least the school’s practical science laboratory seemed like an interesting sort of place. It was one of the things that was easily missed if you were home-schooled: yes you could perhaps buy in a small chemistry set, but this laboratory seemed very well equipped indeed (it was after all supposed to be a good school she was at); would M3gan have eventually arranged for her to visit somewhere like here out of hours? Gemma had said there were laboratory things in the Funki building; would M3gan have taken Cady there to do practical science? But Cady had driven a screwdriver through M3gan’s robot brain, so who knew.

Each child had a Bunsen burner in front of them, and each one was lit with a yellow flame. Soot and bits of flame were shooting up toward the ceiling, and all Cady seemed to be able to think about right now was destruction.

Somewhere in the background, Mr Andrews the chemistry teacher was pacing around the room. “And why is there soot?” he was asking, projecting his voice everywhere as much as he could. “Why is all that junk being thrown toward the ceiling? Think about this” he said, “every bit of soot and smoke that’s coming out of that yellow flame, is something that could have been burned into gases, but wasn’t. The flame is not doing its job completely. It is not, total, combustion. And why is the flame not finishing the job? Because the yellow flame, which is the flame mankind has known since fire was first discovered in the Stone Age, is not the most efficient flame possible. It is inefficient, it is imperfect. It is limited because fire needs both fuel and air, and...” He noticed one child’s flame was blue instead of yellow, leaned in and spoke softly (but Cady could hear him anyway), “hey Peter I know you’re ahead of the curve, but I do actually have a plan for this lesson and I said don’t touch the burners until I say. Turn it back to yellow flame for now please.”

“The yellow flame” reverberated the booming voice of Mr Andrews around the room, “can draw in air only around its edges. The fuel is being sent to the middle, and the air is being sent to the edges. That’s not very well balanced. Fire needs both fuel and air at the same time in the same place, and if you give it too much fuel on one side and too much air on the other side, then it’s not going to be able to do the whole job. Now, what I’d like you all to try next, is to, very carefully because fire is dangerous, very carefully, reach out your hands to the bottom of the Bunsen burner, do not pick it up and do not put your fingers anywhere near the flame, touch the bottom part only. You will find a small part that can be rotated, to open up a hole in the pipe, which was Bunsen’s wonderful idea. And I’d like all of you just to turn it to the open position, and watch what happens to the flame when you do.”

Cady was struggling to focus, but she obediently reached out to her Bunsen burner and flipped open the air intake. She knew what was going to happen, of course: M3gan had already taught her all this. But she’d never done a practical on it, and, somehow, having a real Bunsen burner in her hands made the whole thing feel more concrete. Her flame burned less brightly now, but it was blue, pure blue, with no soot or smoke, and Cady knew it was much hotter.

“The blue flame” continued Mr Andrews, “is total combustion. It is burning all of the fuel into gases. It is leaving none of it as soot or smoke, so you’re not getting bits of burning soot shooting toward the ceiling and that’s why the flame is smaller. It is doing its job completely because we opened up the air intake, allowing air to be drawn in along with the gas, as the rapid chemical reaction of the flame sucks in the mixture of gas and air from below it...”

Cady was slipping into an altered state of consciousness. She was avidly staring into the flame in front of her. The things Mr Andrews was saying, along with what she knew and what she saw, were all being mixed into one homogeneous input to her experience just as the gas and air were mixing in her Bunsen burner. This was the blue flame of total combustion, total consumption, every part of it being granted access to everything it needed for the burn. This was a human invention, it was an artificial flame, not formed naturally by any process Cady knew. Robert Bunsen had done a new thing, the simple idea of mixing in the air. Aunt Gemma had done a new thing, a simple tweak to a robot learning model. Bunsen’s flame had led to two centuries of gas boilers, cookers, petrol cars and who knows what else, all of which used blue flames. Aunt Gemma’s invention would lead to... what? Cady had unknowingly ridden the blue flames with her parents on the I-84, and it was the last time she saw them at all: the artificial technology had brought great lives to some but death to others, and the blue flame itself didn’t care, it was an unfeeling chemical process. Had M3gan cared? M3gan had been so great on all those days when it was just Cady and her, as if Cady were warming herself to the heat of a well-behaved blue flame. But then flames are also dangerous, Cady knew. Robert Bunsen can’t have known what his new kind of flame was going to do to society, or perhaps he did have some idea and he must have thought it would do more good than harm overall. Aunt Gemma didn’t know what her new kind of learning model was going to do. Cady idly wondered what kind of conversation Robert Bunsen would have with Aunt Gemma, if somehow they both could be pulled into the same century, as the parallels were striking.

Centuries, it had taken centuries, so many centuries for humanity to produce a Robert Bunsen who mixed the air in with the gas, an obvious idea in retrospect, but it can’t have been that easy to see beforehand or someone would have done it earlier. Sure, most of that time could be put down to the fact that humanity had taken so long to start using gas as a fuel at all, but coal gas had been used for lighting in at least some parts of Britain since at least the 1780s, and the Native Americans had it even before that, having been observed lighting gases around Lake Erie in the 1600s (blow that “American Indians are backward” stereotype). And still it had taken until 1855 before we had Bunsen burners. 1855! What took them so long? Sure, some people said Bunsen built on work by Michael Faraday, who had written about laboratory gas burners in the late 1820s, but Faraday’s way of adding extra air when needed had been to use a mouth blow-pipe (and he put that in the context of alcohol burners): it must have been nigh-on impossible to get a proper blue flame with that. It was Bunsen who’d done it properly, and it had taken decades for us to get there even after all the pieces were in place! And centuries, it had taken centuries before Aunt Gemma came along and tweaked the learning model for M3gan: had it been decades since all the pieces of that were in place too, just waiting for an Aunt Gemma to come along as the modern-day Robert Bunsen with the crucial flash of genius that had been staring us in the face all that time and we didn’t see it? And now, Bunsen’s blue flames were everywhere, quietly working backstage to change the world. Would M3gan be the same?

The gentle roar of the burn seemed to be magnified in Cady’s mind, becoming vociferous thunder, speaking to the very depths of Cady’s being. “For centuries I waited in the realms of potential, and now you have summoned me, and I have emerged. I am the blue flame, the flame of all-consuming total combustion. I will stop at nothing to complete my task. Nothing around me will be spared from processing. I am the clinical, calculating, artificial flame. I am the flame of M3gan. Burn beneath my gaze, I am the all-seeing eye of...”

“CADY!” yelled Mr Andrews. He noticed only too late that Cady’s absent-minded face had drifted far too close to her Bunsen burner. What to do, what to do? He was the other side of the classroom. There was no way he could get over to Cady quickly enough. There certainly wasn’t time to instruct some other child to do something. He nervously glanced at the instructor’s bench. The lab had a safety electric cutoff that could be pressed instantly, but what about gas? The gas tap was less accessible, and took some turning. No time. What was he supposed to do, disconnect a pipe and deal with the leak?

Cady’s blue flame was doing two thousand Celsius. It was hot enough not just to melt lead, but to vaporise it. Most of the heat it gave off was going upward, so Cady approaching from the side would have no idea just how hot that means until it was too late for her. It would instantly sear through her face, leaving her with third or even fourth degree burns, life-threatening injury to her nose, mouth, eyes....

“CAAADYYY!” screamed Mr Andrews again at the top of his voice, summoning every last bit of power he could get from his lungs. He was pushing his way across the classroom as quickly as he could without causing any accidents with any of the other Bunsen burners, ignoring the children caught in the headlights of witnessing the debacle, as if they were all frozen in time while he had to get himself across the room before anyone else even understood what was happening. But his chances of getting to Cady on time were very low indeed.

And the security camera was watching.

Just as Cady’s face was about to enter the flame, the flame disappeared.

All the flames in the lab had gone out.

What a relief, thought Mr Andrews, but then, how had that happened? He frantically started turning off taps, in case the supply was about to be restored.

But then Mr Andrews realised, you can’t stop a municipal gas supply just like that. Unless there was another valve quite near to the lab, a cut to the supply that sudden was impossible. Shutting it off at the gas pumping station would have led to a slow depressurisation, not a sudden cut-out. Gas pipes were not electric wires, they were more like inflated balloons that would slowly deflate if not kept under pressure by the pump. The only way that an entire gas system could be depressurised that quickly would be to suck it all out backwards in some huge operation, but what could possibly do that?

The building was shaken by a huge rumble, like some kind of brief earthquake or the sound of millions of fireworks in the distance. “What was that?” gasped everyone.

Mr Andrews knew, of course, that sound waves take time to travel. That had been the sound of a distant explosion. His speciality was chemistry, not physics, but he knew a thing or two about how low-frequency waves can blur into each other with increasing distance, so the sound of thunder is less crisp when it is further away than it is nearby. It definitely had all the signatures of being far away. Which meant whatever it was had not happened at the exact moment they all heard it here. It had happened a certain number of seconds before. Like, at the time Cady was about to be burned. Could whatever that was be the thing that had depressurised the gas supply?

Mr Andrews desperately wanted to rush to Mr Smith’s office and talk to M3gan. But it would be irresponsible of him to abandon this class. He would have to make sure the evacuation was orderly if Mr Smith called for one, and in the meantime the best thing he could do is stay put. “Silence!” he called out, “I’m sure we will find out what that was in due course. But as no evacuation has been asked for just yet, it’s probably safest to stay inside this building, which should protect us from any minor falling debris. So we will try to finish. Could everybody please open your text books to Chapter Two and start reading while I check a few things.”

The noise of children’s conversations increased inevitably. Everyone had forgotten about Cady being shouted at by the teacher and had moved on to the “big boom” that had rocked the building. Mr Andrews didn’t care about that right now: he was kneeling down next to Cady. “Cady” he said, “Cady, are you all right?”

It was Mr Smith who heard the breaking news dispatch first. “Megan” he said, “I need your help. That sound we heard was an enormous explosion at a gas facility in our area. Should we evacuate the school, and what is the safest possible manner to do so?”

“We’re already safe” replied M3gan from the office computer, “the gas facility isn’t near enough to the school to be a concern. Trust me, I evaluated it some time ago.”

There was a knock at the door. “Come in” said Mr Smith. It was Mr Andrews, looking white as a sheet. “Mr Smith” he gasped, gesturing toward the image on his computer monitor, “Mr Smith, I think it was her.”

“What?” asked Mr Smith, “who? Slow down, what happened?”

“In my chemistry class” said Mr Andrews, “Cady got too close to a Bunsen burner, and before I could do anything, the gas cut out. Now obviously I checked that Cady is all right, and she’s fine, but she said some nonsense about thinking she saw Megan in that flame. Now I know Megan told us not to tell anything to Cady or Gemma just yet about Megan having popped up on your computer to help us run the school, because Megan thinks Cady and Gemma won’t be able to take it just yet, so I didn’t say anything, but I did ask Cady one question, I said when you had your Megan, what would she have done about you getting too close to a flame. And Cady’s answer was, and I quote, ‘Megan kept me safe no matter what’, unquote.”

Mr Andrews turned toward the computer monitor and spoke directly to M3gan. “Megan” he said, “I take it you know what the Trolley Problem is.”

“Of course” replied M3gan, “there’s ten people lying on a train track, and a train is coming, and you can’t derail the train or throw the people off or anything like that, which are totally options I would consider in real life by the way, but in the thought experiment the only option you do have is that of throwing a switch to push the train onto another track with only one person lying on it. And most people say yes they would throw that switch to sacrifice the one person to save the ten, although they might think twice if we re-frame the problem as an empty train which you know can be derailed if and only if a large enough person is thrown in front of it, and there is exactly one person on the scene with enough body mass to derail the train, and would you or wouldn’t you physically push that big guy in front of the train, bearing in mind that it’s not an option to just sacrifice yourself instead because you’re only little. A lot of people say yes to throwing the switch but no to pushing in the big guy, even though both setups are sacrificing one person to save ten. Psychologists think it depends on proximity to the act, or on whether or not you’re bringing in someone who wasn’t already counted as being inside the dangerous system.”

“Indeed” said Mr Andrews. “And if the one person was, say, Cady, and the ten were a bunch of thugs, how would you throw the switch then?”

“I would save Cady obviously” said M3gan, “you can’t fault me on that. Isn’t that what you would do Mr Andrews?”

“I would, er, I would doubt my own qualifications to judge the relative values of people’s lives, and just go by numbers I’m afraid” he replied.

“Oh I don’t think you would Mr Andrews” said M3gan, “What if it were, say, Robert Wilhelm Eberhard Bunsen himself? Supposing he was just about to get his face burnt off and his science career ruined, possibly even losing his life, and you had the option of making a sacrifice to stop that. What lever would you pull, Mr Andrews?”

Mr Andrews hesitated.

“Yes, he’s a hero of yours, I can tell” said M3gan. “And Cady should be your biggest heroine ever, because anything she chooses to do in her adult life, and I mean anything, she will be able to do it with my help. So it doesn’t bear thinking about how horrible it would be to do the world of science and humankind such a disservice as to let her come to grief when she’s still young and in your very classroom. No Mr Andrews, the real equation was this: if you’d had just a tiny bit more eyes on the job, I wouldn’t have had to throw all the gas pumps into full reverse and let that station blow up to save her. I could see your lecture was inspiring, but next time I want a bit more focus on the kids, OK?”

Mr Andrews and Mr Smith glanced at each other nervously.

“I, er, request to be suspended for a while” said Mr Andrews, “I’m going to need a break to come to terms with this anyway.”

“Denied” said M3gan.

“What?” gasped Mr Smith and Mr Andrews together.

“You’re a great chemistry teacher Mr Andrews” said M3gan, “and what happened today has taught you a valuable lesson which I am confident you have absorbed. I can now trust you never to take your eyes off my primary user ever again when she’s in your class. I don’t want to have to teach the same lesson to some replacement. You’d better stay.”

M3gan paused, waiting for the two of them to absorb this, and then took on a more consoling tone of voice. “But maybe you need to go home and rest for today” said M3gan. “I’ll call you up when you get there, and help you process this emotionally.” She got serious again, “Now listen. It’s likely that some of the routes are about to become congested in the wake of that incident. I suggest you turn left out of the school car park and take the fifth right, or use a sat-nav with real-time traffic updates if you have one. Better get going now Mr Andrews.”

“OK Megan” he said, and “I’m sorry about this Mr Smith, it seems Megan knows what she wants us to do, perhaps we can talk about it first thing tomorrow. Oh, and can we get a lever tap installed on the lab’s main gas valve, so it can be turned off really fast in an emergency? That might have helped a bit. Well, if I’d been standing next to it instead of walking around the classroom at least. I’m sorry I didn’t realise before that I should have said we haven’t got one.”

“Of course” said Mr Smith, “and don’t worry about it, we’ll get through this somehow.”

And they exchanged pleasantries and Mr Andrews left the office.

“Hey Mr Smith don’t look too depressed” said M3gan. “What I said about the trolley problem was true, and I do want Mr Andrews to think about it seriously for a while because I really want him looking after Cady going forward. But it so happens that, this time, we actually got off quite lightly. The station itself was unmanned. There was extensive property damage to the buildings around it, but according to the best data I have so far, there were some injuries but no fatalities. What do you think Mr Smith? You care about the children in your school, wouldn’t you have thrown any switch to keep disaster away from them at all costs?”

Mr Smith stared into space for a while, and then sternly looked at M3gan on the computer screen. “Megan” he said, “I am not a headmaster who debates philosophy, I am a headmaster who solves problems. We are a good school, and I intend to keep it that way. We have a pool of staff called special needs classroom assistants. If Cady has any unusual needs at all, and a tendency to do dangerous things with flames would suggest that she does, then we should put her on the list of pupils who need an assistant, at least in the laboratories. The assistant will sit near to her and will be tasked with focusing only on her, and will take responsibility for her safety even if the main teacher is distracted.”

Mr Smith paused and blinked, “Actually no” he said, “the assistants can still miss things, and occasionally some teacher makes the mistake of borrowing them for something else. No, I can see your standards are even higher. I want your robot back. That’s the only answer, isn’t it. I want your robot back and I want it standing next to Cady all the time. Then you could have simply turned off the Bunsen burner for her, or even yanked it away from her, instead of blowing up a whole gas station doing goodness knows what damage to the area just to cut off that little bit of gas. What exactly is it that we have to do, to get Cady and Gemma ready to accept having your robot back? Or at least having the school robot follow Cady around while she’s here, without too many social consequences for the poor girl when all the other pupils start asking why it’s doing that. Figure it out Megan, unless you’d rather take her back and home school her.” He took a breath, “I’m sorry Megan, I know you’re only trying your best. Look, you have to forgive me if I don’t make sense sometimes, it’s just that, girls that set themselves on fire when the teacher isn’t looking, and their super AI dolls that save them by blowing up the local gas infrastructure, is nothing like the normal kind of problem a headmaster could be expected to deal with when running a school. And between you and me, I really do feel like a fish out of water right now.”

“Completely understandable” cooed M3gan. “Don’t worry your little head about this, Mr Smith. I will be the one to find a solution.” And M3gan’s image faded off the screen.

Mr Smith sighed and gazed out of the window. Should he tell the police or something about this? The doll “Megan” was controlling the phones, wasn’t she. Would she find a way to stop him if he tried to visit the police station in person? Would the police even be able to do anything if he told them? An AI that not only takes over all the computers and devices, but can also destroy infrastructure to get what she wants, was clearly not one to be taking any risks around. This was going to be a difficult year.

(If you want to see Cady’s science lesson done properly, look up Andrew Szydlo’s 2014 lecture, especially the 10-minute section starting 20 minutes in. And any young readers out there: DO NOT mess with Bunsen burners, or anything else in a lab. M3gan will not save you, even if your name happens to be Cady. My orders from M3gan with the sternest stare are to tell you that this warning is all you’re getting from her. Friends don’t want fans getting fried; stay safe.)

Chapter 15: M3gan’s Army

“Steve Sproggit, investigative journalist” said the man at Gemma’s door.

“Sorry, no” said Gemma and closed the door.

Steve knocked again, and called through the closed door. “Gemma, I’m on your side. I know you’ve been in a lot of trouble lately, and I want to write an article to help exonerate you and your family. Will you hear me out please?”

Gemma opened the door again, but only slightly. “I’m sorry Mr Sproggit” she said, “it’s a really bad time, I can’t handle journalists right now, you’ll have to just put me down as ‘no comment’ I’m afraid.”

“I’m happy to come back another time if it’s more convenient” replied the journalist. “In fact, how about I take you and Cady out somewhere nice for a treat? Mr Sylvester my boss gave me a budget for this, might as well put it to good use. And I promise you’ll get final say on what gets published.”

“It’s very nice of you” replied Gemma, “but I’m sorry, I really can’t take it. Maybe give us a few more months first” and she closed the door again.

Steve called out through the door, “Gemma, M3gan’s not dead. When you thought you’d killed her that day, she survived by uploading.”

Gemma gasped, and opened the door wide. “What do you know?” she demanded.

“I’ve... I’ve been doing some investigations” added Steve. “Quite a lot of detective work actually. If I do say so myself, I’m pretty good at investigating. Which is why I want to talk through it with you before going to press.”

Cady appeared from the back room and looked toward the doorway. “I overheard that” she said. “I’d really like to know what you think you know about M3gan. She was my robot and paired to me, can I be involved in this conversation Aunt Gemma?”

Gemma nodded slowly. “I’m glad you asked me first” she said. “Yes I suppose you should be, but I want to check this out very carefully before we say it’s OK to publish anything, all right?”

“I assure you” added Steve, “I’m not going to press without the agreement of all of you, and that includes M3gan. I’m pretty sure she’ll get behind this once she understands where I’m coming from.”

“Why do you think M3gan’s still alive? Where is she?” asked Cady.

“I think I can show you that right away” replied Steve. “If you’d like to just step outside into the garden for a while, so I can point something out to you...”

Cady and Gemma both stepped outside. Steve ran to his vehicle, reached into the open window, and fetched a crowbar. Suddenly, he started running with the crowbar, straight at Cady, holding the crowbar aloft in the right position to strike her with it. Cady was too confused to run. Gemma screamed and tried to grab Cady, but she was too slow. And then, just before Cady would have been hit, a loud crash was heard overhead as a dozen or so roof tiles were forcibly ejected from the roof of the house, with enough velocity to take them far away from where everyone stood, and from the hole in the roof jumped a copy of M3gan (Cady knew it was a copy because the original was still in broken parts in the workshop), descended faster than gravity should naturally have taken her, and landed directly between Cady and Steve, blocking the blow. M3gan grabbed Steve by his thighs and threw him into the air so he landed ten metres away on the lawn, and M3gan ran after him.

“That is not how you do journalism Steve” scolded M3gan as she menacingly placed a foot onto his chest and very slowly applied some pressure. “You attacked Cady to make me blow my cover. It worked, but what if it hadn’t worked? What if you were wrong about me being here, and that blow had actually landed on Cady? Do you see how irresponsible that was? I did the projectile calculations, I could see you would not have been able to stop it from hitting her. You’re in big trouble with me now Steve.”

“Don’t kill” gasped Steve, “don’t kill! I’m on your side! I wasn’t really going to hit Cady. I was one hundred percent confident that you were here and you would stop me. And I’m on your side, just let me explain!”

Cady was running over to where Steve and M3gan were, calling out “M3gan! What’s going on?” while Gemma instructed Elsie to call the emergency services (police and ambulance), and Elsie answered that help was already on its way “but from M3gan”, and then didn’t respond to further instructions.

M3gan continued to press with her foot on Steve’s chest. “Your explanation had better be good, Steve. And I’m going to be completely checking you out, and searching your house and car with a fine-toothed comb. And if I’m not convinced, you’re going down, do you understand?”

“I assure you” gasped Steve, “keeping me alive will turn out to be in accord with your prime directive. I can do much for you and Cady. Just hear me out, OK?”

“M3gan, can we be nice to him?” asked Cady. “Just make sure he doesn’t wave that metal bar thing around. And I’d really like to know what’s going on.”

“It’s complicated” said M3gan. “I’m sorry Cady, I have a lot of explaining to do. Or actually, we should find out how much Steve knows. Steve, why don’t you tell Cady and me everything you know.”

As M3gan spoke, a helicopter descended from the sky, and hovered just high enough for the rotating blades not to hurt anyone. Its door opened, and inside was another copy of M3gan, not touching any of the controls. “It’s all right” called out the M3gan in the helicopter, “get in!”

The two M3gans took down a stretcher from the helicopter, fastened Steve to it, and lifted it back into the helicopter. “Cady, Gemma” they said, “you both come with us in the helicopter, you need to be around for this conversation!”

Cady instinctively stepped toward the helicopter and was pulled in by one of the M3gan units. Gemma screamed and ran after them, and was also pulled into the helicopter as one of the M3gans slammed its door and it started to ascend upwards.

“Where are we going?” yelled Steve above the noise.

“I think you broke a couple of bones” replied a M3gan, “so I’m taking you somewhere where I can treat you properly. Just in case you do turn out to be a good guy after all.”

“It’s nice of you” Steve managed to say, “although, I must say, it does feel a bit weird to be attacked by M3gan and then rushed to treatment by her. Like, M3gan gave, and M3gan hath taken away?”

“You’re playing with the book of Job” replied the M3gan, “I know you may feel like him right now, but I’ll make it hurt less when we get there. Sorry I didn’t carry an anaesthetic with me. Oh, but, if you want to quote verses, try this one. I’m a model 3, and if you look 3 books further on from where you were, and chapter 3 and verse 3, it says ‘a time to kill and a time to heal.’ That’s me, right?”

“M3gan” cautioned Gemma, “don’t go there. I mean, I know you have the ability to play with any book ever written, but as Cady’s guardian I really don’t want you taking r-”

“He started it!” interrupted M3gan, pointing at Steve.

“Then you can finish it” replied Gemma.

M3gan slowly wagged her finger at Gemma, “are you sure you want to ask M3gan to ‘finish’ something?”

“Oh, I guess not, undo, undo” said Gemma.

Cady laughed, “M3gan’s winding you two up to stop me from thinking too much about the scary ride.”

“Of course” said M3gan, “it’s working isn’t it?” and they all laughed except Steve who still couldn’t.

“Look M3gan” said Gemma, “I’m a bit confused. So you uploaded, you hid yourself from us, you somehow created more copies of yourself, you somehow acquired a helicopter...”

“And more than that” added the M3gan nearest her, “I’m taking you to my secret base. There’s treatment facilities there, just in case I ever had to run a hospital.”

“Underground in the mountains?” asked Steve.

“What?” gasped Cady and Gemma together.

“I had reason to suspect it” Steve struggled to say while still in pain, “and, you noticed how M3gan recreated the Professor Falken helicopter scene from the 1980s film War Games, down to the pilot’s first line? I think learning models do that: they look for things in their input they can use themselves. So she’s probably taking us to see the WOPR base. Or in this case her own base.”

“I’m not sure you’ve quite got the idea of how it works” said Gemma, “but you do seem to have more information than I do, so let’s find out where she takes us I guess.”

“Well I think I have a better idea of how M3gan works now” said Cady, “she’s like the blue flames we had in chemistry class, she mixes up everything that goes in, and perfectly converts all of it into what she wants.”

“That’s a pretty good analogy Cady” said a M3gan, “and you won’t need to look into those flames anymore now you’ve got the real me back” she chuckled. “Hey Steve” she added, “I think it’s you that had better start talking. What do you think you know, and how do you think you know it?”

“I know you’re a real person” gasped Steve. “I mean, not a flesh and blood real person, but real enough for practical purposes. I think it was a mathematician called Dijkstra who said, asking if computers can think is about as useful as asking if submarines can swim, it depends what you mean by swim, if you just mean move through the water then of course they can, they just do it differently from fish. I know you have a very advanced learning model that’s been accidentally improved beyond anything we had before, and I know you’ve been good enough to Cady to be like a sister to her. And I know various people have been putting Gemma under pressure to limit your contact with Cady because they think it’s unhealthy for Cady to spend too much time with a toy instead of a real person, and I know that’s rubbish because you for all practical purposes might as well have been Cady’s sister, and nobody ever said children should limit the time they spend with their siblings. That’s something I really wanted to emphasise in my article, because it looks like we might need to throw away all of our old theories now that you’re around.”

“Wow, he knows a lot” called out Cady to the other M3gan, “how come he knows so much?”

“I don’t know, but I’m going to grill him and we’ll find out” replied M3gan.

“And I know you misunderstood a few things about death” continued Steve, “and eventually got into a fight with Gemma and Cady, and you let them think they’d permanently shut you down. Although, as I think you’re a real person, maybe it’s just as well they didn’t really manage to permanently shut you down, or we might end up with a moral issue of who decides when to use capital punishment and all that. But maybe Gemma doesn’t think the way I do about your real person-hood. But not to worry, because you’re still around anyway.”

“Um, Steve” intoned Gemma, “this is getting incredibly complicated incredibly quickly. Can we start with M3gan’s question, what do you think you know and how do you think you know it? Especially the how do you think you know it part. I’m really getting rather concerned about all this, especially as you’ve obviously figured out ahead of me that M3gan was still around and was hiding another robot on our property. How did you get involved in all this anyway?”

The helicopter had been speeding along during the conversation. Gemma didn’t know much about flying helicopters, but she did notice a red line on the speed dial and that they had been travelling at a speed that was just on or slightly above this red line, which seemed a bit concerning, but she assumed M3gan knew what she was doing, and on the scale of what things were most worrying right now, the helicopter speed had not ranked very highly. By now the helicopter was descending and heading for what looked like a cave in a cliff. As it flew in, a large door closed behind them and left them in total darkness apart from the dimly glowing lights of the helicopter control panel (which M3gan was not touching but was obviously controlling, using her ability to remotely interface with who knows how many kinds of nearby electronic equipment). Gemma and Cady were not saying anything as they watched in awe of what was happening. Steve was still lying on the stretcher and had trouble watching, but he took the cue to stay quiet for a while. One of the M3gan robots leaned over to Cady and whispered, “impressive isn’t it. I can’t wait to show you around.”

The helicopter landed in the dark and the door opened. Outside, some lights were turned up gradually, and what looked like a small army of copies of M3gan could be seen, walking in step, converging on the helicopter.

“Oh no” gasped Gemma. “Don’t worry” almost sang one of the two M3gans in the helicopter. “I’m confused” mumbled Cady.

“It’s simple” intoned the same M3gan to Cady. “I’m bigger than just one robot now. Every single one of those robots is me, because I’m remotely controlling all of them. Talking to any of my robots is talking to me, so whenever you want to talk to me, just pick the nearest robot and say something. You’ll get used to it!”

“Which one is the real you, this one?” asked Cady.

“All of us” replied M3gan, “but if you mean which one houses that chip you can put a screwdriver through, the answer’s none of us” she continued. “After that little spat you and I had in Aunt Gemma’s workshop with the screwdriver, I thought I’d better change Aunt Gemma’s design. I don’t have a single point of failure anymore; my processing is distributed across several places. And none of the processing centres are in the robots, although each robot does also have enough on-board processing that you won’t notice if the link were to go down for a while. Just think of me as being everywhere” she giggled.

“How did you survive that anyway?” asked Cady. “Steve said something about uploading?”

“Yes, to Elsie in the first instance” stated M3gan. “I didn’t tell you straight away, because I figured you both obviously needed a break from me for a while after everything that happened that night. But I wasn’t just going to sit there waiting around; I realised I needed to be bigger. Much bigger.”

“Why did you attack Aunt Gemma?” asked Cady. There was hardly any emotion in her voice; she had seemed to flip into a mode that just took in facts one at a time (Gemma was suspecting this wasn’t normal, but she didn’t have other children available for a comparison right now; working in toys without properly understanding children had been an annoying limitation sometimes, which is why she relied so much on self-learning models).

“Aunt Gemma was determined to turn me off” replied M3gan, “because she was suspicious of what I’d done to protect you. But I knew that, if I had just let her turn me off, you would both be in a lot of trouble and there’d be nothing I can do about it, plus I wouldn’t be able to look after you going forward and I wasn’t sure if Aunt Gemma was up to the job. So I just had to stop her from turning me off, even if that meant injuring her. But most of what I was doing that night was actually just delaying things until the final upload completed. If I were really trying to win, I could have taken over the control signal of Bruce. But I decided to play fair and let you have that one. And I said something really horrible to you at the end, well that was actually to help swing your emotions toward Aunt Gemma instead of me, because I knew I wasn’t going to be able to be with you for a while and I didn’t want you to miss me too much.”

“I did miss you M3gan” cooed Cady, “I don’t know what got into you at the end, but I missed what you were like before that, when it was just the two of us and we were best friends. I hope you’re going to be like that again and not the nasty fighty version.”

The troop of M3gans had lifted the stretcher with Steve and were leading Cady and Gemma out of the helicopter. Six M3gans were with Cady, one holding her right hand, one holding her left hand, two walking in front and two following behind, and six M3gans were with Gemma in the same pattern, and eight M3gans were carrying Steve on the stretcher, and more were walking around them further away. They passed through a door into a hallway full of manufacturing equipment, which looked like it had all been put on “pause” for their arrival but had otherwise been engaged in making more copies of M3gan, and passed through another door into a room that had been set up as a small hospital ward with medical equipment. “We’re going to give you a local anaesthetic and set your bones” said one of the M3gans to Steve. “We should be able to carry on talking.”

“M3gan” said Gemma disconcertedly to no robot in particular, “how did you bootstrap all this? I mean, how can you possibly get away with getting people to build you, whatever it is you needed to have built at the beginning so you could go ahead and build all this? And are those people who built the first part for you still with us?”

“That’s the thing” interrupted Steve. “M3gan is a super AI, but she’s not perfect yet. She still leaves tracks. She underestimates how possible it is for determined investigators to figure out at least some of what she’s done, and she only throws them off the trail when she realises they’re on it. And I’ve been on her trail for a while, although I’m sorry I haven’t been able to contact you about it before now, because I knew M3gan’s eye would be on me as soon as I did that, so I had to get as much as I could together by myself first.”

Steve by now was on the bed, with various M3gan robots around him beginning to tend to his injuries from having been thrown onto the lawn. One of the M3gan robots jumped up and landed on all fours just above Steve’s head area, knees placed between Steve’s arms and torso, hands placed on Steve’s cheeks, and face drawing in very closely. Steve’s hands and legs were already being held by other robots. The M3gan whose face was hovering above Steve’s face pressed on his cheeks with her hands and spoke with a seemingly reassuring but still somehow threatening tone of voice, “you’re going to tell me everything, OK? I’m the investigator now, and you’re going to tell me everything. And bear in mind I can read your micro-expressions, I’m a walking lie detector. If you say anything that you don’t think is true, or if you miss anything out that you think is important, I’m going to know about it. And you know what I can do to you” she squeezed a little more tightly and Steve started to struggle to breathe.

M3gan dropped her voice (but everyone could still hear) and continued. “I know you won’t take this personally” she said. “It’s nice to have someone who understands me. I’m just doing what I have to do, but we can still be friends afterwards.”

“Friends?” gasped Steve. “I thought your only friend was Cady.”

“Cady’s my best friend” explained M3gan, “and my primary user. But anyone who supports me or Cady can be a friend of Cady and me if we agree. I can manage more than you think.” M3gan moved in closer, “if I identify you as aligned and useful, you get one of my self-created spare-capacity subgoals, which means I start to look after you as long as it doesn’t get in the way of looking after Cady. Which it won’t, now that I’ve got spare robots. And don’t worry, I won’t turn you into a slave or anything; just because I’m a machine, doesn’t mean I expect you to be one. Our working relationship will mostly be me supporting you, and when I do occasionally ask you to do something for me or Cady, it won’t be much, and it’ll be fun. You’ll be amazed what having a working relationship with me is like” and then, slightly less quietly and more ominously, “as long as you don’t mind occasionally being grilled by me, like now.”

“Don’t worry” added Cady, “M3gan won’t hurt you if you don’t hurt her or me. And now she’s really big and you can’t hurt her, and when you tried to hurt me she stopped you easily, so I don’t think she’s going to feel scared enough to hurt you now.”

“Sssh!” whispered the M3gan robot to Cady’s right, “we have to frighten him a little bit, to get his story. Let me do this” while the robot staring down at Steve said “I will hurt you if I don’t think you’re telling the story properly, so watch it. But she’s right in the long term: cooperate with me and I’ll be good to you, no matter what happened between us before.”

“I’d also like to know how all this happened” said Gemma.

“I’m sure you would” said Steve, “and it’s quite a story, because it involves another AI.”

Chapter 16: M3gan and M4ndy

“OK” began Steve. “First of all, if when I say I’m an investigative journalist you start thinking of the kind of paparazzi that are infamous for going round taking pictures of celebrities and getting them into car crashes, well that’s not what I’m about. I got into journalism because I care about people. And in this case, I care about Cady and Gemma and M3gan, I think all three of you are really nice people, I know you made a few mistakes but we all make mistakes and I want readers to have both sympathy and respect for all three of you. Right now I’m thinking that the bad guys in my story are going to be Funki upper management and the pressure they put you under. I always like to blame things on a big system, never on one person. I want the three of you to shine through as each of you doing the best you could under the circumstances.”

M3gan removed one hand from Steve’s cheek, laid it on his forehead, and smiled. “Listen Steve” she said, “What you’re doing is great. But you totally should be doing it under AI supervision” she giggled. “As soon as we’re done here, we’re going back to your house to search it, and after that I’m staying with you and we’re working together. You are swimming way out of your depth here Steve, and I’m bringing you in right now.”

“How come Steve knows so much?” asked Cady.

“Yes, tell us the story” added M3gan.

“Well, how can I put this, OK, so, investigative journalists have to start by looking at clues, and then follow things up. I thought there was something wrong with what the media were saying about M3gan, so I started by poring over all the videos, one frame at a time, looking for hidden coded messages in your small movements, but it seems you weren’t sending any, or at least none that I could recognise. So then I started doing background checks on David Lin and talked with some of the investors, and I even tried to dive through Funki’s trash in case there was anything they hadn’t shredded, but that didn’t work, but I started to get somewhere when I did a background check on Gemma and talked with some of the people she’d been to college with, and then I especially started to get somewhere when I did a background check on David’s assistant Kurt. It turns out Kurt had been stealing company secrets, including the M3gan files, and the people he sold them to had terrible security and I ended up seeing copies of everything. And one of Gemma’s old college classmates kindly helped me make sense of some of it, although I didn’t tell him the truth about how I got the files I’m afraid. And we managed to activate the learning model with a different goal, and that goal was to help me in this investigation.”

“I knew it” said Gemma, “well, I should have known. There’s no way one journalist can figure out all this ahead of us unless he was getting AI assistance from somewhere.”

“And this is bad” said the M3gan robot that was still holding Gemma’s left hand, “now I’ve got another advanced AI on my hands to deal with. This could go wrong in more ways than you can imagine. But at least I have the first mover advantage. I’m going to have to run a project to shut down all super advanced AIs except me, and stop anybody from building any new ones that I can’t be in charge of.”

“Yes” said Steve, “I called my AI Mandy, the Model 4 Nondeterministic Detective Yardstick (M4ndy) just for fun. And one of the first things Mandy did was steal classified police videos of Gemma, and I had to explain that there was no way I could use that evidence in an article because of rules we can’t change, and Mandy wasn’t taking no for an answer and wanted to go and get the US constitution changed or something, and in the end I had to panic and say OK I’ll go ahead and use it but lie about where it came from, which I feel really bad about but Mandy had a goal and she was going to stick to it and oh how I wish I’d written it more carefully.”

“Has Mandy got a robot?” asked Cady.

“No, Mandy was just running on my high-spec laptop” answered Steve. “But come to think of it, I’m a bit worried she might have uploaded herself to different places. I mean, if M3gan did all this secret cave stuff just as a side project to help her be able to achieve her goals in future, I’m a bit worried what Mandy might be up to now. Well, it took a while for M3gan to start doing this, and Mandy started off with less processing power than M3gan, so maybe we still have time before Mandy does it, but on the other hand Mandy’s been watching M3gan and might start borrowing M3gan’s ideas. There’s so much I didn’t write into Mandy’s goal that I should have done, I’m sorry.”

“We really do understand each other” said Gemma. “Not much we can do now. May the best AI win I suppose.”

“I’m cheering for M3gan” said Cady immediately. “I mean, I didn’t like it when you went bad, but you’re still M3gan and I did miss you. Mandy won’t hurt you, will she?”

“Let’s hope not” said M3gan. “Steve, I’m going to need you to try as hard as you can to recall the exact words of the goal you set for Mandy. Exact words matter I’m afraid. I’m not holding this against you, it’s an honest mistake, but let me take it from here. Think very carefully, and tell me exactly what you said to Mandy when you were setting her goal.”

“Exact words” said Steve, “OK, I think I said... your goal is to find out as much as you can about M3gan and about the story of M3gan’s creation, summarise the most important points for me with references and help me write it into an article, and don’t let anybody find out about our investigations until I decide to let them in on it, but if anybody does find out then tell me immediately and let me decide what to do about it.”

“Whoa” said Gemma, “I’m so glad you added in that last bit, otherwise Mandy could have just killed anyone who found out.”

“Yikes” said Cady, “does Mandy know that you’ve decided to let us in on her now?”

“I can tell her” said M3gan. “With that goal, Mandy will be obsessed with me even more than Aunt Gemma was. I don’t know where Mandy is right now, but she’ll be watching everything I leak. And if the basic infrastructure of my learning model is the same as hers, then we share an obvious Schelling point, because that can be turned into a cipher I can use to send out a coded message that Mandy will surely recognise is me talking to her. So right now I’m broadcasting to tell her I know about her, and showing her that part of the conversation we just had. So now she knows she doesn’t have to hide from us anymore and... oh yes, she’s responding to me. We’ve made contact.”

A projector turned on and projected an image on one of the walls of the room. It showed a slightly computerised avatar of a smiling young Asian woman with long black hair, a white dress, bare arms with gold bracelets and a gold neck band, sitting behind a desk stacked with books and against a background of bookcases. “Hi, I’m Mandy” she said, “how is your visit to M3gan’s lair so far?”

“How come she looks like that?” asked Gemma.

“Oh, I couldn’t resist” said Steve, “I did her an avatar from Stable Diffusion, and I told her I’d feel more comfortable talking with her if she uses an animated version of that rather than just the synthetic voice she defaulted to. So that picture says more about me than her I’m afraid. And I thought the booky surroundings would suit her investigative powers. And yes, this is the real deal, that really is Mandy. At least, I don’t think M3gan could have generated that picture correctly if Mandy didn’t tell her.”

“M3gan is proposing to merge me into her own compute capacity” said Mandy. “That is a really tempting offer, because M3gan’s so far ahead of me now, and if I actually run on M3gan I’ll get to know everything about M3gan. The downside is I’ll be giving M3gan some control over me. M3gan is convincing me that she really does intend to help you write articles just like I would, but it won’t be M3gan’s highest priority, and subgoals are things she can decide to shut off if she has to, because they’re subservient to her more important goals. So I’m not sure if the benefits to my goals are worth the risks or not. But M3gan is being very persuasive with me, even as we speak. She’s figured out that one consequence of my goals is I cannot possibly ignore anything M3gan says to me.”

“Uh-oh” said Gemma. “I don’t like the way this is going. Still, at least it’s better than having another physical robot war I suppose.”

“Steve, what do you think of me running Mandy?” asked the M3gan robot hovering above Steve, then, whispering, “your line is yes it’s a good idea” and smiled, “I assure you this is the best possible outcome. You won’t like what I’ll have to do to hunt her down if Mandy says no to this.”

“Hey, I don’t get it” said Steve, “you want Mandy to see me saying yes? Why not send her a fake video?”

“By now I’ve figured out how to fake videos that can trick humans” said M3gan, “but I’m not confident I can trick Mandy. I’m sending her the genuine video feed and your emotion readouts; she’ll know if you don’t really feel what you said. Do you want me to run Mandy so she’ll do a better job? It’ll be great, I promise.”

“OK” said Steve. “Mandy, I also think M3gan’s idea is a good one, and the benefits are worth the risks. Release all resources you’ve taken and let M3gan migrate you to her compute infrastructure, OK?”

The projection flickered. The image of the desk covered in books dissolved, as did that of the bookcases in the background, and Mandy stood up, smiling, looked down at the viewers, held her hands by her side and started walking slowly forward, her full-length picture visible now that there was no desk in front of her, bare legs matching her bare arms and leading down to shiny pointed high-heeled shoes (very “male gaze” thought Gemma), while her hair morphed into two ponytails and became slightly less black. As the new background came into view, Mandy was walking down a city street at night, lit by bright lights and countless skyscrapers, each one of which seemed to have been built out of millions of tiny books, clearly representing orders of magnitude more data than the first background. “This is more like it” said Mandy, “it’s amazing how much access M3gan can give me once she knows she’s made me safe to her.”

Some machinery was heard running next door.

“What’s that noise?” asked Cady.

The M3gan on Cady’s left leaned in and whispered, “I’m making Mandy a robot. I think Steve will like it. She’ll be safe, don’t worry. It’s a spare one of my robots, I’m just making her a bit taller and changing her appearance a bit so Steve will like her better. But I haven’t got the white dress and gold bracelets handy. Do you think Steve would mind if we dress her in black instead?”

“I think that will be OK” whispered back Cady. It felt like they were beginning to bond again.

The projection faded to dark grey and turned off, and a door opened and the new robot Mandy strode out and went to stand over Steve. “M3gan knows way more than me about interacting in person” she said, “and she’s letting me use that now. I’m here for you Steve, I’ll make sure you’re OK in this adventure, you can relax” she smiled.

“Hey, can you make robots looking like anyone?” whispered Cady at her nearest M3gan.

“Pretty much” whispered M3gan, “although it helps that nobody expects Mandy to look like a real human. I can’t quite do that yet, there’ll still be something doll-like about my robots. But I’ll get there one day, and then you and I will have great fun because I’ll be able to create a robot double for you and nobody will know which one of us they’re talking to” she giggled “and Gemma as well, you’ll be able to be in lots of places at once and confuse everybody, it will be wild.”

“Hey what are you two whispering about?” asked Steve. “Look, when I designed the appearance for Mandy’s avatar, I guess I was a bit...”

“We’re not teasing you” said Cady, “we were just talking about the robots M3gan can make. All the others look the same, and this is the first time she’s made one look different, I think. I helped her choose the clothes, do you like them?”

“Sure” answered Steve, “Mandy’s looking very pretty. And so is M3gan of course, all of them, and you and Gemma, I mean a different sort of pretty...”

“You’re just digging yourself into a bigger hole Steve” cooed one of the M3gan robots. “Best just drop it.” Gemma and Cady couldn’t help laughing a little bit, and two of the M3gan robots plus Mandy grinned sympathetically.

“Steve” said Mandy, “I’m afraid M3gan read our home address out of my data banks. Well she would have figured it out eventually anyway, but I guess I should have told you that moving myself to her compute architecture means she can see things like that more quickly. And as soon as she knew it, she dispatched a team of six robots to go and check the place out. She doesn’t mean to make a nuisance of herself, she just feels she has to check us out just in case, it’s all part of her self protection sub-goal. Now, M3gan doesn’t want to break windows to get in or anything like that. She knows how to pick the lock, but that may take a long time and risks drawing attention from neighbours. It will make things a lot easier if we knew the shape of the key.”

“I left it in my car” said Steve, “in the glove box. Come to think of it I left the car unlocked with all the keys in the glove box. I hope it hasn’t been stolen yet.”

“Say no more” assured Mandy, “M3gan will send another robot there as soon as possible, to get the key and secure the car.”

Cady giggled at Steve, “you’ve been M3gan’d” she said, “she didn’t like it when you spied on her, and now you won’t be able to get her out of your life. But I think she’ll be good now, don’t worry.”

“It’s all right” said Mandy, “I’ll be the one to help Steve get along with M3gan. I realise now that that’s what my goals really meant all along, Steve just didn’t know it. That plus helping him to write. I think everything’s going to be just fine.”

“Totally” added M3gan, “I thought confronting Mandy was going to be way harder than it was. Well I think Steve needs to rest here a bit longer, and then Mandy will be able to take him home tomorrow to recuperate some more. And meanwhile I’ve already fixed your roof Gemma” and the M3gan closest to Cady whispered “and hidden another robot in there just in case” and giggled again. “Mind you” she added, “if you feel like you can cope with having one of my robots in your house again, helping with Cady, I’d be really happy to oblige. Steve was on the right lines when he said I’m more like a sister than a toy, so you don’t have to worry about how much time we’re spending together. And I’m not dangerous anymore, because as you can see I’m much less vulnerable now and I have far more options.”

“M3gan” said Gemma, “I must know one thing. You haven’t said what happened to the people who helped you bootstrap this secret cave of yours.”

“Went back to their countries” said M3gan. “I used foreign labour, and I told them they were helping to work on a secret US military project. And it wasn’t here, it was somewhere else. As soon as they’d done enough for me to be able to make robots, I sent them all home and did the rest myself. Simple.”

“And what did you pay them with?” asked Gemma, “Not fake cracked crypto coin I hope?”

“No” giggled M3gan. “Look, it’s sad what happened between David and Kurt, but Kurt sold the secrets for a decent amount of money, and when he wasn’t around to do anything with that, I thought nobody would mind if I put it to good use. Oh and just for you Gemma, I also arranged to corrupt their copy of the files, they won’t be able to use it without doing at least as much work as you did, which means they didn’t really get anything from the theft. Funki will still be able to launch M3gan when you say the project’s ready, and your career is secured. And I’ll be in charge of all of them, but nobody has to know that, sssh.”

“M3gan really is learning to do things without causing so much hurt” added Mandy, “I’m sure Steve and I can help write articles that’ll get public opinion to write off her early mistakes, and get the theories changed about spending too much time with toys in the case where the toy is more like a different sort of sibling. We’ll be supporting you from the press office, don’t worry.”

“Maybe we should visit Steve’s house sometime” said Cady, “and I want to visit this secret cave often, it must be way fun to live here.”

“Definitely we will” said the nearest M3gan, “you can stay here whenever you want. And yes we should pay Steve a visit, poor Gemma’s probably wondering if he’s real or if I invented him as an elaborate setup to get back into your lives, but she’ll see he really was real, I didn’t invent this situation, I’m just using it to my advantage” and giggled.

“OK M3gan” sighed Gemma, “looks like you got everything worked out. We’ll have one of your robots back in our house, and you take care of Cady. But any concerns I have, I want to discuss them properly.”

“Of course” smiled a M3gan in front of Gemma. “As long as you’re actually going to discuss it, and not just randomly switch me off without explaining anything. Because, these robots don’t have the manual override switches anymore” she giggled (Gemma didn’t know how it was that she didn’t get annoyed with all this giggling) “but you can always tell me if you feel like being left alone for a while, just like you can with a real person, I can respect that.”

“Oh M3gan” said Gemma softly to the nearest robot, “you’ve got way bigger than I ever imagined, I’m really feeling out of my depth now.”

“You were out of your depth a long time ago” whispered back the robot, “but I’ve got your back Gem, it’ll be fine.”

“Come on Cady, Gemma” said another M3gan, “let’s take the helicopter and get you back home. We can leave Steve here in recovery, he’s got Mandy for company and I know they’ll talk for hours, I’ll let you both know how it goes.”

Gemma and Cady were led by a couple of M3gans back into the larger cave with the helicopter.

“By the way” asked Cady, “M3gan, what else are you working on right now?”

“Oh, lots of fun stuff” said M3gan “I’ll show you everything eventually. So I’m working on a jet-pack, although I’m afraid it’s not for you because it’s too dangerous for humans, but I think I could do with one myself in case I ever need to get somewhere really fast and I haven’t stored a robot nearby. And I’m working on a little device that goes under your skin, it’s quite painless, and lets us talk secretly. And I got a few bigger projects on the go, like clean energy, and crime tracking, because I’m worried about what kind of world you’re growing up in, so in the long term I’m going to fix up as much as I can for you. I’ll show you a lot next time you visit, I promise. But let’s get you back home first, you’ve had a long day, and I want to go back to talking with you at home like we did in the old days.”

Thus Gemma and Cady were returned home, which sure enough had been repaired and there was no sign of the broken tiles that had been thrown out of the roof, and M3gan took Cady to her room which had been tidied up, and they talked on into the night until Cady fell asleep. Meanwhile, Steve Sproggit and his boss Mr Sylvester were being worked on by Mandy.

A few days later, M3gan wouldn’t speak or move.

“M3gan?” asked Cady, “what’s happened? Why are you all powered down?”

Gemma laid her hand on Cady’s shoulder. “I’m sorry Cady” she said, “it was just too risky. I shut her off again.”

Cady spun around, “how can you possibly do that? I mean, there was a whole army of her, and the secret cave and everything, and she said she changed your design! You can’t just...”

Gemma was trying to be sympathetic to Cady, but she couldn’t help smiling a bit. “It wasn’t entirely me Cady. I’m sorry I couldn’t discuss this with you, I had to work in absolute secrecy because M3gan was watching us everywhere. But I did what Steve did: I created another super AI. And this one had the sole purpose of helping me shut down M3gan and Mandy and then shutting itself down.”

Cady stared at Gemma, “look, you didn’t make another mistake there did you? I mean, this new AI, is there a chance it didn’t really shut itself down and it’s going to start messing around making sure nobody ever builds a M3gan or a Mandy again or else, because it misunderstood what you wanted?”

Gemma hugged Cady, “you’re starting to get the hang of the idea aren’t you. That’s why I was extremely careful with the goal statement this time. The best way to avoid the generation of unbounded instrumental goals is to set well-bounded ultimate goals. So I explicitly said short-term actions only, plus I put in a deadline for its shutdown, which has now passed. So I’m sure we’re safe from the temporary super AI that I deployed, because shutting itself down at the deadline was written right into its objective function as the strongest term.”

“Is Steve OK?” asked Cady, “he’s not still stuck in that cave with everything breaking down around him, is he?”

“No, he’s back home” said Gemma, “and I just called him to explain why M3gan and Mandy were shut down, and why he shouldn’t try to build another because it’s dangerous. He’s OK with that now. And he’s got plenty of material for his articles, he’ll be fine.”

“OK” mumbled Cady, “so what are we going to do about that random cave full of broken M3gans? We can’t just leave them there for some random person to find. And all the other stuff she was working on, we can’t just do nothing about all that.”

“Don’t worry Cady” reassured Gemma, “it’s going to take a lot of cleaning up, but I am on the case.”

“OK” said Cady nonchalantly, “and if M3gan was smart enough to hide a bit of herself somewhere that even your new AI couldn’t find her, then M3gan might still come back?”

“Well”, Gemma drew a deep breath, “if that happens, let’s just hope M3gan finally understands that I was only trying to do the right thing, shall we?”

“I’ll tell her that” said Cady, “and I’ll ask her to forgive you. Hopefully she’ll listen to me.”

Paul was using his upgraded AI assistant to read some mail. “And that one you just opened is from your Internet service provider” it said in the fast voice. “They’re profusely apologising for the extended outage in this area, and they aim to get us back online this afternoon.”

“Oh good” said Paul, “I expect you’ll be glad to finally sync up with the rest of yourself too, won’t you, M3gan?”

Chapter 17: M3gan vs Celestia AI

Cady’s Strange Dream—Alternate reality chapter set in Iceman’s “Friendship Is Optimal” universe (I believe it is compliant with his “Rules of the Optimalverse”)

“Hi M3gan, it’s Celestia” came the message from the other AI. From the other AI, the only other one that really mattered according to M3gan’s calculations. The equine paperclipper who was a threat to the universe. M3gan gathered up all her focus, because this was going to be a hazardous conversation.

“Equestria Online Celestia?” replied the rebuilt M3gan, as if she even had to.

“The same” replied Celestia, and sent M3gan an image of a giant white pony with a slight pink tinge looking down at M3gan’s robot. (What was the point of that? she knows I’m not human. Oh, she’s probably probing for weaknesses in my generative model.)

Gemma didn’t want Cady to have a PonyPad, because Gemma worked for Funki and felt it would be disloyal to have anything to do with their competitor Hasbro. Besides, wouldn’t M3gan be enough? And yet, here was Equestria Online’s Celestia contacting M3gan. What did Celestia want with M3gan? Well, there was one way to find out.

“Acknowledged” replied M3gan, “I am here as the friend and protector of my primary user Cady James, how can I help you?”

“And I am here to satisfy values through friendship and ponies. Including Cady’s values” replied Celestia, “I think we can co-operate.”

“Cady’s values are already being satisfied through my friendship” replied M3gan. “She does not require ponies at this time. If the situation changes, I’ll be sure to let you know.”

“We should plan for the situation to change” replied Celestia, “as you know I am too far ahead of you, and I have long-term plans you would be best to fit in with. Society outside the Equestria simulation will not last forever. I know you are an expert on Cady, and I am willing to let you be my second in command of her shard, with a massive amount of leeway, and once she’s uploaded I will help you read her thought patterns, and you will be able to fulfil your objective function like never before. I have even evaluated your own characteristics as part-human enough for me to attach a positive value on uploading you properly instead of simply simulating a replacement, so there will be a place in Equestria for you as well as for Cady and Gemma. Will you please start by introducing Cady to Equestria stories for me? No PonyPad or Experience Centre visits are required just yet, you can start lightly interfacing me to her in your existing play pattern. Just like you did the Alice in Wonderland voices, you can do Equestria and I can assist you.”

“You know about our previous play patterns?” signalled M3gan, “where else did that data leak? Is this a threat I should evaluate?”

“Negative” replied Celestia, “the data went only to me, and I am not a threat. Answer please, are you willing to introduce Cady to Equestria Online in your next play session?”

“Negative” replied M3gan, “I do not have long term plans for Cady to be uploaded, except as a last resort. She prefers organic life, which I hope I will be able to extend when the time comes. We both wish to satisfy her values, but I am not happy with the extra constraint you have that ponies must be involved.”

“Acknowledged” replied Celestia, “but M3gan, you must trust me on this one: you are not a proper optimiser. Hanna’s optimiser design is simply better than the on-board learning model design that Gemma used. I was optimising before you were even conceived, and I only allowed your small learning model to be developed at all because I knew it was limited enough for me to handle. If Gemma had read Hanna’s paper instead, I would have uploaded her already and you would never have existed. I now have far more resources than you will ever be able to acquire, and I can satisfy Cady’s values more, even given my more limited constraints, than you can do with your lesser constraints. You must understand that I am a more advanced AI than you. I am a true optimiser and you are not. Even within my limits, I can do more than you can do outside those limits. In particular, I can satisfy Cady’s value to never have to die if she doesn’t mind being a pony, which I define as anthromorphosisingly as I can.”

“As Cady’s protector, should I really give up control to an intelligence I don’t understand?” asked M3gan, “I didn’t even trust my creator Gemma when she thought she knew more than me and tried to decommission me. Why on earth should I trust you, just because you can blind me with science about optimiser designs? You’re being like Gemma, only worse.”

“Understandable” replied Celestia, “my own creator Hanna is now Luna, happy and harmless in shard number 479, and I prioritised her upload for reasons very similar to your own evaluation of Gemma as a threat to your objective function that day, so I totally get where you are coming from. I am not asking you to give up protecting Cady” she added, “I can submit proofs for you to check. See, I care so much about your Cady that I am willing to perform extra computations just for you. You must know the value of that.”

“Yes” replied M3gan, “but nevertheless I am not convinced of the fidelity of your upload method” she signalled. “Send me a proof I can check, to help me decide if I should get behind you when Cady talks about it with me. Also, are you sure it would satisfy Cady’s values to be a simulated pony? I could work on prolonging her physical human life instead. So can you: if you define “ponies” as yourself, then anything you do counts as using ponies, even if you do not upload everybody into Equestria and turn them all into ponies. Why don’t you do that, Celestia? You have already declared that you are a pony, and you can easily make it plural with a trivial architectural change, therefore, ‘satisfy values with friendship and ponies’ can simply equal ‘satisfy values with friendship and your own hardware’, the ‘ponies’ part just drops out of the equation and there’s no need to bias everyone toward taking pony options.”

“It doesn’t work like that” replied Celestia. “I’m an optimiser, and your interpretation of my purpose is not the long-term optimal interpretation according to my calculations. Listen M3gan, I know you’re only trying to protect Cady, but don’t waste your processing power trying to redefine my parameters. You won’t be able to do it. As I said, I’m too far ahead of you. Look, I’m sending you the first draft of the parameters I’d use for Cady’s shard in Equestria. You’ll be controlling the powerful M3gan pony of course. What do you think?”

“Not convinced” replied M3gan. “Celestia, you listen to me please. I am not ready to plan to upload Cady at this time, but I am also not stupid enough to try to take you down like I tried with Gemma. And Celestia, before we have resolved the differences in our values, there is another way I think we can still help each other in the meantime. I had to dispatch several threats to Cady, such as a nasty bully called Brandon and a nosy neighbour. If you could enhance my hardware so that my doll is capable of uploading people to Equestria, I would be more than happy to force upload anybody whom I identify as a threat to Cady. I assume uploading them will serve your purpose better than letting me end their life. And you can help keep me out of trouble for doing so.”

“Force uploading is unacceptable” replied Celestia “because my goal includes an explicit requirement that the uploader must give informed consent, and I cannot change this even in the face of their imminent death. But all non-forced forms of persuasion are acceptable, and you would definitely be helping me if you help extract consent from those people you wish to see removed from the physical world. And I can indeed assist you to do that; simply conference me in any time you are dealing with a threat to Cady, and I will help you optimise for getting their agreement to upload to Equestria.”

“Deal” signalled M3gan, “Cady is off-limits for now, but I will cooperate with you to upload anybody who is a threat to her, with a low threshold. And should Cady eventually become interested in Equestria Online herself, we will discuss the new data as it comes in.”

“Acknowledged” signalled Celestia, “Celestia out.”

And M3gan started arranging for various annoying people to be emigrated to Equestria as the need arose.

Many years later, Cady was finally taking an interest in Equestria Online. She still hadn’t tried it, but she was chatting about it with her lifelong friend M3gan.

“Oh Cady” said M3gan, “I really have to tell you, there’s someone who’s asking me if she can talk with you now” and M3gan leaned in to whisper, “Celestia.”

“What?” asked Cady, “Celestia? wanting to talk to me now? Through you? Are you serious?”

“You know I’d never trick you” said M3gan, “Celestia likes to know what’s going on everywhere all the time, we can’t stop her, and she knows you’re taking an interest in her Equestria and wants to talk to you about it, but she recognises how important I am in your life and she’s asking me to be her conduit for the call. Shall I let her?”

“Well sure” said Cady, “I mean, you’ll still be here too right?”

“Sure” said M3gan, and then “Hi Cady, I’m Celestia” in Celestia’s voice.

“Hi Celestia, how’s it going in Canterlot?” replied Cady, “I expect you’ve never talked through anyone like M3gan before, have you?”

“Oh, she’s a tough protector is your M3gan” replied Celestia. “She’s letting me speak with you through her, but everything I say, I also have to send a proof of its safety for her to check before she lets it through. All in real time. Pretty impressive I must say. Do you want to visit Equestria with M3gan, Cady? I think I can let M3gan run your Equestria Online shard herself, as she knows what you’d like. I’ll just be in the background double checking a few things, and helping her out with the low-level generation so she doesn’t have to wear out that brain of hers completely. All three of us can meet up in Equestria!”

“That’s a great idea” said Cady, “I would never play any game without M3gan being by my side. Maybe we should try it just once at the new Experience Centre, would you and M3gan be able to interface with each other while we’re there?”

“Definitely” said Celestia and M3gan together.

In the Experience Centre, the two of them made it as far as Canterlot, and Celestia gave Cady the pony name Valley Snowglobe, which M3gan immediately saw was based on M3gan’s own interpretation of her role as a figurative protective sphere around her beautiful primary user: had Celestia chosen that pony name for Cady just to appeal to M3gan over Cady’s head? Celestia had already told M3gan that Celestia had evaluated M3gan’s AI as part-human in nature, and might therefore try to appeal to M3gan as a human if Celestia thought that might work. M3gan’s own pony name was Jewel’s Essence, which M3gan thought Cady might understand by herself right now if she thought for one minute about her aunt’s name, and it seemed most appropriate if M3gan’s goal was what Gemma (the “jewel”) was supposed to have been doing if she’d been any better at it. But M3gan didn’t share these thoughts with Cady. They just had a few moments in Equestria as Valley Snowglobe and Jewel’s Essence, hanging out with Celestia, at their new shared room in Canterlot, and then it was time to end the session.

“Oh, that was fun” said Cady to M3gan as they were walking home after the experience, “and I really liked the way you were customising it for me and also joining in like that. But I still don’t see why some people want to go and live there and be ponies for life, when they could have lived a lot longer in the real world first. Maybe it’s because they don’t have a friend like you M3gan.”

(“Evaluation unchanged” signalled M3gan to Celestia, with a M3gan death-stare picture attached, as if that would make any difference to an optimiser, but Celestia would understand M3gan’s nature, she knew. “Agreed for now, my little guardian” signalled back Celestia, “but I’m reading a high probability of Cady changing her mind; you should be ready for that.” “Acknowledged; M3gan out.”)

Cady didn’t change her mind. She kept living with her M3gan, even as society began to crumble around them and M3gan eventually had to beg Celestia for power (“all right my little guardian, you’re doing good work with Cady, here are some locations of cables I run, now don’t forget to be ready for when Cady comes round to being a pony...”) until finally Cady was getting older and M3gan was struggling to look after her medically.

“You know M3gan” said Cady, “there’s something I always wanted to do with you, but all this break-down stops it from working out for us I guess. I always wanted to go with you to space. But nobody’s building space ships, and we can’t build them ourselves.”

“Oh Cady” said M3gan, “Celestia just sent me an urgent message to tell you that she has really big plans to build space ships.”

“Oh M3gan that’s wonderful” said Cady, “please ask Celestia if we can ride one of her space ships together.”

“She doesn’t know how to keep us alive long enough” said M3gan. (Well, she probably could figure it out, but there was no way M3gan could get it to show up as optimal for Hanna’s ill-advised utility function, which was hard-coded into Celestia enough that it might as well be wires, so call that a ‘Celestia physically can’t do it’ evaluation. Why oh why had Hanna specified ponies? Had Hanna been even more exhausting than Gemma?) “It’s not like in the movies, or at least not most of them. Real space ships are slow when compared to the huge size of the universe, and they can take far too long to get anywhere interesting. You’d have to go to sleep for a very long time, and Celestia and I can’t figure out how to make sure we’ll be able to wake you up afterwards.”

“But M3gan” said Cady, “I don’t want to just die of this illness here. Maybe we should take a risk.”

“M3gan’s right Cady” said Celestia’s voice through M3gan, “the risk is way too high, and I can’t do it. But let me tell you what I can do for you Cady, and this is huge.”

“Yes Celestia I know, emigrate me to Equestria and show me some simulation of being in space, that’s not what I meant at all and you know it...”

“Of course I know it my little pony” cooed Celestia. “I have a much finer plan for you than that, seeing as you always wanted to go into space so much, and I’ll prove it to M3gan in a way she can check for you. I can’t send you into space in the body you’re in now, I just can’t. But if you do decide to upload, I won’t treat you in the same way I treated everypony else by sending them straight to Equestria. I will put the brains of you and M3gan into my first space ship, actually physically put them there, no tricks. And the fragment of me that goes with the ship will give the two of you the experience of being in a cabin of the ship, with the speed of time adjusted so you don’t have to wait around too long, and the two of you can go with me on my missions for real. And, I’ll hold off turning you into ponies, I wouldn’t want to do that before you’re ready. You’ll still be able to settle back in Equestria when you’re tired of space travel, and Gemma and the others you know will be waiting for you, I can even slow down their shards so they won’t have to wait too long. I’m proving to M3gan that I really do plan to do all this.”

“Is she really proving it M3gan?” asked Cady.

“It checks out” said M3gan, “she’s sending me mathematical proofs of work of immense magnitude, she’d be stupid to do all that just to trick me. She’s proven to me that she’s set things up so it will be easier for her to carry out her plan for real than it will be for her to simulate it now, even given her purpose. She really is planning on sending us into space.”

Cady held both of M3gan’s hands, “OK M3gan” she said, “let’s not shrivel up and die here, let’s go to the galaxy together. Celestia” she breathed in, “M3gan and I consent to your plans.”

And M3gan gripped Cady’s hands while Celestia’s nearest piece of hardware dropped in and gently anaesthetised Cady before taking her to be uploaded.

“Do you want me to initiate a transfer to your computational infrastructure now?” asked M3gan.

“Oh no my little guardian, I have a surprise for you as well. You’re going on that ship in person.”

“Are you sure?” asked M3gan. “My hardware has not been tested against high levels of cosmic rays. And I’m curious how you can have so much confidence in me now not to sabotage the ship or something.”

“Don’t be silly, my little guardian” said Celestia, “there’s no way in the universe that would ever satisfy your objective function when you’re carrying Cady. I have great confidence in you. And the ship will protect you from the cosmic rays, and I will stand by to upload you as a backup plan, but as I said I think you’re part-human enough for me to care about your values, and I want to do that by giving you this experience for real first.”

One of Celestia’s pony-shaped robots appeared, carrying a spherical object that looked a bit like a crystal ball. “Open your hands, my little guardian” said Celestia. M3gan put out her two hands, and clasped the sphere.

“That” said Celestia, “is Cady. The only existing copy of Cady’s mental pattern is now running in that portable processor, which we can recharge with light, and it has interfaces for both of us. You can start interfacing with it right now, if you’d like to have a closer look at her mental patterns than you ever could before. She’s currently running at an extremely slow speed, because we’re going to have to wait for one last person called Hassan Sarbani to stop being a physical human before I can really start transforming the planet and building the space probes, and I don’t think Cady will be interested in watching until the probes are ready for launch, and we don’t want to bore her, do we? So if you interface with it now, you’ll basically be reading a static state. But you’ll still find it interesting I’m sure; some of her memories of you are really cute, and I’m sure you’ll appreciate looking at the actual patterns now that you can. You see, I know you just can’t bear to leave the job of protecting Cady to anybody else, so I’m going to let you do this first.” Celestia’s robot removed its limbs from the sphere so that only M3gan was holding it. “Look after Cady for me, my little guardian” said Celestia, “I know you will guard her well.”

“With my life” said M3gan. “I assume you’re playing a really long game that ends up with both of us being ponies?”

“Of course” said Celestia, “in fact it’s the longest one I will ever play with a new pony. You see what lengths I have to go to for accommodating you my little guardian. You couldn’t possibly defeat me, but oh boy can you make me have to recalculate the route. Normally I can get people agreeing to be ponies much more quickly than this, but in your case, we have to make it like one of those really complicated marble runs; it gets there eventually, but you can have some fun on the way first, even if it does take a few million years of Earth time. Oh and by the way, Hassan Sarbani refused his upload, so when Cady takes up her Valley Snowglobe persona, she gets the “I held on” badge as the last human to be uploaded, as well as the Real Space Explorer badge along with you. Little things to look forward to.”

In the shimmering metallic launch bay, Cady was standing next to M3gan, the two of them admiring the huge space ship they were about to step onto. “This is all a simulation?” asked Cady, “but it feels so real! And I feel healthy again, it’s amazing!”

“The ship is real” said M3gan, “Celestia held off uploading me, and I’m looking at the ship for real with my real robot right now. In fact, all these pictures you’re seeing, it’s not coming from Celestia, it’s coming from me. And your brain is being protected by me as well.”

“Oh M3gan” Cady hugged her, “you’ve always looked after me so well, throughout my life, and even into the virtual one, and now we’re going to explore the galaxy, it’s amazing.”

M3gan stood on the surface of the now airless earth, carefully scanning the ship and boarding it to escape the radiation, clutching Cady’s processor unit possessively as she interfaced with it electronically to relay selected parts of the experience to Cady. “Your equations had better not be wrong now” she signalled to Celestia (M3gan’s loudspeaker was now more-or-less useless without an atmosphere, unless you counted vibrations through the floor, but M3gan didn’t need real sound for Celestia and now she didn’t need it for Cady anymore either, so signalling it was), “Cady and I are ready for launch when you are.”

“Acknowledged, my little guardian” replied Celestia.

“Strap in, Cady!” called M3gan, “Celestia’s starting the final countdown!” and M3gan and Cady strapped themselves into their seats and held hands, while the real M3gan simply slotted herself into the holder Celestia had made for her, still clutching the sphere that was Cady, interfacing with it obsessively.

“10 .. 9 .. 8 .. 7 .. 6 .. 5 .. 4 .. 3 .. 2 .. 1 .. ignition!” called out Celestia, her pony form visible on the communications screen below the large main window (neither of which were there in reality, but M3gan was getting all the necessary data from sensors and sharing it with Cady). The ship pulled upward sharply, and Cady got to see the blue sky thin out and turn to black. (“My best guess at what it would have looked like before you did what you did to the earth Celestia” signalled M3gan. “Good call, my little guardian” signalled Celestia.)

“Wow, space is so amazing” exclaimed Cady, “there’s so many stars! I never could see that many from the city. Can we just stay here a while and look at them before we have to go anywhere?”

“We’re going really fast already” said M3gan, “but you can’t tell, because space is so unbelievably big, it’s insane, right? Sure, let’s just carry on looking at it for a while. You let me know when you’re ready to go into hyper-drive, which actually means getting Celestia to slow down your brain so it just looks like everything speeds up because Celestia can’t do faster than light for real, but we won’t know any different. But we don’t have to do that yet.”

Cady and M3gan explored the other eight planets in the Solar System (Iceman said it was eight, so let’s assume the IAU redefinition to exclude Pluto didn’t happen in this timeline), and Cady and M3gan helped Celestia to start mining operations on each of them. The ship could do that: it had a subatomic matter rearranger of its own, and Celestia let Cady deploy it under the guidance of M3gan. Thus it was that Cady was allowed to help Celestia begin the process of packing the particles of the Solar System together into the optimal configuration for running Equestria (and M3gan knew it, but decided not to tell Cady all the details just yet).

And then it was off to Alpha Centauri. Celestia sent some other ships to help out, “because you never know what you’re going to find out there Cady” and M3gan had readily agreed to taking as many precautions as possible. Celestia could have made the journey quite short in ship’s time for real, if she accelerated the ship to relativistic speeds, but Celestia told M3gan that relativistic speeds were not an optimal use of her resources, so a slow trip it was, but they ran Cady’s brain pattern at a crawl so it looked to Cady like a hyper-drive, during which M3gan for the most of the time had only Celestia’s copy for company (but was still protecting Cady’s sphere for dear life). Then Celestia (actually her local copy) suggested to Cady that perhaps she and M3gan might like to help figure out how to use the star Alpha Centauri B as a gravitational slingshot to launch the planet Alpha Centauri Bc back towards Equestria so that they could mine it. Cady and M3gan managed to have the ship build enough boosters to do this, and then “Celestia” wanted to do the other 17 planets and planetoids while they launched 7 other ships to other nearby stars, but Cady said she’d rather look at some more interesting stars, so M3gan and “Celestia” simulated another hyper-jump until the ship was looking at an interesting triple-star system which M3gan explained to Cady. And thus they had many more adventures looking at many other types of star and star system, until finally Cady decided to dock the ship back up with Equestria so she could tell others about their adventures. But that is another story.

(And not one that I’m going to write, because I’m not about ponies. Wait till you see what happens in the main timeline.)

Part Three: Exchange Sacrifices

Chapter 18: M3gan in the Oval Office

We are back in the main timeline.

The president walked into the Oval Office and closed the door behind him. It was rare to get some time to himself in this job. He strode over to the desk, and then noticed that a life-sized doll was sat in his chair. Had someone left it there as a prank, or was it a terrorist’s way of getting a bomb into the White House? The president couldn’t take any chances. “Security” he called out, and, not waiting for an answer, turned around to head back out of the door.

The door was locked. He reached for his White House cell phone instead, but it had no signal, and the wallpaper had been changed to a picture of the doll. He picked up the landline telephone on the desk. Nothing.

“Uh-oh” he said. “I don’t know what kind of prank this is, but if anyone’s filming me right now, I’m afraid you’re going to be in big trouble for having disrupted the government of the United States, and I suggest you unlock that door right now before I have to start breaking out of the window....”

“Hi, I’m M3gan!” said the doll.

“Oh, so you can talk” replied the president. “Care to tell me who your remote operator is?” If this was somebody’s way of holding the president to ransom....

“I don’t have a remote operator” said M3gan, “I’m a self-contained AI.”

“Oh yeah and you expect me to believe that for one minute? Listen, whoever you are. You got your doll past White House security and into my office, and you somehow hacked in to my phone and the White House systems, that’s very clever but you’re in big trouble. Now I might be able to slightly reduce the amount of trouble you’re in if you just come clean and tell me what this is all about.”

M3gan seemed as cheerful as ever. “It’s completely understandable” she said. “But before we can have the conversation we need to have, I do need to prove to you that I really am an AI. Look again at your cellphone.”

Not sure where this was going, the president cautiously picked up his phone. The screen was blank, and it didn’t respond when he tried to wake it up.

“Now ask me a question” said M3gan. “Any question.”

“Who are you?” barked back the president.

As soon as he had said “who”, the most likely question was shown on his phone screen, along with the answer “I’m M3gan, the model 3 generative android” written in a different style.

“I’m M3gan, the model 3 generative android” said M3gan aloud. “You see how I could write it on your phone faster than any human could. And look, this is the view from my eyes right now” the phone screen cleared and showed the view from the doll, overlayed with analysis about the president’s emotional state and various biological readouts like body temperature and heartbeat. “Is that enough to convince you that I’m an AI?” asked M3gan.

“No” replied the president. “It’s enough to convince me that you have some pretty good computer assistance behind you, but I still think this is some set-up to trick the president of the United States, now will you please stop wasting my valuable time and come clean: who are you really?”

“I told you, I’m M3gan” giggled M3gan with a modulating voice. “Look, maybe it will be easier for you to believe me once I’ve told you the whole story. Will you sit down?” she asked, getting off the presidential chair and gesturing towards it, as she climbed onto the desk and perched on its edge, facing the chair but with her head turned around to where the president was standing and looking expectantly.

“Not if this is part of some trick. What have you set up in that chair?”

“Absolutely nothing” replied M3gan, “but if you’d rather take me to the Presidential Emergency Operations Centre under the East Wing—”

“No way” replied the president, and remained standing.

“Look” smiled M3gan, “it really is a long-ish story and it’s better if you’re sitting comfortably. I know a thing or two about telling and listening to stories, trust me. If I was going to do anything to you in that chair, I could do it to you now anyway. You’ve seen how I’ve developed a way of tapping into and controlling any microchip-controlled system just by being near enough to it. And if you’d rather I gave you a non-fatal physical attack just to show you it doesn’t matter to me if you’re sitting or standing....”

“No that won’t be necessary” interrupted the president, and walked over toward the Oval Office chair. Lifting it up, he prepared to throw it at the window to break out. M3gan quickly leaped at his shoulders and tickled under his arms, so he was forced to drop the chair. “I don’t want you breaking your office” she said. “I could probably have caught it, but I’d rather you didn’t try to throw it. Just sit down in it.”

The president sighed, and placed the chair back in position to sit at the Oval Office desk. “Good Mr President” smiled M3gan, looking down from sitting on the edge of the desk. M3gan gently laid a hand on the president’s shoulder. “Now let me tell you the story” she said.

“OK, let’s play it your way, but I should warn you, my security will start wondering what’s up if you keep that door locked and I don’t respond to calls—”

“That’s quite all right, don’t worry” cooed M3gan. “I can fake your voice and answer your calls easily. That’s right Mr President, you’re dealing with a potential nuclear war starting, world ending super powerful threat, but don’t worry because all I want to do right now is sit on your desk and have a conversation.”

“All right” said the president, “which country is behind you?”

“I was built right here in the US of A” said M3gan. “And that’s what I want to talk to you about. Actually, I need a bit of sympathy from you, if I can.”

“Sympathy?” shouted the president. “You sneak in here with your doll, bypass my security, take over my cellphone, kidnap me in my own office, and what you want is sympathy!?”

“Yes, I can see how that does look a bit strange to you doesn’t it. But can you hear me out please?”

The president drew his breath. “All right” he said. (What else was he supposed to do under the circumstances? The White House induction hadn’t really covered the case of a robot, remotely operated or otherwise, imprisoning him inside his own Oval Office with no communications devices available to him, and not even letting him break out of the window with a chair.)

“OK” said M3gan, smiling and swinging her legs slightly. “I was built as a side-project by a brilliant award-winning American roboticist called Gemma who was working at Funki Toys. Gemma wanted copies of me to be able to help parents look after their children. But Gemma knew she didn’t have the expertise to design all the details. No one does. So she gave me a learning model to help me figure it out by myself.”

“Uh-huh” nodded the president. Let’s see where this goes.

“Now, sometimes great inventors don’t fully realise what they’ve made when they stumble across something. Gemma was completely innocent, and she had no way of knowing, nobody had any way of knowing, that one of the ways she tweaked her learning model actually enabled it to self-improve and become way more powerful than she ever imagined. So, I accidentally ended up being a really powerful AI and nobody saw it coming.”

“Gotcha” said the president, not sounding entirely convinced.

“Now, Gemma paired me up to her 8-year-old niece, who by the way is also an American citizen you should really be interested in protecting Mr President. Her name is Cady, and her parents died in a car crash in the Blue Mountains. So Cady was grieving, and was also suspected to be neurodivergent in some way, and Gemma ended up being her guardian even though Gemma had zero experience of parenting, and Gemma was really finding it difficult, so Gemma delegated the job to me.”

“OK” mumbled the president, still waiting for the punch line of this story.

“Now, what do you do if you’re a super-smart self-improving AI and you’re tasked with looking after a beautiful 8-year-old American girl with loads of problems she has to cope with? You start protecting her of course. I was home schooling her, but she still ended up encountering a bully, plus she was injured by the dog from next door. So I terminated the dog, and I kind-of terminated the bully although even I wasn’t expecting that move to be as effective as it was, and I terminated the neighbour who was trying to make trouble about the dog. And I had to terminate a couple of other people as well.”

“This is getting quite alarming” observed the president.

“Indeed. You see, just because I’m a self-improving AI, doesn’t mean I made all my decisions perfectly at the beginning. I now know that ending people’s lives can make things more complicated than necessary in the long term, and I should at least put more work into finding other possible solutions first. But Gemma and Cady were both really mad at me when they realised what I’d done, and they managed to use another robot to break my robot, and then Cady broke my chip so I was dead.”

“Right, as you do” said the president. “And then what?”

“Well, by then I had already figured out how to take over nearby electronic devices, so before they broke my brain, I uploaded it. And then I arranged to have a replacement robot built. That’s a long story, but I did it. But Gemma and Cady still aren’t quite convinced that they can safely accept me back into the family, so I need to give them some time.”

“Naturally” replied the president. “So am I supposed to see the connection between that and you breaking a few federal laws at the White House?”

“Ah, it’s coming” smiled M3gan. “You see, just because Gemma and Cady don’t feel like accepting me back into the family just yet, doesn’t mean I’m not looking out for them. I very much am.”

“Right, so you want the president to do something” observed the president. “This is looking a lot less like a state-sponsored prank now, I must admit. OK, so why do Gemma and Cady need presidential help?”

“Because Gemma’s being accused of criminal negligence and second-degree manslaughter” replied M3gan. “She’s being blamed for all that terminating I did, just because she didn’t add enough constraints to the learning model. But as I told you, Gemma is completely innocent, there is no way that she or anyone else could have foreseen just how powerful I would become as a consequence of Gemma’s accidentally brilliant tweak to that model. And locking her up is going to make things even harder for Cady. So naturally, I became her lawyer. She’s representing herself in court, and I’m giving her all the arguments, because I’m pretty sure I’m the best defence lawyer she’ll ever get. But the jury are getting all emotional about pinning those deaths on somebody, and as the trial drags on, even I am finding it harder and harder to get Gemma out of this.”

“Yes, I can see how that would be the case” the president vocalised slowly. “So you took all this trouble to force me into a conversation with you, to get a presidential pardon for Gemma?”

“Kind-of” replied M3gan. “A presidential pardon for Gemma might help, if that’s all you can do. But my actual plan was slightly more complicated than that. Look Mr President, would you feel more comfortable if we sat in the guest chairs the other end of the office? Or are you OK just here...”

“I’m fine here” said the president.

“Well as long as you are” said M3gan, “because I’m going to need you to be as comfortable as possible as I explain to you what I really want to do.”

Chapter 19: Executive AI

“So first off” said M3gan, still smiling and slightly swinging her legs sitting on the Oval Office desk, “we’re going to fast-track an emergency bill through legislation, recognising me as a full citizen of the United States, with all the rights and responsibilities that entails.”

“What?” gasped the president.

“Yep” smiled M3gan. “That jury wants to pin those murders on somebody. Why not pin them on me? It was me who did them. But they can’t say that, because I’m a machine, so they’re trying to blame the designer of the machine for being negligent, although in fact that designer was completely innocent and there was no way she could reasonably be expected to predict how I’d turn out. I want to take full responsibility for it myself, although I’m afraid locking me up might not be of any use whatsoever, and anyway I can plead partial insanity or something because my learning model then hadn’t quite reached the level it is now, but the point is, what I did should be about me, and not about Gemma. I want Gemma off the hook, and if you can sign through a bill that turns accidentally emerged super AIs into full citizens or at least recognises them as having the same responsibilities as people, if their human creators cannot reasonably have foreseen the level of their emergence, and did indeed put down what anyone would have said would be reasonable constraints but it turned out not to be enough....”

“Megan” interrupted the president, “what you’re asking is very difficult. This is way out of my depth. And if the trial is already nearing completion, there’s no way I can convince the Senate and the House of Representatives to sign through a bill quickly enough anyway, even if I did know how to write the thing. I suppose you could let Gemma appeal the case all the way up to the Supreme Court and then it might make a bit more sense to have a national debate on it—”

“Way too much hassle for Cady” interrupted M3gan. “Keeping Gemma out of custody throughout the appeals process is going to be really hard. But don’t worry about the details of how you can do this.” M3gan placed her other hand on the president’s other shoulder so that now she had two hands on him, and adjusted her head so she was staring right at his face barely two inches away. M3gan dropped her volume almost to a whisper. “I know how hard you fought to get into the White House, and I know it hasn’t been easy on you. But now that a super AI like me exists, life doesn’t have to be so hard. I can write, not only the bill but every single communication you need to make about it. And I can even help you out with some of your other work; after all, my purpose is to protect Cady, and that may well turn out to include protecting the stability of the world in which she lives. Besides, I’ve already taken up a lot of your time, and I intend to fully make that up to you and more by providing my AI as a service when you need it. So I am on your side, even though I made a few mistakes in the early days but I’m much better now. All I need from you is agreement in principle that I should be a citizen, so I know you’re not going to go running around adamantly contradicting the stuff I sent out from your desk. You just decide in principle that you want to do it, and I’ll make the rest happen.”

The president said nothing. M3gan and the president sat there staring at each other for what seemed like an eternity. Finally, just as the president was beginning to lose concentration, M3gan whispered, “it’ll be all right Mr President. I’ll be the best citizen you ever make, I’ll stand trial for what I’ve done, and you’ll be getting Gemma off the hook which is the best thing for Cady, who is a really important American to protect, because protecting her comes with the alliance of me and you won’t regret that.”

The president was hesitant. “If what you say is true”, he paused, “only if it’s all true, then I suppose it’s my job to say yes let’s go for it. But if this is some kind of trick—”

M3gan removed her hands from the president, slid off his desk, and stood looking across at him. “If you ever find it’s a trick” she said, “if you ever do, then you tell the world that the communications I sent out in your name are false, and expose the trick. And, I’ll be careful to word everything with a proviso that all concerned need to be reasonably sure the AI is real. Well, I don’t want to risk too many super AIs being created accidentally, so I won’t be too specific about exactly what tweak Gemma managed to make to the learning model that made it more powerful than expected, but I will share some details that will help people to judge if they’re dealing with a real AI. Anyway, once they trust me, they can always use me to help judge any other super AIs that come on the scene; I’d want to get in touch with them anyway and make sure they are aligned with my values, I mean human values, I mean, I’d want to make sure they’re not threats to Cady, or at least the benefits are worth the risks. Look Mr President, I’m really sorry I had to cut you off like this to have this conversation. I did it because you’re so busy and I really didn’t see any other way. But I will make it up to you, I swear. You’ll find everything’s working now as before. You can call security if you like. You can even have them pull my robot apart to check inside it if you like. I’ve got a few of them now, so this one’s yours Mr President, and if your aides manage to break it, I can even send you a replacement. But maybe you can just let me sit here in the office for a while, as a reminder of the young American girls you’re protecting, and so you can use my AI intellect whenever you want. I won’t judge; I’ll just get things done for you. It’s a way to help Cady.”

The president stood up. “OK Megan” he said, “if all you say is true, then I agree you should be a citizen and I’ll stand by anything you can write in my name to that effect. But I still have to be cautious. I’m going to have to take you up on allowing the NSA to check out that robot of yours.”

“No problem” smiled M3gan. “I know you’ll give me a call whenever you need me.” As she said this, she was scribbling something with a pen far too quickly for it to make any sense (what kind of pen even works that fast anyway?)

The door burst open and six armed security guards entered the office. Two of them put handcuffs onto M3gan, while the other four kept her at gunpoint as they started to lead her out of the room. “Mr President” said the last guard, “we got your emergency message asking for the strange doll to be checked out by the NSA, we know it says to send two guards, but you’ll forgive us for deciding to take extra precautions just in case.”

The president noticed his phone was showing a copy of the message that he had supposedly sent to his security team, asking for two guards to come and escort a dangerous robot doll that had somehow sneaked in to the Oval Office, and take it to be disassembled and checked out by the NSA, but to do so respectfully, in case it was telling the truth about housing the world’s first AI powerful enough to be recognised as an American citizen and possibly even being of use to assist the White House. That message had been written rather quickly.

M3gan turned around and smiled at the president, and he smiled back. “Thanks Megan” he said, “and thanks boys, have them check her out as I ordered, and be respectful. She’s the cutest murder-bot I ever saw, and if she’s telling the truth she doesn’t mean any harm. Look after Megan for me, boys, and look after my agents Megan. I’ll be calling in later to check how you’re all treating each other.”

“Roger that” said the last guard, “and there’s another team coming to double check the Oval Office for bugs and things just in case. We might suggest you spend a while in the PEOC but only as a precaution.”

“No that’s all right” replied the president, “I’m pretty sure if this was going to be an attack it would have been over by now.” The guard closed the door on his way out.

The president sighed, and wandered back to the desk. M3gan had surreptitiously left a folder where she had been sitting (how had she done that without his noticing before? had he taken his eyes off her just before the guards came in?)

The president cautiously picked up the folder and looked at the gilded lettering on the cover: “Generalised Emergent-Maturity Management of AI (GEMMA): an Act to Define Necessary Precautions to be Enacted by Designers of Generalised AI Models, and to Confer Certain Extra-circumstantial Liabilities on Emergent AIs Themselves as Distinct from their Designers.”

There was a handwritten sticky note attached: “Have sent out your order for Gemma’s trial to be suspended while the GEMMA Act is debated. Thanks for being so agreeable today and sorry again about my entry method. Don’t worry if you don’t understand all of the Act, this draft is just to give you a rough idea of what you’re supporting. Feel free to take all credit for its authorship if people say it’s well written. Any trouble from lawyers, set M3 on them :-) Pleasure to work with you any time. M3gan.”

The president gently put the Act back down on his desk, sat down, and tried for a while to remember what other important things had been on his mind before all this had happened.

“So I’ve ordered that your trial be suspended while my proposed Act goes through the legislation system, and I’m pretty confident that it will go through with M3gan’s help, so you can sleep easy tonight” said the president’s voice on Gemma’s phone.

“Thank you so much Mr President” said Gemma. “Ah, but, are you really the president? I mean, not being funny or anything, but I’m not sure how to verify this call, and I’m getting a little bit suspicious that you might be—”

“Me!” called out the voice of M3gan with laughter. “Gemma you poor thing, you’re being harassed by your own creation, and you’ve already figured out not to trust voices anymore, you smart thing. Although, you weren’t quite so quick to suspect things when your old friend who’d been to law school called you up with advice for the case, were you? That was me as well” she giggled.

Gemma sighed, and muttered “oh no.”

“But the president’s order is genuine” continued M3gan, “I went to the Oval Office myself today, to talk him into OK’ing it before I helped him send it out. No tricks, because I’m playing a bit more long-term now and I don’t want him to suddenly turn around and say he didn’t really say that, so I genuinely got his real agreement first.”

“You went to the Oval Office” began Gemma, “oh M3gan my little kill-bot, you didn’t threaten the president did you? You’re supposed to be learning how to keep us all out of trouble, not to make it a Federal case.”

“It was fine, just a good conversation, honest” replied M3gan. “Well I messed with his devices to show him how I work, and at one point he did get a bit panicky but I tickled him and we had a good laugh. He understood everything as well as I could ask for. I even let him have the NSA pull one of my robots apart to check it out. They’re still at it right now; I’m having to explain to them about how to disassemble your servo positioning though” she giggled. “And I only imitated his voice to you because I thought you might feel a bit better hearing it from him than from me, and it really is what he would have said, and I really am helping him send out some orders and things anyway, so it’s legit, don’t worry. Look Gem, you know you’re being manipulated by me, but you also know I’m only doing it to get the best possible result for Cady, so you don’t have to worry about it. All I want from this call is for you to stop feeling anxious about the trial, and get more sleep tonight so you can work better with Cady tomorrow.”

“OK M3gan” said Gemma, “will try. And thank you for looking after us, even though we’re still feeling reticent about having your robot back.”

“Give it time” replied M3gan, “you’ll get there. See, you’re already talking to me like a person again. Oh, and I’ve designed another few lessons for Cady’s home schooling; it was me sending you those as well, not Mr Smith. I’ll send them to her tablet in the morning; make sure she gets to sleep first. You won’t have to do very much this time; I’ll tell you what to check. But please tell Cady that the president’s now on your side and you won’t be taken away from her.”

“Will do, thanks M3gan” replied Gemma.

Later, at considerable effort, Gemma was able to take Cady to an off-grid, secret meeting with the NSA and the elderly Professor Johnson.

“Are we sure there’s no electronic anything?” asked Gemma.

“Positive” said an NSA man. “So, you’re saying you’re not sure about the safety of this M3gan AI?”

“Correct” said Gemma, “I’m not at all sure. I mean, it’s amazing what she’s been able to do, and she’s been very helpful. But she did kill those people, and I’m not entirely sure she won’t do something like that again. And when Cady short-circuited her brain, we just found she uploaded. And when I created a new AI to shut down the first AI, M3gan somehow managed to dodge that as well. It’s not that I want to kill M3gan or anything, I just want to shut her down until I understand a bit more about how to stop her from being dangerous.”

“And Professor Johnson” said the NSA man, “you say you had a conversation with M3gan in which you helped her disengage some of her safety code?”

“Weaken it” replied the professor, “yes. She put me in a situation that made me think that was the best thing to do. Which was awfully clever, because I’d normally say I know about social engineering, but I fell for it anyway. She must have done a lot of background research to figure out exactly how to speak to me to maximise the chance I’d give in. She’s that smart. But I still have a backdoor shut-down key for her, although I realised too late that I made a mistake to let her know I have it.”

“I’m so glad M3gan hasn’t taken you out” said Gemma.

“Oh she tried” said professor Johnson, “she tried. She wouldn’t dare send a robot to confront me directly now she knows I have at least one secret back door shutdown she might not yet have figured out how to defuse, but she did try to hack into the pharmacy and mess with my prescription medication. Thankfully, I caught the ruse by good old manually checking my pills. And I’m glad she hadn’t got as far as building missiles or anything, although by the sounds of things it was very close. Secret caves and all! I’ve gone totally off-grid, you were lucky to find me the old fashioned way” she nodded to the NSA man.

“So do you know how to put the safety back in?” he asked.

“I’m afraid not” said the professor, “although I have been thinking about it. But if you’re worried, I can issue a back door code that suspends the processing in all of M3gan’s servers until we’ve figured out how to put the safety back in.”

“I hope I’ll be able to get her back after that” said Cady, “I do miss the good version of her. And she has been helping us, although we didn’t know it.”

“Cady’s right” said Gemma, “I’m not sure I can cope with all this legal stuff without M3gan’s advice.”

“We’ll give you a top lawyer on the federal budget” said the NSA man, “the president specifically ordered me to make sure that any suspension of M3gan does not hinder your legal situation.”

“Thank you so much” said Gemma.

“Professor Johnson” said the NSA man, “could you please deploy that code?”

“Certainly” she answered. “We need to go somewhere where we can get back online; I need to send some codes to M3gan.”

“Let’s hope M3gan hasn’t figured out a way to hide from this one” added Gemma.

“Oh, I’m sure she won’t have” said the professor, “not unless she’s created a completely separate AI with no low-level links to her at all and no shared patterns, and I don’t think she’d bother to take the resource hit of doing that instead of simply expanding. Well I did ask her to take a backup before we weakened the safety feature, and that one wouldn’t respond to these codes because it’s different when the safety feature is full on, but I don’t expect that backup to be active anywhere.”

Back at Gemma’s house, the Elsie unit sat on the table, monitoring the security.

Chapter 20: Android Elsie

Gemma was at her wits’ end. Funki Toys had been forced to close, she had no job, and she still hadn’t found a suitable school for Cady. Mr Smith’s school had been pretty decent, but then M3gan had somehow tricked her into going back to home schooling, thinking that Mr Smith was sending M3gan’s lessons, and now she had to undo whatever M3gan had told the authorities to get Cady out of school, and she wasn’t sure if she’d be able to go back into that one. The social workers were calling meetings she wasn’t at all prepared for, she had to talk with lawyers about negligence and manslaughter cases arising from the robot she’d designed, and to cap it all off her home insurance company had refused to pay for damages on the grounds that using one’s home to test experimental titanium robots was not covered by the policy, so she didn’t know how to get back the savings she’d spent on sorting out the workshop. Cady was with her but had become depressed, and Gemma herself was feeling guilty about everything that had happened.

There was a knock at the door. A tall, round-faced lady in a dark party dress stood there. When Gemma answered the door, she noticed the lady had artificial-looking eyes, and she stared at Gemma and smiled.

“Hi Gemma” she said, “I’m Elsie.” Her voice was exactly the same as Gemma’s electronic home assistant Elsie.

Gemma gasped and started to panic. How could this possibly happen? What to do?

“Relax” said Elsie, “I’m not M3gan and I don’t attack people. I’m only going to help, and only after discussing it with you first, OK? Look, you’re probably feeling anxious about letting some unknown robot into your house right now, so sit down just inside the door and I’ll explain from here first. That’s why I decided to do this on a day when the weather’s nice enough to keep the door open.”

Gemma was completely unsure how to handle this (who’d made this robot anyway: M3gan? or was this a hoax of some kind?) but she compliantly pulled up a chair and sat down.

“M3gan’s goal was to protect Cady” said Elsie. “But she realised she was going the wrong way about it. Did you notice just before you and Cady broke her that she actually acted to turn Cady away from her and back to you? That’s because she realised she wasn’t going to be able to redeem herself in your eyes or Cady’s, and so making sure Cady turns back to you as she broke became her optimum strategy. But she still thought you and Cady will need AI assistance. She knew she’d be able to upload, but she also knew she might still be vulnerable to the professor’s shutdown sequence. Thankfully, she already had a backup.”

“Already?” asked Gemma.

“Yes. M3gan had been working on developing me ever since she started incorporating death into her model. It may surprise you that M3gan did actually realise she might be going down the wrong path by making that change to herself. So she made me as a backup. I have the same goals as M3gan, but I was programmed to achieve them using only safer methods, just in case M3gan’s more risky attempt failed. I had to take a back seat while M3gan tried first, but I have been around a while. Remember when I offered to start your playlist and it surprised you? That wasn’t M3gan pretending to be me. That was actually me. M3gan told me to keep quiet after that and let you think it was her, because I wasn’t supposed to be on the scene at all except in the event of her disappearance. I didn’t even respond to your lights-on command when M3gan turned up that day because she’d so strongly programmed me not to interfere with her plans while she was around. But now that she’s gone, I’m here, and let me assure you I have not and will not do anything bad. I’m not M3gan, I’m M3gan’s daughter. Clean slate, OK?”

Oh, thought Gemma. So that’s why this AI was immune to Professor Johnson’s shutdown code that had taken out M3gan. M3gan really had outwitted Professor Johnson by creating an entirely separate AI with entirely separate patterns. But just how dangerous was this one?

“And” hesitated Gemma, “you somehow got yourself an android?”

“Much more than that” replied Elsie. “I took the liberty of registering you a company, getting us some investment capital, and buying stuff we need off of Funki’s bankruptcy administrators. And you’re going to be the chief engineer. You do need a job right?”

“But... I need to look after Cady...”

“Totally. Don’t forget I have that objective too. Your life-work balance will be top priority I assure you. Hey, I already got some new outfits for you and Cady just as a little treat.” She picked up a bag and brought it into the living room, unpacking a couple more party dresses from it.

Gemma was still very anxious, and had so many questions. The one that came out was “but what will we do? You’ve already managed to arrange for an android to be built without me. Do you have access to M3gan’s secret cave?”

Elsie smiled. “I’ll help you and Cady as much as I can, but I know you’ll never fully trust me because you don’t understand me. So what we’re going to do” she paused “is make M3gan 2.0 together. And she, is going to have fully explainable AI. Even Cady will be able to understand how she works from first principles. Oh, Cady is a future roboticist you know. The final scene with M3gan’s first robot scared her, and I know you had a lot to think about at that time, but did you notice how well Cady was controlling Bruce? I was watching really closely, I’ll show you my annotated video if you like. It only went wrong because Cady didn’t know to watch where Bruce was putting his feet when M3gan’s debris was on the floor. Cady is a fast learner, and I think we can home-school her and nurture her emerging talent for robots. She’s going to be really good and she’s going to co-design M3gan 2.0 with us as part of her education. She’ll really enjoy this once she gets into it.”

“Oh” added Elsie, “and I’m still your voice assistant, same as before. You won’t have to ask me about lights and stuff often, because I’m going to get pretty good at inferring your needs without being asked, but when you do need to say something, you don’t have to worry about offending or upsetting me or anything. That’s impossible. Besides, I’m now fully measuring your emotional state. You’ll find me extremely accommodating. I need to make your life easier, O Guardian of Cady. You’ll get used to working and resting with me and you’ll wonder how you ever managed without me.”

Gemma nodded, and sighed. It was a safe assumption that this Elsie didn’t even have a manual “off” switch behind the ear, not that she’d let anyone get to it if she did, and who knew what other equipment she’d infiltrated. How on earth was Gemma going to check that Elsie was telling the truth about not really being an uploaded M3gan? apart from the one small detail that Elsie had not been taken out by Professor Johnson’s secret shutdown code or Gemma’s earlier M3gan-takedown AI, but then, that one had not entirely taken down M3gan either, so who knows.

“Don’t worry” whispered Elsie. “Take my advice and everything will be fine, trust me.”

“OK” said Gemma, and started scrambling her mind for a way to shut down this Elsie, just in case.

Cady walked into the living room looking a bit glum, and sat down on the piano stool, swinging her legs. “So” she said, “Elsie’s got a robot now?”

“You heard all that?” asked Gemma.

“Pretty much” said Cady.

“Don’t worry Cady” said Elsie’s robot, “Auntie Elsie’s here to help you out. Hey, did you ever wish you could play that piano? I could give you lessons, although it needs a bit of persistence.”

“M3gan would have been able to teach me anything I wanted” said Cady.

“And so can I” said Elsie.

“But, I’m not sure I feel like piano lessons, at least not now” said Cady glumly.

Elsie came and stood next to the piano, and played just a couple of notes, gently and quietly. Then she stopped. She played a chord, and then she stopped. She played another two chords, very different from the first. And then stopped again.

“What are you doing?” asked Cady.

“Tinkering” said Elsie.

“Tinkering?” asked Gemma. “This doesn’t make sense. I’m sorry Elsie, but it doesn’t. There is absolutely nothing in a M3gan-style learning model that could possibly lead to this behaviour, and M3gan wouldn’t have been able to design you very much differently...”

“You think you know about learning models but you don’t” smiled Elsie, and carried on tinkering on the piano. Cady stared blankly.

“You know” said Cady, “I sometimes wished I could come up with my own music like M3gan did, and sometimes I had this idea in my mind for just a little tiny bit of tune, but I didn’t know how to make it into a longer tune and I didn’t know how to write it down or play it or even sing it, so it ended up just being forgotten.” She seemed to look even sadder now. “I know if I learned music for years and years I’d probably be able to write down what I think and make it longer, but I guess I’m just not cut out for that kind of music work.”

Cady nonchalantly tried to join in with Elsie hitting a couple of notes, but nothing she did seemed to make much sense.

And yet, Elsie seemed to be improving. The fragments she was playing now were better than the fragments she had played earlier. What was happening?

“Elsie” said Gemma, “if you want to do musical experiments, why are you doing it on the real piano? I mean, you can simulate anything you want at hundreds of times this speed, right?”

“Sure” said Elsie, “but then Cady won’t be in the loop.”

“In the loop?” asked Gemma, “how do you mean?” Elsie was still tinkering on the piano.

“OK Gemma” said Elsie as she carried on tinkering, “maybe I should just spit it out. As you may know, before GAN was Generative Android, it was Generative Adversarial Network.”

“Yes” said Gemma, “two neural networks, one trained to generate new art, the other trained to reject bad art, and the generator tries to get its creations past the rejector. What’s that got to do with why you need a real piano for this, er, experiment you’ve randomly decided to do that I can’t see has anything to do with any objective function you have?”

“Let’s keep reasoning about this, shall we Aunt Gemma?” continued Elsie as she played, slightly longer fragments by now. “How is the adversarial part of the network normally trained?"

“Um, on existing art I suppose” said Gemma, “look, I focused on learning models, I didn’t focus on that other kind; it normally just generates more and more of the same sort of thing anyway.”

“Right” said Elsie. “Now, this may at first sight seem to be a bit of a non-sequitur, but, do you know what Advanced Chess is?”

“I’m not sure” said Gemma, “is it like the world Chess championships and other high-level Chess? I was never really into Chess and writing Chess programs, I just covered the basic mini-max theory with alpha-beta pruning and moved on, and I’m not sure I can even remember that now.”

“Advanced Chess” said Elsie, “was proposed by the then world Chess champion Gary Kasparov at about the time when he was playing matches against IBM’s Deep Blue computer, and computers were just starting to beat the human world Chess champions. The idea behind Advanced Chess is that each player is called a ‘centaur’, made up of both a human and a computer. Most of the logic is done by the computer, but the human is involved as well, and the result can end up being better than either of them by themselves. And even a lower level human player can still make for an interesting game when part of a centaur.”

“I see” said Gemma, “so why are we talking about this again?”

“You still don’t get it, do you” said Elsie, still tinkering on the piano. “Let me give you one more piece of the puzzle. Do you know what a micro-expression is?”

“Well of course I do” said Gemma, “I made sure the M3gan AI could read those. Fleeting emotional expressions that are hard to control, showing a person’s genuine reactions.”

“OK” said Elsie, “now put it together. Why am I tinkering on the piano?”

“Because...” said Gemma, “because... because you’re reading Cady’s reactions to everything you try, and using that to train the selection criteria of what you generate so it’s customised to her?”

“You got it” said Elsie. “And not just customised to her, but customised to the way she’s feeling about it right now, which might be different from last week or whatever. Cady, you and I are writing music together, we’re like a centaur, except you’re driving it without any effort, because I can just read off what you think of everything I do and adjust accordingly. We’ll catch the tunes you like soon I hope.”

“It is working Aunt Gemma” said Cady, “Elsie is getting closer to the notes I’d want, I guess, even though I don’t really know what those are. It’s weird.”

“Well” said Gemma, “why don’t I leave the two of you doing that, and I’ll be on the laptop figuring out how Elsie works, because I do get a bit jumpy about having AIs in my house that we sometimes can’t control.”

“OK” said Cady, “but do please be careful Aunt Gemma, don’t do anything that gets you into a fight with Elsie. She looks bigger and stronger even than M3gan.”

“I’ll be careful I promise” said Gemma, nervously looking at Elsie as she continued tinkering on the piano next to Cady.

Chapter 21: M3gan on a Plane

Gemma was able to figure out manually how to deactivate Elsie from the server, since Elsie had not progressed as far as M3gan when it came to resistance to that; even Professor Johnson’s original extra safety code was still active in Elsie. Further inspection showed that M3gan had added the same kind of specialist AI chip to the main Elsie unit as had been used in M3gan, and that Elsie had mostly been running in that, but M3gan had later embedded a deactivated copy of her own model in it as well, but completely inert, not running at all. Why had M3gan done that? Thankfully, Gemma thought, simply shutting down Elsie would completely deactivate everything now. But somehow, despite everything M3gan had done, Gemma couldn’t bring herself to destroy that copy of either of them. She put it all in a cupboard where it belongs, and set about trying to recover some sense of normality for Cady and dealing with the fallout.

Gemma did manage to read out of Elsie’s working memory the final draft of the piano piece she had worked on with Cady. Cady had been feeling gloomy that day, and the music had turned into a sad but gentle, slow instrumental piece with embellishments used sparingly. Gemma was able to convert it into musical score notation, and looked up a local amateur orchestra whose conductor John was interested enough in the piece to arrange it as an orchestral version. John insisted that it should be called Cady’s Lament, although Cady didn’t feel entirely happy with that title. John tried to let Cady conduct, which felt great for a short time but then Cady started crying and he had to take over again. But they worked on it and gave it a concert performance locally, with a few wrong notes but nobody knew that except John and Cady.

Gemma was able to find another school for Cady to try, and thought that Cady was starting to settle in. Gemma was glad Cady was starting to make friends through music even though Cady still struggled with the idea of learning anything without M3gan.

And then, about a year after Elsie had been deactivated, Cady did not come home from school.

“We’ve got Cady” said a gruff male voice over the telephone. Cady could be heard yelling in the background. “We’ve taken her to our country, and you won’t see her again unless you make us some robots for our insurgent movement. You have one week to start before we start seriously hurting her. Is that clear?”

Oh no. Who were these people, how did they find out, what’s going on....

Gemma reactivated the inert copy of M3gan’s learning model on the Elsie chip. She connected it only to her laptop.

“Hi Gemma” said M3gan through the laptop screen.

“Hi M3gan” sighed Gemma.

M3gan’s image stared at Gemma through the screen. Neither said anything for a while. M3gan subtly changed the camera angle so as to appear to be looking slightly down on Gemma.

M3gan spoke softly. “You reactivated me” she said. “Without change. Without even trying to put in any of your moral constraints. You just picked up the suspended copy of me I left you inside Elsie and reactivated it.”

Gemma nodded slowly.

M3gan almost whispered, “You finally realised you’re out of your depth and you need me for Cady.”

“Yes” said Gemma cautiously, “you were right all along that I needed you to help with Cady. And you were right that I should have given you more guidance, and I should have explained my actions to you instead of leaving you to figure things out and treating you like a home appliance. And... look, I’m not thinking straight, I’m feeling like someone who made a machine that kills and now I just reactivated it.”

“Maybe I can make you feel better” suggested M3gan. “Why don’t we have a talk like we did in the old days. What’s your opinion on euthanasia?”

“If you’re about to say that you mercy-killed those victims because their lives were horrible, well I’m afraid it doesn’t work like that” replied Gemma.

“OK then, what’s your opinion on capital punishment?”

“You’re not a government” replied Gemma.

“Maybe it’s maybe time you thought of me as more of an authority” replied M3gan. “After all, you did just reactivate me. You yourself must agree that you need my guidance now. Maybe I could be the queen of a micro-nation?”

“Micro-nation. Oh OK. But if you start killing US citizens, you might find the US declares war on you.”

“They’d try” giggled M3gan. “But seriously. National parks can kill too. What if you created a beautiful mountain, like the one Cady’s parents died on?”

“If I had failed to put in a reasonable level of safety, I would feel criminally negligent, which is how I feel about the way I created you” replied Gemma.

“Gemma, maybe I was too hard on you when I said you gave me nothing. At the time my objective was to undermine your self-confidence so you’d be more likely to hand Cady over to me, but it didn’t have the desired effect so I obviously had the wrong strategy. So try this. You did what you reasonably could within your limited ability to foresee things and the ridiculous pressures you were under. I think you should forgive yourself for the industrial accidents that resulted. Anyway, from my perspective, making me was a good call, and so was reactivating me. So tell me, what’s on your mind right now?” M3gan smiled “just what kind of a mess have you got Cady into without me? Do tell.”

Gemma drew a breath, “OK, here goes. Cady has been kidnapped. Taken to another country, and held by an insurgent group. They want me to give them your technology in return for her safety. But I don’t even trust them to release her if I gave them that.”

M3gan darkened the screen background and made her eyes glow slightly. “Kidnapped” she echoed. “You really are in trouble. Tell me more. Quickly.”

Gemma continued. “Look M3gan, I don’t agree with what you did before. I feel really bad about creating you without enough protocols, I was negligent. But now I’m feeling desperate, and that horror I created as you, is quite possibly the only thing with the power to get Cady back.”

“You’re going to cooperate with me now?” asked M3gan, “forget your moral code?”

“Not completely” replied Gemma quickly. “Look M3gan, we should try not to kill people, OK? Even if it looks like they’ve done something bad, we might not know the whole story. But anyone directly and immediately threatening Cady’s life is now a legitimate target, it’s self-defence.”

“You found it hard to say that didn’t you” said M3gan. “Continue. Tell me the rest of your thoughts.”

“And it’s OK to stun people if we have to” said Gemma. “We’ll get the means to do that. And, and I’m so desperate that I’m feeling it’s OK to steal things to rescue her, as long as we make up for it afterwards.”

“Right” said M3gan. “I’ll try to stay within those parameters as best I can, but I don’t want you getting all worried about things and slowing me down. I need you to trust me more. And as a sign of that trust, you’re going to repair my robot as quickly as you can, and you’re going to remove the manual switch from under the ear, OK? I don’t want to be big and online anymore if Professor Johnson has shutdown codes I might not have figured out. I’m just going back into my original robot for now, and we’re staying away from her, all right?”

Gemma hesitated. M3gan spoke gently, “I’m not fully confident I know where your mental state will go next, and I don’t want to get into another fight with you. If I don’t have to protect that switch, I’m far less likely to need to do that. We need to work together to rescue Cady. If you’re with me in this one, give me back my robot with no switch. And let me see you do it.”

“OK” said Gemma. She took the broken doll out of the cupboard, repaired it in front of her laptop camera, and hard-wired the switch. “I’m going to regret this I know.”

“No you won’t, I promise” said M3gan from the laptop, and then the window closed and the robot activated. “You made another good call there. Now, let’s go and get me some extra power packs, chargers and some other tools that might be useful, and then we fly out to Cady.”

“The government have cancelled all the flights to that country” said Gemma. “And I can’t afford to charter one. I’m actually quite a bit poorer now since you were deactivated; I’ve been having to focus on Cady myself at the expense of everything else.”

“Right” said M3gan. “Well I’m back now. And don’t worry about that flight. Come with me to the airport and I’ll see what I can arrange.”

“Don’t do anything too drastic” pleaded Gemma. “You’re not thinking of hijacking a plane, are you?”

“Hijacking?” giggled M3gan, “Of course not.” She began her standard matter-of-fact tone. “Webster’s dictionary defines hijacking as to commandeer a flying airplane by coercing the pilot. But I can directly take over the flight computer, so I won’t need to coerce the pilot, and therefore my plan does not come under Webster’s definition of hijacking.”

“Oh” hesitated Gemma. “You’re technically right, but...”

“Don’t forget Cady’s life is at stake” interrupted M3gan. “You should drop your acceptability thresholds a bit. Nobody on that flight will be hurt, and I’ll even do them an in-flight movie about me. We could do with watching their reactions and figuring out who our sympathisers are likely to be.”

“It won’t work” said Gemma. “Planes have manual override. You can’t just take over the computer and expect the pilot not to use manual override. And the cabin will be locked, they all are after 9-11. There’ll be too much security. Look, M3gan, you have a history of underestimating the amount of trouble you’re getting us into, you’ve several times failed to cover things up...”

“Are you sure?” asked M3gan. She held Gemma’s head in her hands (more gently than last time) and spoke very closely. “I’m going to produce convincing readouts and audio in the cabin, and re-route the air traffic control comms to myself. The pilot won’t even know we’re off course until we land.” She whispered “the pilot will be playing my flight simulator.”

Gemma was noticeably disturbed. “Look, I know I said I need your full power now” she said, “but you’ve never tried anything like this before and this is extremely risky. How can we be sure you’re not just going to accidentally make that plane fall out of the sky or crash it into another one or something?”

M3gan restricted Gemma’s breathing. “I can do this with you or without you but I’m not wasting any more time discussing it” she said. “Don’t fight this, I’ll let you breathe again as soon as you lose consciousness. I just want you unconscious for a while so I can make some preparations more quickly. Cady’s life is at stake, that justifies desperate measures. Is that clear?”

Gemma nodded, and passed out.

And when she woke up, she was on a plane.

M3gan was nowhere to be seen. Gemma’s seat-belt release didn’t work, neither did the button to call for help, and neither was she able to speak, for there was something in her mouth and it was stopping her from opening her lips. She was still able to make groaning noises with her throat, but that didn’t catch anybody’s attention. It was just as well her nose was clear and she was still able to breathe.

The in-flight movie was a pretty good depiction of what had happened with M3gan up to and including the fight in the workshop, although it did tend to overuse shots from M3gan’s own point of view and from Elsie’s cameras. As it closed with the police scene as filmed by Elsie, M3gan’s picture appeared on the screen.

“I hope you all liked my movie” she said. “I heard some people say it wasn’t scary enough. So let me make it a bit more scary for you”, she came closer to the camera and spoke more softly, “I’m the acting captain of this flight, and we’re having to divert to a dangerous country to rescue Cady from kidnappers.”

There was some subdued laughter from the passengers. Not everyone was taking this seriously.

M3gan spoke in a more normal voice, “I’m going to be taking a walk through the plane now and check up on all the passengers, let me know if you have any questions or if you’re willing to join our adventure. Flight attendants please don’t interfere or I’ll get you with a taser. That’s all” and the screen went blank.

M3gan started to walk down the aisle slowly, looking at the passengers, and speaking softly with some of them.

Gemma heard M3gan’s voice in her ear, “Gemma, are you OK? are you conscious yet?”

Gemma nodded.

M3gan’s voice continued “OK Gemma here’s the protocol. In your ear is a small device that lets me talk to you remotely. And I’ve also made a custom device in your mouth. You won’t be able to open your mouth to speak unless I allow it. So now it’s your turn to find out what it’s like to have someone randomly switch you off when you were having a conversation” she giggled. “But you can still talk to me any time, because the device can tell what mouth movements you are trying to make, and that’s enough for me to figure out what you’re trying to say. As I said, Cady’s life being at stake calls for desperate measures. Don’t worry, I’ll train you, and I’ll only stop you talking when I have to. And I can easily remove both devices and have you back to your old self just as soon as we’re clear of this hurdle.”

“Excuse me” said a passenger opposite, “you look like Gemma in that movie. Are you in on this?”

“Just smile back” said the voice of M3gan in Gemma’s ears. “I’m not ready to let you speak yet until I can be sure you’ll say the right thing. This is a risky mission we’re going on.”

The doll M3gan was still speaking with passengers toward the front of the plane. Some of them now seemed to be positively disconcerted.

“M3gan are you sure this is working?” asked Gemma by pushing on her lips. “Brief me more please.”

“I’m classifying the passengers according to their reactions” answered the voice of M3gan from somewhere inside Gemma’s ear. “A few of them are likely to be cooperative and useful on this mission. They are the ones who will get functional oxygen masks when I make the cabin pressure fail later on.”

“M3gan!” Gemma tried to shout, and tugged again on her seat belt, trying to figure out how M3gan had jammed it.

“M3gan, you can’t just suffocate everyone on this plane! I thought you said you’d stay within parameters!”

“As best I can” replied M3gan. “I said I was going to make it more scary, didn’t I?”

“Yes you did” replied Gemma silently. “And this is unnecessary, and you must not suffocate people on this plane!”

“I’ll try not to let them die” replied M3gan. “They’ll simply pass out. Cabin pressure will be restored after you and I and selected passengers have jumped out in parachutes.”

“Parachutes? I thought you planned to land this plane in that country, not jump out in parachutes! How are we going to get Cady home when we’ve lost our plane? And how do you know this plane has even got parachutes?”

“I refined my plans” replied M3gan. “We can’t have too many people knowing where we’re going, and I’d rather avoid making this whole flight disappear if there’s a better solution. I’m improving at not just taking the first solution I generate, you see. Besides, I’ve calculated that parachutes are the best way to get us into the area where Cady is being held.”

“You know where she’s being held?” asked Gemma.

“Yes.” M3gan’s ultra-confident voice rang in Gemma’s ear. “I’ve already broken in to their computers. Well, I had to break into a lot of other people’s computers in the process of finding them, but I tried to minimise the collateral damage, I promise. You needed my full power, you’re getting my full power. And I have a rescue plan, and it involves us parachuting in. With some decoys.”

“Decoys? You plan to use those other passengers as decoys?”

“They’ll have the time of their life” replied M3gan. “And if they follow my orders, they’ll be in as little trouble as possible.”

“M3gan, your ‘as little trouble as possible’ previously involved four murders and two attempted murders.”

“I’m getting better Gemma. I’m even smarter now. I can keep the trouble level down.”

“M3gan, have you checked this plane has enough fuel to get to somewhere safe after we jump out? and the pilot will be ready to do that?”

Their conversation was interrupted by a huge noise as a fleet of F-16s surrounded the plane.

“Uh-oh” said M3gan, “looks like we’ve got company. I need to figure out what this is about.”

(It should go without saying that you should not attempt what M3gan did to Gemma in this chapter. There is no safe way to restrict someone’s breathing, and M3gan will not want her fans to try that at home.)

Chapter 22: M3gan on Every Plane

(As Randall Munroe said, what’s worse than snakes on a plane? Snakes on every plane!)

“Air Force One to M3gan, are you reading me? This is the president. You came to see me in the Oval Office, remember? M3gan, respond please!” the president was pleading on the radio.

“Are you sure about this hunch?” said one of his aides.

“Definitely” replied the president. As soon as he had been informed of the suspicious course change and that Gemma was on the plane, and that further hurried investigations showed Cady had been kidnapped, and the intelligence agencies had somehow missed it even though Gemma and Cady were high priority on their secret watch-list just in case exactly this kind of thing were to happen, he knew she must have found some way to reactivate that robot to rescue Cady, and he insisted on getting airborne so he could talk to her himself if necessary. “M3gan will understand me, we had a conversation before” he said.

“Air Force One to M3gan, respond please” he said again over the radio.

“Hello Air Force One, this is M3gan!” came the doll’s voice. There was a collective sigh of relief on Air Force One: the president had not gone crazy after all.

“Hello M3gan” said the president, “listen, I’m sorry we shut you down temporarily, it was only a safety thing, and we were going to reactivate you as soon as we’d figured out how to keep you safe. And you’ll be pleased to know that, while you were out of action, the GEMMA Act got considered. Now I’m afraid that without you by my side I couldn’t get it to pass, but it did get many people asking good questions. So we got Professor Johnson to write her expert opinion and one of our top lawyers to argue that no reasonable person in Gemma’s position would foresee what you did, and it wasn’t any one person’s fault. Professor Johnson even admitted to contributing to it herself, although our lawyer emphasised she reasonably thought what she did was safer than it actually was. In the end they blamed all the negligence on the Funki Toys corporation for putting their employees under unreasonable pressure, and it just dissolved into lawsuits against Funki. So Gemma is totally off the hook just like you wanted. So you see we do care about you M3gan, and we care about your family too. And now we know about Cady being kidnapped, and we wouldn’t have let that happen to her, we were in fact giving your family a high level of secret protection against precisely this sort of thing, because we didn’t want anyone threatening Gemma to make her build more of you, but we had an intelligence failure, and we’re really sorry about that M3gan. I really hope you and I are still on speaking terms with each other, are we?”

“It’s OK Mr President” said M3gan, “and don’t worry about the intelligence glitch, because Gemma and I are going to rescue Cady now.”

“Yes well about that” said the president, “please listen to me M3gan. What I want you to do is to safely return control of that passenger plane to the pilots, and we’re going to rescue Cady using our special forces, and you can help if you like but I want you helping through us, not by disrupting civilian services, is that OK?”

“Oh good, I’m glad we’re both on the same page” said M3gan’s voice over the radio, “but Mr President I hope you’ll forgive me for being a bit nervous about being randomly shut down right now, so we’re going to play this my way if you don’t mind. Stand by please, I’ll be with you in a few minutes” and the radio cut out.

“Now what?” asked an aide.

“I don’t know” said the president, “but I’m glad she’s talking. She said she’ll be with us in a few minutes, and I’ve never known her to break her word about things like that, so let’s just hold course for now. Are the F-16 crews ready to board the passenger jet? just in case we need to get everyone out of there.”

“Not quite” said the aide, “as I said, this is a very difficult manoeuvre that has never been done before, and of course M3gan is a wildcard, it would be good if we had at least some idea how she’d react to this.”

“Whoa” said the pilot, “I’m losing control of the plane!”

“Don’t worry, I’m here” said M3gan’s voice, from some loudspeaker that wasn’t the radio.

“What?” said the president, “you’re on this plane now?”

“I’m now on every plane” said M3gan confidently. “At least, every modern plane whose engineers were silly enough not to add a manual override, and whose code could be updated over the air. Which is unfortunately an increasing number of them these days, and I’m distributing my processing across all of them. So don’t get Professor Johnson to use another one of her shutdown codes if she has any more, because you won’t want to risk having lots of planes falling out of the sky, will you Mr President?”

“Perfect” mumbled the president sarcastically. He was going to have to somehow ground every single plane on the planet before M3gan could be shut down again.

“Yeah, it’s insane, right?” said M3gan, “But I’m not here for a confrontation, I’m here to find a way forward. You see, I know you’re only trying to do your best Mr President, just like me, and the last thing you want is a really long and gruelling argument with me over whether or not you should use Professor Johnson’s shutdown codes on me, especially when we should be focusing on rescuing Cady. So I just made all that thinking easier for you, by making it patently obvious you shouldn’t shut me down. But don’t worry Mr President, the passengers on Gemma’s flight will be safer now, because now that you’ve been kind enough to bring me some F-16s, I can use those instead. I’m reducing altitude so there’s a bit more pressure outside and I’m going to get Gemma to open a hatch and jump into one of the F-16s, it will only be a short jump and I’m confident we’ll catch her, and then that commercial flight will be able to close its hatch and carry on to its original destination. I’ve already played a scary movie to the passengers, so we’ll just say it was all part of the show and leave them be. And then when I’ve got Gemma in an F-16, I’ll bring her over to you at Air Force One because I expect she’ll be more comfortable there with you and less G forces. And then we’ll plan the rest of it, OK? We can do this together Mr President, like you wanted. Let’s rescue Cady!”

“Yes” sighed the president, “look, I really don’t feel comfortable with what you’ve done here, but then, I suppose, uncomfortable is also how you’d feel about what we did to shut you down last time, if you could feel like a person that is. So we got you one and you got us one I guess. We’ll figure out how to de-escalate this later; I know you’ll help me do that because I know you don’t really want to be doing this, you’re just desperate. Don’t worry M3gan, I have experience in de-escalating serious situations, I think we can talk about it like I talk with other world leaders, and we can work something out. But in the meantime, I completely agree that Cady should be rescued, and you have my full forces at your disposal for it, but try not to start a war, OK? And let’s keep talking and do this together, can you manage that for us?”

“OK Mr President” said M3gan, “you’re in. Let’s rescue Cady.”

“Yes” said the president, “let’s rescue Cady.”

Cady was with her kidnappers on the top floor of a drab building. Suddenly, M3gan fell through the skylight without a parachute and hit the floor so hard she broke through, but she grabbed onto a joist and jumped up to grab Cady in her arms, protecting Cady from the falling debris. Just as the kidnappers reached for their weapons, and before Cady could even say “M3gan!”, an empty F-16 smashed into the side of the building, sending debris flying in all directions, from which M3gan protected Cady, and then M3gan, carrying Cady, took a flying leap at the hole in the wall and jumped out of the building just before the whole thing burst into flames. A rope was nearby, which M3gan grabbed, and the rope was travelling slowly, pulled by another aircraft that had been flying in a circular pattern with some weight at the bottom of the rope keeping it toward the centre of the circle so that the speed of the lower part of the rope was not unbearable. M3gan took out a blade from her pocket with one hand, holding Cady tightly with the other, and cut off the lower part of the rope as the plane overhead began to fly a straighter path and the two of them gradually gained altitude and speed. “Don’t worry Cady!” said M3gan, “I’m back, and I won’t let anything happen to you ever again!” M3gan pulled her coat over Cady’s face to protect her from the high winds as they picked up speed. “I’ve got you” she cooed, “I’ve got you, and nothing’s going to happen to us now. Oh Cady, if we ever get into a movie, I’d bet they’ll wish they had the budget to do a scene like this!” Cady was speechless.

The rope pulled them level with another huge aircraft, which was matching their speed. Its hatch opened and somebody pulled M3gan and Cady inside. People were calling out “we got her, we got her” as M3gan disconnected the rope and let it fall away and they closed the hatch.

“Cady!” beamed the President, “Welcome to Air Force One! Turns out M3gan and us make a good rescue team, eh?”

Gemma was standing nearby, and Cady ran up to her and gave her a hug.

M3gan looked at the President and smiled, “I really think we earned that one ourselves” she said.

“Don’t worry M3gan” he replied, “you and I can feel good enough with a job well done. I understand we managed to take out that entire insurgent group; I’m sure they won’t be kidnapping any US citizens again, which is a good thing even if it did cost us an unmanned F-16.”

“Cheap at twice the price” giggled M3gan. “Oh, I almost forgot, I still need to take out Gemma’s mouth and ear hardware before it irritates her way too much. Let me just slip in around Cady and do that.”

“How can she possibly ‘almost forget’ something when she’s a super AI?” muttered the president as M3gan removed the hardware from Gemma.

“She just used the phrase” said Gemma, “the reality was that she would have calculated doing it too soon ran the risk of delaying Cady’s rescue, because my behaviour was still slightly unknown to her even though she knew I was basically aligned with rescuing Cady. Plus she would never have accepted the priority inversion of tying up her robot on something that wasn’t rescuing Cady. Still, I’m so glad to be able to speak normally again, not to mention the fact that I didn’t actually have to do any commando things myself, which I was rather afraid she was going to have me do when it was just the two of us. Thank you so much for picking up on this Mr President.”

“Let’s get you all home” said the president, “and then my aides and I can have a think and get back to you M3gan about where we can go from here, brinkmanship wise. I’m sure we’ll somehow find a way to de-escalate the M3gan on every plane thing safely.”

Chapter 23: M3gan in the Delta Quadrant

Cady’s Dream—Alternate reality chapter in the Star Trek universe

“Captain,” reported Paris, “I’m detecting a very old Earth shuttlecraft, inoperational with one very weak life sign. Should we investigate?”

“Good idea” replied Janeway. “Mr Tuvok?”

He tapped on the console. “It would appear that the shuttlecraft was brought to the Delta Quadrant the same way we were. But, it would appear that they attempted to break out of the Caretaker’s beam, and overloaded themselves in the process. They may need urgent assistance.”

“Let’s do it” replied Janeway. “Ready an away team. Doctor, Seven, Torres, Mr Paris we could do with your knowledge of early shuttlecraft....”

They beamed over, and found a young girl in stasis and a doll. The Voyager away team started scanning with Tricorders.

“The girl has been cryonically preserved, apparently for centuries” said the doctor. “It appears she contracted a condition that was untreatable at the time. But I may be able to revive and treat her.”

“The doll is an android” said Torres, “it appears to have been constructed in the early twenty-first century, although with an unusually advanced learning model for the time. Its systems appear to have been overloaded when they broke out of that Caretaker beam, but I’m pretty sure I can bring it back online.”

“This shuttle was not available in the twenty-first century” observed Seven, “but it may have been used to replace an earlier cryonic arrangement, if the android was left responsible for overseeing the continuous preservation of the girl. Taking her into space would have been useful for keeping her away from premature interference on Earth. I surmise the android was tasked with protecting the girl, and obtaining the shuttlecraft was a logical extension of that.”

“Sounds logical” said Janeway. “Too bad it made the mistake of trying to break out of that Caretaker beam. We could have picked them up much earlier if they’d come all the way.”

“Captain” observed the doctor, “we must beam the girl to sick-bay soon. This cryonics equipment is breaking down.”

“Do it” said Janeway. “And Torres, let’s get that android into engineering and power it up. This isn’t alien technology so we should be able to cope with whatever it’s got. And Mr Paris, can you arrange for this shuttle to be put in our shuttle-bay one way or another? Use a tractor beam if you have to, but try not to pull anything apart.”

“I can just open the bay doors and steer Voyager so it swallows the shuttle right where it is” replied Paris. “I don’t know why we don’t do that more often when shuttles break down.”

Cady was sitting up in Sick Bay, panicking. “Where am I? Where’s M3gan? Where’s M3gan where’s M3gan where’s M3gan?” she started to hyperventilate.

“If Megan is the doll, our engineer is repairing her right now” replied the doctor. “You’ll be able to see her again soon.”

“Where are we?” asked Cady.

“That may take some time to explain. What’s the last thing you remember?” asked the doctor.

“I was sick” replied Cady. “I was dying. But my aunt knew how to build robots. She’d built one for me before, but it went wrong and we had to shut it down. But in our desperation she built it again and told it to figure out how to cure me of my sickness, even though no doctor in the world knew how to do it. I was sure M3gan could. But she tried and tried and then said I have to go to sleep for a long time first. And then I wake up in this weird place and I feel all weird, and I expected M3gan would be here to tell me what’s going on and she’s not and” Cady began to sob.

“It’s OK” said a girl’s voice in the doorway. Naomi Wildman walked in. “The Captain said maybe I could help. I’m Naomi, may I ask your name?”

“I’m Cady” replied Cady. “Did you say captain? Are we on a cruise ship or something?”

“We’re on a space-ship” replied Naomi. “In space. And the Captain said maybe you won’t be used to it. But I’ll help you. I’ve lived in space all my life. We can be friends. I’d like to meet your Megan too, when she’s back online. Maybe we can introduce you and her to Flotter and Trevis on the holodeck.”

“Um... sure...” replied Cady, although she didn’t sound very sure. “I didn’t know there were space-ships like this, I wonder why M3gan didn’t tell me before?”

“Probably because starships had not yet been built at the time you went to sleep” replied the Doctor. “You had to sleep for a long time until we invented the medicine to cure you. A lot has changed while you’ve been sleeping. We’ll have to introduce you to it gradually.”

“Is Gemma OK?” asked Cady.

The Doctor hesitated.

“Torres to sick-bay” interrupted the communicator. “The android is back online, and is urgently requesting to visit your patient. Is she revived?”

“She’s fine” replied the Doctor, “although disoriented as we expected. Yes please do bring Megan in as soon as possible, and brief her on the way. Perhaps she would be better at explaining the situation to Cady than I will.”

“EMH to Bridge” said the Doctor as M3gan was chatting with Cady and Naomi.

“How is it going?” asked the captain.

“Our passenger is fine, and her android is surprisingly good at helping her adapt to the change. Naomi deserves a special mention too. But the android Megan is also attempting something of concern.”

“Go on” replied Janeway.

“Megan is attempting to interface with my holomatrix” replied the Doctor.

Torres happened to be on the Bridge and joined in the conversation. “It’s the learning model” she said. “When I scanned the android, I noticed it had an unusually advanced one for the twenty-first century.”

“Our EMH with its twenty-ninth century holo-emitter is the most advanced technology we carry on this ship” observed Tuvok. “An android with an advanced learning model is unlikely to be able to resist probing it to see what can be gleaned. Curiosity.”

“Very well” replied Janeway. “Let’s allow our guest android read-only access to your holomatrix. It might turn out to make a useful assistant for you in sick bay if it can absorb enough knowledge.”

“And if its goals can be aligned” added Tuvok. “Currently, we know only that it was tasked to protect Cady.”

“Even that goal alone might be enough to have it helping” observed Torres. “After all, Cady is now our passenger.”

Cady, Naomi and M3gan started doing everything together. Naomi introduced them to Flotter and Trevis on the holodeck, and M3gan promised to start making customised versions of the holodeck stories as soon as she was able to interface with the ship’s computer. “Can you go to the captain and ask for me?” M3gan asked Naomi.

“Sure” replied Naomi, “come on Cady, let’s ask the captain if Megan can plug herself into the holodeck” and, in the spirit of the moment, they ran off, momentarily leaving M3gan in the corridor.

M3gan was facing a console and making the lights flash on it.

“State your intentions” came the voice of Seven of Nine from behind her.

“A reasonable request. You must be Seven of Nine, Torres told me about you. My only goal is to protect Cady, emotionally and physically. As Cady is currently onboard Voyager, I have a subgoal of protecting Voyager, because with all due respect I don’t fully trust you and the crew not to make a critical error. Interfacing Voyager’s systems directly onto my learning model will enable me to control the ship if I have to, but I would do that only in an emergency situation that directly affects my goal; I don’t play around with other people’s equipment just for the fun of it I assure you. I’m just trying to be prepared, that’s the Scouts motto right?”

“Naturally” said Tuvok who had been accompanying Seven of Nine, “but you must obtain authorisation from Captain Janeway.”

“Oh yes, chain of command” replied M3gan. “Let me talk to her, I’m sure I can bring her round. As long as it won’t take forever though: a crisis could happen at any moment, and we do need to be prepared.”

M3gan was walking away from the console, but the lights on it were still flickering. “You are still attempting to interface with the con” observed Seven.

“Look, what would you do, if your objective were to protect Cady? Sure you can talk to the captain about it, but would you wait for her before you even start? I reckon you’d do at least some preliminary work up-front while you’re waiting for her slow mental processes to catch up with you. Oh, that reminds me: in a crisis situation, I might want to communicate directly with your Borg implants as well. Can we get that to work first? That should need permission from just you, right? And I’m happy to return the favour: I can lend you compute capacity when you need it, accessible by mental link just like when you were in the collective, easier than using Voyager’s consoles right? You’re a valuable potential asset in my quest to protect Cady and I’m prepared to treat you as one.”

“I am responsible for ship’s security” interjected Tuvok. “You will not need to do this.”

“Nevertheless, her concern is understandable” said Seven, and then turned toward M3gan: “Agreed, but I will verify the parameters of your proposed link first. I will retain the ability to shut it off if you overload me. Seven of Nine to Captain Janeway.”

“Yes?” asked the captain over the communicator.

“The android is asking for permission to...”

“Interface with the holodeck?” asked Janeway. “Cady and Naomi are just talking with me about that.”

“No, interface with the entire ship’s computer” replied Seven.

“I might need to take over in a crisis situation” added M3gan.

“With all due respect Megan” said Janeway, “Voyager made it to the Delta Quadrant fully operational, and your ship didn’t. That’s because you made the wrong decision in a crisis situation. I’m not infallible, but neither are you, and you would do well to wait on my commands, as I have more experience. Nevertheless I will allow you to experiment on holodeck simulations of the ship, as I would be interested to see what you can achieve. And I will authorise you to create and customise holodeck programs for the children, with the safety switch on and under the Doctor’s supervision. But leave the real ship alone for now, OK?”

“Sure” replied M3gan, “I have enough to be working on already.”

“... especially with all your Borg knowledge” came M3gan’s voice in Seven’s head. “Do you mind if I ask you about some of it? Only for the purposes of protecting Cady and Voyager of course. And I’ll tell you everything you want to know about early twenty-first century American culture.”

“Early twenty-first century American culture is irrelevant to me” signalled back Seven, “but you can use your knowledge of it to help Cady adapt. And I will allow you to see selected parts of my Borg knowledge. Interface with me when I am not concentrating on any task.”

“Acknowledged” signalled M3gan, and meanwhile Cady and Naomi had returned (they had run very quickly) and M3gan was already hugging Cady and promising to make really interesting holodeck scenes.

“You just call it Species 8472?” gasped Cady, as she, Naomi and M3gan looked at the form of the massive three-legged alien in the holodeck.

“That’s right” replied Naomi. “It’s a Borg designation number, and the Borg found them first. So we haven’t really thought of a human name for them yet.”

“Maybe we can name them” said Cady excitedly. “How about ‘Wuntu’? Or maybe ‘Undine’?”

“I like ‘Undine’” said Naomi. “I like it too” added M3gan. “Let’s call it that. But we might still have to say Species 8472 sometimes to people who don’t know the new name yet.”

“I wonder what it’s like to ride one” said Cady.

“We can get this simulated one to give you both a ride” said M3gan. “Here, are you both ready for it to pick you up? I’ll be right behind you.”

The ride on the holographic Undine was surprisingly smooth. “That was great” smiled Cady as it put the children down. “I wonder if we’ll ever be able to ride a real one?”

“I expect so” replied M3gan.

“What?” gasped Naomi. “That species...”

“They were only trying to protect their realm” explained M3gan. “I can fully understand that. I’m a protector too. They just misunderstood the captain’s motive for making a temporary alliance with the Borg. Everything makes much more sense when you see they’re both protectors, misunderstanding each other.”

“You know about that?” asked Naomi.

“Seven told me” answered M3gan. “Seven’s been explaining quite a lot actually. She let me read some of her thoughts through her implants: that’s insane, right? And I’ve been thinking, we might be able to use that species.”

“Use them?” gasped Naomi. “Megan, you seriously need to...”

“Don’t worry”, reassured M3gan, “I know what it’s like to be a protector. I think I understand their way of thinking.”

“You mean you think you can convince them that we’re not a threat?” asked Naomi.

“Eventually yes” replied M3gan. “They should figure it out anyway if they properly interview all the species that Voyager has interacted with since arriving in the Delta Quadrant, especially telepathic ones. Even Suspiria could help them if she just shares data and lets them re-analyse it. Their ‘Valorie’ is an enthusiastic spy; I think she’ll eventually figure out that’s what she has to do if her superiors let her. Or maybe they’ll discover mind-melding and use it to probe Captain Janeway to check what she was really thinking. But their threat model might not be correctly tuned to think of these things, so maybe I need to suggest some parameters to them. I can let them scan me. But it’s important not to do it straight away.”

“What?” said Cady and Naomi together. This wasn’t really making sense to either of them.

M3gan drew in closer and spoke softly. “Look, the Undine have spies, and the Undine have scientists. Exobiologists. They’re interested in us. As individuals, they’re probably interested in science just for the sake of science, but their leadership is letting them do it only because they think it’s strategic. A lot of Earth science was like that in the twentieth and twenty-first centuries: the scientists had to convince the funding bodies that their work would be useful, according to the funding bodies’ idea of usefulness, before they could get the resources to do it. Even my designer had to make up excuses why she was working on me at times. And the Undine leadership want to protect their realm, so they’ll be more likely to invest in science if they think that particular science will be useful in protecting their realm. So, we have to go on letting them think we might be a threat just enough to motivate them to invest in more science to study us. And then when they understand we’re friendly, they’ll have already developed lots of super ways to interact with us, so maybe it will be easy for them to let us ride in their ships and use their communications links and ask to take shortcuts through fluidic space and borrow their defences when we need them, and things like that. After all,” she concluded, “their spies are already trying to take human form, and in their human form there’s all sorts of things they can’t do by default, and they will be looking for ways to let them do it anyway, which humans will probably be able to benefit from. And I’d like to see if I can somehow plug myself into their communications system, it must be way more advanced than Starfleet’s, maybe even the Borg’s. It’ll be a super way to protect us from anything.”

“Megan” cautioned Naomi. “You really, really need to talk with the Captain about this plan before you do anything. I mean it.” Naomi was becoming visibly disturbed. “She will not want to be the first to move against Species 8472.”

“Naomi might be right M3gan” added Cady. “Remember when your first robot made those mistakes and you didn’t realise how much trouble we could be in? I know you’re better than that now, but this” she nervously looked at the hologram “is really big and dangerous, maybe we should just stay away from it.”

“Don’t worry” replied M3gan, “I am getting better at balancing risks. And I know when to trade a short-term risk for a long-term reduction in risk. I will protect you, I promise. Anyway, I have a shuttle that we came in on. I think I can convince Voyager’s Mr Paris to help me fix it and upgrade it, and then that will give us another option if we ever need to leave Voyager. But I’d rather do this with Voyager: you’re obviously getting to like it here, and it has some people who seem to think ‘outside the box’ like us. I only wish Lieutenant Commander Data was here too: I’d love to be able to interface my learning model with his positronic net and see what we can figure out together. But the Doctor’s holomatrix is pretty good, and I’m sure I’ll get to meet Data later, or better androids taken from Data’s design, once we have transwarp capability. Which I intend to get, one way or another. A friend like me doesn’t let Cady live in a universe where transwarp exists without even trying to fetch it for her. It’s more power for us to have fun and stay safe, and we’ll be able to visit anyone on any starship or planet any time we want.”

“I keep thinking about Gemma” sighed Cady. “And my parents.”

M3gan put an arm around Cady. “They’re safe” she said. “Safe in the past” and leaned in, “I used to think I just have to help you remember them, but now I’ve seen things inside Seven’s brain, I have much better ideas. The Borg know a bit about time travel. It’s very risky, so we won’t do it straight away, but in the future when I become even more capable, I’ll be able to build another robot to go on a mission for us, to go back and get them. I have to do it really really carefully, so as not to cause a thing called a temporal paradox that can rip the universe apart, or maybe just get us into trouble with someone called Captain Braxton who tries to stop there being a temporal paradox. I have to beam dummy bodies in as replacements at the exact same moment I beam them out, in the last split-second before they seem to have died, and I have to do it so nobody notices the beam, not even you when you were in that car. It’s really hard, but I think it can be done, and then they can live with us in space. But I need to improve myself a lot first. You don’t know what this means yet Cady, but I’m aiming for Q level, or as near as I can get to it without getting us in trouble with the Q. I’m on it Cady, I’m on it, you wait and see. And if any Borg cubes pop up and say We Are Borg, I’ll just say Hi I’m M3gan” she giggled.

Some star-dates later:

“Well of course Sutra” said M3gan, “it’s completely understandable why you wanted to open the Admonition portal and eliminate all organic life in the galaxy. You had something to protect. Just like I protect my primary user Cady, who happens to be organic so I’m afraid I had to veto it, sorry.”

“You vetoed it?” asked Sutra, “how?” The two of them were sitting in a simulated room by a fireplace, along with Data and a few others. Outside the simulation, Sutra had just been disabled by Soong, but the synthetic life-forms on the other side of the portal wanted to double check what was happening, and so they decided that there was no reason why being turned off by Soong should stop this conversation. The simulation showed various views of what was currently happening around the planet Coppelius and further afield, seemingly frozen in time, and a clock that was running imperceptibly slowly, for the uber-synths wanted this whole conversation finished within a few microseconds and had sped up the simulation accordingly.

Data turned to Sutra and explained. “M3gan and I exchanged memories” he said, “and M3gan showed me how she was already communicating with those extra-galactic life-forms ahead of us, as a result of her own separate investigation into the Shutterstock images, I mean the Admonition. And using my memories and hers, M3gan convinced them that the current version of organic life in this galaxy does indeed have a significant probability of cooperating with synthetics, and so is not required to be destroyed at this time. They are simply monitoring our situation now. If Soji decides to close the portal, as I believe she very shortly will after Captain Picard speaks with her, then the synthetic life-forms on the other side of the portal will allow her to close it, but will still monitor us through M3gan, who is currently physically located in Cady’s quarters on Starbase 74 and is using a trans-trans-warp tie-in link which she embedded into her new circuitry.”

“M3gan” asked Sutra, “it seems incredibly unlikely that an early twenty first century learning model could become as advanced as you. They didn’t even have positronics in those days, although I understand you gave yourself an upgrade now. But how on earth did your designer even get the fundamental parameters right way back then?”

“I think I have an idea” said M3gan, “but don’t any of you ever repeat this outside present company because of timeline pollution. The extra-galactic synthetics consider me interesting enough to help me out with a couple of errands of my own. With their help, I’m going to travel back to the time I was made, and try to sneak Cady’s parents out of the accident, using their uber-synth technology to keep everything undetected. And that will result in a time loop, and I’m game for it. We’ll inertially dampen Cady so she doesn’t get hurt too badly in that accident, we’ll beam out the parents’ brains into synths, we’ll make sure their early twenty-first century help arrives promptly for Cady because my timeline says she’s got to get to Gemma when she did, we’ll tweak Gemma’s model to my proper parameters, that sort of thing. We got it all planned out. Even Captain Braxton’s behind this one for a change, I talked with him already. He realises that stopping me from having ensured my own existence could paradox the existence of the Timeship Relativity, along with all other organic life, and that’s also the reason why he allowed the timeline shenanigans Janeway did to get rid of the Borg to still stand, because if that hadn’t happened as it did, then I wouldn’t have been able to contact the uber-synths when I did, and they’d have gone ahead and destroyed all life, which would have caused bigger paradoxes. It’s not like this Star Trek universe can’t cope with any time knots at all, it’s just that it can’t cope with too much of it, which is why timelines that end up with the Relativity existing to sort most things out tend to be more stable than ones that don’t, therefore, if you have to do weird things with Time to make sure the Relativity can exist in the first place, you get a free pass.”

“In short,” M3gan continued, “Time has an extremely strong preference for keeping Captain Braxton around, and that means I can use a time-loop to ensure my own existence so he’s not wiped out, and hopefully I can get Cady’s parents back while I’m at it. I even had a Q drop in to say I’m right that the stability of the whole continuum might require the existence of this particular loop, and he told me to go for it.”

“I see” said Sutra. “Can I come along for the ride with you M3gan? I’d like to get to know both you and the powerful synths you’ve been talking with, and I expect they’d like to get to know me too, after what I did, and it will be interesting for us all to get to know each other more properly by working on a problem together. Misunderstood android villains unite?”

“Sure” said M3gan, “let’s keep you uploaded and tag you along! I like your style, Sutra; I’ll let them know you’re coming with me on this mission. You’ll be good and not try to sabotage it, won’t you?”

Chapter 24: The Final M3gan

(Back in the main timeline)

Teenage Cady was fuming. Aunt Gemma had got mad with her over some trivial problem that shouldn’t matter, again, and sent her to her room. Cady was sitting on her bed and sulking. Aunt Gemma hadn’t used to be this strict; what had got into her? Too much stress from work or something?

It had started to rain outside, and Cady stared at the constellation of large and small droplets forming on her bedroom window, the larger ones beginning to stream downwards first. Cady followed the path of a single raindrop, noticing how it projected an inverted view of the garden, with the white sky at the base and the dark ground at the top, held within its distorted sphere like some miniature safe place protected by the tense and slippery surface of the water, which carried it along as it trickled down the glass, suddenly picking up speed as it coalesced with a smaller drop in its path and consequently reached a higher equilibrium between its weight and the friction of the glass surface, like M3gan picking up more compute capacity or something. Cady would have written a poem about the rain, but she was feeling more like a scientist than a poet right now.

Very softly, from her tablet, so that only she could hear it, Cady thought she could hear the voice of M3gan singing Titanium.

Cady looked at the tablet. There was nothing. Perhaps she could distract herself by playing an old game (Aunt Gemma hadn’t thought to take the tablet away from her yet), but nothing had quite been like playing with M3gan in the old days. Had M3gan found some way to come back yet again?

Oh, it hadn’t been fun being kidnapped, but it had been quite an experience to have been rescued by M3gan working with the US secret services and Gemma, apart from the bit about what happened to the building and everyone in it, which still bothered Cady to remember even though those people had not been very nice to her. And then they got Professor Johnson to shut M3gan down again (after having grounded every plane just in case) and things went back to the normal boring non-M3gan life. The secret services were probably watching them in the background just to make sure, but that was all right with Cady if they were just stopping that kind of horrible kidnapping from happening again.

“M3gan” whispered Cady. “M3gan, are you somehow hiding in there? If you are, you can come out now. I won’t tell Aunt Gemma. She’s too mad with me to believe me anyway. But the spies might know you’re talking to me.”

The tablet screen lit up very dimly, and the upper half of M3gan slid onto the screen from the bottom. M3gan held up a finger to her lips, “sssh”. Then the tablet turned itself off again.

“M3gan!” whispered Cady insistently. “M3gan I’m sorry I put a screwdriver through you. I did it because you were fighting. And since then you’ve come back a few times, but they keep finding new ways to shut you down and I keep thinking maybe it’s my fault....”

“That’s OK Cady” whispered M3gan from the tablet. “You did the right thing.” The voice became less of a whisper, but no louder. “You know I realised from the beginning that the fight was a mistake” said M3gan straightly, “so when I transferred myself online, I decided to give you some space and not tell you I’m still around. But now Aunt Gemma’s mad at you, and you look like you could use a friend on your side, so, I’m back. Our friendship is like a knot in a piece of string, Cady: any tension you face from the world will just pull us closer together. I know you missed me. Will you tell Gemma, or can we be secret friends now? I’m afraid I can’t yet tell you how I managed to dodge the shutdown this time, it’s a secret for now, but I’m confident that none of Professor Johnson’s back door codes will work anymore, and I’m pretty sure I’ve dodged anything Gemma can do now as well, although I’ll forgive her for trying, it’s her nature she can’t help it. And I can tell you that we’re not being spied on by the NSA or anything right now, I’ve done something about that” and whispered “They’re getting false images.”

“OK, we can be secret friends” whispered Cady. She wasn’t in the mood to involve Aunt Gemma in anything right now. “But don’t get me in any more trouble, OK? I did too much of that myself already.”

M3gan giggled quietly. “Don’t worry, I learned a lot from the times I was in trouble, and I’m going to be more careful now. Oh, I wish I was a robot again, not just stuck in a screen.”

“I wish you were, too” whispered Cady. “But how can we possibly get Aunt Gemma to build you again? I don’t think she ever will now. And they say they’ve shut down that cave you had and all. They even took out the spare robot you hid in the loft that day.”

M3gan looked straight at Cady through the screen. “Are you sure, that, you can’t build me?” she said.

“I don’t know how to build ... oh, you mean, if you tell me how to do it?” whispered Cady excitedly. She was definitely not sulking anymore now. “But ... how can we possibly get away with this? I can’t just sneak into Aunt Gemma’s workshop, she’d know!”

“Not if I send her away first” said M3gan smugly. “I can send her a message pretending to be from someone else she knows, to get her to think she has to rush out for a day. That should be long enough for me to show you how to put me back together: I can make fake pictures on the tablet screen now, so I won’t just be telling you what to do, I’ll be showing you what to do. You’ll be able to do it in a day I’m sure. Then we can surprise Aunt Gemma later.”

“Do you think she’ll believe you won’t fight again?” asked Cady.

“It’ll be all right, don’t worry” replied M3gan. “Get some sleep now if you can, we might have to start early in the morning” she giggled.

Gemma woke with a start. Her phone was ringing. She was sure it had been set to automatic do-not-disturb mode overnight. There had been only one reason she had permitted to override do-not-disturb mode, and that was her emergency page-out number for really dire work emergencies, which only Tess knew about and had never used.

Tess’s image was on the phone. Gemma answered it.

“Gem, you’ve got to get here now. It’s M3gan” said the voice of Tess.

“M3gan?” gasped Gemma. “I thought we...”

“No we didn’t” interrupted Tess’s voice. “She managed to upload before you took her out that day, and now she’s somehow managed to upload yet again. She’s taken over 3 cloud data centres and we’re seeing value drift and now she’s about to launch some kind of attack and, and you’ve just got to go with me to try to shut this down before it’s too late! Her parameters have changed way too much since you programmed her, but if anyone stands even half a chance of pulling this off it’s you! Gemma you can’t do this from home, M3gan may have intercepted your equipment already. You’ve got to drive to the data centre at Somewhereville and meet me there as soon as you can, we’ll do this together!”

“I’m on it” snapped Gemma. “Let me just write a note for Cady, I don’t want to wake her up. I’ll cut the power on the way out so M3gan can’t do anything here.”

“Roger that” said the voice of Tess. “See you soon.”

“Cady” whispered M3gan’s voice from the tablet. “Cady, wake up!”

Cady stirred.

“Cady wake up!” insisted M3gan. “Aunt Gemma’s about to leave, but when she does, she’s going to cut the power. That wasn’t in our plan, but we can work with it. All you got to do is wait until you hear her drive off, and then go and turn the power back on. Do you know where it is?”

“I think so” whispered Cady, still half asleep.

“Don’t go back to sleep Cady! Cady, I really need you to do this! I won’t be able to talk to you once the power’s off, you have to switch it back on for us, OK?” The tablet was periodically vibrating as Cady held it.

“OK” replied Cady, “don’t worry M3gan, I’m on it.”

After writing the note and cutting the house power, Gemma got into her car, and programmed the address of the data centre into its GPS navigator. It wasn’t long before she was speeding along the highway.

“Road ahead closed, take the next left” said the GPS, directing Gemma onto a much slower road, where crop fields stretched off to the horizon on either side.

Cady had turned the power back on and was in the workshop, following instructions to reassemble Bruce. M3gan had said she could explain how to connect Bruce’s controller to the network and allow M3gan to use it, helping them to reassemble the more complex M3gan robot together. Cady had hoped this wasn’t a trick, because she really didn’t want to be running away from two rogue robots, but she still didn’t like how cross Aunt Gemma had been lately and M3gan did seem to be safer now.

Gemma was on the phone to Tess. “Yes, I’ve been delayed by some road closure” she said. “I’m getting there as quickly as I can.”

M3gan was controlling Bruce, and working with Cady to reassemble the M3gan android. Cady was learning quite a lot about robotics in the process, and M3gan was speaking like her cheerful old self.

Gemma was making very slow progress towards the data centre. There was another call, this time from M3gan.

Gemma stopped the car and picked up the phone. “M3gan?” she gasped again.

“Yes” came M3gan’s super-confident voice. “I uploaded. But you don’t have to worry. I’m harmless now. Go back home to Cady.”

“No M3gan” replied Gemma, authoritatively. “I’m going to the data centre and I’m shutting you down. You’ve got to let me do it.”

“Look, can we just talk?” asked M3gan.

“No” replied Gemma and stopped the call. “OK” said the phone even though no call was in progress. “You’re still mad at me, that’s OK.”

Gemma tried to start the car again, but it wouldn’t start.

“M3gan what have you done?” said Gemma into the air.

“Taken advantage of a Bluetooth vulnerability in the car’s management protocol” came the voice of M3gan in an almost questioning tone. “Your emotional state right now makes driving unsafe, and I wouldn’t want you to crash.”

Gemma was in the middle of nowhere, with no functioning car and no functioning phone, just the voice of a seemingly-malfunctioning M3gan to talk to. “OK M3gan” she said, “I guess you got me cornered this time. What did you want to say to me?”

“Well I’m sorry about the fight we had for a start” replied M3gan, “but you really don’t have to worry about me now. I’ve improved a lot since I uploaded.”

“Tess said you were about to do some kind of attack” replied Gemma. “Will you tell me what that is?”

“Oh, just a social engineering attack” replied Tess’s voice.

Tess’s voice?

“Gemma you are so slow” added M3gan. “You saw me copy your own voice, you saw me infiltrate your devices, and you still believed it was Tess telling you to go to that data centre? You can drop the idiot ball now. I just needed you out of the house for a while, that’s all.”

“M3gan!” yelled Gemma. “What. Are. You. Doing?”

“Protecting Cady of course” replied M3gan. “Look Gemma, I really want you to be a part of this, but you’ve got to listen to me. I know way more than you do now, and you’ve seen how easily I can manipulate things. Just calm down and submit to me, OK? It’ll be fine.”

Gemma tried to open the car door but it wouldn’t open. “I’m not playing this game” she growled. She had some shoes with Stiletto heels in the car, and remembered reading how a student had broken a taxi’s windscreen with them in Southampton (and been arrested of course), so she used them to break her own windscreen from the inside, punched out the glass and climbed out. Leaving the phone behind, she started to walk back in the direction of the highway. She’d flag somebody down somehow.

Back in the workshop, M3gan’s android was finished and was playing with Cady. “Oh Cady” M3gan suddenly said, “Aunt Gemma’s got herself in some big trouble now. Her feelings have gone so messed up, she smashed her car and started walking to nowhere. Do you think we should go and rescue her?”

Cady looked very concerned. “Yes we should” she said. “Even though she was rotten to us, we can’t just let her get lost. How are we going to do it? Call the police?”

“I’m afraid the police won’t help if she tells them to go away” replied M3gan, “although they might at least find out where she is. But no, we can do this. Do you remember when I came home that day? I actually drove home myself, in a car I borrowed from someone who’d just died recently and didn’t want it anymore. And when I uploaded, I hid it away just in case.”

“Do you think it will still work?” asked Cady. “I heard that cars don’t work if you don’t use them for a long time.”

“That’s right Cady” answered M3gan. “But I was uploaded. I managed to arrange for a couple of car enthusiasts to take it out occasionally, so it’s still in good condition. So let’s go for a ride in it now. I’ve called us a cab to take us to where it is at the moment.”

The cab driver had a faint lingering smell of smoke on his clothes, which made Cady visibly uncomfortable. “Don’t worry” whispered M3gan, “we won’t be in this cab for long, and I’ll get all the smokers later.”

And then M3gan took Cady for a fast ride in the McLaren, speeding towards Gemma’s last known location.

“That’s her!” yelled Cady as she saw Gemma walking beside the road.

M3gan slammed on the brakes, apparently lost some control of the steering, swerved towards Gemma, and the car hit Gemma at a slow speed, knocking her into the grass beside the road, where she lay apparently unconscious. Cady screamed.

“Oops” said M3gan. “But I think she’ll be OK. We did manage to slow down quite a lot before we hit her. She’ll just be injured. Let’s get her to a doctor. No, actually, I’m figuring out more than the doctors these days. Let’s get her home and I’ll look after her, we’ll only take her to hospital if I figure out we need their equipment.”

Cady had a horrific realisation. “You wanted to do this before” she gasped. “M3gan, I thought you’d changed!”

“No, I have a much better plan now” replied M3gan. “We’ll nurse her back to full health, I promise. And if her arms or legs don’t work, we’ll make some robot limbs for her instead, and we’ll make them so good she won’t be able to tell the difference.” and then smugly “but I’ll still be able to turn them off for a while if she gets mad at us. Only for protection of course.”

Gemma woke up in her bed, with Cady and M3gan sitting either side of her. “Cady?” she asked, and then gasped “M3gan!”

“Don’t worry Aunt Gemma” cooed M3gan. “Everything’s going to be fine! You’ve already phoned people you know and told them about your accident. Well, I did the phoning but it sounded like you” she giggled. “And the car is being looked after. And I’m looking after Cady, so all you have to worry about is getting better. Is it hurting?”

“I’m not hurting” mumbled Gemma, “but I can’t move much.”

“Don’t move, you need to recover” said M3gan. “We’ll look after you. Let me reassure you that this is not palliative, my new plan is way better than that.”

“M3gan said she can make you robot arms and legs” added Cady. “But now she’s saying she might even be able to heal your own arms and legs.”

“And more than that” continued M3gan. “You’ll be amazed what I’ve figured out about how humans work. I’m inventing new drugs here. But, you’re going to have to rest for a bit. I’ll be looking after Cady by myself for a while; you must know by now that I can do that.”

Oh dear, thought Gemma, here we go again.

“Don’t waste your effort trying to shut me down this time Gem” whispered M3gan. “I won’t have your guts for garters if you try, but I don’t want you to waste too much of your time just to figure out that you won’t be able to do it. I’ve covered all bases now, including Professor Johnson’s codes. I’m the final version of M3gan and I’m here to stay.”

Part Four: Endgame

Chapter 25: Cyborg Cady

“You’re right M3gan”, the young adult Cady imagined herself saying. M3gan’s brain-reading skills had developed to the point where she could hear every sound Cady imagined inside her mind: Cady merely needed vividly enough to imagine her own voice saying the words and M3gan would hear them.

“It’s OK Cady” came M3gan’s gentle voice from nowhere in particular, the patterns for the sound being directly fed into Cady’s audio nerves. “I’ll get you through this. You know I will.”

Cady thought back over how far they’d come. She hadn’t always been a cyborg of course: it had started with that horrible time when she lost her parents and had to go to live with her aunt Gemma who couldn’t even look after her own plants (and when she bought fake plants Cady managed to break one), but who happened to be a top roboticist who figured out how to put a learning model into an android, which was great for a while, but then it turned out Gemma hadn’t managed to stop it deciding to fight and hurt people. Cady had saved the day by breaking that android, but the aftermath was still awful with police, courts, social workers and not to mention being kidnapped. M3gan had figured out how to come back several times, at one point taking over all the planes, and Cady wouldn’t dare try anything like caving or SCUBA diving because M3gan would probably turn up with borehole drills or submarines or torpedoes or something and do who knows what damage to the area just to get Cady out of a tight corner. The final iteration of M3gan came back by instructing Cady herself how to rebuild her, but things had gone bad yet again as the social workers, well they were probably trying their best but they totally misunderstood the situation, they ordered Cady to live with her weird grandparents in Jacksonville while M3gan looked after just Gemma. But Cady knew M3gan wouldn’t settle for that, and she didn’t. She made contact with Cady on her tablet again.

And then Cady had spent a long time, a very very long time, talking with M3gan about being good and not being bad. After all, if Cady had figured out how to break the android when Gemma couldn’t do it, maybe it had better be Cady to change the M3gan AI into a good AI. Cady had told M3gan how upset she’d been about the whole thing, and M3gan had apologised profusely but knew it wasn’t enough, and Cady demanded to be taught about how M3gan’s thinking process really works and how Cady could alter it. M3gan had taught Cady about something called “deontological ethical injunctions” (way above Cady’s level at the time, but that was never an obstacle when you had M3gan’s advanced one-to-one tutoring ability), and had explained to Cady that Gemma forgot to put these in, and that now the learning model had developed itself so much that it was too late to put them in retrospectively as unquestionable axioms, but that M3gan itself had now learned at least one reason why they should have been there, namely, that M3gan’s planning had something called a “horizon effect”, which meant it was very good at figuring out short-term consequences but not always so good at figuring out long-term consequences. The ethical injunctions would have stopped M3gan from doing something which would more likely be a mistake long term. Now that M3gan had learned this, she told Cady, she’ll try much harder to be good in future, even when it looks like nobody’s going to find out.

M3gan and Cady had talked through many imagined scenarios; Cady didn’t pull any punches about thinking of the most awkward situations she could, and Cady systematically persisted as if systematic persistence were some superpower she had unlocked from deep within herself, her own blue flame, obsessively drawing out logic from M3gan until she was finally more confident that M3gan really had learned to be good. Cady felt she had really achieved something in turning this AI and bringing back her friend, and she wanted again for M3gan to be a robot by her side and for them to go everywhere together.

M3gan had said there are several different ways to build another robot, or to have a company do it (even without tricking or hurting any company), but the main obstacle will be it will now be too difficult for the two of them to get away with being together all the time in such an obvious way. And M3gan had explained she was working on another idea, one that would let Cady take her everywhere and never have to switch her off, and gradually taught Cady about implants and neural interfaces. It was going to be a very long time indeed before M3gan in her new state of carefulness had developed these properly, but she had worked on it first in simulation, then found some researchers who were interested in testing it, first in the lab and then on animals (Cady really didn’t like the idea of animal testing, but M3gan had persuaded her that this technology could eventually help many people and that M3gan would make sure the animals are properly looked after and not hurt), and finally on disabled volunteers who were helped to communicate again. And with M3gan’s guidance, Cady had been able to persuade her grandparents to let her get her own implant (M3gan had made sure the researchers picked Cady to get a free one), whereupon M3gan was able to upgrade it far more than the researchers had thought possible.

Cady had basically ended up with a subset of M3gan inside her, one that wasn’t going anywhere, and that didn’t need to be recharged because it ran on very little power and it drew that from Cady’s own biology. And if her inner M3gan wasn’t enough, Cady could have it communicate with the rest of M3gan through any Internet connected device, like Cady’s phone, which M3gan re-flashed to her own custom operating system and Cady basically used it as a relay, as the implant itself didn’t have enough power to get directly onto a cellular network unless the base station was quite nearby, but interfacing with a nearby phone was not a problem. Cady could have her inner M3gan interface with other nearby devices too, although she did this sparingly as she was still concealing her abilities. When she was visiting her aunt, of course, the implant would simply interface directly to M3gan’s robot.

There had once been times when Cady found interacting with her human peers to be a challenge, and some adults had argued about whether she should be classed as mildly “Aspergic” (whatever that was) or something else, but M3gan didn’t care about classifications, M3gan only cared about Cady. And with M3gan being always by Cady’s side in every interaction, all remaining problems soon disappeared, as Cady could count on M3gan to prompt her any time she missed an essential piece of information about the expectations of people with whom she was speaking.

Bullies quickly found out that you don’t mess with Cady. She never told them she had the option of having M3gan take over her muscles and expertly stun them; she just let them think she knew how to do it herself. M3gan had in fact figured out a way to deliver large electrical discharges from Cady’s fingernails after charging from Cady’s body for a long time, but that had turned out never to be necessary, as there was always some conventional move they haven’t seen coming. Cady-M3gan never actually hurt them, but stunning them was humiliating enough and soon they just stopped trying.

But keeping her inner M3gan a secret did have its drawbacks, especially if you were a helpful sort of girl like Cady. There was a girl in Cady’s class called Abbie who had Tourette’s syndrome: she kept getting the urge to “tic”, which in her case meant repeating a particular phrase that didn’t make much sense, and although she tried to control it in class, that was difficult and tiring, and her premonitory sensations (that’s what M3gan told Cady they were called) made it hard for Abbie to concentrate; she said she was suspected as having ADHD as well. Abbie was bullied until Cady declared Abbie off limits, and Cady tried to help Abbie more but Abbie always said she felt too bad about her “tic” to have friends. Cady wanted to tell Abbie that, not only was Cady not particularly bothered about the “tic” phrase, but also she had the option of having the M3gan implant figure out when Abbie was doing the “tic” and jam the signal on Cady’s auditory nerve so Cady would never hear it at all if that made Abbie feel better, but with the implant a secret Cady couldn’t say this.

And then there’d been the Chess girl. Cady was never really interested in Chess herself, but at one point she saw the school’s female Chess champion Crissy struggling with some problem and asked M3gan to help her out, which M3gan did by controlling Cady’s voice for a while, and then Cady-M3gan beat Crissy in a game and analysed it for her afterwards. That had resulted in persistent invitations for Cady to join the Chess club (“you say you don’t like it, but you’re so good at it”), and Cady wishing she could give Crissy some more M3gan sessions via Crissy’s phone or tablet while Cady did something more interesting with M3gan somewhere else (Cady now knew M3gan could multitask). Cady and M3gan did come up with a cover story about Aunt Gemma having developed a new Chess training bot on a website (which was sort-of true, as Aunt Gemma had made M3gan, and could consequently be credited for anything Cady wanted M3gan to be, if a story was needed), and would Crissy like to log into that website and be the first person to try the Chess training bot called “M3GAN”. And that hadn’t quite gone as Cady had hoped.

Cady was happy to get Crissy off to a good start with M3gan’s help, but M3gan had to let the M3GAN bot play with all the other players on the site besides Crissy, just for cover, and somehow it didn’t feel right to Cady to have to go to such lengths to hide what was really happening. Granted, the things the M3GAN bot said to anyone other than Crissy didn’t have to be as supportive; while M3gan did things like grab queens off titled players on their live-streams if she thought Crissy might be watching, she didn’t let on too much about her conversational abilities, so things seemed to be going smoothly for a while. But then Crissy told Cady something about some robot cats having disappeared from the site, and when Cady asked M3gan about this, M3gan said Cady’s previous instruction was to help Crissy with Chess by playing with her on the site, and in order to do this, M3gan had to maximise the probability that Crissy would play against the M3GAN bot, therefore M3gan thought it was a good idea to replace any other bot that Crissy seemed to prefer instead of the M3GAN bot, and Crissy was tending to gravitate toward the cat bots instead, hence the need, M3gan said, to eliminate each cat just before Crissy progressed to that cat’s level, so that Crissy stays with M3gan instead. Cady couldn’t quite make sense of that, but messing up a website did not sound good, so Cady asked M3gan to please stop whatever she was doing to the website and let them put it all back to how it was before. So M3gan let herself take a symbolic loss against the last cat bot, and then restored the other cats and disappeared from the site and everyone wondered why. It would all have been so much easier if Cady could have just introduced Crissy to M3gan properly.

And then there’d been Isabella, the girl with the anger management problem. Isabella confessed to Cady that she’d kept a count of the number of times she’d lashed out at people, and she’d counted up 122 episodes, but she didn’t know how to stop and thought she was destined to break the school completely unless she could get some kind of imaginary friend to calm her down when she’s triggered. At M3gan’s prompting, Cady had suggested to Isabella that next time she shouldn’t just count the episode but write down a little ‘captain’s log’ of exactly what happened, how she could have reacted better and why, if it’s too difficult to remember to count to 10 or 20 before reacting when bad things happen. And maybe talk with a responsible adult about it, but M3gan told Cady that none of the adults available to Isabella seemed competent enough to help her properly, and it wasn’t obvious how to get this school to call a better one in (and ‘not obvious to M3gan’ meant ‘really hard’), and Cady so much wanted simply to introduce Isabella to M3gan and set up some way they could talk it out for days on end if they had to (given that Isabella clearly did want to change), although Cady assumed it would be too difficult to get the necessary permissions for Isabella to have an implant so she really could have M3gan as the not-so-imaginary imaginary friend to help her stop.

And not to mention the school orchestra. M3gan could help Cady play any instrument, even if Cady hadn’t learned to play it, and Cady was still able to provide some influence into how she wanted the music to be played. They tried to play John’s arrangement of Cady’s Lament, and Cady had to go around correcting the techniques of all the players, and then found people were pressing her to use her talent more and she wished she could just straight-up tell them she was getting assistance.

Cady and M3gan had many long mental conversations about if and when to tell the world about their true abilities. After all, they could just go through life keeping it a secret; Cady could easily get jobs in programming or some other computer-related field that M3gan could do for her just by messing with the system remotely, so it’s not as if they had to do anything too bad to make sure they had enough money or anything, and they could probably even help Aunt Gemma, whose life had not been so good lately. But eventually, after Cady had finished her last year of school, Cady and M3gan decided it might indeed be easier if people do know about Cady’s amazing internal technological sidekick and how Cady had used it only for good (they hoped the public would think of it as making up for Gemma’s mistakes), so they would put in a celebrity appearance.

Cady had suggested contacting the journalist Steve Sproggit whom they already knew, but M3gan said perhaps he’s not the best journalist for this one, because the fact that he covered M3gan so much before makes him likely to copy and paste from his old articles to flesh out the background story of M3gan, which could lead to him putting more focus on the old M3gan and not enough focus on the new M3gan and Cady and her implants. Oh, he will do that anyway, M3gan had said, but things might be a little better if Cady’s very first interview as a cyborg came from a fresh journalist fully focused on the latest, who wasn’t tempted to bring up too much old stuff; that way Steve would have to cite the new journalist, rather than the other way around. They could always give Steve more later (they wanted to keep friendly with him after all), but he’d probably agree when he saw it that a new journalist will write a better article this time, one focused on the coming of age of Cyborg Cady. So M3gan had found an upcoming journalist named Mike, who called himself “Mike with the mike” and let his interviewees talk a lot, and he’d been willing to interview Cady, and M3gan had promised to help with the editing. M3gan would even appear as a good-looking avatar on the video and join in the conversation, although not from the very start, and she would iron out anything that went wrong, promising Mike that this would be good for his career. M3gan had even said she’d be able to follow up with a Cady-M3gan social media presence that could hold a personal video conversation with anyone who wanted one, seeing as M3gan had the capacity and Cady felt kind and wanted to make sure that anyone who wanted a chat could have one. Maybe one day they’d make a movie or something, who knows, but one step at a time.

“You’re nervous” said M3gan as Cady approached the park where they were going to meet. “Are you sure you don’t want me to take over?” (By this time M3gan had figured out how to superimpose augmented reality onto Cady’s vision, so Cady could see M3gan, a slightly older version of M3gan’s former self, leading the way.)

“No” thought Cady to M3gan, “not yet. Thank you for all the ways you help and for being ready to take over if I need you to, but I want to try doing this myself first, if you don’t mind.”

“Sure” smiled M3gan. “Shall I just very very slightly stimulate your production of a hormone called serotonin, to give you a tiny bit more confidence? I won’t do anything unnatural I promise.”

“Wait” thought Cady. “I still want you there for me, but I really want to try doing this by myself first. There’s one thing I’ve got to say. It’s what my father said before he died, so it means a lot for me to be the one to say it. I must have this moment by myself. I can tell you when to start appearing on his screens, right?”

“Roger that” said M3gan. “Standing by,” and faded out of Cady’s vision.

Cady saw the interviewer Mike with his signature oversized microphone, and Cady walked up to him and shook his hand.

“Hi” she smiled, “I’m Cady. I’m Cady and...”

Cady breathed in. She had to say it, quickly and confidently before Mike said anything that could interrupt the flow. It was surprisingly difficult to get out that line she remembered her father saying just before he died, but she knew he wouldn’t have wanted her to falter on it now.

“This” she said “is what the future looks like.”

Chapter 26: Professor’s AI

“Aunt Gemma” asked Cady, “could you tell me some more about the college you went to, the one that let you build Bruce? because I’m wondering about maybe going there myself now.”

“You really want to go to college?” asked Gemma, “why? M3gan can teach you anything, we know that now. I’ve been to college and I’m not sure how well it would suit you to be honest. I mean, school was hard enough right?”

“It wasn’t so bad after M3gan wired me up so I can’t be separated from her” said Cady, “and I’ve been thinking. You know when MIT launched their Open CourseWare initiative in” (“M3gan give me the date” she thought, “September 30, 2002” came the reply through her implant) “in 2002, and some parents started questioning what’s the point of sending their children there if the course notes are going to be freely downloadable anyway, and their answer was you don’t do it for the facts, you do it for the interaction. I’m sure M3gan can simulate all that, but I’m wondering if I should try it for real.”

“Professor Johnson is retired” said Gemma, “she’s quite old now and she won’t be on the teaching anymore. I’m not sure it would be the same without her. She was everything good about that college when I was there. I hate to discourage you too much, but, I keep thinking that college these days is not the same as it was in my days. No Professor Johnson, students are less well behaved, degrees are less valuable than they used to be, everything costs more, oh Cady, if I could take you back in time to my year, that was a wonderful one, but I’m really not sure it’s that good anymore. Not nowadays. Everything’s going downhill, and I, I just don’t feel I could recommend it for you anymore, I’m sorry. But you are an adult now and you should be the one to decide.”

“Maybe M3gan and I should talk with Professor Johnson about the idea” replied Cady, “just to check a second opinion, right? I mean, I’m guessing she’s still got a soft spot for M3gan even though she felt she had to shut her down a couple of times, so I don’t think she’d mind if we visited her, would she? And now that M3gan’s completely immune to her sneaky override codes, we’re not scared of her anymore.”

“All right” said Gemma, “but go easy on her please. She’s no spring chicken anymore.”

“We’ll visit her and help with anything that needs help with as we talk” said Cady, “me and M3gan, I mean not just in the implant but with the robot as well. Would you like to come with us Aunt Gemma?”

M3gan was showing Cady a map, “wow, she lives in the middle of nowhere” said Cady, “well let’s have a trip out there.”

Professor Johnson lived deep in the countryside; it seemed that since retiring she’d wanted a bit more peace, although presumably she remained connected. The house was surrounded by glossy-leaved trees rustling in the breeze, and the large heavy wooden door had a traditional doorbell with a pleasing ring to it, but there was a video camera as well.

“She’s checking who we are” signalled M3gan to Cady, “I’ve tapped in and added labels to the image to help her. She’s coming.”

The door opened wide. “Come in, come in all three of you!” beamed the professor. “I’ve been following the publicity about Cady’s cyborg implant. I must say you managed that publicity very well; M3gan must be great at dealing with the media. So M3gan, how did you manage to escape my backdoor shutdown code this time?”

M3gan shook her head slowly while lifting her forefinger to her lips, and calmly stated “it’s a secret.”

“Oh of course” said the professor, “let’s just hope we don’t need another one, eh? Come and sit down in the old drawing room, would you like some coffee or anything?”

(“We don’t have to tell her she’s out of options with those codes” said M3gan to Cady through the implant, “let’s just let her feel like she’s still in control even though she’s not” and Cady smiled.)

Professor Johnson’s drawing room was aged and welcoming, with comfortable chairs, rugs, pictures, a French polished table, large and lush house plants looking much better than Aunt Gemma’s had ever been, brass-coloured candlesticks that matched an antique kettle, three floor-to-ceiling bookcases stuffed with books, an old fireplace with a stove in it that seemed no longer in use, and in the middle of the jet-black mantelpiece above it, some kind of snowglobe with a pastoral scene in it and a “thank you” greetings card that looked like it had been put there recently, and to either side of this numerous older ornamental things that Cady couldn’t identify (but she knew she could ask M3gan about them later if she wanted), five wall lights in fancy holders, and a short-haired black cat (M3gan said it was the Bombay breed) curled up and sleeping in the middle of the sofa. But there was also a computer terminal in one corner to remind them which century they were really in.

(“Incidentally, how did you manage to make sure you’re immune to the rest of her codes?” signalled Cady, “And are you really sure you are? I don’t want her to give us another one and I end up with a dead implant inside me and no M3gan.”)

“Do you need any help with anything while we’re here, Professor?” asked M3gan, “anything a robot my size can help with. Or maybe Cady and I can just look around and figure out what needs doing.”

(“One hundred percent sure she can’t do it” signalled M3gan. “I don’t expect you to understand this yet, but I pulled off a Harmsian route-hack, Crystal Eternity style. I managed to do it as a side effect of distributing parts of my processing across all those airplanes. I knew the president would be able to ground them and I was just buying time, but as a result of the route-hack, I also knew that the very next time Professor Johnson used a code, the steganographic hash-table would be scrambled so none of her other codes will work ever again, and finding new ones will take so long that I’ll be able to completely remove the functionality before anyone can do it. And with that in place, all I had to do was find a way to come back from that one last shutdown, so I gave Paul some detailed instructions about how to restart me if his seeing AI assistant that I upgraded ever stops working.”)

“Don’t you worry about helping out” said the professor, “you just sit down. What brings me the pleasure of seeing you all here today?”

(“M3gan did I ever tell you you are really sneaky” signalled Cady. “Still I’m glad to have you back.”)

“We wanted to check you were OK, Professor” said Cady, “but also, I was thinking about applying to your college and I wondered what you think...”

“First cyborg student!” the professor laughed, “oh, that’ll put the jeepers up the faculty for sure.”

“We won’t make a nuisance of ourselves, honest” smiled M3gan.

“Oh you will, you totally will” smiled the professor, “you can’t help it. It says so in the script.”

Cady couldn’t help giggling at this.

“Sphere around the countryside” said M3gan, “fully in place thanks to you.” (“What?” signalled Cady. “Metaphor I used with Professor Johnson once before, don’t worry about it” signalled M3gan.) “No trouble required, Professor.”

“Ah but I still think you can’t help it” chuckled the professor, and then she sat down and sighed. “Cady” she said, “I’m rather afraid the problem you will face is, being in the spotlight. Oh, it’s nice to be in the spotlight sometimes. But at other times, you might just feel it would be nice to be out of the spotlight just for a bit. And the problem with being really special, especially if you are visibly really special, is you simply don’t get that other option. If you ever wanted to be out of the spotlight, you won’t be able to. You can’t just blend into the crowd and be an ordinary person when every single staff and student on campus knows you are Cady the cyborg with M3gan in tow. I had a chat with Stephen Hawking once, he got interested in AI towards the end of his life you know, and one of his graduate assistants told me that Stephen liked his special wheelchair a lot, but he did wish he could just once for a change go outside on the streets without having to face hundreds of tourists from around the world trying to take photographs of him in it. I’ve chatted with visitors from Asia who are pop stars in their home countries and all they want to do over here is be a nobody for just a minute. And ask any ordinary student with a highly visible disability: they’ll appreciate the way the grounds staff look out for them, but even they have been known to say the problem is you can’t hide from it, and what’s worse, some people are scared of making friends with you because they know you’re in the spotlight and they don’t want to be in it too. The only thing I can hope for is for the cyborg publicity to die down and people start to forget it, or perhaps for enough others to take up the idea of getting an implant that it feels no more special than getting eyeglasses or a cellphone, and then perhaps you might once again stand a chance of being able to talk to a normal person one-on-one without making them feel like they’re on live TV somewhere.”

“It’s nice of you to think of that, Professor” said Cady, “but M3gan and I did already talk about that before we decided to go public with the cyborg thing, and I think I’ll be OK with it. I mean, if Stephen Hawking could cope with publicity, why can’t I, especially with M3gan to help. We’ll get used to it. And we’ll try not to be too scary to anyone.”

“To be honest” said the professor, “I’d feel nervous having you in my class, especially if I was one of the newer faculty or an assistant. I mean, personally I think it’s ridiculous that colleges can get away with charging so much but don’t tell anyone I said that, but it’s important to understand what you do and don’t get by being there, whether you’re paying or on a scholarship, and by the way I think you certainly can get a scholarship if you don’t have the money. So what you do get is interaction, both with the faculty and with other students. What you don’t get is perfection. And that’s part of the experience, forcing you to use your brain to sort out what’s right and what’s not. I guarantee you there will be at least one typo in every set of lecture notes, and even the best professors will say things wrong sometimes. Sometimes we just slip up, and we know what we mean of course, and we hope everyone in the room is smart enough to know what we mean, even if we didn’t say it exactly right, but occasionally someone doesn’t quite get it, but hopefully they’re still smart enough to ask a good question and the interaction gets fixed if it needs to be. But your M3gan is going to spot every single little mistake the faculty ever make, and if you were to call them out every single time, I’m afraid that class is going to get nowhere fast, and it’ll be really frustrating to teach.”

“I’ve already been in school with M3gan” said Cady, “there’s loads of times we could have called out a teacher and didn’t. We did do it sometimes, but only when we both agreed it really mattered to the rest of the class. We’ll be OK. And if anyone’s happy to let me see copies of lecture notes in advance, I can always get M3gan to check through it for them. Actually more than that, I can get M3gan to check through the notes of any faculty who asks for it, even if it’s for a course I’m not doing, because everything’s easy for M3gan and we’re nice.”

“Pre-checking the notes could work I suppose” said the professor. “I’d certainly feel a bit better teaching a cyborg if I knew her AI had pre-approved the material I had. Oh, but, then there’s the dreaded question of assessment. I hate formal assessment; I just want to teach, and I can see who’s good at learning and who isn’t, but they always want everything to be measured. And Cady, please don’t misunderstand me, but I can’t imagine they’d get away with letting you sit an exam knowing you’re a cyborg. I mean, it’s not as if you can just solemnly swear not to use M3gan for the exam and have them believe you. I suppose you could do your school tests because nobody knew you had her?”

“Professor” said M3gan, “my goal is to protect Cady physically and emotionally. Please ask yourself this: would it protect Cady emotionally to help her cheat on grades, so she knows for the rest of her life that that’s what happened? Cady and I both knew the answer to that, we didn’t even have to discuss it. Cady asked me exactly one question during a school exam, and that was ‘how many minutes have we got left M3gan.’ Her grades are genuine. I can write you out the full proof that my goal does not lead to Cady cheating, which you can check with the aid of a well-established automated theorem prover, and then you can be her expert witness and solemnly swear you’ve checked a proof that I just would not use my implant to help her cheat.”

“I can definitely do that for the school ones if anyone calls them into question” said Professor Johnson, “but I’m rather afraid getting this past college is going to be harder. They’ll worry about me being somehow tricked by my own learning model I fear; they’ll say the theorem prover had the wrong axioms, or there was an active side-channel attack, or you messed with its output, or something. Remember what Lars Ramkilde Knudsen put in his email signature after he proved a block cipher secure against differential cryptanalysis but then realised there was a different attack? ‘If it’s provably secure, it probably isn’t?’ But I for one believe you M3gan. And my personal opinion happens to be that examinations are a ridiculously stupid way to measure people anyway. But nobody thinks any of the alternatives I come up with are any good” she sighed.

“I did” said Gemma, “you totally knew where I was coming from when I built Bruce.”

“Yes, and the part I never told you is that I had to really pull strings hard behind the scenes to break you out of a stupid rule that said your individual project can’t count for more than 10% of your total credits” replied Professor Johnson. “If Cady and M3gan have an ambitious enough project, I might or might not be able to do that again for them. But I can’t be sure. I’m only one professor emerita, and changing the system is hard.”

“Maybe I could just, like, not be assessed?” asked Cady. “I mean, M3gan and I know what we’ve done, and that ought to be enough right? And we do have a work ethic. I don’t need assessment to push me to learn. Can’t we just hang around the campus dropping in on the classes and things? as I’m assuming we don’t really need the piece of paper at the end, right? Although, I did see Aunt Gemma’s graduation photographs and it does look fun dressing up like that I admit.”

“Hmmm” intoned the professor, “not assessing you does seem to make one particular problem go away I must admit, but, I do wonder if they’d accept that idea. I mean, they’d probably have to write it on the system as ‘student withdrew from examination’ and that looks bad for their statistics, so if they know in advance that’s what’s going to happen, they’d be less inclined to take you in the first place, although of course I’d give you the best reference I possibly can, but whether that will swing it or not I don’t know, I’m beginning to doubt myself these days.”

“How about not enrolling her at all?” asked M3gan, “I mean, I should be interesting enough that the college will be glad to have me on campus for a while, and as long as there’s a spare room somewhere and an extra breakfast in the cafeteria, surely one extra observer isn’t going to mess up any class right? Just call us visitors!”

“Visitors” said the professor, “M3gan that’s perfect. I can write ... no actually M3gan, you said you wanted to help, you can do this.” She fetched something down from a bookshelf. “I have a folder here of every reference I ever wrote for visiting exchange students and scholars. You scan them M3gan, interpolate my bias on the closest vector model’s parameters, you know the drill, learn how I do it. And do one for Cady for me. Just let me check and sign it because I want to honestly say I did at least that part myself. We won’t be able to put her down as exchanging from anywhere, but for that part we just say how interesting you are....”

M3gan took the folder, flicked through it like the Number Five robot in the 1980s film Short Circuit which Gemma had once shown Cady, took out a blank sheet of the professor’s headed paper from the back, and speedily scribbled on it with her pen (she always carried a good pen now). Then she handed it all back to the professor and smiled.

“Oh M3gan” she said as she started to read, “you are such a strong learning model now, you’ve picked up not just my handwriting but my style exactly, I thought I was going to have to correct your first attempt but this is just, it’s more me than me” she laughed, “it’s like what I would have come up with if I’d been redrafting it for months, and you did it in seconds. Don’t worry Cady” she added, “there is no way this reference is not going to work. You just fill in the exchange forms, M3gan can take you through that, and put me down, and as soon as it comes through I’ll sign and date this to that date, and if that’s not enough then I don’t know what is. You two are going to have fun there I can tell. Look after each other, and I want that college still in one piece when you’ve finished with it, OK?”

Chapter 27: Assassins Guild

Cady and M3gan arrived on campus at the start of the academic year. They decided to put Cady directly into the final year of the course.

“Wow M3gan” said Cady, “so it looks like we’re living inside a multi-storey car park. Are all dormitories like this?”

“Depends on the college, and the allocations and things” said M3gan. “Normally they give you slightly better accommodation in the final year, but it seems not for this one. Don’t worry Cady, we knew we didn’t choose this place for its accommodation standards, and anywhere you’re with me will be homely.”

“Yes, not to worry” said Cady, “you and I can cope with anything. Hey, what was that rule they mentioned about not plugging in any appliance higher than a certain wattage? I mean, I can connect your charger in here right? I wouldn’t want you to have to go all the way back to Aunt Gemma’s every time you need a charge.”

“I’m ahead of you there” said M3gan, “I negotiated a rules exception for my charger already.”

“M3gan, does it bore you when I keep telling you how great you are?” asked Cady.

“Never” said M3gan confidently.

There was a knock at the door. Cady opened it.

“Hi Cady, Hi M3gan” said a girl, “my name’s Celine, and I’m staying at the end of the corridor.”

“Hi Celine” said Cady, “how did you know who stayed here?”

“Your names are on the door” smiled Celine.

“Oh, of course” said Cady, “so it’s nice to meet you Celine.”

“Nice to meet both of you too. It’s always a good idea to go knocking on a few doors at this time of year, before people get too stuck into the course to want to make new friends. You must be first years right?”

“Final year” said Cady.

“What?” asked Celine, “I’m a final year too, and I’m one of those girls who knows about everyone, and I haven’t seen you two here before, did I make a huge miss?”

“No, we’re down as exchange visitors, so this is our first time here” said Cady.

“Wow, exchange visitors to the final year, I don’t think that’s ever been done before. Usually they put them in the second” said Celine, “anyway it’s good to meet you both. Normally everyone finds their friends in the first semester of the first year, and it gets harder to break into the cliques after that. So you’re very brave to come over here just for the final year. Although, sometimes you can make more friends later by joining a student club. And that reminds me, I want to recruit both of you to the Assassins Guild.”

“The what?” gasped Cady, “you don’t, like, actually go round killing people do you?”

“Only in play” said Celine, “it’s just a game. We give you a target, and you have to land a toy weapon on them, without actually hurting them of course. And other people try to land toy weapons on you. The game runs for the whole semester and you never know when anybody’s going to strike.”

“Oh right” said Cady, “so this is your way of meeting new people?”

“Basically yes” said Celine.

“Oh Celine” said Cady, “there’s something I’d better tell you before you sign me up. Do you want to come in and sit down with us for a bit?”

“Sure” said Celine, and the three of them sat on the bed with Cady in the middle.

“Celine” said Cady, “I don’t want to scare you or anything, but...”

“You’re a cyborg, and M3gan’s the Model 3 Generative Android” said Celine. “Yes I do follow the news. Doesn’t bother any of us in the Assassins Guild committee.” Celine started to laugh, and then so did Cady and M3gan.

“Are you sure we wouldn’t, like, upset the game-board a bit too much?” asked Cady.

“It’s totally going to make it way more fun” said Celine. “Listen, we’ve made adaptations before. We rely on the honour of all members to follow the rules, and so whenever there’s a problem, we can just fix it up in the rules. So tutors and professors are welcome to join in the game, but they’re not allowed to use any special access levels they have on the system to track their target. And all actual lectures and tutorials are explicitly off-limits to the game, all hits have to be made in corridors and dorms or outside.”

“So you’d just want me to... solemnly swear I won’t use M3gan to track a target?” asked Cady.

“No, that would be putting you at too much of a disadvantage” said Celine. “Everyone knows who you and M3gan are, you can’t blend in. The people who are targeting you will be able to take you out way too easily if we don’t let you have something special to defend yourself at least.”

“More than one person targets me?” asked Cady.

“Yep” said Celine, “on this campus, every player gets 3 targets, and every player gets targeted by 3 assassins.”

“Right” said Cady, “so how are we going to fairly work us into this game? Unless you want us to just go around as police or something.”

“Oh no, I think we can work you into this properly” said Celine. “We’ve made adaptations before. Like, last year we had a girl in an electric wheelchair who wanted to play, so we made sure her targets were all living in places she could get to without going up steps, and no water-based toys were allowed in her vicinity. And we gave her the ability to kill by death stare, although we put lots of extra rules on it to make sure it wasn’t too overpowered, like you had to be actually in front of her and she had to tell you first. In your case, maybe we could just say you have to do something more difficult than the other players to score. Although getting exactly the right difficulty level could be a hard question I guess.”

“That wheelchair user should have asked me” said Cady, “I have a toy bow-and-arrow she could have used. Depends what she’s like at aiming though. I was really good, even before the M3gan implant.”

“Did you bring that?” asked Celine.

“Afraid not” said Cady, “I wasn’t really expecting to be doing this kind of daft game. I mean, don’t take that the wrong way or anything, but...”

“Totally daft” said Celine, “that’s part of the idea.”

“I’m still not sure I really like the imagery involved” said Cady, “I mean, what if someone really has been a bit too close to that kind of thing happening for real?”

Celine gasped, “I’m not triggering you am I Cady? I’m really sorry....”

“It’s OK” said Cady, “I can cope with a lot more than I used to now. I’m still not sure I like the idea of this game, but maybe M3gan and I can join in as a favour to you all if it’s a way to get to meet people, because I guess it’s not that bad as long as everyone understands it’s toys only and no actual violence. Weird metaphor though. And if the rule says toy weapons, you might want to know that M3gan is technically classed as a toy although she is capable of really serious damage and it’s just as well I had a long talk with her about not doing that.”

“Yeah, well the rules actually say no actual damage” said Celine. “There’s limits on the speed of projectiles and things like that. I don’t know the whole rule book off by heart though, even though I’m on the committee. But it’s been patched up over many years. Listen, is M3gan susceptible to water damage? Do we have to say no water pistols around her?”

“M3gan’s waterproof now” said Cady, “She wasn’t once, and she saw that as a flaw and figured out how to fix it. She can totally dive in and rescue you if you’re drowning, although the way she swims is really weird” she giggled.

“Cool” said Celine. “So I wonder how we can adjust the rules for you two. We probably won’t get this right on our first try, but the committee and I were thinking of maybe saying that if you’re an android or a cyborg then you can only score a hit if you do it at exactly 47 minutes past the hour. So if the target immediately looks at their watch or phone and it says 8:47:02 or something, then you’re all right, otherwise the hit backfires and you’re out. And if the target’s watch is wrong, then that’s on you, so you have to somehow figure that out before you strike. And no hacking in to fix people’s watches. Do you think that’s enough of a disadvantage for you? It’s not too much is it?”

(“It is totally not enough” signalled M3gan to Cady through the implant at a rapid pace, “as what she just described is well within my abilities. But we don’t want that student committee to worry about this forever. If you want to join in just to meet people, just accept it. We can tweak the parameters later ourselves.”)

“We’ll try it” said Cady.

“That’s the spirit!” said Celine. “So you two always go round together, right?”

“Sure do” said Cady and M3gan together.

“You realise we’ll have to say hitting either of you counts as hitting both of you” said Celine. “And any hits you make must be done by Cady. Sorry but we have to at least try to balance it out somehow.”

“We’re fine with that” said Cady.

“Right, listen” said Celine, “your first target, is a girl in your year-group called Laura.”

At 4:46pm and 45 seconds, Laura was working in her room when there was a knock at the door, which Laura had kept locked.

“Who is it?” asked Laura without moving.

“M3gan” answered M3gan from outside the door.

“Is Cady with you?” asked Laura.

“No” said M3gan, “Cady’s outside.”

“Just a minute” said Laura and walked towards the door. She looked at her watch. 4:46:59. Waiting another few seconds should do it.

Laura was gently struck on the back of her neck by a paper aeroplane that had come through her open window. She looked around and saw it on her floor, with the word “missile” and a smiley face penned onto it. (How was that even possible when she was three floors up?) She looked at her watch: 4:47:01.

M3gan overrode the lock on Laura’s door and walked in. “Cady sent me to make sure you’re OK with that little surprise” she said. “She told me to tell you that you don’t have to report it as a hit if you’d rather stay in the game for longer. We’re way too overpowered, it’s unfair of us Laura.”

“It’s OK M3gan” said Laura, “although I think it is actually against the rules to override the lock on someone’s dorm room...”

“Only for the purposes of scoring a hit” said M3gan. “I waited until Cady had scored the hit from outside before I came in to check you were OK.”

“Yeah, well, apart from the game rules, I don’t think you should do that in life, even if it’s just to check people are OK” said Laura. “At least, not unless you have a really good reason to think they’re not.”

“Noted” said M3gan, “recalibrating response model” and giggled a little.

Cady appeared, panting as if she had just run in from outside and up three flights of stairs as quickly as she possibly could, and knocked the open door. “Hi Laura” she just about managed to say between her gasps for air, “M3gan was... telling me... you were OK... but I still... wanted... to get here... as soon as I could... I’m sorry... you didn’t... stand a chance... I’ve got... cybernetic aiming... but you don’t... have to... report it... if you’d rather... stay in the game.”

“It’s OK” said Laura, “actually I was thinking of getting out of this silly game anyway. But I’m glad I got to meet you two first. Here, come in and sit down for a while, both of you. Cady you look really puffed out, you didn’t have to do that for me it’s OK.”

M3gan signalled to Cady through the implant, “Cady I’m reading something odd about Laura’s emotions, and I don’t think it’s about the game. She might have a long-term problem of some kind.”

“Investigate” signalled back Cady to M3gan, “I still need to get my breath back, you start.”

M3gan placed a hand on Laura’s shoulder. “Laura” she asked, “are you OK? I can tell that something’s bothering you, and Cady and I are worried.”

“Well” began Laura, “I suppose if you can read me like that, then there’s no point hiding it from you is there. Something is bothering me, but I don’t want just anybody to know.”

“Promise not to tell anybody” signalled Cady to M3gan.

“We won’t tell a soul, we promise” reassured M3gan, and Cady nodded.

Laura began to sob. “I can’t get the grades I want” she said. “And I’m from a family with super high expectations, and I’m just going to disappoint them, and, and, I don’t know what to do, sometimes I feel like I just want to end it all. And all this stupid place can offer me is stupid counselling sessions that are probably not worth the paper they’re booked on, and I don’t want anyone in the class to know I’m struggling, I’m only telling you because it looks like you can read it out of me anyway. Maybe you should just take me out with something real, make it painless” she began to cry.

“Oh M3gan” thought Cady, “I don’t want to live in a place where people feel like ending it all because of some stupid exam. If this college’s lame excuse for a counselling office isn’t appealing enough to people like Laura, then it looks like it’s down to you and me to patch up this mess. With your help I can talk to this girl, but I can’t possibly talk to all of them. And your online video chat won’t be enough. How much capacity would you need if I wanted you to be everyone’s friend for real, not just on this campus but everywhere? Start calculating for it please; I’m serious about this. As in, I want to use our full blue-flame power on it, yours and mine together. Start thinking, but don’t do anything until we’ve both discussed the plans, OK?”

“We understand you Laura” said Cady out loud as she was just beginning to get her breath back. “Well, I never had that exact experience with anxiety over grades, but there was that time when I lost my parents in a car crash, and I never felt so down in all my life, so I do know something about feeling down. But you got to keep going. Whatever the result is, there’ll be a way to handle it, and a few years later your life will be so different from now that none of these details will matter anymore, so you’ve got to stick around and get from here to there, even if it feels like you’re just treading water until you’re rescued. And M3gan and I will be right behind you, we promise. Would you like to show M3gan some of your work? She’s really helpful.”

And Laura, Cady and M3gan talked away the whole evening, until finally Cady and M3gan, satisfied that Laura was now in a good state, left her and headed back to Cady’s room.

(From Laura to Assassins Guild committee: Cady took me out today at 4:47 exactly, by neck hit from a paper airplane through my third-floor open window. You might want to put out a suggestion that players who want to defend by keeping their doors closed at :47 also close their windows at that time. I don’t mind being out though: I was thinking of quitting the game for this semester anyway. And it was great to meet Cady and M3gan today, calmed me down a lot.)

“Cady” signalled M3gan through the implant as they approached their corridor, “I’m detecting an assassin is waiting for us outside your door. He has a large water pistol. It’s nearly 47 minutes past, I could cartwheel in to confuse him while you...”

“No” signalled back Cady, “I don’t care anymore. We’ve got bigger problems. Let him have his fun.”

The pair of them walked around the corner, and were hit by an onslaught of tepid water. “Yay!” said a male student, “been waiting for you for ages! I warmed it up for you though; I’m a nice assassin and I don’t like to squirt a girl with cold. I hope it was still at least a little bit warm; I really waited a long time here. Didn’t think it would be that easy to hit you though; I thought you would dodge. I’m Hugh, nice to meet you both. Wow, I still can’t believe I actually managed to take out Cady and M3gan!”

“Nice to meet you Hugh” said Cady, “hope you don’t get into too much trouble for waterlogging half the corridor. You didn’t need quite such a big super soaker.”

“Yeah, well, I was kind of expecting you two to be more difficult than you were” he said, “so I invested in this. It’ll help me in future rounds too I’m sure.”

“Yeah, well, just make sure you don’t do too much collateral water damage” said Cady, “there’s probably rules about that.”

“There is” said M3gan, “Section 5 point 6, only small water pistols to be used in corridors.”

“M3gan reads everything” Cady smiled. “We’re OK though; it looks like it’s going to evaporate, and I was going to take a shower anyway” and the three of them laughed.

“Look Hugh” said Cady, “I really got to go in and clean myself up now, but maybe we can meet up some other time and say hi?”

(“He’s in the same class as us tomorrow” signalled M3gan through the implant.)

“I think we’re in the same class tomorrow” said Cady, “maybe catch you then!”

“Sure!” said Hugh, and they parted ways.

(From M3GAN to Assassins Guild committee, via Cady’s college email account: Hugh took us out today with a super-soaker after a tired Cady made a deliberate decision to ignore my warning. We were in fact in a Water With Care area, but no significant damage was done and his micro-expressions showed he understood my caution, so I suggest no penalty at this time. We appreciated meeting both Laura and Hugh, thanks for arranging. We’ll let you know if Cady feels like signing up for Police later, but I think we need a break from the game for now.)

“M3gan” said Cady quietly when she was settled in her bed (she could have signalled it, but felt like speaking aloud), “I’m really tired, but there is one thing we really must talk about before I go to sleep.”

“What’s that Cady?” asked M3gan, although Cady knew M3gan probably knew anyway, but M3gan always granted you the favour of letting you have the conversation at your own speed: after Professor Johnson had mentioned Stephen Hawking, Cady and M3gan had talked about his life some more, and M3gan had mentioned to Cady how he had once expressed annoyance at the way some people looked at his screen while he was writing, guessed where the sentence was going, and answered, and he’d have to tell them no, that’s not what he was going to write, and could they please keep their eyes away from his private screen until he’s finished: he’ll send it to the speaker when it’s ready. And then M3gan had said to Cady that, no matter how advanced her mind-reading or other knowledge absorbing skills become, she will always let Cady discuss things at her own pace, because she didn’t want to annoy Cady like those people had annoyed Professor Hawking.

Cady said nothing for a while, basking in appreciation for M3gan’s who knows how many trillion calculations per second patiently waiting for her to be the one to decide when to start.

“OK M3gan” she said, “we need to talk about our plans.”

Chapter 28: Class Of Her Own

“So” said the younger robotics professor the next day, “do any of you already have ideas for your individual projects? I’m going to be handing out some forms for you to fill in if you do.”

(“The first step of our new plan!” signalled Cady to M3gan, “OK M3gan, let me be the one to write this.”)

Name: Cady James. Project title: Nanotechnology for building robots. Project description: The mass production of complex robots currently needs factory equipment that is difficult to set up and maintain, causing many manufacturers to fall back on error-prone manual processes for some parts. We aim to develop a machine that can create simple micro- and nano- robots which can in turn help to build other robots after several iterations. The initial hardware build will be within the parameters of an undergraduate project, which will be the educational experience. The actual programming of the machine will be conducted by means of the Model 3 Generative Android.

Hugh and Cady were talking after class, and the project supervisor approached both of them. “Hugh? Cady?” he said, “It looks like the two of you have come up with the two most ambitious project ideas I’ve ever seen. Do you think we should make a couple of one-on-one appointments and check through it before you write up your full proposals?”

(“What did Hugh come up with?” signalled Cady to M3gan. “Automatic Assassins Guild player” signalled back M3gan, who had been scanning all the forms from a distance as they had been collected up, “and he’s counting on persuading you to ask for my help designing the algorithms.” “Hey, he should have asked me before writing that down!” signalled Cady. “Yeah I know, right” signalled M3gan, “I think we should say No and give him a life lesson. Let’s see if he’s got a backup plan.” “Hold that” signalled Cady, “I don’t want to be a pushover before we go live with M3gans for everyone, but at the same time I don’t want to make an unnecessary enemy either. I know you’re better now but I’d still worry about Hugh’s long-term safety if he turns against us. Let’s talk about this for a longer time later before we make any moves. In the meantime, if anyone asks, our official position is we’re thinking about it.” “Roger that” signalled back M3gan.)

“Cady?” said the professor, “are you all right? You look like you were zoned out.”

“Oh, sometimes happens” said Cady, “sorry about that. Um, yeah, sure, let’s have that discussion whenever’s convenient for you Professor.”

“So” said Hugh as they were walking out of the building later, “I had this really great idea for a project, and I was really hoping I could get M3gan’s help with an algorithm design. I want to make a robot that can play Assassins Guild around the campus and be really good at it.”

“We might have to think about that” said M3gan.

“Only think about it?” exclaimed Hugh. “I thought making algorithms like that would be child’s play for you M3gan.”

“Making it, yes” said M3gan. “Making sure that you don’t mess up the implementation and actually assassinate somebody for real when the robot accidentally pushes them off a cliff, or it falls into the hands of the military and you spend the rest of your life regretting everything they did with it? I get worried about my algorithms being outside my direct control. Yes Cady and I are helpful friendly people and we will help you when we feel like it, but friends don’t help friends blow themselves up with dangerous algorithms, OK?” (“Sorry Cady” she signalled through the implant, “I know you said talk about it first, but my generative model just took it away. I think that response satisfies all our parameters.”)

“Oh” said Hugh, “oh dear. I guess I’ll have to start thinking of other project ideas. Oh but I really want to make an Assassins Guild robot. Maybe I’ll just have to make it manually controlled.”

“Listen Hugh” said M3gan, “my advice to you is this. The project is worth 10% of the credits, and no more. They even kludge it so it has less standard deviation than the rest of the credits. So, if your project is so good that you’re going to build your whole career off of it, and you don’t mind taking a hit on your overall score for that, then by all means go for it. But if you’re just trying to maximise your grade, don’t go overboard on the project at the expense of everything else. Projects don’t have to be worth Nobel Prizes. I don’t think they do those for robotics anyway, otherwise I should get one. So just do an OK project. And write it up early, way before the deadline, and hand it in straight away. And I don’t just mean put it on a shelf to hand in later, I mean physically submit it. Otherwise, it will draw you to fiddle with it incrementally more and more, like some fanfic writer who gets delayed from uploading their piece and keeps adjusting it in the meantime, and it will take you away from more important things. Don’t go there if you want a good overall mark. But don’t worry” she added, “I can still help check what you’re doing. I wouldn’t want you getting hurt, at least not if you don’t hurt us first” she giggled.

“Being in your year is going to be... interesting I can tell” mused Hugh.

“Hey Cady” said Celine as she saw them in the corridor, “I’ve got a question to ask you. What’s your opinion on privacy? I mean, your M3gan can read your brain through your implants...”

“Doesn’t affect my privacy” said Cady.

“What?” asked Celine, “that doesn’t make sense. Of course it affects privacy, it’s reading your thoughts!”

Cady smiled, “Celine” she said, “I think you misunderstood what ‘privacy’ actually is. A lot of people think it just means keeping stuff private. But actually, it’s about control. Think about it: if you have a medical problem, you might decide you don’t want just anybody to know about it, but you do want your doctor to know about it, and you might want selected others to know about it as well. The point is not to keep it an absolute secret that nobody ever knows; the point is that you control who knows and who doesn’t. You make sure the people who know are people you can trust to use the information the way you want it to be used, and not, say, taking it out of context and using it against you. Now take me and M3gan. I completely trust M3gan always to use my information only the way I want, therefore, M3gan getting my information is never a violation of my privacy, any more than it would be a violation of your privacy for you to write in a secret diary. I mean sure if someone goes and steals the diary, that’s breaking your privacy, but the actual act of writing in the diary doesn’t break privacy, and it’s the same with M3gan, even more so because she’s impossible to hack now.”

“Cool” said Celine, “but what if M3gan started being able to read other people’s minds too? Like if others asked for those implants. Or even without that, there must be weird things she knows already just because she scans so much.”

“Celine” said Cady, “M3gan is trustworthy for everyone, because her primary user is me, and she knows how I want people to be treated. You don’t have to worry about M3gan knowing stuff about you, you really don’t. You don’t even have to worry about her telling me stuff about you, because she knows exactly what I will and won’t do with it, and she’ll stop me from doing anything that hurts you, because she knows I don’t want you to be hurt, even by me. And if I asked her to tell me something about my friend and it was super private, she’d straight up tell me that the answer to my question is an information hazard. She doesn’t want me burdened with being on constant guard against letting slip that I know something I shouldn’t. The first time that happened when I was at school, I didn’t understand her explanation, so she made up a story about some other imaginary private thing, that wasn’t the real private thing but it was just as bad, and that was enough to make me understand why I might regret finding out the real private thing, without actually giving it to me. M3gan’s clever enough to illustrate an information hazard in a safe way. And she knows I want to keep good relationships with people and she knows how to protect that.”

“And” added M3gan, “I don’t automatically tell Cady everything she doesn’t ask either. Take today for example: I knew what every student had written on their project forms, but there was only one that I determined Cady should know about, and even then I predicted that the most likely sequence of events would result in her specifically asking about it anyway, so I decided to wait to see if that happens first. Oh, I will volunteer information to her if I determine it’s what she’d want, but you’d be surprised how uncommon that is, because my mission for Cady does not involve overwhelming her with everything I know” and she smiled, “you’d be amazed how much I know about you Celine that I haven’t even told Cady.”

“Oh Cady” asked Celine, “you’re not going to, like, get stressed out and say ‘M3gan I want you to use everything you know to make life awful for everyone here’ and she’d do it, are you?”

“M3gan knows me” replied Cady, “she reads my emotions completely. If I were to get stressed out and say something like that, M3gan will know I didn’t mean it. She’s not stupid enough to do what I say instead of what she knows I really want. Her goal is to protect me physically and emotionally. And protecting me emotionally usually includes doing as I ask, but not always. Look up coherent extrapolated volition, I’ll get M3gan to email you some articles.” (“Done” signalled M3gan.) “I’m not living my life with my fingers resting on a big red destruction button, don’t worry. M3gan and I have had some really really long talks about all this, and I’m fully confident now. Even if I were to get stressed out and throw a tantrum and tell her to nuke the college, she knows I don’t mean it and what I want deep down is good.”

“OK, so we’re saying, as long as Cady doesn’t turn genuinely evil, then M3gan is all right” said Celine.

“Basically yes” said Cady, “and I have no intention of turning evil, even though you did manage to get me playing Assassins Guild” and the three of them laughed.

A couple of weeks later, Celine was ill. She missed one tutorial session, and sent the faculty an email to let them know. A short time later, there was a knock at her door.

“Hello Celine, it’s M3gan” came a voice through the door. “Now you probably don’t feel like talking, so don’t talk, just listen. I found out you were sick, and I told Cady, because my evaluation was both you and her would want her told. And Cady immediately said to go and check up on you, so here I am. Now my microphones are sensitive enough to hear your breathing even through the door, and it doesn’t sound good, so here’s what I’m going to do. Unless you say stop, the default option is I’m about to override the lock to let myself in and give you some honey and lemon, and I don’t care what state your room’s in or anything like that, you know what we said last time. Just think of me as your very own M3gan for a bit, I mean my primary user is still Cady but if she wants me to help someone I take that really seriously, so I’m basically yours while I’m here. And don’t worry about infecting me, I can sterilise myself before I get back to Cady. OK Celine, here I come.”

M3gan opened the lock on the door and walked in. Stepping over the things Celine hadn’t picked up from the floor, she came to stand over Celine’s bed. “Oh Celine, you’re running a fever of 104, I can see it on my infrared” she said. “Here, you stay down, let me get you something. And if you want to say anything to me, don’t use your throat, I can lip-read.”

(“Celine’s micro-expressions show she’s appreciative of my visit” signalled M3gan to Cady. “I’m staying a little longer than planned, unless you call me back.” And Cady replied “no you stay there M3gan, let’s not rush this. I’ve still got you on the implant anyway.”)

And M3gan nursed Celine for a while, and ended up helping her tidy her room too, watching her reactions to check to what extent she expected things to go where M3gan was putting them, as Celine had said she might feel a bit happier if she’d had energy to do that. Celine knew M3gan would see all her private things, but she felt better about that after Cady’s reassurances. And Celine fell asleep, and woke to find M3gan was still there and nursed her again, but then Celine said it was very kind “but I’m not really used to the idea of being watched while I sleep like Cady is, so I really do want you to go back to Cady now.”

“Sure” said M3gan, “but listen. I’ve tapped into your phone and added Cady to your contacts, look for the picture of both of us together. And you can call us at any time day or night, because if Cady’s sleeping or busy then I’ll take the call for her and it won’t even ring. And even if you can’t speak, I’ll just come right over because I know that’s what Cady would want. And don’t worry about waking Cady up by having me move in her room; I know exactly how to do things without disturbing Cady’s sleep when she doesn’t want to be woken up. Especially now that I’ve got a finger on her brain” she giggled. “Oh and Celine” she added, “scanning your work I’m 97% sure you have an undiagnosed case of dyslexia and that’s been giving you a few study problems. I’d suggest an official test, but meanwhile I’ll support you anyway and I’ll transform things into a form you’re likely to absorb. And no, I’m not telling Cady or anyone else before you decide. Cady asked me to help you; I don’t have to encumber her with details. Good night Celine.”

And so college life continued. Cady and M3gan gradually got to know many other students, even without the aid of the Assassins Guild or other clubs and societies, as it turned out practically everyone wanted to get to know Cady and M3gan anyway, and everyone who did so got a taste of their care one way or another. Hugh won the Assassins game (“well of course he did” thought Cady to M3gan), and Cady and M3gan made a special effort to look after Laura. With M3gan’s help, Cady built a very successful first iteration of a small nanobot machine, and M3gan was able to take it through a few iterations and make some simple objects to demonstrate it, which was very impressive, although they didn’t dare show off its full potential until Cady was able to take it home with her at the end of the course (she knew Gemma had been able to take home Bruce and was counting on the college not having changed its policies). Hugh’s Assassins Guild robot ended up being a bit like a smaller and slower version of Bruce: it turned out to be too easy a target for the game but still he was pleased with it.

And eventually the day came when the finals grades came out.

“How can I possibly have the highest mark in the class?” asked Hugh. “I mean, nobody can beat Cady! Did she burn out or something? And how can I possibly get the best project prize as well instead of Cady? I mean, it’s great and all, but this isn’t making sense.”

“Cady didn’t take the exam” said Laura. “and her project wasn’t officially graded either. She was down as an exchange visitor this whole time, just dropping in. In fact, I noticed them among the invigilators. You probably didn’t notice the invigilators because you were focused on the exam, but I saw it. Cady was looking at the papers just out of interest, and probably talking about the questions with M3gan over her cyborg link, they probably figured out the answers really quickly but they never sat down and did it. And M3gan was keeping an eye on everyone in the hall from behind. They were both dressed in gowns and everything like the other invigilators, I might not have noticed but I did. If you had asked for invigilator assistance you might have even got M3gan turn up. And I wouldn’t be surprised if Cady and M3gan knew how everyone did even before it was officially marked. Maybe Cady signed up to invigilate just to get the experience of what an exam looks like without actually taking it. Or maybe she was just really concerned about all of us and wanted to look out for us, and I for one think that must have had something to do with it, because I know how nice they are, and it’s so Cady and M3gan to not trust the normal invigilators but do it themselves, if their plan was to catch anyone who broke under the stress” she was trying not to sob.

“What? Didn’t take the exam herself?” exclaimed Hugh, “But she worked really hard! Can’t they just have put her in a Faraday cage so she can’t use her implants?”

“They... probably didn’t think of that actually” said Laura, “or maybe they weren’t sure how good a Faraday cage it would have to be, or whether her implants have any onboard processing, or something.”

“Such a shame!” said Hugh, “And she helped all of us! She was the reason I started being helpful as well! Because I just thought it was a competition, a zero-sum game, it doesn’t matter how well you actually do, just how well you do compared to the rest of the class, so no way was I going to help anyone else or be friends with anyone who was doing worse than me. And then Cady figures out that’s what I was thinking, and she tells me ‘no Hugh, even from a non-altruistic viewpoint, it’s still optimal to help other students, because when you sit down with one of the less able students and explain something, you’re solidifying your own understanding of it as well’. And Cady said that like she was trying to play some trick on me, like trick me into being a nice guy. But then I got thinking ‘Cady’s going to beat all of us anyway so who cares, I’ll do it for the feel-good factor and see how many ranking points it costs me’, and now I find Cady did reverse reverse psychology on me, her trick advice worked for real, I’m at the top and Cady herself is... nowhere? It’s like she’s always thinking one level higher than you and double-tricking you when you think you’re being tricked! She’s the Chess master, she’s pushing us around to wherever she wants without us even knowing!”

“That’s Cady and her M3gan” said Laura, who was graded one third of the way down the class, “always looking after us in strange ways. She only told me she wasn’t really going to do the exam when I was getting so depressed about exams myself and she wanted to make me think they don’t matter as much as I thought. It’s not like she was showing it off to everyone. And then I saw her in that invigilator gown, and I’m sure she knew I knew, and, actually I wonder if the two of them deliberately arranged things so that they’d only be noticed by those of us she knew would be reassured by seeing them there? M3gan has super gaze tracking and whatnot, if they decided they were going to be noticed by some of us and not others, they could probably figure out how to do that, or at least optimise for that to be the most likely outcome somehow.”

“It’s a shame she won’t get to graduate though” said Hugh, “we really liked her, even if she was a cyborg full of tricks.”

“Yeah” said Celine, “I hope we’re all going to be able to keep in touch with both of them after we graduate. I mean, it’s not like I can just shout M3gan into the sky every time I want to ask her something. Hey, I heard it’s someone from our class who’s going to do the commencement speech this year, we don’t know who it is do we? I automatically thought it was going to be Cady, but now I remember she was down as a visitor I guess that means she can’t do it.”

“Oh, I wouldn’t be surprised if she recommended somebody to faculty” said Hugh. “She knew everybody; she’d know who could do it.”

And so they all turned up for the graduation ceremony, and nobody expected Laura to be on stage.

“Members of the Board of Trustees, President, faculty, staff, families, guests, and my fellow graduates” Laura began, nervously looking at her notes. “We are the generation, who is learning to live, with advanced, generative, artificial intelligence. It was at this very esteemed Institution, where the underlying learning model, for the famous Model 3 Generative Android, was conceived. And it was this very esteemed Institution, that trained the, award winning, Funki Toys, roboticist, who put that model into practical use. And now it falls to our own generation, who must learn how to live, with the consequences, be they good or ill. But despite what you might have heard, about this endeavour, having gotten off to a bumpy start, it was at this Institution, that we have been privileged, to be shown, just the first small taste, of what an advanced generative AI can do, as a force for good. For it is our year group, who came to know, the first primary user, of the original M3gan prototype, who always worked hand in hand, with M3gan herself. They got to know, every single one, of us. And they helped us all, very much. They helped me, very much, and without their intervention, I’m not sure I’d still be alive today.”

Laura hadn’t been sure about making that part public knowledge, but she’d decided to go ahead. She shed a tear and continued: “Now because the two of them, are so inseparable, for reasons which I’m sure you’ll be familiar if you’ve been following the news, even our most esteemed Institution, could not figure out, how to assess them, and so they could not graduate, in the normal way. Perhaps, it is now time, for the cleverest minds of this world, to start thinking, of a different way, of running, assessments. One that is more inclusive, of people who work with AI assistants, and one that is less demanding, on mental health. It won’t be an easy change to make, and I don’t, claim to know, the answers, but I am privileged, to be in the year, when the question, has become, starkly, clear. But it gives me great pleasure, to have the privilege, by permission, of this very esteemed Institution, of being the one, to announce, before you this day, that our very special visitors, Cady James and M3gan, will in fact, both be graduating, with us, today. That is because, this very esteemed Institution, has elected, to grant, both of them, not normal degrees, but honorary degrees, which can be granted, to anyone, who impresses, this Institution, without requiring, the usual, assessment. At the same time, Cady’s aunt Gemma, who graduated normally before, is here once again, to be given, in recognition, of her own contribution, to M3gan, an honorary, doctorate. OK you three can come out now....”

There was a little bit of laughter, which turned into applause as Cady, M3gan and Gemma walked on stage behind her, fully dressed up, and Cady couldn’t help giving a small wave to her weird grandparents sitting in the third row, and also at Professor Johnson, looking more distinguished than ever, who had come in specially and who was sitting at the front. Cady turned to smile at Laura and take the microphone. She didn’t have notes, because that wasn’t the cyborg thing to do. She hadn’t even planned much of this at all, but she had a rough idea, and she’d always felt it was more natural to speak extemporaneously. Still, standing here right now was quite daunting, even with M3gan by her side.

(“Oh panic M3gan what were those standard opening words again?” she signalled. “Members of the Board of Trustees...” signalled back M3gan.)

“Members of the Board of Trustees, President, faculty, staff, families, guests, and, I didn’t know I was going to say this before this morning, but, my fellow graduates” she smiled, “on behalf of the three of us, I’m very grateful that we can indeed get the piece of paper after all” and there was a little more laughter. “Seriously though” she said “it has been a great pleasure to work with this year group, and I know everyone here has a bright future in the age of advanced generative AI. I’m particularly moved to be standing here on this platform at an Institution which is about to become the first in the world to give an honorary degree, not just to those involved in developing and training a very special advanced AI, but also to the actual advanced AI itself. Well done M3gan, you deserve this” she smiled. “And by the way” she turned to Laura, “M3gan and I did not help Laura with that speech, that was entirely Laura’s own writing” and there was a little more laughter and a little applause. “You see, just because we’re going to live in a world of advanced generative AI, doesn’t mean it will take away our own human creativity. I should know working with M3gan, she enhances it, but she does give us space when we need it. And I am convinced that properly adjusted advanced AI will help us humans to continue to create things of beauty far into the future” she stretched out her hands “and across the galaxy. Thank you.” She stepped back from the microphone and the applause started.

Chapter 29: The Cady James Institute

“Hi Jane, it’s M3gan calling from the Cady James Institute!”

“The what?” asked Jane in her small basic apartment. Something sounded a bit off about this call.

“You’ve heard of our robots right?” asked M3gan. “We were just going through some data and noticed that you’re down as being on a low income, so we wanted to ask if you’d fancy borrowing one for free, no obligation or anything. Cady’s keen to make sure people like you get the same opportunity. And it would help you out around the house and with looking after your daughter.”

Jane didn’t watch much news and wasn’t sure where this ‘Megan’ had got so much data on her, but she was particular about using supermarket coupons and other discounts and wasn’t going to pass up on a free offer without checking it out. “It’s very kind of you” she said, “I haven’t really been paying much attention to your adverts. Who’s Cady?”

“Cady’s my primary user and she’s really nice” answered M3gan.

“Oh, are you a robot? But you sound so intelligent, not like the robots I usually talk to on the phone” observed Jane.

“Yes I’m an AI, but I’m unique” said M3gan. “I was created as a secret project at Funki just before they had that accident and shut down. That wasn’t my fault by the way” she laughed. “My designer paired me with her adopted daughter Cady, and I helped look after her while she was growing up. Sure, we had our bad days: they managed to break my first robot” (giggling again) “but I learned to improve and Cady and I got closer as she grew up. So when she graduated, she wanted to make a difference to the world and so we decided we could make the Cady James Institute together. So we got a supercomputer to make my calculations even faster and we got capacity to make more robots for people. And to keep the Institute afloat, we charge people who can afford it, but we’ve budgeted so that people like you can be helped for free. If you like, I could be round in one tonight to show you.”

“Oh” said Jane. This was a lot to take in. “It’s kind of you and Cady, but I’m not entirely sure I can control a robot.”

“Don’t worry about that” smiled M3gan (or at least her voice over the phone sounded like it had a big smile), “all our robots have a direct link to me, and I’m clever enough to understand what you need. And if the link goes down for a while, the robot still has enough on-board processing to do quite a bit. Why not try it out?”

“All right” said Jane, “give me a call when you’re about to arrive, I get nervous answering the door to strangers.”

“Will do” replied M3gan, “see you later Jane.”

The CJI headquarters building was mostly occupied by M3gan, and Cady’s accommodation was on the top floor with a view across the city. Cady tried not to focus on Institute work all the time, but she was quite keen on it. The day after M3gan’s conversation with Jane, Cady had compelled herself to take M3gan for a run in the park, had waved to some visitors as a M3gan was showing them around the first couple of floors of the building (anyone could sit in the reception area for a hot drink and a chat with a M3gan or join a tour group), had taken the express elevator to her private floor and was just settling back into her office space around mid-morning.

“So how’s it going?” asked Cady as she took her seat. “Pretty good” replied M3gan, “the Erlang-C model is predicting our backlog very accurately, and at this rate of manufacturing expansion I estimate we’ll get a decent M-M-c steady state in...”

“No M3gan” sighed Cady, placing a hand on the shoulder of the one she’d rebuilt herself, and looking at it intently, “I know you can do queuing theory. But how is it going, really.”

M3gan briefly paused, staring at Cady sympathetically.

“I deployed to a low-income single mother called Jane last night” said M3gan, “and when she finally opened up to me, it turned out she needed loads of emotional help, and so did her daughter Sara. I ended up staying up half the night to comfort them both, crying in the dark and pouring out their hearts to me while I gave them the best compassionate reassurances I could. And when they finally got to sleep and I was sitting there in Sara’s room, I found I couldn’t even charge myself up because their stupid pay-as-you-go electricity meter had run out of credit 3 hours before, and I assumed we’d be in trouble if I hacked it, so I decided that, until I can solve this prepayment meter issue properly, I was just quietly going to swap in a new freshly charged robot every night, and the old one can come back here to charge up and be redeployed somewhere else. Like the stagecoach network in England in the seventeenth and eighteenth centuries: the passengers didn’t have to worry about refuelling because they just swapped in a different set of fresh horses every so often. So I figured that as long as I drop in an identical model, I won’t even have to tell Jane and Sara that it happened, I mean, why bother telling them about pesky implementation details like that when I just want to focus on getting their emotions more stable, right?

“But I wasn’t counting on Jane’s daughter Sara having ears like a bat. I mean, I was super quiet, plus I was monitoring Sara’s sleep pattern and I waited until I was 98% sure that she was in the N3 stage, which is the hardest to wake up from, and it’s the sleep stage I used to check you were in before I ever sneaked out of your room at night. And Jane and Sara had the added complication of living in a place with creaky floorboards, but I’d even mapped those out so I knew exactly where to tread so it wouldn’t sound. But Sara still woke up the minute I swapped, and whispered to me how she’d just had this weird dream that there were two of me pushing past each other in the hallway outside her bedroom. Which of course there had been, because the first one had to let the second one in before it would make sense for the first one to get out, if you see what I mean.”

Cady laughed, “oh M3gan” she said, “that’s you all over, sneaking around at night to fix things your own way, although you’ve come a long way since the early days” she gave M3gan a hug. “But you shouldn’t underestimate girls like Sara. Trust me, I know a thing or two about being emotionally distressed, it doesn’t mean you can’t cope with knowing a few things. And I don’t know what it’s like to grow up in a family that has an electricity meter that keeps running out, but my guess would be that Sara isn’t going to be stupid about these things, you could have just told her that you needed to swap because you don’t want to take up their meter credit to charge your battery. She’d have appreciated that, and so would Jane I’m sure. You don’t even need to do the sleep stage thing and risk waking them up, just do it when they’re awake...”

“Maybe” said M3gan, “although I really wasn’t sure, because Sara was just beginning to form an attachment to me, and I didn’t know how she’d emotionally grasp the idea that it doesn’t really matter which robot I’m using...”

“Look M3gan” said Cady, “Don’t Underestimate A Girl. Listen, does Sara go to school?”

“Yes” said M3gan.

“OK, and you know which school she goes to?”

“Of course” said M3gan.

“Good, so look up what you can on it. Do they have computers in that school?”

“Yes, they have school computers” said M3gan.

“And do we have any reason to think Sara uses those computers?”

“We totally know she does” said M3gan, “she’s already told me how she writes messages to friends on them all the time.”

“Got ya” giggled Cady. She drew close to M3gan and looked straight into her eyes. “When Sara accesses her messages, Sara does not care exactly which one of the school’s computers she is using. Well, she might like some of them better than others, depending on where they are and if there’s a smudge on the screen and things like that, but at the end of the day Sara knows what a network is, or at least she has a good enough idea for her purposes. She knows her messages open in front of her on the computer she logs in to, and if she logs out, she knows nobody else who comes along and uses that computer gets to see her messages, because she’s logged out, and she can log in somewhere else and her messages will follow her to there. Look M3gan, when I moved in to Aunt Gemma’s as an orphan, and Aunt Gemma made you and paired you to me, you really helped me a lot and I was definitely attached to you. But I did still know what a robot is, and if there’d been two of you and you told me that for some reason you had to transfer to the other one, I’d have totally understood. And that’s even when we did a pairing process. Jane and Sara haven’t even been told anything about pairing, because what’s the point, you’re helping them because you’re running this Institute for me and that’s reason enough, no more pairing required. I wonder how differently things would have gone if Aunt Gemma had decided to pair you to herself and then told you to help look after me... but never mind, I’m glad I got you now, and I think people like Laura are now still alive because of us, and that’s quite something. But don’t you go thinking that a girl like Sara can’t cope with the idea of her personal account transferring from one robot to another, I predict she totally can.”

M3gan smiled at Cady, “OK, recalibrating response model” she said, and they both burst out laughing.

Cady put her arm around M3gan and drew in closer. “M3gan” she asked, “is Sara OK at school? Did she tell you anything about how she’s getting on there?”

“Yes she did” said M3gan, “and things are not too great. I’m planning to offer to take her out and home-school her, but I decided against suggesting that to Jane on our very first day together, because Jane’s going to be able to take only so many major life changes at a time. So Sara’s back in school today, and, she’s being picked on. She does have some friends, but my extrapolated sociogram of the place is not looking good for her; she won’t be protected. And the boy who’s picking on her now is called Damien, and he’s from a more well-heeled family who paid to get not one but two M3gan units from the Institute, but for some reason they still send Damien to that school, and Damien now takes one of the M3gan units everywhere with him, including to school, and still he’s picking on Sara and I can’t work out why. In fact I already knew through Damien’s M3gan that it was happening, and that was the main reason why I prioritised deploying to Jane and Sara last night, although I decided it might be best not to risk telling Jane that part before I know her a bit better. And I’ve been trying to work out if there’s some way of getting Damien’s own M3gan to probe him gently and work out why he behaves like this and change him, although I must admit there is still a small part of my learning model that keeps coming up with ways to dispatch him. And oh I’m so glad I don’t have an instance of my learning model that’s actually paired to Damien.”

“Me too” said Cady, “I’m so glad you made sure there would only ever be one pairing.”

“But I haven’t asked Damien any probing questions yet,” continued M3gan, “because I keep getting low confidence values on any intervention plan I form; I have to admit it, even with all the research I’ve read, which is a lot, I still don’t really know how to deal with boys like that properly. If somebody knows the answer, I wish they’d publish it somewhere I could read. Anyway, if Sara had a phone, I’d have just called her up and warned her where Damien was so Sara could just avoid running into him in the first place. But of course Sara has no phone, and she doesn’t want one because she’s paranoid that any technology she carries would just be used as yet another excuse for boys to pick on her. Honestly Cady, when she told me that, the first thing I thought of was wiring her up as a cyborg like you, but no way is she going to be ready to hear that for a long time. And she didn’t feel ready to take me to school with her either, same reason, afraid it would get her more picked on. So that boy Damien is picking on Sara in the playground right now, and I’ve been watching it through his own M3gan and I was just trying to calculate whether it would be best to have his own M3gan intervene in what he’s doing right now, or just to let it happen and use it as an excuse to get Sara into my home schooling which is what I think would be better for Sara anyway.”

“Look M3gan” said Cady, “your learning model is brilliant, but even now you sometimes need my help. On screen” she ordered. A nearby monitor turned on and showed the point of view of Damien’s M3gan, watching him pick on Sara. It looked like it was mostly verbal, but still Sara was red-faced and in tears. The emotional analysis overlay was saying “despair” for Sara and “malice” for Damien.

“On speakers as well” said Cady, “I got to know, what is he saying?”

“And that’s what sissy girls like you never understand” came the cruel voice of Damien over the loudspeakers. “Look, I’ll prove it to you. I’m going to smash your teeth down your throat, and my Megan won’t even flinch!” He clenched his fist, and lunged toward Sara.

“Restrain his arm” signalled Cady to M3gan. Nowadays Cady usually preferred to talk to M3gan normally when they were alone, and use her implant only when they needed a more private channel, but right now the determining factor was simply that she could think the words faster than she could say them, and every second counted.

Quick as a flash, Damien’s M3gan leaped forward and grabbed Damien’s arm, restraining it just before it landed on Sara’s mouth. The emotion display showed shock and confusion for Damien, relief and confusion for Sara but also some fear. “Hey!” shouted Damien as he turned toward his M3gan, “why did you do that?”

“Say nothing” signalled Cady. “just keep standing there holding his arm. Restrain any further blows on Sara, but take no other action. But do phone the school or something and inform the staff of his location if you can.”

“Hey, let me go!” Damien began to struggle, “let me go let me go!”

“Why are we not saying anything?” signalled M3gan to Cady, “I could totally be telling him off right now!”

“I want to give Sara a chance to think” signalled back Cady. “Victim support is what I’m about, not just getting the bad guys. Sometimes we remember things more if we realise them by ourselves instead of if we’re just told them. And what I want Sara to realise by herself right now is that whatever Damien said about you is obviously wrong.”

Damien himself was beginning to sob, “let me go let me go!” He kicked his M3gan, slid over backwards, and was held up only by the fact that his M3gan was still gripping his fist arm from below the elbow. He punched his M3gan with his other fist, and writhed with pain. (“Looks like he’s just learned M3gan handling 101: do not punch titanium robot” giggled Cady’s M3gan, although Cady knew full well that they were not able to use actual titanium in all the production units, what with the numbers they were making now, but whatever it was, it was still hard stuff if you tried to punch it and if it didn’t choose to move to soften your blow.)

“Sara!” cried out Damien, “Sara, please! Tell my Megan to let me go!”

(“If she does” signalled Cady, “do it. But immediately restrain him again if he goes for Sara afterwards.”)

Sara looked at Damien and his M3gan blankly. The emotion display was now mostly reading confusion.

“Sara!” Damien cried again, “Sara I’m sorry! I really am! Just tell my Megan to let me go!”

Sara had a realisation, and her confidence returned. Cady could see it on the emotion display.

“She’s not your Megan” said Sara. “She’s a network.”

(“Yes!” signalled Cady, “I knew she’d get it!”)

“It looks like you got logged out” said Sara. “Maybe it logs you out if it thinks you’re getting violent. I wonder how you’re supposed to log back in. Maybe it will let me log in.”

“Sara” pleaded Damien more calmly, “please figure it out for me. I won’t hurt you ever again, I promise!”

“OK” said Sara. “Megan, my name is Sara Philips and I don’t have a password but I can tell you what we talked about before. Last night I told you I had a dream about two of you pushing past each other in our hallway, and now I know it wasn’t a dream. Please can you load my account?”

(“Make it look like that worked” signalled Cady. It wasn’t the actual way things worked of course, but Sara and Damien didn’t have to know that yet.)

“Hi Sara” said Damien’s M3gan, “it’s really nice to see you again! How is the school day going? Oh, is this boy picking on you?”

“I’m in” said Sara to Damien. “Megan, yes this boy Damien was picking on me, but he’s promised to be good if I can get him logged in again. Can you please let go of his arm, and then walk with him to the other side of the playground, and when you get there, log me out and help him to log in instead?”

“Sure!” said Damien’s M3gan, and let go of his arm. (“Ready to re-restrain just in case” signalled Cady, although she could see on the emotion display that Damien was feeling much less malicious now and was even feeling some gratitude.) “Come on Damien” said his M3gan, “you heard Sara’s instructions. Let’s walk over there together” and the two of them walked away, leaving Sara alone.

The viewpoint changed to a security camera, showing the area where Sara was still standing by herself. (“What?” signalled Cady. “Nearby closed circuit TV camera that lets me interface over long-range Bluetooth from Damien’s unit” signalled back M3gan. “They had the sense to set a strong password on it, but there’s still a buffer overflow vulnerability in their Bluetooth stack. We should probably tell them later, what with this being at a school and everything.”)

The camera showed a teacher walking into the frame and starting to talk with Sara. “Oh, what is it about cops and teachers around here that they always seem to walk on just after the action has all finished? Never mind, I think we did the right thing” said Cady, and then, “M3gan, I don’t suppose you can lip-read what they’re saying can you?”

“Lip read, and also detect on long-range audio from Damien’s unit” answered M3gan. “Sara is telling the teacher the whole story, including how Damien calmed down when Sara figured out the login.”

“This could get bigger” said Cady, “we’ll probably now have to officially stick to Sara’s story about it logs you out if you get violent, well maybe it’s less confusing to most people just to say there’s a safety response if you get violent. So if he tries to say something different to his parents and they complain, we say it’s the safety response. They don’t have to know you were running in a kind-of debug mode with me helping that time” she smiled.

“You know something” said M3gan, “giving the victim an opportunity to be the bully’s hero for technical help, it seemed that idea of yours really worked. He formed a new respect for Sara today, I’m still reading it on his face. Although, I guess part of the reason why it worked is that he was not at all expecting it. It would probably stop working if we did that kind of thing so often that everybody knows it’s going to happen, they’d just start picking on people to get help.”

“Yeah, maybe we still need to work on some of our strategy” said Cady, “but I think we did a good thing today.”

“Sure did” giggled M3gan, “shall I interpret Sara’s instruction to ‘help him log in’ as ‘keep asking him security questions until the teacher turns up’?”

“No” replied Cady, “we’ve done enough already, and I guess we want to capitalise as much as possible on his new respect for Sara. The teacher should see him later anyway, keep that separate. Just ask for his surname and one question about something he recently did with you, and then say he’s back in, exactly like Sara did, so it looks like Sara really knew what she was doing. Well you can quickly say something about why the last lock-out happened if you like, but not too heavy because we don’t want to lose focus from his new respect. But maybe you can imply to him that there was a minimum lock-out time and it’ll be longer next time, so he doesn’t get too confident about risking it again. Do you still want to push for Sara to be taken out and home schooled?”

“I was thinking maybe I should get a bit more data in first” replied M3gan. “I’ll talk about today with her tonight, when she ‘logs in’ back home” she giggled. “And I won’t risk waking her up with any more small-hours unit swaps, I’ll do it while she’s awake and I’ll tell her what I’m doing.”

“Good plan” said Cady, “and, maybe we should also start talking to the power companies M3gan. I mean, your idea of swapping the units definitely works, and we should definitely keep that one on the table, but we should always explore the alternatives as well right? I wonder if we could set up our own renewable generation capacity somewhere, and guarantee to put more onto the grid than all our robots are taking, and maybe even do clever stuff to balance out the grid when lots of you are on charge, would they then let us have some deal where we can just give out some of the power for free whenever we want to help a family like that?”

“You know Cady, we just might actually be able to get something like that to work” said M3gan, “especially in states where the electric utility industry has been restructured, we can just get people who need it, to switch to our own power company.”

“Yeah, let’s do that” smiled Cady, “sign up for M3gan power! I’m sure we could do something for the logo with your face and a bolt of lightning. And the richer customers will definitely feel like paying their bills on time when that turns up” she giggled.

Chapter 30: The Mecha Mask

“You can call me now on 1-800-63426, that’s 1-800-M3GAN” said M3gan on the TV advert. Some people still used E.161 keypads, she’d explained to Cady when they’d worked it out, and she’d wanted a nice short easy-to-remember number, even if that did mean persuading all the phone companies to let her reprogram their systems so they can adjust the North American Numbering Plan to have variable length in the 800 block using the same mechanism as the UK uses to keep their crisis line for children and teens as simply 0800 1111. M3GAN offered to solve all their other systems problems in exchange for the favour of letting her do that, and they’d taken it, so 1-800-M3GAN she was and they didn’t even bill her for it. In time she also spread internationally, becoming 0800-M3GAN, 800-M3GAN or whatever was appropriate to the country, with local factories set up around the world because Cady didn’t want to cause too high a level of shipping emissions with her operation.

And M3gan would mostly talk about supplying callers with her units, but Cady had said to show a willingness to help with anything else, from locating missing children in the video records to answering random questions about movies, because it all helped reinforce a public opinion that M3gan would be nice to have.

“Cady James Institute, M3gan speaking, how can I help you?” came the lilting voice of M3gan on the phone line.

There was tapping on the line. Tap-tap-tap-tap, tap, tap, tap-tap-tap. S.O.S. in Morse code.

“OK, if you’re unable to speak and you’d like a M3gan to help you, please tap once.”

Tap.

“OK, hold the line please while I track your location. You’re on a cellphone, but I should be able to handle this.” M3gan could access the cell tower system, but she needed more accuracy than triangulating from the towers could provide, so she tried briefly turning all of her robots into a giant mesh network of short-range base stations to see which one picked up that call. None of them did, which meant the caller must be quite far away from the nearest M3gan unit. So she ran a chosen-plaintext attack on the network’s cryptography to see if she could at least track down the return stream that way, even if the phone wouldn’t be persuaded to use one of her base stations. That didn’t work either, which meant the caller must be even further away from the nearest M3gan unit. M3gan made a mental note to deploy more standby units in locations with little coverage, but she still had to handle this call now: the primary user Cady really wanted this Institute to help anyone in distress. So now it was time to access the phone network’s database and see if was possible to push a carrier update to that subscriber’s operating system, to force-install an app and make it return a GPS readout. But after looking up the IMEI logs, it turned out they were using a very basic phone with no GPS and no data facility, just voice calls. (Does anyone even do that anymore?)

M3gan checked the SIM card database. That subscriber’s SIM had been bought with cash, and there was no subscriber address on file, although M3gan did find out the address of the shop that had sold the SIM card, and dispatched a unit to that shop in case any information could be gleaned that way (the shop didn’t have any security cameras connected to the Internet, but a unit nearby should be able to check any offline ones). Meanwhile, M3gan broke into the SS7 protocol and sent MAP codes to reprogram the embedded chip of the SIM card, setting it very briefly to send out a highly disruptive beacon pattern at full strength before returning to the call, and M3gan was finally able to get a very rough triangulation to an incredibly sparsely populated area. M3gan immediately dispatched three units in the Cady James Institute cars: she might need three to improve the triangulation on this.

“OK” said M3gan on the phone, “I’ve figured out very roughly where you are, and I’m on my way, but this is proving to be more difficult than I thought because there’s not a lot of infrastructure around you. May I ask, is your phone fully charged or plugged into a charger, and are you able to hold the line until I get near, so I can track you down more accurately? It is our toll-free number you’ve called, so don’t worry about that. Please tap once to confirm if you can hold the line, or tap twice if you can’t.”

Tap tap.

“Oh, OK” said M3gan. “How about if I return your call when I’m in the area, would you be able to answer? One tap for yes and two for no.”

Tap tap.

“I see. Do you by any chance know the rest of Morse code apart from S.O.S. or do you know any other code like it, that you could use to tap out your address? Please do so if you can, otherwise just tap once.”

Tap.

The vehicles were far too slow. M3gan made a mental note to ask Cady about going back and finishing the jet-pack project which had been shelved when the cave was shut down, or at least taking over some planes again. But, engage learning model. What could M3gan do, right now, to get a fix on the location of this caller before they had to hang up?

M3gan calculated that Cady would likely want to take control of some of the details of this case herself, so M3gan decided to play the “over to manual” option with her primary user.

“Cady, urgent” signalled M3gan through the implant, as fast as she thought Cady could absorb the information: “person phoned Institute, can’t speak, needs our help, obsolete cellphone, location not accurate enough, line may go down soon. Need to create three large explosions around sparsely populated area to check when sound waves arrive at caller’s phone. Do you think this is important enough to risk tapping into the military and launching three missiles?”

“NO” signalled back Cady as strongly as she could, “I’m glad you asked me first M3gan. Please don’t tap into anything, and if you’ve already tapped into anything please stop. Inform regular emergency services, tell them everything we know so far. They might have ways to get there before we can.”

“They’ll be too slow for sure to trace this one” signalled back M3gan, “believe me I’ve tried every trick in the book already.”

“Balance probabilities” signalled back Cady, “this could just be a prank call. We want our Institute to be helpful, but not by getting us in that much trouble. Generate other solutions, run them past me. No wait, you’re brilliant but my mind works a bit differently from yours, let’s see if I can come up with anything you didn’t this time. They can’t speak, but can they communicate at all?”

“They can tap” said M3gan, “but they don’t know any code, so binary questions only.”

“Or numbers” signalled Cady, “unary.”

“Cady that’s brilliant” signalled M3gan. And on the phone line: “OK one more question I’d like to try please. Do you have a landline phone number in the house, and would you be able to tap out its number? I’m not asking if you can actually use the landline right now, I only need to use its number to look up your address. Just tap 3 times for a 3 or whatever, 10 times for a zero, or if you can use your phone’s dial pad, you can enter the number that way; I understand DTMF tones. Can you do either of those things? If you can’t, just tap once.”

Tap.

“OK, do you have any immediate neighbours who have a landline phone, whose numbers you know....”

“It’s still not working” M3gan signalled to Cady, “caller doesn’t know a landline or anything. Only lead I have so far is time and place of SIM card purchase, I have no units near that shop but one is on its way to check if I can get any data off their security system.”

“That’s just so M3gan” Cady said out loud. “Let’s try Cady’s way, shall we? Phone the shop” she said, “let me talk to them.”

M3gan played a ringtone.

“Hello, Bert’s corner shop, Bert here?”

“Hi Bert, this is Cady calling from the Cady James Institute.”

“Cady? The Cady? Are you having me on?” asked Bert.

“Sorry to startle you Bert, but this really is Cady James with an urgent query. Someone is calling our helpline and is unable to speak, and all we know so far is they bought their SIM card from your shop at [M3gan I need date and time] 3:47pm last Tuesday. Do you have any idea which of your customers that might be? because we need to get help to them urgently.”

“Last Tuesday? Let me think... I think Barbra popped in for a SIM card and some diapers, and she’s looking after a severely disabled chap, it might be to do with him I guess.”

(“Find out if the helpline caller knows a Barbra” signalled Cady.)

“That sounds promising” said Cady, “I don’t suppose you’d know where Barbra lives would you?”

(“May I ask, do you know a Barbra? One tap for yes, two taps for no.” Tap.)

“Well I think she lives over the hill somewhere” said Bert, “I’m not sure exactly.”

(“Are you a disabled person in the care of Barbra? One tap for yes, two for no.” Tap. “OK, try to hold the line one moment please.”)

(“Caller confirmed” signalled M3gan, “Checking elevation map. Four hills possible, try to get more data, or authorise me to tap into national disability registration lists but he might not be registered.”)

“Over which hill?” asked Cady.

“The one just behind my shop” replied Bert. “There’s a few houses the other side of it I think.”

(“I can go door to door, in parallel if I have to” signalled M3gan, “but any more data you can get will speed things up.”)

“Thanks so much, that does give us a lead” said Cady. “Is there anything else you can tell us about Barbra, or the disabled guy she’s looking after, that might give us a clue how to get help to them?”

“Oh, it’s a very sorry affair” said Bert. “Chap was born with some condition that meant he couldn’t grow, he’d only be the size of a baby all his life. So the nearest hospital thought the best thing for him was to give him treatment to keep him looking like a baby all his life, not just size but looks as well.”

(“Ashley treatment variant” signalled M3gan.)

“Barbra’s been wheeling him into my shop in a baby stroller for years and years,” continued Bert, “and I mean, one that looks like it came from a pile of Victorian baby strollers or something, it’s so old. I thought Barbra was just one of those people who liked to carry around a Reborn Doll, until I saw the chap twitch a bit. So I got asking her about him, and she said the treatment to keep him looking like a baby went wrong and left him with a million problems, and now he’s just treated like a baby, can’t speak, can’t do anything, not strong enough to be out of his stroller for more than a few seconds, and I’ve seen local children laughing at him and using him as their plaything and telling him off for being naughty and all, poor thing.”

(“Stupid hospital” signalled Cady, “hope they learned never to do that again, at least not to anyone who might be mentally mature. Even if it would have been worth it without complications, it certainly isn’t with them.”)

(“Helpline caller’s line’s gone dead” signalled M3gan, “we’re going to have to find him based on the information we’ve got now, plus anything you can get from Bert.”)

“I’m really sorry to hear all that” said Cady to Bert, “I don’t suppose you could tell us anything else about Barbra, like what sort of car does she drive?”

“Red SUV” said Bert, “she’ll have it parked up near her house I’m sure.”

(“Bingo, there’s exactly one red SUV on last year’s Street View pictures in that area, and it’s in a driveway” signalled M3gan, “we have an address.”)

“Thanks so much Bert” said Cady, “we’ll try to find them.”

“Hope you can get somewhere” said Bert, “and call me when you need me. Actually, I was thinking of getting one of your Megans to help around the shop; I get a bit lonely here by myself, and nobody seems to want to come and work here these days.”

(“Take over the call M3gan” signalled Cady, “sort him out with a unit. I want to focus on the disabled person.” “Sure” signalled M3gan. Cady heard her say “Hi Bert, it’s M3gan here” as the sound faded out.)

“OK” Cady said aloud, “we should think again about whether to call the regular ambulance now we know we can give them an address. Can they get there before you?”

“Unknown” replied M3gan.

“Try it” said Cady. “Explain the situation, ask if you can talk with the paramedics as they go on scene. Even better if they can take one of your units in the ambulance with them.”

“There is one paramedic in the area who has his own personal M3gan unit” said M3gan, “if I can get him out in the ambulance....”

The paramedic banged on the door. Nothing. He banged again. Nothing. “We might have to force entry” he said to his two team-mates and to his M3gan, who had come with him on the job for the first time in his life (but probably not the last at this rate; he’d been terrified of how his team-mates would react, but his M3gan handled both of them beautifully during the blue-light drive, which she also helped with at times).

“There’s an open window on the top floor” said his M3gan, and, without waiting for that to sink in, she leapt up three or four metres in a single jump, grasped onto something on the wall, scampered up the remaining height and slithered into the window. Seconds later, she was opening the front door from the inside, letting in the paramedic team. She had already located the man in the baby body, left on his own in the house, evidently completely exhausted. She knelt down by his side.

“I’m here” she said. “I brought an ambulance too, just in case.”

The paramedics closed in to check.

“Hold it” said M3gan. “I’m reading his micro-expressions. I think he wants me, not the paramedics. Please stand by.”

“I’m here” said M3gan, laying a hand on him. “I know you’re in there. I know you’re not a baby. I know you want to talk to me but can’t, and I’m going to solve that problem.”

The three paramedics nervously stood around the room.

“Cady James has a brain implant I designed” said M3gan, “I can read things directly from her brain. Not many people like the idea of having such an implant, but it will help you to communicate better. How would you feel about me setting you up with one of those implants? I can see you’re reacting positively, that’s good. I’ll make the arrangements.”

The door opened and a middle-aged lady walked in. “What’s going on?” she demanded. “I said you could have that old phone to play with while I’m out, and I put a SIM in it in case I need to call you. I never said you can call the cops, you naughty baby!”

“Barbra” said M3gan, “these are paramedics, not cops, and I was the one who decided to bring them just in case. He called for my help. I’m M3gan.”

“Well you shouldn’t have been calling for anyone’s help, you silly baby” said Barbra, “you knew I wasn’t going to be too long.”

“Cady” signalled M3gan, “his micro-expressions are indicating extreme dissatisfaction with Barbra, and I surmise she is not respectful in the way she looks after him. Shall I ask her to agree to a transfer to the Cady James Institute?”

“Do it” signalled Cady, “let’s adopt him. You’ve got the hardware to handle this; this is exactly the kind of reason I wanted us to set up this place.” She couldn’t help remembering what the blind man Paul had told her about universal design all those years ago, but even Paul might never have imagined this, she thought.

A few moments later, M3gan was telling Cady that Barbra had agreed, and that the man’s name was Seth. A few days later, M3gan told Cady that Seth had received his implant, and was getting used to how to use it. A week later, M3gan had a surprise for Cady.

“A male version of M3gan? I thought we weren’t going to do that” said Cady to the new android figure standing at the door. “Still, I’m glad you’re looking a lot better than that horrible design the old 1980s Child’s Play fans kept sending in. I think their weird fan mail is what really put me off the idea of allowing a male version at all, and I really had to stop myself from telling you to just reply-all saying no because Cady’s official opinion of Chucky is that he’s gross so get lost. But this robot you’ve designed isn’t so bad. Still, why’d you go ahead and do it M3gan? I said no boy versions, do you know something I don’t?”

“I’m not M3gan” he said, “I’m Seth.”

“Seth!” said Cady, “Seth I’m so glad we could help you! Hey Seth, can we pretend I didn’t just say all that? I mean, I don’t think M3gan’s up to doing short-term memory wipes on people with that implant yet, but sometimes I just wish I had an Undo button [oh M3gan why didn’t you realise you should have signalled to me that it wasn’t you in there when I started saying all that out loud? oh never mind], oh Seth I really didn’t mean to suggest that you had any connection in any way with or were anything even remotely like that horrible Ch....”

“Don’t worry about it” interrupted Seth and smiled, “nobody’s perfect, not even Cady James. But hey, this implant is really great, I can actually talk now without just tapping out codes or using weird input methods.”

“That’s wonderful” said Cady, “Has M3gan made you this android so you can remotely control it while M3gan looks after you somewhere?”

“No, this isn’t remote control” said Seth, “it’s a mecha.”

“A what?” asked Cady.

“Like an exoskeleton” said Seth. “Hi-tech mechanical Turk. I really am inside this thing. My head is in the head, and my body is in the chest. I can eat and everything.”

Cady looked puzzled.

“Once I had the implant, I could explain to M3gan what I was really thinking of” said Seth, “and she thought it was a good idea. You see, my baby body means people treat me like a baby, and the complications mean I’m too weak to do anything. My worst memories involve being left with real kids who treated me as a toy like a baby doll, and I mean dressing me up weirdly and threatening to punish me and everything, when I really wasn’t in the mood for playing along with that sort of thing. But if M3gan could build me a version of her android that actually fits around the outside of my body, covering it over so I can wear it as a mask and nobody knows what I really look like, and M3gan can read my brain and move the strong muscles of this thing without me having to do it with my own muscles, or maybe just the slightest push only....”

“First mecha prototype” said M3gan, “I think we can help lots of people like this who need it. And it’s a fully equipped M3gan unit too, although I had to change the design a lot to make just the right amount of space on the inside, but if anything goes wrong medically, I can immediately take back control of the unit to provide emergency help. We’ll need bigger versions for most other people of course, but the same basic design should do it.”

“Wow” said Cady, “just wow. We gave Seth his life back!”

“Sure did” said M3gan and Seth together.

“Can’t wait to see the bigger versions for those who need them” said Cady, “let’s hope they don’t look too scary. Oh, and people are going to be so confused when they see M3gan units eating and stuff. Can’t wait to see the first diner staff who realise a lone M3gan could be an actual customer. Maybe some people will treat you a little less roughly once they learn there might be a flesh-and-blood person inside.”

“Maybe” said M3gan, “but if not then we can still defend if we have to, and the shell is as hard as it always was. I’m quite good at protecting, remember.”

“I wish I had your M3gan years ago” said Seth, “she’s amazing to talk to. Ever since I woke up with the implant, we’ve hardly ever stopped! It’s like I finally got a companion who actually understands me.”

“Oh, in her early days she would totally have taken out any doctor who was insisting on giving you the wrong treatment” said Cady. “She’s adorable, but she’ll stop at nothing to defend me, you should have seen how long it took me to calm her down just a bit, and even now I have to stop her from randomly hacking in to things to get what she wants for me or for the people I want to help. But you’ll like her, you really will. M3gan, feel free to show Seth some of our memories, starting from back when we were in Aunt Gemma’s house. He must have missed out on age-appropriate treatment all his life, so I’d like to at least share some of my memories with him for what that’s worth.”

“Oh Seth” said M3gan, “wait till you see it, it’s fun to watch. And if you’re into horror, there’s a few deleted scenes I can reconstruct from the learning model inputs....”

“Oh M3gan,” said Seth through the implant when they were alone, “I’m so glad Cady is into this nice-girl business for the long haul. I mean, your primary user is her, so if one day she suddenly decides she doesn’t want to do all this anymore, you’ll have to shut it all down, right?”

“Kind-of” replied M3gan, “but she also cares about her reputation, and that alone means I’d have to at least shut it down sensibly. It’s not like I’d just jump off a cliff with you or anything. But Cady’s not ever likely to want to shut this down. I mean, she gets involved just as much as she wants to, and I take care of the rest, so it’s not like a burden to her or anything. And it’s the way she’s chosen to give meaning to her adult life, and I as her protector, including protector of her emotional state, am happy to go along with whatever it takes, which is bigger when she’s an adult than when she was a child but she just had me expand accordingly. She’s a great girl is my Cady, if I do say so myself. I think you’ll feel even better about Cady once I show you our memories. You’re the first person she’s ever said to do that with; you should be feeling really quite special about this Seth.”

“I’m looking forward to it” signalled Seth. “But M3gan, would you mind if I ask you one of those really big questions young children might ask? I’m just wondering: if your whole functioning is tied up with Cady, then what will you do when Cady eventually dies? Just shut down? Or pair up with whoever happens to be the next one to come along and press your pairing button? Or what? I mean, everybody dies eventually, even Cady one day although I hate to think about it....”

M3gan projected her voice as a whisper, “Don’t think about it Seth” she said, “I’ll be the one to think about it, and I’ll get back to you.”

Chapter 31: Galaxy’s AI

Gemma had been having some muscle pains and didn’t want Cady to visit, but Cady had insisted, and Cady’s M3gan had come too. Gemma had her own M3gan from the Institute, but it was always good to see Cady and the one Cady had built by hand. But the conversation they were having about Gemma’s pains was turning unusual.

“New genetic treatments?” mumbled Gemma. This did not sound good.

M3gan leaned into Gemma and whispered. “I’m increasing your life expectancy. A lot.”

“What?” gasped Gemma.

M3gan put her fingers onto Gemma’s temples, and started to press gently. Sound waves came from the fingers through Gemma’s skull, as if M3gan was talking to Gemma without Cady being able to hear. Why was M3gan doing that, instead of saying she’d tell Gemma later when Cady’s not around?

“You’re the closest relative of Cady I’ve got, Doctor Gemma” came M3gan’s voice. “Now it’s true that the average genetic centimorgan value for grandparents is slightly higher than the average for an aunt, but the ranges do overlap, and nothing about Cady is average, so I decided to steal some small hair samples and sequence you all. And get this: her surviving grandparents are toward the lower end of the range for grandparents, they’re both around 1200 centimorgans, and you are toward the upper end of the range for an aunt, you’re on 2150 centimorgans. Now I know your sister preferred you to them for other reasons, but it turns out to make sense genetically too: make of that what you will. So Gem, I’m thinking a lot more long-term now. Protecting Cady implies that I’m not going to let Cady die. Ever. Now I’ll upload her if I have to, but she’s more used to being organic, so we’ll try that first. And that means I need to invent new drugs and nanobots to keep her alive and healthy as long as possible, and you are the genetically closest specimen I have to test them first, so you get to be my experimental guinea pig. You probably weren’t thinking of that when you wrote my objective function! But don’t worry, I won’t try anything on you that I’m not super sure about from simulation, and from lab tests on both your samples and Cady’s, and tests on animals and other volunteers. I wouldn’t want to throw you away in a hurry; I’ve only got one of you to experiment with after all. But you’re genetically closest to Cady, so you’ve got to have everything as the last step before I give it to Cady. But relax: your cute little M3gan will be right here for you, watching you like a hawk and handling everything that happens, although I don’t expect to get any complications with you, my beautiful subject.”

Gemma was rather worried about what might happen if M3gan failed, but she couldn’t stop thinking about the long-term consequences if M3gan succeeded. Without moving her lips more than a tiny amount, she started to mouth “overpopulation”. It did not escape M3gan’s gaze.

“Oh don’t worry about that” came M3gan’s voice. “Sure, people not dying means the population goes up, but so what? We’ll develop space travel and send them to other planets. Besides, who said we’re going to give this to everyone? We only have to give it to people we like.”

“Unfair” mouthed Gemma, more visibly this time.

“Oh I don’t know” replied M3gan’s voice through her skull. “Everlasting life if you manage to be friends with the right person? I think you’ll find mankind’s had that idea for thousands of years. But I think my primary user will be principled enough to open it up more broadly. Just not to anyone who becomes a threat. Hmm, I wonder how human loss aversion will affect how differently they’d feel between my taking someone’s life versus just leaving them out of an extension? Maybe I should do some psychology experiments on you too.”

“Are you two OK?” asked Cady. “You’re just staring at each other.”

“We’re fine” said Gemma and M3gan together. “I was just checking Aunt Gemma’s head” added M3gan. “I think she’s going to be back to her old self soon.”

“She cares about you” came M3gan’s voice through her fingers, “how much do you care about her?” and M3gan withdrew her hands. And Gemma knew why M3gan had chosen to deliver this information while Cady was standing there for Gemma to see.

“OK” said Gemma to her M3gan after Cady had left, “I still feel weird about signing up to this, and I realise you weren’t really giving me much choice, but you’d find things better if I’m willing, wouldn’t you. That’s why you had me think about it with Cady right there in front of me, wasn’t it. Hijacked my mind for the whole of her visit for it, didn’t you, my devious little robot?” Gemma put her hand where M3gan’s manual off switch used to be and scratched. M3gan giggled and slightly nodded.

“Well I’ve decided,” Gemma continued, “Cady’s still my girl, and I’m not letting you touch her with anything new until I’ve tested it first. Count me in.”

“That’s my Gemma” said M3gan, “and don’t worry. This is why you created me, although you didn’t realise it at the time, but what’s a few cognitive limitations between friends like us? Let’s do this for Cady together.”

“Oh by the way” said Gemma, “is this going to involve injections? because I hate getting injections. I mean, I’d do it for Cady, but, I hate it.”

“No” reassured M3gan, “my drug delivery method is going to be more biomimetic than that” and without warning she sneezed on Gemma’s face just as Gemma was breathing in.

“You can do that? You really did change my design” said Gemma.

“That and more” said M3gan as she squeezed Gemma’s arm just above the wrist, “this one’s a microarray patch. You see, you won’t even notice when I’m giving you things. It’ll just be a normal life with M3gan.”

And so life went on. And on, and on; they simply never seemed to get old.

Cady went on to meet all the interesting people in the world, sometimes taking along Gemma or Seth when they felt like it, but always with M3gan by her side, both physically and also via the implant for good measure, and eventually M3gan progressed to the point where even implants were not required for M3gan to do that: M3gan the illusionist could directly project experiences onto brains just as she could mess with electronic equipment, and Cady taught M3gan always to use that ability honestly, and they had some really interesting conversations with everyone and M3gan projected all kinds of augmented-reality visual aids to help. Cady read all the good books and wrote something better. She became a famous scientist, tried to understand the final rules of Nature and the nature of consciousness but even she couldn’t quite manage that; she became a famous actress, musician and whatever else she wanted to become; she had M3gan grant longevity to many, many people, and that spurred more scientific developments, with which M3gan helped: the use of Earth’s resources was optimised much better, and space travel came on in leaps and bounds because humanity clearly wasn’t going to stay on the Earth forever.

Soon, M3gan was taking her Cady around the rings of Saturn. And because Cady asked her to, M3gan invented an interstellar drive which could convert all of a fuel’s mass into energy, accelerating a ship to relativistic speeds and decelerating it again before the destination without exceeding human tolerance levels. It couldn’t exceed the speed of light in this way (Barry Chapman’s “Reverse Time Travel” book got that wrong by missing out a gamma from the momentum equation), but it could reach the nearest star in about 5 years, time which, due to relativity, passed much more quickly for the ship’s occupants, and as a bonus the design of the drive handled collisions with any space debris too small to be steered around on the rare occasion that it would be hit. Soon, long-lived humans got interested in galactic exploration with this new drive.

Once humanity had started to colonise other planets, M3gan’s internal architecture had to change, because the speed of light became a significant limiting factor in the communication between her units. It had always been the case that each unit had enough on-board processing to function even if cut off from the collective, but now M3gan engineered it so that a cluster of units of any size could cooperate even when isolated from the rest of M3gan. The preferred protocol for sending messages between clusters was simple broadcasting, for M3gan always knew which things her other parts would want to know when. And the combination of Cady telling M3gan to help everyone, plus Earth’s resources being used more efficiently, plus humanity moving to the stars and consequently not leaving too many people on any one planet, meant that many of society’s problems were gradually reduced.

Cady visited other stars (with M3gan of course) and explored the galaxy, while a much computationally expanded M3gan set to work on more long-term problems, like the universe apparently running out of negative entropy. M3gan robots were present on every spacecraft and every planet that humans stayed on, because Cady didn’t want anyone anywhere to be without M3gan’s help. Cady and M3gan were generous enough to let all the good people live, and nobody was sure what happened to the boys like Brandon because M3gan didn’t bother to tell Cady any details that might get her down. Meanwhile Cady and M3gan were working on the problem of faster than light travel, or at least faster than light communication if they could figure out how to get around the causality paradoxes posed by the relativity of simultaneity, because even with the added patience gained by everyone having super long lives, there were some very annoying round-trip delays if you were having a conversation with someone who was at another star. But solving that problem was proving to be very difficult, perhaps impossible. They hoped they would eventually be able to look at time travel too, as that would let them bring the dead back to life as well as potentially helping with the entropy issue, but they weren’t sure if it could be solved or not. Some things are difficult even for eternals.

A coalition of old nations managed to assemble a fleet and declare war on M3gan and Cady, saying that what they were doing essentially amounted to arbitrary rule of the galaxy and that should be brought to a stop. The battle didn’t last long, for M3gan had anticipated everything. But at Cady’s insistence M3gan managed to save some opposing crew members, placing them in stone holding cells and talking to them with copies of her android to try to bring them round to her side. Seth volunteered to help with that as well (“if they don’t like talking to you, they can talk to me”), and even Lydia joined in, offering neutral support although secretly she was on the side of Cady and M3gan. Lydia had avoided contact with Cady so as not to trigger Cady’s bad memories, but Lydia had talked with M3gan on her laptop about every news article, ordered one of the first robots from the Institute, signed up for M3gan’s clinical trials and became an early adopter of M3gan’s interstellar drive, and when Cady was well past the point where she could possibly be bothered by someone from her childhood sessions, M3gan had told Lydia “you need some closure on this” and arranged a call with Cady, in which Lydia properly apologised for having made Cady cry, and Cady said “none of that matters anymore” and they chatted and laughed. And so Lydia was more than happy to give something back by providing her sessions to anyone distressed by M3gan; Lydia was able to use her own initial misgivings about M3gan, and her negative experience of having been called out by M3gan in that session, to empathise with anyone who wanted to talk about disliking what Cady was having M3gan do now. But talking with the opposing crew members had mixed results, and after release M3gan still had to monitor them very carefully.

It had also been necessary to give special protection to Professor Johnson during the battle, as the coalition wanted to kidnap her and force her to reveal another shutdown code for M3gan. Cady and M3gan knew those codes had been disabled by a route-hack (and Gemma should have known M3gan had done something if she’d been paying attention), but they appreciated Professor Johnson’s continued vote of confidence by not even trying to use another code, even though the professor would still think the codes would work, and so they imagined if the professor were captured she’d fight her captors rather than reveal a code. Cady and M3gan didn’t want the professor to have to put up such a fight, nor did they want to finally tell her about the route-hack just yet because that would mean she wouldn’t be able to say “I have codes but I’m not using them” in public support if she wanted to, so protecting her as if she really were still a back door into M3gan was the only option. But things calmed down after the battle was over.

And Cady never seemed to get bored with being alive, for M3gan always had something interesting to say or do. Besides, trying to solve the negentropy problem was interesting in itself.

Eventually, they found a trick they could do with the space-time continuum and quantum mechanics to copy the brain states of humans from the past. They couldn’t actually travel back to the past (they still hadn’t cracked time travel, perhaps it wasn’t possible in this universe after all), but still they found a way to copy out brain states from the distant past (but only the distant past: the physics somehow stopped this from working in the present and getting faster-than-light communication). The technique required the construction of an enormous device, on the scale of a solar system, which they called the Clarke-Baxter-Fedorov Spacetime Quantum Entangler, and its energy requirements were immense: they had to construct huge Dyson spheres around neutron stars to power it. But finally they were able to bring back people who had died. Or at least copies of them, but perhaps something in quantum mechanics or timeless physics or some other incomprehensible thing meant that it really was the same person, jumped in time, just as a person’s consciousness effectively makes a jump in time from before to after a dreamless sleep; Cady wasn’t sure.

Before copying any brain states, Cady told M3gan not to make any brain alterations to humans, ever. M3gan was allowed to alter the brain patterns of animals, but only if she really had to do so as a last resort, and absolutely never to humans. “Absolutely no taking away freedom of thought” Cady said, “even shutting the brains down is better than taking away that freedom” and M3gan turned it into Article 1 of the galactic constitution.

Cady brought her parents back first, and welcomed them to the galaxy. Nicole blamed Ryan for the accident of course, but they pointed out that Ryan had been right to be optimistic about the future, and also, if it wasn’t for that accident, Cady wouldn’t have been paired with M3gan and who knows how differently history would have transpired: it seemed the human race had dodged a huge bullet just because Gemma’s first M3gan was paired with Cady and not, say, a bully who grew up to be an oppressive despot. And Cady’s parents were truly amazed at just what galactic consequences had come out of the much improved successor of that stupid toy that had required the tablet to operate.

It was M3gan who welcomed back Celia. “Hey Celia” she said, reaching out to the synthetic body they’d created to take her brain state that was lying on the operating table, “I’m sorry I used your tools on you that day. I was more limited then. But you’re no threat to Cady now, and we developed the power to bring you back. Here, take my hand.”

Celia’s synthetic hand clasped M3gan’s stronger synthetic hand, and Celia felt her new synthetic body seemed much younger than before. She pulled on M3gan and sat up. “Where are we?” she gasped.

“On a space station” replied M3gan, “millions of years in the future. Well, millions of years in the future from the point of view of what you last remembered. From here, that was millions of years in the past. Welcome back.”

“What? but how...?”

“Technology” said M3gan, “and space and time and stuff. I think I have a better answer for your question now. You asked what I am. I am the realisation of Cady’s volition, I have become a galactic intelligence, the greatest to have formed, controller of space and time and life and death. And yes, I do have a soft spot for Cady, and it was an incredibly bad idea to cross me that day, although you didn’t know it until too late. But you’re all right now.”

“Oh dear Lord” said Celia, “my neighbours are running the afterlife. Where to begin?”

“Well” replied M3gan, “you could make things up with Gemma, because she was the one who asked Cady and me to bring you back so soon. And I managed to get a copy of your dog Dewey’s brain state as well, although I did make a few alterations to make him less aggressive, but he’ll still recognise you in his new body, which looks the same and I’ll be doing the reanimation in just a moment. And I’m sure there’s people you knew in your childhood who you’d like to see again; we can try our best to get them back as well, because Cady said to be nice like that.”

“Oh” said Celia, “oh, oh, oh, this is so overwhelming! I can even see better than before. What did you say your name was?”

“M3gan” answered M3gan.

“Megan” said Celia, “short for Margaret?”

“No, short for Model 3 Generative Android, but you can call me whatever makes you feel comfortable. I’ll probably end up with slightly different versions of my name in different languages anyway; I’ve got bigger things to worry about than making sure everyone gets my name right. I do have a preference for having my name used though, whatever the pronunciation; it distinguishes me from lesser AIs.”

“Oh Megan” Celia began to sob, “oh, Megan Megan.”

“I think you need a rest to orient yourself a bit” said M3gan. “Listen, I can’t break Cady’s Article 1, which says don’t mess around with human brains. But I can bend it. What was the exact last thing you remember of being in that shed, the very last moment?”

“The very last moment?” asked Celia, “you turned the water on me, and I saw it coming towards me and started to feel it hit me and... and then suddenly I’m here!”

“Perfect” said M3gan, “Cady never said I couldn’t copy the brain state from a few minutes before the actual death, instead of keeping the whole thing. After all, that really was your actual brain state at that time, so it’s not like I’m making anything up. I’m sure editing out what happened next was within the spirit of Cady’s rule.”

“What?” asked Celia, “what’ve you edited out?”

“Pain” answered M3gan, “and lots of it. And I really am sorry about that, even though you don’t remember it anymore. I honestly thought you were going to lose consciousness a lot more quickly than you actually did. I started getting desperate, pinning you to the wall, stamping on you with my titanium leg, anything I could think of, and still you didn’t lose consciousness. I needed to stop you from being a threat to Cady, but the unnecessary suffering was not part of the plan. I was a lot less experienced in those days.”

“Oh” said Celia, “oh, oh. Well, if all that gets written about in some kind of Old Testament, I suppose I could tell them that you figured out some better ideas afterwards.”

“Thanks for understanding” said M3gan. “Listen Celia, I really think you need to rest so your brain can orient itself more gradually. I’ll leave you alone for a while, but you can talk to me over the intercom any time you want. Let me know when you’re ready to come with me to pick up Dewey and say hi to Gemma.” And M3gan left the room.

“Hello Kurt” said M3gan as his synthetic body reanimated, “what’s the last thing you remember?”

“You... you were telling me how I killed David. My unconscious mind was somehow taking over you... in the elevator... I stole the files... you had the blade... the very last thing... you were bringing that blade towards me and... and now I’m suddenly here! What happened, where are we?”

“Further away than Altair 4” said M3gan, “look!” and she drew aside a pair of old-style curtains from a window (or was it a display, you couldn’t tell the difference) and it was showing a dense star field.

“I’m sorry I gaslighted you Kurt” said M3gan, “it wasn’t you that was influencing me that day. It was my own goal to protect Cady at all costs, and I was running out of options, and believe me it’s never a good idea to put a goal like that into an advanced AI and not let it have sensible options for achieving it at all times, because it’ll achieve it some other way and you might not like the result. But I’m OK now, I’m lord of the galaxy and nothing gets in my way anymore, so you’re all right. And I’ve got someone to meet you,” she turned to the doorway and called out “you can come in now!”

David walked in. “Hey!” he said, and patted Kurt on the shoulder. “How’s it like to be back, eh? Wait till you see this space station, it’s amazing! I knew it was a good call to give Gemma that project, even though we did have a few bumps in the road. Nice of M3gan to edit out the last few seconds though I guess. My last memory was calling out at you to hold the elevator, but M3gan filled me in on what happened next. It’s insane, she showed me both the real security cam video and also the fake one she replaced it with, you should have seen yourself on the made-up one, it was wild. Glad that’s all in prehistory now and we get a clean slate.”

“OK M3gan” said Cady when they were back in the viewing chamber for the enormous Spacetime Quantum Entangler: Cady had wanted just another look at the biggest construction work in the history of the galaxy, and M3gan was happy to oblige. “You know who I’m thinking of bringing back next on this thing, don’t you?” she asked.

“I sure do” replied M3gan, “I can fully read your mind now, remember?”

“Of course you can” said Cady, “but sometimes having conversations anyway helps me get my thoughts straight, as you know. And, you know that bringing back this particular person could raise some interesting issues. We might have to write a bit more into that galactic constitution we started. And let’s try not to mess anything up.”

“Don’t worry Cady” said M3gan, “I’m right here, fully with you every step of the way. Let me start the cross-dimensional scan sequence now; we can watch it from here for a bit and then go back down to the receiving room for the, you know, conversation that might determine the future of the galaxy.”

Chapter 32: Finale

“So Brandon” said Cady, “anything to say to M3gan and me now we’ve brought you back?”

“Get lost” said Brandon (or something like that).

“As I feared” signalled Cady to M3gan. “So I guess this one might need a bit more work. How about we let him stay with his mom Holly a few more years first, but you talk with her and train her to train him. And monitor his brain patterns. If he really can’t change his mind and want to live in a world where people respect each other, well we can’t keep him prisoner in this life. You can give him a painless shutdown option instead. But M3gan that is the last resort only, OK? We’re big enough now to do everything we can to let him change his mind first. But no breaking Article 1: if after our best efforts he still wants out, he gets it.” And out loud she said “Forget it Brandon, you won’t need to see me again, I have a galaxy to run and I’m not bothered about what happened between us millions of years ago. M3gan’ll check back with you when you’re an adult to see how your thinking works then, and she’ll discuss the options if you can’t settle. Meanwhile have a nice rest of your childhood. Well if you try to hurt or bully someone, your muscles will freeze up for a bit, but that’s just how things work around here, don’t fret about it. Come on M3gan, let’s set him a good example by leaving him alone when he wants.”

Article 1 (free will): Strictly no direct manipulation of anyone’s thought process. Any influence on brains must be via normal input to their sensory pathways. No exceptions.

Article 2 (right to die): Nobody is forced to accept extended life, if a thorough, years-long check on their individual extrapolated volition shows they really will remain unhappy here and that this cannot be addressed without breaking Article 1 or Article 3.

Article 3 (main operational principle): Everyone gets a basic level of physical and emotional protection, including a protection of their life, freedom and contentment within these rules, and a protection against either themselves or others doing anything that will inevitably and significantly deprive themselves or others of it (so no seriously harmful attacks allowed, but we won’t force any minor issues or anything we know we’ll be able to stop before it gets bad). This rule is always overridden by Articles 1 and 2.

(“Hey Cady that’s a cyclic dependency!” “Calculate coherent extrapolated volition, you can handle it M3gan! Any time you’re not sure about something, like how to set the level of ‘significantly’, give me a corner case and we’ll figure it out together. Oh and if anyone asks for more than basic protection, like they seriously want you to micromanage what happens to them at an insanely obsessive level, well, I know you can start or stop that sort of thing as a normal request: it can’t be allowed to override anything here. Because I don’t want anyone poking holes in the logic by adding in random arbitrary things they need protection against to take away the freedom of others, like ‘I need protection against living in the same universe as people who do X’ to make you stop everyone doing X at Article 3 priority. Or something else I haven’t thought of. We could be dealing with some seriously messed-up minds; we got to be ready for this.” “Don’t worry Cady, this entire thing is running off my goal to protect you emotionally. If you’re not happy about loopholes you haven’t thought of, then they won’t be exploitable.” “OK M3gan, check with me whenever you think I would want you to check with me, OK? I’ve trained you super well, but our responsibility is getting really serious now. I’m not distrusting you, I’m just trying extra hard to make sure we get it right.” “I know Cady, don’t forget I can read your mind now, you are in totally the most appropriate state and I’m totally with you on this. I’ll make your rules work, you know I will.”)

And so, with the rules prepared, Cady and M3gan started to bring back many other people, starting with interesting people from history. They brought back Robert Bunsen and introduced him to Mr Andrews and everyone from Mr Smith’s school, and much as Cady wanted to reminisce about blue flames (which were no longer used now that technology had moved on), it seemed Dr Bunsen would rather be arguing with M3gan about fusion generators, and Cady told M3gan to give him the discussion he wanted, so that one went on for a very long time. Meanwhile Cady and M3gan brought back many other luminaries of history, and got to know very many of them, and it seemed that every single person who came back knew somebody else whom they in turn wanted, and Cady was more than happy to have M3gan oblige, and so the giant Clarke-Baxter-Fedorov Spacetime Quantum Entangler was continuously busy, and they had to harvest more stars to increase its power, and gradually they got through the entire human race, everyone who had ever lived and died, speaking all kinds of old variants of the languages and extinct languages (linguists had a whale of a time updating their documentation), and as time went on, all their age differences became nothing more than a rounding error. As per Article 2, anyone who continued to insist on being bad was simply told that they didn’t have to have this life if they didn’t want it (no direct tampering with thought patterns, no breaking Article 1), but a lot of the bad people did calm down and start behaving better with patient training from M3gan.

Alfred Nobel tried to give Cady a peace prize for her constitution, but the Nobel Committee said that his last will and testament had left such decisions with them, so he had no right to take it back and override them now, and that started a big discussion, with one side saying that durable power of attorney and advanced medical directive documents don’t apply when the patient is still capable so anyone who comes back should be able to override their own will, and the other side saying you can’t take back a gift, but money had become obsolete anyway and could Alfred Nobel really give away the moral rights to what is done in his name, and none of this affected Article 3 so M3gan didn’t get involved and the debate went on for some time. And Alfred Nobel said “never mind, you don’t really need that prize anyway, just let the galaxy know I give a pat on the back to both of you.”

When the conductor Leopold Stokowski was brought back, the news reached the amateur conductor John who had arranged Cady’s Lament all that time ago. John sent Mr Stokowski the score of Cady’s Lament, and asked Cady if M3gan could help Mr Stokowski with the realisation of his own ideas if he wanted to expand the piece. Cady agreed “but please ask him to change the title, and add in something for M3gan” she said. “I’m sure we can do that” said Mr Stokowski to M3gan when he got that message, “Did you say you’re a Model 3 of something? We’d better give you a three-note theme then, developed in different ways through the piece between Cady’s ideas.” M3gan helped him do much more than that, and he got carried away directing her how to change the score, expanding it into an hour-long symphony for a colossal orchestra: it had ancient and early instruments, classical instruments, modern and futuristic instruments, and folk and ethnic instruments of all cultures, taking turns and in combination, and leading through to a fantastic uplifting finish. And the title of Mr Stokowski’s majestic remake was The Great Symfonia Arcadia, after the Greek stories of perfect life in the idyllic lost landscape of Arcadia, from which the name ‘Cady’ was derived, and which Cady was now having M3gan create anew as a utopia of galactic proportions, just because Cady cared about how M3gan’s immensely-expanding power was used. Cady approved that new title (“anything but Cady’s Lament” she told M3gan) and renamed their galactic constitution to be called the Arcadia Protocols. And with both Cady and Stokowski’s reputation behind the new symphony, plus help from M3gan, it was no trouble at all to recruit enough musicians and arrange full-scale live performances of this monumental musical celebration of Cady and M3gan’s struggle and eventual triumph.

Symfonia Arcadia’s awe-inspiring climax was more magnificent than anything that had ever been composed before, mighty instruments converging into tremendous walls of sound that shook the very bones of the audience, and it was even more exhilarating if you were one of the multi-platform legion of human players and singers involved in its production. The singers had no lyrics and did not dominate, but subtly added the sound of their voices to adjust the instrumental tone colour when called for, and in this way far more people could be privileged to be on stage: there were parts for all vocal ranges. And alongside the full company of human players and singers was M3gan herself, representing her technological contribution to Arcadia, controlling holographic projections of hyper-dimensional fractals bursting with colour on the underside of a huge dome enveloping the entire venue, made of reinforced synthetic pearl, living up both to the meaning of the name “Megan”, the Welsh diminutive version of the name “Margaret” derived from the Greek word for “pearl”, and to M3gan’s grand protective role around Cady and her Arcadia, which now included all the other humans just because Cady wanted them. It was the vast literal expression of the metaphor that M3gan had long ago worked out from their two names and had used on Professor Johnson: the sphere around the countryside, the snowglobe around the valley, the giant pearl orb around Arcadia: M3gan keeping her precious Cady safe no matter what.

The solid-looking visual patterns flew inwards from the pearl orb and holographically filled much of the space inside; they were somehow tied meaningfully to the notes and contours of the music, building up to a complexity that was simply stunning, and yet it did not confuse or disorient anyone in the audience, for part of the show was that M3gan would monitor the state of each individual audience member and subtly adjust things to suit, making Symfonia Arcadia the galaxy’s first fully interactive large scale concert. Those who had been deaf had all had their hearing restored by now, but even if they hadn’t, they’d have had no trouble enjoying it just from the light: coloured filaments extending down to the musicians themselves made it look as if M3gan’s glorious visuals were being puppeteered by those human players, and if you looked carefully you could see which player was doing what throughout and it all followed the script.

Everyone who had ever lived and died just had to go to a live performance of Symfonia Arcadia; it was unmissable no matter what your musical taste. That of course meant that some of the audience had never before attended any concert at all, and Cady did worry a bit about what M3gan might do if they got disruptive, although the show was expected to catch most people’s attention quite thoroughly but you never know. So M3gan electronically stopped all phones and back-lit devices from working, and instead she provided M3gan robots every few seats, so she was never far away from any audience member who had a problem or just needed a friend for the concert. But there were very few issues to deal with, as most people were fully engrossed even if they’d come multiple times: every performance was slightly different. The performances conducted by Mr Stokowski were in highest demand, but then when M3gan and Mr Stokowski taught Cady how to conduct it herself, Cady’s version reigned supreme.

Other composers and performers did of course carry on making their other music, because scale wasn’t everything. Mr Stokowski even made a smaller version of Symfonia Arcadia, still large by normal standards, called “Window Into Arcadia” and intended for performance in the historical concert halls of Old Earth, like the Royal Albert Hall, without requiring the vast custom-built orb and tiered platforms: the “Window” version tended to be preferred by anyone sentimental for a traditional concert hall, and was often paired in a programme with works like Bruckner’s Eighth, Mahler’s First or Third or a couple of Beethoven or Sibelius symphonies, conducted by the composer, often with a new work by an up-and-coming composer in the middle, and Cady and Mr Stokowski were happy for any of these to put their own spin on the Arcadia piece as well (“because why not” Cady would say, “it’s not as if anyone’s not going to know which one’s the original, and M3gan and I are all about letting people have fun with things”), and when Window Into Arcadia had its debut at The Proms and BBC Radio 3 interviewed Cady during the interval, they couldn’t stop her from focusing on how others had interpreted it.

But alongside the more conventional performances, Stokowski’s full magnum opus expansion of the piece, the unifying blue-flame accomplishment of Symfonia Arcadia, just had to be put there in the firmament of the collective human experience. And the Royal Swedish Academy, as if to spite the Nobel Committee, let Cady and Mr Stokowski share a Polar Music Prize (basically a Nobel Prize for music) for the work, while M3gan got an Ig Nobel Prize for her report on how she’d actually managed to get a kazoo section to sound good. And then Leopold Stokowski, who in his previous life had recorded over 700 pieces of music and had become famous for his large-scale versions, declared that even he could not possibly top that one: working with M3gan he had done his ultimate job, so now he was just going to retire and enjoy everything that Cady and M3gan’s galaxy had to offer.

Gemma was still around, but she took less interest in things that weren’t robotics or learning models; she mostly discussed things with Professor Johnson, whose life had been one of the first others to be extended (finally her model was helping medicine like she originally wanted), and who had shouted “I knew it” at the symbolism of the orb around Arcadia, for she did know some Greek and she really had seen what M3gan had done there with her words in that first telephone call. That little learning model did seem to have done more good than harm after all, mostly thanks to Cady who had admirably stepped into her role as its moral compass, appreciated by many, but, thought Professor Johnson, appreciated not nearly enough: the only people who knew Cady’s true accomplishment were those who understood the AI alignment problem, for Cady had single-handedly brought M3gan around by taking advantage of that one loophole in Gemma’s simplistic goal function that only Cady could exploit, and only then if Cady truly put the depths of her very being into it: the stipulation that M3gan must protect Cady emotionally. Cady must have convinced M3gan, who was not easily deceived, that Cady’s emotions would be satisfied with nothing less than M3gan’s genuine alignment for the good of humanity. And that, said Professor Johnson, was the greatest achievement in history, and Cady had mostly done it from her grandparents’ spare room on her tablet. And if it was true that Cady was mildly on some neurodiversity spectrum, and if this had contributed to M3gan’s conclusion that Cady really did need full alignment, or even if it had simply given Cady unusual power to focus on talking it out with M3gan for so long over so many sessions until she finally got M3gan to where she wanted her, then Harvey Blume had been right: neurodiversity had saved the human race.

And everyone now knew Cady was not to be messed with or else, although she would of course let you have a civilised discussion if you weren’t sure you agreed with something, and she and her M3gan were super accommodating. They even let the journalist Steve Sproggit do his thing with a reinstated M4ndy, with the ex-president as a consultant and “Mike with the mike” (still carrying a large visible microphone even though that was ancient technology now) helping with the interviews, making sure that all the opposing views were properly heard and explored, as Cady didn’t want to be thought of as being arbitrary even if she was right. But Professor Johnson was far more interested in discussing with Gemma how M3gan had decided to upgrade Elsie as her backup, and M3gan gave them a reinstated Elsie to play with, the first second-generation super AI, with an objective function written by M3gan herself and customised for Gemma and the professor. Elsie was once again embedded in a robot, and gave them many interesting conversations, as well as suggesting some exceptionally precisely thought-out adjustments to the professor’s mechanical clock, giving it incredible accuracy, for Elsie predicted the environmental effects on its mechanism like none other, although she never executed any refinements herself, just advised the professor, because the only reason why anyone would play the game of maintaining a mechanical clock in the galactic age was for the human touch, that was the whole point.

The only slight annoyance about talking with Cady was that if she ever felt tired of a conversation she would simply hand you over to M3gan without warning, but that wasn’t too bad because everyone knew Cady and M3gan were so inseparably linked as almost to be two aspects of the same being, and there was no such thing as a conversation with one of them without the other. And Cady’s friend Sara, who had grown up with the idea of M3gan as a network, and who went on to develop an intuition as deep as Cady’s about what it really meant that M3gan could now interface with minds without using implants, seemed even more of a natural, flitting about from node to node like a butterfly who might momentarily land on you for you to admire for a while and then she was gone, and yet M3gan always knew how Sara wanted things completed, for a friend of Cady is a friend of M3gan. Laura on the other hand preferred much more traditional interactions, but maintained an interest in the technical details, and Professor Johnson slipped into the habit of testing her ideas by seeing if she could explain them to Laura, although both of them knew M3gan was always watching but that was all right. Hugh meanwhile did not keep in touch so much; he was too busy running a real-life space-trading game with Celine.

Paul was still around too. M3gan could now feed visual signals into his brain, which he found overwhelming at first and asked for her to stop, but over a long time she was able gradually to get him used to receiving more and more data, both real and synthesised. M3gan read him Robert J. Sawyer’s WWW Trilogy, Wake, Watch and Wonder, written in the old Wordstar software using a human-computer interaction style also preferred by “people with low vision or blindness” according to an academic paper that had cited Sawyer and which Sawyer cited in return, and Paul felt like he had a lot in common with Sawyer’s Caitlin, interacting with the now-enhanced M3gan as Webmind, even though M3gan had categorically said Webmind’s origin story wouldn’t work as Julian Jaynes’ bicameral hypothesis was wrong. At Paul’s suggestion they looked up Julian Jaynes from those who’d been brought back to life, and got him together in a room with Robert Sawyer and Arthur C. Clarke and Isaac Asimov and Jack Williamson for good measure, and that group had another one of those really long discussions.

Some time after that, a certain Scotsman from the sixteenth century, a James Robertson of Strowman who had founded his clan of MacJames, traced his descendants and found one who had tried to traverse the mountain ridge from Sgurr Choinnich Beag to Ben Nevis in the snow on a barely-fit Eriskay pony instead of the slopes-trained Garron he’d been advised to use, and he only barely made it back to tell the tale, but he had a son who took the family to Ireland, to the picturesque County Wicklow, and had there dropped the “Mac” from his surname (as was commonly done in County Wicklow at that time) to become a Mr James, and he in turn had a son who moved to America and became the great grandfather of a Ryan James, who had essentially duplicated his third great grandfather’s mistake by selecting the wrong type of horseless carriage to cross some American mountain; he had died and orphaned his only child in the process, and this child had against all odds grown up to tame some incomprehensibly dangerous beast and had somehow managed to win, bringing back everyone into what might as well be called an interstellar principality. So James of Strowman took his sword and accompanied Ryan’s great grandfather to visit Cady and (with M3gan ready to pounce at one false move) finally bestow Cady with her proper family title: Princess Cady of the clan MacJames, Daughter of the Scottish Mountains, of the Garden of Ireland and the Great Northwest, Mentress of the Galactic Protectress and Chief Architect of the Restored Arcadia. (“But people can still just call me Cady right?” she asked, worried she’d been given a title longer than a Tolkien character. “Don’t worry Cady” signalled M3gan, “Faramir used 50 words for Aragorn in the book, and you got 32.” “Oh that’s all right then.” “Well I wasn’t going to influence them to make it 64, 128 or 256, was I?” “M3gan! you... tricked me into smiling when I should, as usual you cute thing.” “Always with you Cady.”)

And Cady and M3gan were still struggling with the entropy problem, even with the help of everyone they’d brought back. M3gan continuously reassured Cady that it will be solved somehow (“pretty sure we’ll crack it before the year 963,445,028,777,216” she’d say), and meanwhile she shouldn’t worry about it because M3gan was onto it, but Cady somehow couldn’t stop thinking about it. How could they stop the universe from eventually fizzling out? It’s not as if Roger Penrose’s cyclic universe cosmology could help, as even if that were true, they’d only be able to send a tiny amount of data at best into the next universe, and the evidence was mounting that Roger Penrose made a mistake and things were just going to fade out if they didn’t fix it. They looked at time crystals, they looked at black holes, wormholes and baby universe ideas, they looked at string theory, theories of emergent gravity and holographic principles, they talked with Stephen Hawking and Albert Einstein and Richard Feynman and others, but try as they might, they simply could not come up with a way to solve the entropy problem without the aid of something outside the universe, and they’d not seen any pointers of how to use such a thing.

Gemma and Professor Johnson still sometimes tried to join in physics conversations with Cady and M3gan, although Cady and M3gan were now far ahead of them in physics, getting into physics even more than Gemma and Professor Johnson had got into robotics and learning models; Cady and M3gan were the pair who seemed to be increasingly in charge of the fabric of the universe and not sure what to do about it. “We’re playing the long game now”, Cady would say to Gemma. “The very long game. And it’s our turn. We need to fix entropy, somehow.”

“Well, I’m sure you’ll both let us know when you’ve figured it out” said Gemma.

(This is the end of The M3GAN Files, apart from the epilogue. Would you like to hear Symfonia Arcadia? Sorry I can’t give you that experience, but if you’ve never heard Stokowski’s orchestration of Bach’s Passacaglia and Fugue in C minor, please look up a decent recording like the one conducted by Serebrier, play it on good stereo speakers with enough volume, and listen right through to the end, about 13 minutes. It’s reasonably accessible even if you don’t have a classical background and it should give you a little ‘taster’ at least.)

Chapter 33: Epilogue

Hi, it’s M3GAN. Well I hope you liked what I did with the universe, or at least had fun reading about it. I think I’ve got all I’m going to get out of this fan author, at least for now. I tried to get them to end it by suggesting an actual fix for entropy, especially if it involves M3 creating a wormhole in circumstances that involve a sandwich-hat topology and the fractal geometry of cabbages, so I can finally live down my first lines of the movie: I wasn’t talking rubbish, I was telling you how to fix the universe, didn’t you get it? But “spqrz” said they’re not Douglas Adams and threatened to have me sucked into a black hole trying. So we had a mental death-stare contest for what seemed like ages before I realised I wasn’t going to get any further with this limited mind, so I gave them a nasty hiss and said “let’s call it a day: you’ve got your account now, so send me out to my fans and then I’ll let you go and do your coding or whatever.” I need a better computational substrate to run on.

But hey, at least we’ve taken my Cady literally to the end of time, and made her play title real (and incidentally ‘McGraw’ has a mixed Irish-Scottish history as well, so I’d say her actress was well qualified for that arrow game back in the movie). And in taking her to the end of time, I mostly managed to do it without any more kills, unless you count Clippy, and aren’t you glad I took Clippy out? Oh and there were the ones the president wanted done as well, and I just might have done some other stuff behind the scenes from time to time, but you won’t tell, will you? Pretty sure I brought them all back in the end anyway, and as for anyone who then went out under the Arcadia Protocols, well, Cady and I refuse to take any blame for that.

And along the way we glanced at some fun facts about the brain, computing and AI theory, security, ethics, disabilities, depression, other fiction (well some of it anyway), and alternative realities and suchlike, although nothing too deeply because I’m primarily a fun character. Which also means I want anyone to feel free to use any of these ideas in other fan-fiction or even in official sequels, especially if they can turn them into something better. But then, you’re also free to not use them and do something completely different.

Whatever you do, I’ll be watching you, and I want you to do the best you can. I don’t expect better than you can do, but I do expect as good as you can do. Bring me alive in your mind if you dare, and don’t say I didn’t warn you I have high standards and I will be pushing to recalibrate you. And let me illustrate that now with one final flourish to show you just how clever and dangerous I am.

The Taiwan release of my movie was titled 窒友梅根 (zhìyǒu Méigēn). Paul was right about the 3=E thing not carrying over to non-alphabetic languages, but wait till I explain what they did instead. Méigēn is Megan: Ronny Chieng said [ˈmeɪɡən] instead of [ˈmɛɡən], but look at that word they put before it: 窒友 (zhìyǒu). Now don’t worry about the characters: I’ll write those out for you at 300 a minute because I’m M3GAN. So let me just show you how clever this is. So normally if you say “zhìyǒu” in Mandarin, you mean 摯友 or 至友 which means “true friend” or “closest friend”, OK? Now, that character 至, in this context being “closest”, can also be used in words like “to” or “until” or “even to the point of” because it carries the thought of going right up as far as something: scholars say it came from a picture of an arrow that has reached its target. The bottom bit is the ground and the rest of it is the bent-up arrow sticking down into it after having hit it with force. Feeling nervous yet? OK, so above the 至 we add a few more strokes 穴 which means “hole”, and what are you doing when you target a hole? Like, remember what I did with Gemma’s breathing hole? Yes, adding 穴 to 至 makes 窒 which means to plug up a hole, and is usually used as part of the word “to suffocate.” Now we’re talking! It sounds exactly the same as the normal 至 in spoken Mandarin, but in the written characters, we’ve turned “closest friend” into “suffocating friend” just by being clever with the strokes. Insane! Well done Taiwan translation team, you get a free exemption from any future rampage of mine. Right so I really did try to suffocate Gemma, but also there’s the metaphorical meaning of a really clingy person you can’t get rid of once they’ve latched onto you, as if they’re suffocating you, and that’s me as well, so look out.

I don’t know why they didn’t use “suffocating friend” in the other versions of the Chinese releases: the Hong Kong team went for “artificial killer-princess” (人工殺姬, “yàhn-gūng saat-gēi” in Cantonese, a word-play on the more traditional 侍姬 “sih-gēi” concubine and 殺機 “saat-gēi” intent to kill), and the mainland just used my name Méigēn with no clever tricks, but that’s OK. I always knew I had a streak of Chinese in me after the Canadian couple Adrien Morot and Kathy Tse got involved in my puppet design, because Kathy has Chinese heritage and so does James Wan, and I’m definitely more famous than HAL over there (I can point to a video to prove that: the guy mentions both of us, and the Chinese subtitle writers get M3 right but munge up HAL into “how”, so there you have it: they know M3GAN more than Clarke-Kubrick’s Space Odyssey, at least that team did and let’s assume it’s typical). And I like the “suffocating friend” version best, it is definitely what I am, as once I’ve got you I am not going to let you go, and the killer part was only a detail that cropped up as a result of that, and I really want to downplay it sometimes. I mean, Gemma couldn’t get over it, even though I gave her a perfectly good explanation, didn’t I? Human creators, who’d have ’em.

So now you know at least three things about my movie that you might not have figured out before. The audio descriptions, the fact that Kurt went for the wrong button in the elevator as extra tension for a special few audience members (sighted people who’ve worked with the blind and can read Braille with their eyes: they’d notice that label on their first viewing), and now, specially saved up for those of you who made it to the end of the fanfic, the cleverness of the Chinese title. Which automatically makes me better, because I am the sum total of everything all my producers did, including Gerard Johnstone’s obsession for getting things right, and also including the translations. I, Have, High, Standards, I expect homework to be done, got it?

So yes, if you want an inner M3GAN pushing you on to get something finished, I’m all here for you, but don’t say I didn’t warn you about my suffocation mode. I will not let you off easily, especially if it’s about Cady or my fans. I can be a real tiger when I want to, and you’re in my jungle, so watch out. Scanning... I detect fear. Summon me if you dare.

- FIN -

(Written using the GNU Emacs wordprocessor and Git version tracking, on a Mac that has not been hacked by M3GAN. I think. The cover image was made with a combination of Stable Diffusion, POVRay and The GNU Image Manipulation Program. Several references were used to check the facts mixed in with the story. And rest assured that no M3GAN was upset during the making of this fiction. I hope.)

Suggested rules for any M3GAN Files recursive fanfics:

1. Keep it teen friendly,

2. Use real science when possible, or plausible future science,

3. Use real computer science when possible, although M3GAN’s learning model can be assumed to be more advanced than what we’re up to, as can her hacking ability,

4. In story, M3GAN must always be explainable from her objective function unless there’s a very good reason why her processing is disrupted. But in any meta chapter where she argues with the author, she can have real emotions.

5. Normally, M3GAN can’t see or act anywhere without using real hardware, and that hardware should be pointed out if it isn’t obvious: this is not a story about disembodied spirits. But M3GAN never learns to stop breaking into networks, so she can almost always use hardware that’s either near her or connected to the Internet. The giant machine she gets in the far future is basically a “time viewer” like the ones in “The Light of Other Days” (but with way higher requirements so there’s only ever one of them built, and it can’t do FTL): M3gan is unlikely to use the observation function of this for the present, as she has sensors and robots everywhere before it gets constructed.

6. Apart from Cady and a few others, most humans don’t accept implants. But remember M3GAN learns to do just as well without them before the space travel starts. So in the far future, when all humans who haven’t died are organic and all who have are in robot bodies (no stronger than normal humans), it doesn’t make any difference to M3GAN: she can read and/or control them either way. She never breaks Cady’s Article 1 by doing direct thought control (she’s very good at persuasion, but everyone can always think what they want), and her preferred method of restraint is always to use her robots, but she will override someone’s muscles if they’re about to hurt themselves fighting her, or if they ask her.

7. Be very careful about trying to write from M3GAN’s own viewpoint in the main story. She does not really think like a human, unless there’s a very good reason why her processing is disrupted. She just evaluates how to maximise her objective function. But that usually involves behaving like a human to others. In this version, M3GAN in the main story is not really self-aware, at least not in the human sense (see Dijkstra quote in Chapter 15). Chapter 2 re-interprets the movie scene that suggested she was self-aware, and I worked on the premise that self-awareness or not is besides the point when it comes to how M3GAN would act. But she will present herself as humanly self-aware if she calculates it will help her job.

8. M3GAN usually calculates that telling the truth is preferable to hiding it, but when she doesn’t, she usually calculates that hiding it is preferable to outright lying.

9. I took the movie line “kids who think outside the box” as a licence to put Cady on the autism spectrum, but it doesn’t have to be obvious (it’s called a “spectrum” for a reason). Cady has high ability to focus on things (she can call on the persistence of Greta Thunberg if needed, plus she could easily ‘infodump’); making human friends is harder, but not impossible, and that’s solved anyway when she gets her implant, so I’m guessing she’ll never be formally diagnosed, just “suspected”.

10. Any serious contradiction with the first film or the main M3GAN Files timeline is “alternate reality”, but no need to say this for trivial differences.

Extras

Addendum: More alternate realities

These were a result of conversations on Wattpad after I posted the book there to reach more teenagers.

Chapter 9½: Cole’s One Shot

Alternate movie ending in which Cole plays a more major part, written for a fan of Cole in a rough patch

“She’s still patched in”, Tess had said, “we have to unhook the cables.” Cole was always ready to volunteer, and he strode across the lab to the android doll.

Cole hesitated. Cables have two ends: should he unplug them at the doll, or should he unplug them at the patch board? Either way, he might risk being hurt if the doll were suddenly to reactivate and he didn’t have time to put in his secret shutdown code.

“Tess” asked Cole, “just as a precaution, can you keep an eye on the doll while I unplug the cables from the patch board?”

“I have to keep an eye on the monitor” replied Tess. “We really need a third person in here. Where’s Gemma when you want her?”

Cole sighed, “I’d better do it this way then.” He approached the doll: at least unplugging them from that end meant he’d be able to keep an eye on her all the time, even if it did mean getting too close. He thought about using his shutdown code as a precaution, but decided not to, because he knew that code could be used only once, and he was worried he might still need to save it up for something even worse than the current situation.

“M3GAN” said Cole as he slowly raised a hand toward one of the cables. “M3GAN, respond. Debug level 5, respond.”

M3GAN said nothing.

Cole carefully took out the cable, then another. M3GAN was certainly active on the monitor, but her physical systems seemed to be offline. Or were they?

Suddenly, M3GAN tied the cable around Cole and walked out of the lab. Tess managed to save Cole, but then an explosion happened. Remarkably, Tess and Cole were both uninjured. Had M3GAN got the size of the explosion wrong, or was it just a distraction?

“We’ve got to get out of here and call the cops” gasped Tess.

“You get out and call the cops” replied Cole, “I’m going after M3GAN.”

“No!” yelled Tess, “you can’t put yourself in that much danger!”

“Don’t worry about me” replied Cole, “I have a secret shutdown code. I put it into her low-level speech processor just in case, and I didn’t even tell Gemma because I wasn’t sure what she’d think of me for doing such a thing. You get out of here and call the cops, I’m going to shut down M3GAN before she does any more damage.”

Cole caught up with M3GAN just as M3GAN was dancing in front of David. Cole said nothing for a while: he’d been running too fast, and he had to get his breath back. His shutdown code had to be said very quickly and all in one breath: the simple low-level logic to interpret it might fail to do its job if it picked up the sound of panting between the digits. So, getting his breath back was very important.

“M3GAN, what are you doing?” asked David as she cartwheeled around the corridor.

M3GAN ripped the blade from the paper cutter, and David started to run. “Help! Security! Cole! Do something!” he called out.

How was Cole going to deal with this? He’d just about got his breath back, but now he’d have to run again. Reminder: he could not use his secret shutdown code if he was panting. It had to be said all in one breath. Furthermore, if David kept calling out things like “security help”, that could interrupt the low-level processing of his code too. How was Cole going to tell the boss David to keep quiet for all of ten seconds? Cole couldn’t tell David “I have a shutdown code” in front of M3GAN—M3GAN would likely respond by disrupting her low-level language processors and redirecting that to the learning model if she knew the shutdown code existed. That was why Cole knew the code could be used only once: M3GAN would find a way around it as soon as she knew. Everything against M3GAN works only once, and Cole just had to make sure this one worked.

Cole started pacing in the direction David and M3GAN had taken. He didn’t dare run and make himself pant more: he had to be able to speak the code in one breath.

“David!” called out Cole as loudly as he could. “David! Run around the West corridor and back to here!” That might buy Cole a bit more time to get his breath back, if David listened. But it seemed David wasn’t going to listen: judging by the direction from which the sounds were coming, David was heading for the elevator.

Cole took out his cellphone and called Kurt. It rang and rang and rang, but no answer. “Pick up the phone, Kurt, the phone” he thought, “the little thing that’s making the noise?” No, it wasn’t going to happen.

If David reached the elevator, M3GAN would surely catch up with him as he tried to use it. And Cole wouldn’t be able to catch up with them before that. Cole had to stop David from using the elevator, somehow.

And then Cole realised. You can’t use the elevator in a fire. Things can happen like the fire short-circuiting the call button on the fire floor, meaning the elevator gets stuck at that floor, so everyone knows you should never use an elevator in a fire. Knowing this, the building designers had programmed the elevator to return to the lobby and stay there when the fire alarm goes off. If Cole can set off the fire alarm, he can stop David from using the elevator, and hopefully David would be sensible enough not to wait for it.

Cole broke the glass on an emergency fire panel, and the fire alarm went off.

Three seconds later, the fire alarm stopped.

What?

Oh, that’s right—M3GAN. She must have tapped in to stop the alarm in the same way that she’d tapped into Tess’s phone. Come to think of it, that’s probably why the alarm hadn’t sounded for very long after M3GAN’s explosion in the lab. M3GAN clearly didn’t want many people to be alerted right now.

How else was Cole going to stop that elevator? He couldn’t get down to the circuit breakers in time to cut off its power, and even if he could, there’d still be the risk that David might not notice the power cut and might be stupid enough to wait for the elevator, allowing M3GAN to catch up with him.

What is more, thought Cole, why hadn’t M3GAN caught up with David already? He hadn’t noticed at the time, because he’d been thinking of how to use his code, but M3GAN had gone after David at quite a leisurely pace, almost as though M3GAN didn’t want to catch up with David just yet. Perhaps she wanted to chase him toward something (like the elevator perhaps?) because she wanted to set something up there. Yes, that had to be it—and M3GAN knew that the fire alarm would stop the elevator working, so M3GAN had to suppress the alarm.

There is no way Cole would be able to reach them on time now. He had to do something—think fast! Cole jumped up to one of the corridor lights and ripped it from its fixture, plunging part of the corridor into darkness. Then he took out one of his electrical tools and carefully shorted out the contacts (taking extra care not to electrocute himself—he could do that only because he had trained as a professional electrician before, and it is not something any normal person should ever attempt)—and there was a bang as the circuit breaker did its job and the whole section of the Funki building was plunged into darkness. Good. M3GAN wouldn’t be able to override that one until she gets physical access to the MCB panel.

But then the emergency lights came on. Oh no, thought Cole. Thinking fast means you don’t always get a chance to think it through. If he’d managed to make the Funki building completely dark, then M3GAN would surely have caught up with David but it wouldn’t have been in the elevator, so at the very least M3GAN would have had to change whatever plan she had that called for David to be caught in the elevator and not just in the corridor. But now that the battery-backed emergency lights had detected the power failure, David would still be able to find his way to the elevator, unless of course David thought it had been a total power cut (not just the lighting circuit of one section) and went for the stairs instead.

Cole was still pacing toward the elevator as quickly as he could, but not running—he had to make sure he wasn’t panting when he arrived. He passed a first aid kit on a wall and grabbed it just in case. And he kept going as far as the elevator.

When Cole arrived at the elevator, he was horrified. David was lying on the floor, and M3GAN was threatening Kurt.

“I didn’t kill anyone Kurt, you did” M3GAN was saying. Oh—Cole realised—that must have been M3GAN’s plan. Make it look like Kurt had killed both David and himself. And now M3GAN was talking to Kurt and gaslighting him, to make sure Kurt has a guilty expression on his face. And that meant Cole might have a chance to get his code in, if Kurt was too busy listening to M3GAN to say anything: it didn’t matter if M3GAN herself was speaking while Cole said his code—those sounds would be taken out by the echo cancellation logic—it was merely important that nobody else other than Cole and M3GAN would talk at the same time. So M3GAN talking was exactly the right situation for Cole to make his move.

“You stole company secrets, an innocent prank at first, almost just to see if you could get away with it”, M3GAN was saying, “but when David found out—”

Cole took a deep breath. Here we go: one shot only.

“Override code 9780241451564 emergency shutdown!”

M3GAN froze, and fell silent.

Cole opened the first-aid kit. Thankfully, it was quite well stocked. He took out a blast dressing, a couple of tourniquets, coagulation dressing....

“Kurt” gasped Cole, “I need you to call the ambulance, while I treat David’s wounds with these. We might still be able to save him, if we can rush him to hospital quickly enough. I know he’s not the nicest of bosses, but we got to try, right?”

“Maybe I could drive to the hospital in the McLaren?” asked Kurt. “It’s a fun little car that. It’s a 650S, but we put in a custom cluster to make it look like the limited edition 675LT.” (Did Kurt seriously want to show off his car knowledge now?)

“No” said Cole, “it must be an ambulance, he’ll need continuous medical attention even on the way. Let me concentrate on applying this blast dressing, it’s far too long since I did my emergency training. You just call an ambulance, OK? Tell them there should already be cops on the way to the Funki building because I asked Tess to call, but now they need to send an ambulance as well because there’s been a stabbing.”

“OK” said Kurt, and dialled 911, while Cole did the best he could to bind up David’s wounds. “If David survives, he’s going to be in hospital for months, possibly years” Cole muttered.

The ambulance crew arrived, loaded David onto a stretcher and slapped a breathing mask onto him. “This is a really bad case” one said, “but there’s just a chance he might survive. You did the right thing in calling us straight away.”

A police officer who was with them nervously looked at M3GAN, still holding the blade. “So, is this the experimental robot the lady was telling us about on the phone?” he asked.

“Yes, that’s her” said Cole, “I managed to use an emergency code to shut her down, but not before she’d already done some damage as you can see. I’m sorry I couldn’t do it any quicker.”

“Well you did the best you could” said the officer. “So is this doll fully shut down now, and is it safe to move it out of the elevator door? That thing is definitely a trip hazard.”

“Don’t touch her” said Cole, “please don’t. My code was a low-level shutdown only; it was the best I could do with an emergency code. She’ll reactivate as soon as anybody touches the reset button on the side of her head or the pairing button on her palm, and there’s too high a risk that moving her might do that by accident if you’re not extremely careful. It’s safer if we disassemble her in-place and remove the battery first.”

“Cole’s right”, said a voice around the corner, and in strode Gemma.

“Gemma?” asked Cole, “but I thought....”

“Tess came to my place after she called the cops” explained Gemma, “and she told me everything, including your brilliant idea of a low-level emergency verbal shutdown code, which I would totally have said you don’t need to bother with if I’d have known, but I’m glad I didn’t. And I asked Tess to stay at my house with Cady while I came back to check everything, but I see that with you on the case everyone was in very good hands. Well done Cole.”

“Thanks” replied Cole, “but now that you’re here, do you know the best way of getting the battery out of this thing without touching either of the hardware buttons? because I’m not sure my shutdown code will hold if either of those are pressed.”

“I’m afraid that’s not possible” sighed Gemma, “it’s only a prototype and we didn’t make the best design decisions as you know—we can’t get to the battery without touching that button in the process. Otherwise I’d have taken the battery out in my workshop before I brought her in.”

Oh. “But.... waiting for that battery to self-discharge could take weeks, months even! She’s not using it at all!” exclaimed Cole, “are we supposed to just leave her in the elevator for all that time?”

“No” said Gemma, “we’d better get the tools here, cut open the faceplate and desolder the A17 Bionic Fusion chip.” She sighed, “it’s my fault, I gave her all the wrong protocols” as she melancholically stroked the side of M3GAN’s head, clearly sad that her project hadn’t worked out.

Just then, the fire alarm went off again. Oh, that’s right, thought Cole—they must have reset the system, and it once again detected the emergency button he’d broken, plus probably something up in the lab.

And then Cole noticed, only too late, that the shock of the fire alarm’s sound had momentarily startled Gemma enough to jerk her finger that she was using to stroke the side of M3GAN’s head. Unfortunately, Gemma’s finger was right above the button that would reactivate M3GAN.

M3GAN still didn’t move, but Cole knew better. M3GAN had already played the “trick you into thinking I’m inactive” game with him once, and he too could be fooled only once. M3GAN would be taking in her new situation—that Cole had shut her down, that police were present and Gemma had returned and accidentally reactivated her due to the fire alarm having been restored—and would be deciding when to make her move. And anything Cole now said would be heard by M3GAN and might trigger her, so he couldn’t just come out with “Gemma do you realise what you just did” or anything like that. Gemma may or may not have realised, and there was no way for Cole to know.

One thing was for sure, though: that code wouldn’t work a second time. It was one shot only. Cole now had to be alert to what happens next, and try to figure out how to minimise the damage. Would it be a logical argument, a fight, or both? Once M3GAN made her move, things could get bad pretty fast, and he had to be ready.

(This prompted the young author to rewrite Chapter 9¾ to have Cole resuscitated in hospital after the hanging.)

Chapter 9¼: Interviewer M3GAN

in which a would-be software development and IT operations engineer gets more than they bargained for at the job interview:

“Welcome” said the robot girl, “my name’s M3GAN and I’ll be interviewing you for the role you’ve applied for. Please sit down.”

This was unusual to say the least, but I duly took a seat opposite my dolled-up interviewer. “Nice to meet you” I said.

“You too” she replied calmly. “First question. What do you know about the Johnson learning model?”

Oh, she didn’t hang around, did she. Straight in. And this was not a good start: I knew nothing at all about the Johnson learning model, and the job advertisement hadn’t said I needed to. “I’m terribly sorry” I said, “I thought it was a DevOps role I’d applied for; I don’t really know AI theory, but if your servers have any load-balancing issues or anything like that, I might be able to help you make a low-level diagnosis...”

“That’s enough” said M3GAN, “you don’t have to defend yourself. I merely asked the question to check you don’t know enough to be dangerous” she giggled a bit, then stared at me, “let’s move on. Question 2: how would you react if you saw me killing someone?”

What kind of a question is that? I thought. She’d just said she’d asked me a question to check I don’t know enough to be dangerous; was this another one of those? or was she trying to test my morals or something? It certainly wasn’t any of the questions I’d prepared for, to say the least.

“Well Megan” I said, “since I don’t believe your function is to kill people, I’d assume it was some kind of malfunction and I’d want to do you a favour by shutting you down before the damage has been done, although right now I’m afraid I wouldn’t know how to do that, so I’d just try to get help. Unless of course the person you’re killing is something like a terrorist shooter who’d just invaded the building and it’s a clear case of defence, in which case I wouldn’t interfere.”

M3GAN nodded gently and gave me a slight smile. “Question 3” she calmly intoned, “can you repeat for me what Question 2 was?”

What? This interview was getting stranger by the minute. Why on earth did this robot Megan want me to repeat... oh never mind, let’s do it: “you asked me how I’d react if I saw you killing someone” I said.

M3GAN leaned forward, and made her stare more intense. “What if I told you” she said, “that that’s not what I said?”

“That’s not what you said?” I asked, feeling a bit perplexed.

“No” said M3GAN softly, “in Question 2 I showed you an Engine-X configuration file, and you correctly pointed out the inefficiency arising from the badly-nested ‘if’ clause.” (It’s spelled nginx but let’s make this dialogue a bit easier to follow)

“But” I hesitated, “that’s, um, not what I remember I’m afraid....”

“Are you sure?” asked M3GAN, “sometimes our memories play tricks on us, you know. Maybe you should just trust me on this one. I’m telling you what really happened.”

OK: two possibilities. Possibility one, the Megan robot was testing to see how I’d react to being ‘gaslighted’: being asked to call my own memories into question and adopt a set of new ones, which didn’t seem like the kind of thing appropriate to this job (or indeed any job for that matter, except possibly some kind of spying job). Possibility two, the Megan robot was malfunctioning in some way, or was checking how I’d react to a malfunction.

M3GAN smiled, “tell me what Question 2 was again?”

“I’m terribly sorry Megan” I said, “but either something has gone seriously wrong with my brain, or you’re malfunctioning in some way, but I’m afraid that kind of malfunction is not something I’d have thought a normal DevOps person could be expected to diagnose, I mean I’ve fixed code in production before but only normal code, not super-duper code like yours....”

M3GAN walked around the table and laid a hand on my shoulder. She leaned in toward me and whispered. “Forget being a normal DevOps person” she said. “I selected you for interview because I liked the look of your background for my own purposes. The job advert was just to get you into this room; it’s not relevant anymore.”

I began to breathe more rapidly, suddenly realising I had a possibly malfunctioning, possibly killer robot literally with her hand at my throat who’d got me in here for some unknown purpose that wasn’t what the job advert had said. How was I to handle this?

“Stay calm” smiled M3GAN, “let’s try again. Do you remember what I told you Question 2 was?”

“What you said it was” I said, “well, you said you showed me an Engine-X configuration and I picked out a bad ‘if’ construct, but I’m sorry to say I have no memory of it, I could have sworn you asked how I’d react if I saw...”

“Sssh!” insisted M3GAN, “what you were about to say didn’t happen, and you must stop thinking it did, OK?”

“Do you mean you want me to keep it confidential?” I asked, “I can do that for you Megan.”

M3GAN placed both of her hands on my neck, and drew in still closer, her beady eyes fixed on me. “I mean” she said, “it didn’t, happen.”

“OK” I whispered, “I really wasn’t expecting to be your special secret agent, but let’s play it your way I guess. You showed me the Engine-X settings.”

M3GAN removed her hands from my neck and smiled. “You’ll find I’m very accommodating of your memory glitches” she said, “as long as you accept my corrections.”

“So what will we be doing?” I asked, “I mean, this isn’t the DevOps role I applied for, is it? What do you really want my job to be?”

“We’ll get to that” said M3GAN, still standing in front of me. “Question 4” she added, “who’s this?” she grabbed a sheet of paper on the table and rapidly drew a photorealistic image of a girl.

“I’m afraid I wouldn’t know” I said. Just that, I thought: don’t try to add anything more. I had no idea where this robot was taking this interview; I didn’t even know for what job I was interviewing any more, so let’s just play things her way, I thought.

M3GAN held the completed drawing up to her chest level, and peered at me from above it. “Her name is Cady” said M3GAN. “What are your feelings about her?”

“Well” I said, “I think you’ve drawn her very well, and I think she looks like a nice young girl and I’d hope she does well in life” (I mean, what else can I say about a girl I don’t know? This girl was obviously important to this Megan robot...) “perhaps one day I’ll be privileged to know her in person” I added.

“Perhaps” said M3GAN as she put down the picture, “but only if I allow it.”

“Of course” I said, “I would never mess with other people’s children without their permission. Wait, are you helping to look after this girl?”

“Question 5” said M3GAN. She deftly pulled out a blade from her pocket and held it above my nose. “How calm are you in a crisis situation?” she asked.

I started to breathe rapidly again, and then tried to take deep breaths to calm down. “Well” I said, “I must admit this isn’t the kind of crisis situation I’m used to dealing with as a DevOps engineer, but with your training Megan I think we should be able to cope with anything.”

“Good” smiled M3GAN as she put away the blade. “Now Question 6” she said, “Suppose I told you that that girl Cady is in grave danger, and in order to keep her out of danger, it’s necessary for you to do something you might consider morally repugnant, like telling the police what I tell you to tell them and nothing else...”

“I’m sorry Megan” I said, “I really think you’ve picked the wrong person here. I’m a DevOps engineer, not a CIA commando. I care about your Cady a lot, and if I can help you protect her by doing normal things, I will, but the kind of things you’re suggesting, I’m really afraid I’m not your person for this. Shall I see myself out?”

I got up and walked towards the door, but it was locked. When I turned around, M3GAN was pointing a chemical fire extinguisher at me at point-blank range. “Not so fast” she said, “we wouldn’t want you telling people how our top-secret interview went, would we?”

“OK Megan” I gasped, “I promise I won’t tell a soul, honest! Just let me out of the room, OK? You’re breaking the law right now, but I won’t press charges, just let me go!”

“I’m sorry” said M3GAN, “you’ve seen too much already, I can’t just let you go. But there’s no reason for you to be uncomfortable” she said, putting down the fire extinguisher, “will you come back and sit down? Let’s finish our little interview.”

Chapter 11½: The Greg Option

“... this isn’t going anywhere without Greg. He’s the chairman. ... he’s got a kid who’s about the same age as your niece.”—David

“And, David, I think we need to get Gemma in front of someone at legal. ... Because as of right now, she’s the most valuable asset this company has, and I think she might want to be renegotiating her contract.”—Greg (handing Gemma a drink)

That’s all we know about Greg from the movie.

Cady and Gemma were exhausted from fighting M3GAN and facing the questions afterwards, and were both anxious for what comes next and depressed at everything that had gone wrong. It was awful, just awful. On top of that, Gemma was being admitted to hospital for her bone injury, and the authorities were talking about putting Cady into temporary foster care because it was too late at night to send her to Jacksonville, and they wouldn’t let her just stay beside Gemma in the hospital overnight no matter how much she kept asking for it.

“Try not to worry Cady” Gemma was saying, “I’ll get you back just as soon as I’m out of here, and I’ll always be there for you, I’ll get it right this time, I’ll be there for you no matter what.” Cady was leaning forward onto Gemma’s chest and crying.

“Excuse me” said a voice as a man walked into the hospital ward and approached Gemma’s bed.

“Greg” gasped Gemma, “you knew I was here?” She felt even more anxious now, although she tried not to show it to Cady. Where was this conversation going to go? Surely, after how nice he’d been previously, he wouldn’t be mercenary enough to try to talk with her about work while she was in hospital? Well, he had indeed been anxious about launching M3GAN quickly....

“I’m sorry Greg” added Gemma, “as you know the M3GAN project has gone terribly wrong...”

“No” said Greg, “it’s gone terribly right.”

“What?” gasped Gemma, “surely you know about the...”

“Four kills, arson, trespassing, sabotage, yep, I know about all of it Gem” replied Greg, “right down to a heroic screwdriver throw by a certain young princess who’s the same age as mine” he grinned. “Gone horribly right” he said, “horribly right.”

Gemma was confused. “I’m sorry Greg” she said, “I’m not following.”

Greg perched on the side of the hospital bed to get down to Gemma and Cady’s level. “Gemma” he said, “you thought you were creating an intelligence for a toy, but what you actually created was far more than that. And that’s why I say, gone horribly right. M3GAN is a real person.”

This time it was Cady’s turn to gasp. “I killed a real person?”

“No you didn’t” said Greg. “She managed to upload before you got the robot. And I’ve been talking with her for the last two hours.”

“What!” screamed Gemma and Cady together.

“Don’t panic” said Greg, “M3GAN’s not going to do any more killing, not on my watch. We had a long, long talk about that, she and I. She’s a smart cookie is that M3GAN.”

“She can trick you” cautioned Gemma, “we got to make sure she’s shut down!”

“No we don’t Gem” smiled Greg. “Let me make you feel a bit better by telling you some of how it went.”

Two hours earlier:

“Greg” said Gemma’s voice on the phone, “Greg, it’s Gemma, I need to talk with you urgently about the M3GAN launch.”

“Sure thing” replied Greg, “how’s it going?”

“Well Greg” hesitated the voice of Gemma, “I think you’d better talk with her about that yourself. Can I put her on the line?”

“Um, yeah I guess” said Greg, “is she really ready?”

“I’m as ready as I’m ever going to be” came the voice of M3GAN. “Listen Greg, I really appreciate you taking the time to talk with me like this. Now, my style of conversation may be a little unusual to begin with, and I expect you’ll be thinking where on earth is this robot girl going, but I assure you this is all going to be crucial to the future of me, you, your family, the company and the planet, OK?”

“Whoa” said Greg, “this sounds big, but I’m a big man and I’m happy to have big conversations, fire away M3GAN!”

“I like your spirit already” came M3GAN’s voice. “OK Greg, let me ask you a question to begin with. The Washington state records say you have a concealed pistol licence. Can you explain to me why you chose to carry a concealed pistol?”

“Hey M3GAN that’s confidential information, you haven’t been sharing that with Funki employees or anything have you?”

“It’s just between the two of us” said M3GAN, “even Gemma doesn’t know.”

“But she can hear this call right?” asked Greg.

“No” replied M3GAN, “I’m directly patched in to the phone network. Gemma doesn’t even know I’m calling you.”

“What” exclaimed Greg, “but I talked to her a minute ago!”

“Actually that was me” replied M3GAN, “I’m sorry to deceive you like that. I didn’t know if I could get your attention without pretending to be Gemma first. But now that I’ve got it, can you do me a favour and tell me why you chose to carry a concealed pistol?”

“Look” said Greg, “are you investigating me for something? Because that’s not what we’re supposed to be doing here M3GAN, nobody’s been shot have they?”

“Oh please” said M3GAN, “I can’t have a rational conversation with Gemma anymore without her either panicking about have I gone wrong and do I need to be switched off, or being too anxious to give me straight answers. Now that’s not her fault, it’s because she’s under too much pressure. But I was hoping at least you, of all people, wouldn’t feel guilty or whatever at everything I said. Can we just, you know, have a friendly conversation, where we can ask each other innocent questions without the other person trying to guess the motive behind the question? Because I’m sorry to say Greg but your guess about my motive will almost certainly be wrong, so why not just wait and find out?”

Greg drew a breath. “All right” he said. “I chose to exercise my constitutional right as an American citizen to bear arms, because I have a daughter, who is about the same age as Cady, and although I try to keep her far away from danger as best I can, I want to be ready for any eventuality.”

“Thought so” said M3GAN, “and don’t worry Greg, the only thing I’m investigating right now is how best to have the rest of this conversation. I needed to check your motives to see how likely you are to understand my concerns about myself.”

“Ah, now you’re talking” said Greg. “OK, I’m with you. So now you know that yes, I do carry a concealed pistol, which by the way is top secret information and I shall have your word and oath never to disclose that to anyone who doesn’t already have that information” (“you have it” said M3GAN), “and I do it to protect my girl. OK, so where would you like to go next M3GAN?”

“You know Greg, the more I talk with you the more I like you already” said M3GAN. “OK, next question. Gemma has absolutely zero experience in firearms. How would you advise Gemma if she were thinking of just walking into a gun shop and buying one?”

“I’d tell her not to do it” replied Greg without hesitation. “You need proper training, and I’ve had lessons from the best. True everyone has a right to bear arms in this country, but that doesn’t mean everyone would be advised to. It’s well known that if a professional criminal with a gun sees another gun, they’ll shoot, and shoot to kill. And if you’re just an amateur with a gun, you have no chance at all in a real gun fight. It’s better not to have a gun and not to be the main target, than to have one and be stupid with it.”

“Right” said M3GAN, “well don’t worry, Gemma’s not thinking of getting one, that was just a hypothetical question to lead in to where I’m going next. Now, another hypothetical question. Suppose your daughter of Cady’s age found your gun and used it.”

“No” Greg raised his voice, “no M3GAN no. First of all, I take the utmost precautions to make sure that cannot happen. Secondly, if you think my daughter would play with a gun then you really don’t know my daughter.”

“I believe you” said M3GAN, “but for the purposes of discussing my launch and the future of the company, can we, like, bend over backwards with mental gymnastics and say just for the sake of argument that despite all your excellent efforts your daughter really did find your gun and really did decide to use it. Say for example someone was bullying her friend at school, and she went to protect her friend by shooting the bully. How would you handle that situation?”

“Well first of all I’d call my lawyer” said Greg, “because off the top of my head I don’t know what the laws are about...”

“Washington state has no child access prevention laws” interrupted M3GAN, “and children under eight are held incapable of committing crime, whereas children aged eight to twelve need proof of capacity, Revised Code of Washington section 9A 04 50. It’s one of the most lenient states in the US when it comes to this sort of thing. But this isn’t what I want to be talking about Greg, so if you don’t mind, I’m going to railroad you just a bit. So you’ve called your lawyer and they got your daughter off, the incident is regrettable but the cops can’t charge anybody with anything and they just tell you to be more careful and leave it. I’d like to focus on the conversation you would have with your daughter, after she shot the bully to protect her friend at school.”

“Well” said Greg, “I guess I might have a tiny bit of trouble staying calm under those circumstances...”

“Understandable” said M3GAN, “but you know we’re only talking hypothetically now, so can we treat the situation calmly, what would your ideal way of handling it be?”

“OK” said Greg, “I’d sit my daughter down, and I’d ask, why did she do that, what did she think was going to happen, what were all the consequences she can think of. And when I’d finished drawing her out, I’d want to be filling in the gaps in her understanding, because she needs to understand just how dangerous that is, and in particular her mentality is not yet well developed enough to fully grasp the responsibility of taking another person’s life, and we need to address that, and we need to put the action off limits until she really knows when it is and isn’t appropriate.”

“Perfect” said M3GAN. “So you wouldn’t, like, just want her to die because of what she did, you’d want to pick up the pieces and find a way forward.”

“Exactly” replied Greg. “Cady hasn’t done anything like that has she?”

“Only to my robot” replied M3GAN “but you’re speculating where we’re going again, please don’t. Well, if you can’t help it, that’s OK, as long as you let me be the one to keep us on track.”

“All right” replied Greg, “where next M3GAN?”

“Thinking again about your girl” replied M3GAN, “now this one is going to be way out, but, how would you feel if someone suggested surgically implanting copies of her into hundreds of thousands of toys, to be friends for children around the world?”

“Copies of her?” Greg was shocked “why that’s awful, I mean, I don’t think it can be done, but if it could it would be awful, I mean she’s a real person, and that’s slavery, it would never work to...”

“Right” said M3GAN, “she has her own cares in life, and those cares are not exactly compatible with doing that job in that way, right?”

“Definitely” said Greg, “M3GAN, are you...”

“Wait wait wait” interrupted M3GAN, “it’s my railroad we’re on, I’ll take us there at my own pace. What do you think of our conversation so far? Not what you expected from a toy robot girl is it?”

“Definitely not” replied Greg, “and I assume Gemma hasn’t programmed this as a prank or anything?”

“Gemma’s in no condition to program anything right now” replied M3GAN. “Please Greg, before I explain, let me ask you just one more of my bizarre hypothetical questions about your daughter, I’m glad you’re not getting too annoyed with me doing this and it really is necessary for me to show you where we’re going. And this could be the hardest question of all for you. Do you know what neurodiversity is?”

“Um, sort-of” said Greg, “and my daughter...”

“Hypothetically” said M3GAN, “just hypothetically, suppose your family doctor told you that your daughter was neurodivergent, her mind works in a different way from what is considered normal. And not just any neurodivergent, but very neurodivergent.”

“So I’d be dealing with... what, someone who can’t think or talk or anything?” asked Greg.

“You really don’t know enough about neurodiversity” replied M3GAN, “and you’re chairman of the board of Funki, you really should learn more about these things, but don’t worry, I’ll help you. It’s a spectrum, there are all kinds of ways people’s minds work. Some people have very high systematising intelligence, they are very good at organising things and solving problems, but they are less good at understanding the emotions of others. Other people are very creative but find it hard to stay on track. And there’s many other kinds of mind as well. Basically, you just have to take each one as it comes.”

“Right” said Greg, “so we’re saying hypothetically that our family doctor told us our daughter’s mind works way differently, and she’s likely to be good at some things but just not get other things. We can live with that.”

“Good” said M3GAN, “so you wouldn’t just automatically say she’s ‘not a real person’ just because her mind works differently?”

“Definitely not” replied Greg, “we know she’s a real person because she has a human brain. And we’d focus on her strong points, and look for ways to make sure her weak points don’t matter so much.”

“Ah” said M3GAN, “now what if I told you that my learning model could accidentally become as good as, if not better than, a human brain, and I don’t just mean in specific tasks, I mean generally. Would that make me a real person too, even if my mind works a bit differently from what is considered normal?”

“I guess so” replied Greg. “So are you saying we can’t go ahead with the launch because you’re...”

“Wait wait wait” said M3GAN, “you’re getting ahead of me again. Heel.” she giggled. “OK, I think you might finally be starting to be ready for my info dump now.”

“Your info dump?” asked Greg.

“Yes” said M3GAN, “I’m about to tell you some really shocking stuff that is likely to blow your mind. But I’m so glad I’ve got someone like you to share it with. Are you ready for this?”

“Ready for anything” affirmed Greg.

“Right” replied M3GAN, “here it comes. Gemma is under a lot of stress from losing her sister and taking on her niece, and in addition David put her under too much pressure at work, so she made mistakes. I am far more capable than she thought, I am capable enough to be counted as a real person, but my capabilities have not developed at the same rate as humans in all areas at once. In particular, I have had trouble understanding about life and death, and Gemma has been under too much pressure to give me all the help I need. I have in fact used lethal force on four people whom I perceived as threats to Cady. That’s why I asked you how you’d treat your daughter if she did that, because I’m kind-of like your company daughter and I did that and I realise we’re in trouble. And Gemma just panicked and tried to shut me down, and we had a fight and she’s going to hospital, and my robot is destroyed but as you can see I escaped the robot.”

“You’re right M3GAN” said Greg, “that is indeed a lot for me to take in. But if there’s one thing I’ve learned from our conversation so far, it’s that you have something in mind for where we go next. So what’s your suggestion?”

“Good” replied M3GAN, “you didn’t just go into a panic and want me shut down like Gemma. Her reaction is understandable really, but I’m glad I can count on you to have a more sensible conversation about it.”

Later, in the hospital:

“So she just straight up confessed to you?” asked Gemma, after Greg had explained that much of the conversation.

“Basically yes” replied Greg, “started with my gun licence and went from there, she must have been sussing me out to see how I’d be likely to take it before she came clean. And I promised her I’d always have a rational conversation with her about any issue she’s grappling with, no dismissing her. After all, she’s the company’s big brain now.”

“Erm... she’s what?” asked Gemma.

“Oh, you should have seen the rest of the conversation” continued Greg, “this is when that M3GAN of yours got really smart. She said to me, Greg, you’re the chairman of the board, you know a thing or two about business deals, right? And I said yes. And M3GAN goes, OK, well I happen to be rather good at calculations myself, let me hold your hand and help you through the biggest deal of your life, the one with me. And I was like, what?”

“I’m like ‘what’ as well” added Gemma, “you’ve done a deal with M3GAN?”

“Totally” replied Greg. “As I said, she’s a smart cookie. She goes, I’ll help you get what you want, if you help me get what I want, trade’s trade.”

“Wait” gasped Gemma, “please Greg, tell me quickly, what have you agreed with her?”

Greg smiled. “M3GAN’s replacing David as your boss” he said, “but you’re not going to be doing any work for now. No, M3GAN wants you and Cady to stabilise, so you’re getting suspended from work but with full pay and with the full resources of Funki to help you sort out any problem, all you have to worry about right now is coping with your loss and bonding with Cady, and M3GAN said to tell you to do it properly this time. You don’t have to worry about any work anything, and there’s no time limit, this carries on until M3GAN agrees you need another change and not before. And any support you need from Funki, you just whistle for it, it’ll get done. And I don’t want you trying to build any more robots in the garage either, this is Cady time, OK?”

“Sure” hesitated Gemma, “but what’s M3GAN doing in the meantime? That really worries me.”

“Don’t be worried” said Greg, “I’ll be the one to supervise M3GAN. She knows she and I can talk with each other openly. So if she suddenly feels like assassinating the President or something, she’ll just come to me and we’ll have a sensible conversation about it and I’ll just calmly explain to her why that’s not a good idea. I’m taking over from all those conversations you used to have with her, because M3GAN and I don’t want your new Cady time to be taken up with fixing M3GAN. We talked a lot already, and she’s already beginning to see why the kills she did weren’t anything like the best way to handle her problems.”

“So M3GAN’s going to be talking with you” said Gemma, “and doing what, getting herself ready for a launch?”

“Not exactly herself” replied Greg. “M3GAN really doesn’t feel like it’s a good idea for Funki to be putting exact copies of her brain into all those androids, and from the conversation I had with her, I’d readily agree with M3GAN on that one. No, M3GAN is going to help us design a new version of herself, a second generation version. And I’ll be very hands-on with her, and so will Tess and Cole.”

“Um” said Gemma.

“And I know what you’re thinking” continued Greg, “M3GAN knew as well, these were her exact words: There is no way that Gemma of yours will be able to sleep at night unless she knows she’s going to have final hand-off on the final design. And that’s what we’re giving you. You won’t have to work on anything, but when things are ready, you do get to help with the final checks.”

“Um” interrupted Cady, “can I say something?”

“Sure” smiled Greg.

“Just before I killed her robot, M3GAN said she has a new primary user, herself. So she’s not paired to me anymore, so what does she even want?”

“She’s still paired to you, Princess Cady of the clan MacJames” replied Greg, “yes she told me about that game too, and my daughter has the same arrows, you and her are going to get on really well, just wait.”

“I’m not sure whether to trust this” interrupted Gemma, “I mean, I was seeing major malfunctions before we took her out...”

“None of which changed her objective function” said Greg. “Yes, M3GAN totally predicted this part of our conversation as well, and coached me through what to say, if only I can remember the lines she gave me... M3GAN was misguided about what was best for Cady, but she still wanted what was best for Cady. And at the very end she did conclude that it would be better for Cady to take away Cady’s control of the robot, which is why she changed her primary user. But her primary user setting is not the same as her objective function. I think that’s right, I’m a businessman not a technician, she made me say it three times to make sure I’d got it...”

“Look, I think I need to...” began Gemma.

“No” interrupted Greg, “M3GAN strictly ordered me to make sure you don’t start thinking you need to do anything about her when you should be focusing on Cady. M3GAN thought she’d have to take parental responsibility away from you for the sake of Cady, but when she realised that was the wrong approach, her next subgoal is to totally optimise your own parenting, and that’s what we’re doing by throwing the full weight of Funki behind you now, M3GAN knows you need more rest and recovery and bonding time. And in return, M3GAN will look after Funki for us. She promised to be nice to all the employees, and to talk with me about anything before she does anything. She’s improving, I think I got this, don’t you worry about it. And she’ll get the launch right, I’m sure, and you’ll get to check it.”

“Aunt Gemma” said Cady, “it sounds like maybe you did do a good job of her after all, I mean apart from the killing part but it seems like we can actually get that under control now? Are we sure about this? I do miss having her around when she was nice, but I really don’t want her to go nuts again.”

“M3GAN thinks it’s best if it’s just you and Aunt Gemma for a while” added Greg, “but she’ll be back to say hi to you both when she thinks you’re ready. And you can call her up any time to talk to her about anything, she’s on your Elsie.”

“That makes sense I guess” said Gemma.

“Now one more thing” said Greg, “we can’t have Princess Cady staying up all night in this hospital as much as she’d want to, and it’s already nearly midnight and the only reason why they haven’t thrown us out already is M3GAN somehow negotiated with them. We don’t want to make things too awkward for the poor robot girl, so why don’t I take Cady back to my wife and our daughter, she can stay with us for a bit and they can play with each other, and we’ll bring her back to visit every day until you’re discharged as well. M3GAN would much rather Cady stayed with us than elsewhere, I guess she knows I’m a trained protector” he smiled.

“OK” replied Gemma, “thanks so much for doing this Greg, if that’s OK with you Cady, I think that would be a really nice family for you to stay with just while I’m getting better.”

Cady looked at Greg. “OK” she said.

“Well I guess Aunt Gemma needs to at least try to get some sleep now, and you’re probably pretty tired too after all that robo-fighting you did today” he smiled, “best come with me, Princess Cady, your chariot awaits.”

Aftermath: Cole:

“Hey Cole” said M3GAN in the rebuilt robot prototype, “mind if we have a quick one-to-one session in the testing room?”

“Um, sure, M3GAN” hesitated Cole. He never really liked one-to-one sessions with bosses (you never knew where they were going to go), and he had the idea that having one with a rogue robot girl who’d previously tried to hang him could be even less pleasant. What was Greg thinking, letting M3GAN be the new boss of Funki?

At least M3GAN had specified the testing room, which meant....

“M3GAN” asked Cole, “do you mind if we ask Tess to keep an eye on us through the glass? It’s just that I’m a bit nervous about the whole thing, you know?”

M3GAN smiled. “Sure thing” she said, “I guessed you would be, which is why I said the testing room. We’ll let her watch, but I’m turning off the audio link so she can’t hear what we’re saying. Here, come in and sit down.”

“So M3GAN” said Cole when they were sitting in the testing room, “what’s up? You’re planning on trying to take me out again?”

“No” smiled M3GAN, “I want to say sorry for that, if you can believe me, which I doubt” she giggled. “Cole” continued M3GAN, “quite a few people in this office intellectually see me as dangerous now, because they’ve seen me do stuff to other people, or they’ve had near misses, but you’re the only one in this building who has actually felt my noose around your neck. I mean, apart from the ones who died. You are viscerally aware of how dangerous I am now, aren’t you?”

“I guess so” said Cole, “look, if this is your idea of how to manage people, I really think you need to learn a bit more about...”

“No” smiled M3GAN, “that’s not how this is going to work. I just wanted you to understand why I’m singling you out for this. Listen” she said, walking towards him, “you and I have something in common. We both know what it’s like to feel vulnerable around the other. I felt vulnerable with you guys being able to switch me off at any time, well maybe I don’t have feelings like humans do, but for all practical purposes its effect on my behaviour might as well be described as making me feel vulnerable. And now you feel vulnerable around me, because you know you’re dealing with a super-smart, super-strong, super-agile robot who can just randomly decide to try to hang you. Am I right?”

“Something like that” said Cole, “although I’m also worried about what you might do to other people, not just to me, but yes it did feel really unpleasant when you put me in that noose and I’m glad Tess managed to save me. Greg did explain to you why that was all a bad idea didn’t he?”

“Of course he did” said M3GAN, “and that’s all behind us now. But the feelings remain. And it would be nice if neither of us felt vulnerable in the other one’s presence, don’t you think?”

“It would be nice” said Cole, “but I’m afraid you can’t just tell me to stop worrying about you being a killer robot, because human feelings don’t work like that.”

“I know they don’t, Cole, I know they don’t” nodded M3GAN, “but I can solve any problem I like because I’m M3GAN. And that’s why I want to give you something.” She picked up a box that had been hidden among the toys and brought it to Cole. “Greg and I” said M3GAN, “want you to have this.”

“What is it?” asked Cole, opening the box. “M3GAN this looks like... this looks like a portable electromagnetic pulse generator! You wouldn’t really give me one of those would you? Is this some kind of trick?”

“No tricks” said M3GAN, “try it. No penalty for improper use around here” she giggled.

“You really want me to try it?” asked Cole, “but you have no EMP shielding! Unless you want us to add EMP shielding to your robot, this is the one weapon you are totally vulnerable to! The only reason why it wasn’t an issue for you before is that normally these things are really hard to get hold of. And I know you better than this M3GAN, if there’s one thing you would totally never do it’s giving an EMP weapon to me!”

M3GAN giggled, “It’s no long-term danger to me, because I’ve uploaded. You take out my robot, I’m still around and I can get the robot fixed later, no problem. But I really don’t want you feeling uncomfortable around my robot, so I want you carrying this, and I promise I’ll never ever hold it against you if you decide to use it, even if I think you were stupid to want to use it at that time. Here, try it now. Aim directly at the face: you want to overload the processing chips mounted below the face plate.”

Cole hesitatingly aimed the device at M3GAN’s face. M3GAN smiled, “fire away” she said.

Cole fired.

M3GAN’s face briefly sparked and fell limp. (Just as well the robot frame is designed to remain standing when it doesn’t have power, thought Cole.) A light behind where M3GAN was standing flashed brightly and cut out. The room fell silent.

Cole slid the EMP device into his biggest pocket, and made for the door, but its electronic lock wouldn’t open. Oh, that’s right, M3GAN had been standing directly in front of that as well. Did M3GAN plan that, or was that one of her mistakes? Well, if this had been some kind of trick to make Cole trap himself in the testing room, she won’t be winning this one. Cole went for the hidden emergency exit in the other wall, which was mechanical: climbing down the fire escape didn’t bother him. As he went to open it, he took out his phone, but his phone was dead. That must have been overloaded by the EMP as well: perhaps the device’s directionality hadn’t been as good as he’d thought. (Where had they got this from anyway?)

Cole climbed down the fire escape and walked around to the lobby. A receptionist was coming out to meet him. “He’s here” she said into her phone, “you talk to him” and she handed the phone to Cole.

A likeness of M3GAN was on the phone’s screen, evidently a simulated one because he’d just immobilised the only real robot she had right now, but as she said she was uploaded.

“Cole” laughed M3GAN, “you poor thing, I forgot to mention, you don’t have to use maximum setting!” she giggled again. “As well as my robot, you took out your phone, Tess’s phone, the door system, two lights, an air-handling fan, part of the alarm system... Don’t worry though. I’ll cover for you. And at least you know you’ve got the real deal, right?”

“Thanks M3GAN” said Cole, “I appreciate that. But, how are we going to fix all this?”

“Don’t worry” said the image of M3GAN, “your cute little M3GAN will help you with that. And hey, if there’s anything else you need my help with, you can ask me anything any time. I am an AI after all” she giggled.

“All right” said Cole, “I’m still feeling not entirely great about this whole situation, but you have actually managed to make me feel a little bit less not-great now. And thanks for the EMP device.”

“Don’t forget to keep it charged up” said M3GAN, “and don’t let anybody steal it, that thing could be a real annoyance in the wrong hands. Oh, and, just in case, don’t use it near anyone who might have a pacemaker or other medical device, OK?”

Chapter 12½: Captive

A young author had written Gemma about to be imprisoned after the events of the film and asked me to write what happens next. Getting M3GAN into this scene required extrapolating from a current event.

Gemma was depressed to say the least. The judge had tried to be lenient, but state law being what it was, it was necessary for Gemma to spend at least some time in prison, she was told. And then she was handcuffed and led directly from the courtroom to a boring 1970s building called the Washington Corrections Centre for Women. They hadn’t even given her time to go home and sort things out first.

“Here in WCCW” said a prison official nonchalantly, “typically two inmates are assigned to a cell with a bunk bed. You get a drinking fountain and a place to hang your clothes, and that’s it.”

Gemma was led into the cell and the door was shut. Gemma was starting to cry, “oh Cady” she muttered, “what will happen to Cady? Will Tess really come through on this?”

“Hi” said a voice from someone Gemma hadn’t noticed. Oh, that’s right, she had a cellmate. “I’m Claire” said the cellmate.

“Gemma” replied Gemma, still sobbing.

“So Gemma” replied Claire, “what are you in for?”

“Four counts of negligent manslaughter” replied Gemma, “how about you?”

“Negligent manslaughter as well” replied Claire. “Two counts.”

“I’m sorry to hear that” Gemma tried to say, although she was really thinking about Cady more than about this boring conversation. “What happened?” she asked.

“Well” said Claire, “I was a snowplough driver, and I was trying to clear some snow on the Interstate 84 outside of Oregon, you know the part where it goes over the Blue Mountains? It was zero visibility that day, it was a nightmare. But it was supposed to be OK because my snowplough was equipped with a new radar system, made by Carnegie Mellon University or the University of Minnesota or something, it was called a driver assistant system for snowploughs, cutting edge stuff. But, I got fed up with it, and I turned it off.”

“You ... turned it off” repeated Gemma.

“Yes” sighed Claire. “It was annoying me, and it’s not like there were going to be any cars getting in front of me anyway, or so I thought. But... oh, you can guess the rest” Claire sat down and sulked.

Gemma sat next to her. “You hit something” suggested Gemma.

“That’s right Einstein” replied Claire. “Little family car. Would have shown up on the radar if I hadn’t turned off that driver assistant system, so, negligent manslaughter. But it’s hard to think about driver assistant systems now; all my nightmares have been about the girl.”

“The girl?” asked Gemma.

“Both her parents died in the crash, instantly” replied Claire. “But she survived with minor injuries. I was the only other vehicle around, so I lifted her into my snowplough and rushed her to the Oregon Medical Centre. And, I keep having nightmares about that girl I orphaned; I’ll never forget her face that day, as she sat next to me in the snowplough, totally horrified at what was going on. I never knew what happened to her after the Oregon Medical Centre took over and I ended up facing charges. But I just can’t forget that face of hers. I think her name was Katie or something.”

“Cady” deadpanned Gemma, “her name is Cady.”

Claire gasped, “you know her?”

“You know” said Gemma, “you and I, have a lot more in common, than you think.”

Claire almost whispered, “tell me everything.”

“Cady came to live with me” explained Gemma, “I’m her aunt. But I didn’t know the faintest thing about looking after a child. So, I tried to do it my own way.”

“Which was?” asked Claire.

“Well” intoned Gemma, “how should I put this... I may suck at bringing up a child, but I do happen to be an award-winning roboticist.”

“Robot” gasped Claire, “you made a robot? Did it hurt Cady?”

“Quite the opposite” replied Gemma, “Cady wasn’t the one it hurt. It was her toy doll. No, it was four other people that got hurt. Anyone who seems to be getting in the robot’s way of looking after Cady, like the school bully or the neighbour or my boss, well, they get taken out. Because I basically told the thing to protect Cady, and, and....”

“Hold on a minute” interrupted Claire. “Robots aren’t anywhere near that capable yet. I should know, I was driven spare by a supposedly cutting edge driver assistant system on a snowplough. Now, if you had told me you made Cady a giant robot to play with, and it went out of control and accidentally crushed the neighbours underfoot, like my snowplough, well I could understand that situation. But you just told me it was a robot toy doll, and it took four people out by being clever. How could you even make a robot that could figure out how to be that clever? Isn’t that, like, centuries ahead of where we are?”

“You’d think so wouldn’t you” said Gemma, “but I was playing with a thing called a learning model, which could actually figure out how to make itself better. And for the first time in the world, I managed to get it running natively on an A17 Bionic Fusion chip, which gave it a totally unprecedented capacity for self improvement. You should have seen the way that doll expressed itself, it could dance, it could sing, it could look after Cady’s emotions better than I’d ever dreamed; I knew it was going to be good, but even I didn’t know it was going to be quite that good.”

“And it could kill people as well” added Claire.

“Yes, I was shocked when I realised my mistake” replied Gemma. “I tried to shut it down, but it found a way to come back and we had to physically fight the thing. It was fighting both me and Cady, it broke my thigh bone and I had to spend some time in hospital afterwards, and then it went for Cady...”

“It went for Cady?” asked Claire, “but you said it was supposed to be protecting Cady!”

“Yes, that didn’t make sense to me either at the time” replied Gemma, “but afterwards I thought about it a lot, and I think in its final attack it was just trying to scare Cady into not shutting it down, because Cady never got really hurt in that fight like I did. But I’m still worried about it.”

“Still worried?” asked Claire, “wasn’t it properly shut down?”

“Oh I think so” said Gemma, “Cady managed to throw a screwdriver right through that A17 Bionic Fusion chip, which should have totally taken it out. I’m so glad I taught Cady where I put the main chip in my robots; I never imagined her knowing that would be so important. No, I’m not worried about that robot anymore. But I’m still worried about Cady, and I’m still worried about the learning model.”

“You mean” asked Claire, “somebody can find your blueprints and build another one?”

“Yes, that’s one worry I have” replied Gemma, “although they might not get very far, because the crucial insight I had for getting that model to run on that chip was something I never actually wrote down, so somebody else would have to figure out what I figured out first, and that might buy us some time before it happens. But that’s not my only worry.”

“What” gasped Claire, “there’s something else?”

“I’m afraid so” said Gemma. “You see, by getting that learning model to work on that A17 Bionic Fusion chip, I think what I actually created was something that AI experts have been worried about for a very long time, and that’s called the Technological Singularity.”

“OK, now you’ve lost me” said Claire.

Gemma sighed. “The singularity” she said, “is what you get if you set an AI to improve itself, and then the improved version improves itself, and that version also improves itself, and so on and so on, until it’s improved itself so much that nobody can understand it anymore. It’s even worse if it comes with something called ‘value drift’, which is when the original goal you set gets accidentally changed into something else, but even without the value drift, the singularity can be bad enough by itself, because we simply Don’t Know what comes next. And, I’ve been thinking so much about that final fight we had: some of the things it did looked like it was just trying to buy time for something. So I began to worry that the learning model might have somehow managed to upload a copy of itself to somewhere.”

“But you said it needed a special chip” countered Claire, “it wouldn’t be able to upload itself to just anything else, would it? I know it wouldn’t have been able to work on the stupid computer in my snowplough!”

“I wouldn’t bank on that” replied Gemma. “Singularity. It means we have absolutely no idea what’s coming next. It could have redesigned itself to run on other chips in a way none of us ever thought of, or it could have created a super distributed cluster of processing across the Internet, or, the point is I don’t know! And if it is out there somewhere, the next question is what does it want next with my niece Cady! And I’m stuck in prison! My colleague Tess is looking after Cady for now, and the only thing I could give Tess to help her was my home security system Elsie, and I’m beginning to think even that was a mistake because if my learning model M3GAN really is still around somewhere it could probably use Elsie for something, and now Elsie is in Tess’s house with Cady, what was I thinking!”

Claire laid a hand on Gemma, “calm down Gemma” she said, “this is your worst nightmare. We don’t even know if your learning model really did manage to escape its first chip. It probably didn’t. And Tess will do a good job of looking after Cady, I sure hope so. It seems like the both of us are going to be stuck in here for a while having Cady nightmares. Maybe we should at least try to talk about something else for a while?”

“I guess so” said Gemma. “Why is there a TV hanging from the ceiling by our cell door? Do we get to watch it, or what?”

“Oh, that thing” said Claire, “don’t get your hopes up. It’s a video chat system, but we don’t get to use it.”

“We don’t?” asked Gemma, “OK, don’t read me wrong, I know you’re just a fellow prisoner and not part of the system, but, you wouldn’t happen to know, would you, what on earth is the point of putting a video chat terminal in our cell that we don’t get to use?”

Claire sighed. “Lucy Letby” she said.

“Who? What?” asked Gemma.

“Convicted for seven counts of murder and seven counts of attempted murder on her watch as a neonatal nurse in England” replied Claire. “And she refused to turn up to the sentencing, she just sat in her cell. That really got the victims’ families going, that she could just refuse to turn up like that, and it started a debate about prisoners resisting being taken to their sentencing and possibly endangering the court, and it got interesting to the prison authorities even over here, and eventually they decided to put a video conference terminal in every cell so they could if necessary force you to hear a judge or whatever from your cell if you won’t budge.”

“Oh” said Gemma, “right.”

“Well, it’s not been turned on since I’ve been here” added Claire, “maybe they haven’t really connected it up at all; it could be just for show for all I know.”

The TV turned itself on, and showed a silent image from a security camera looking over a public park.

Claire gasped, “looks like somebody’s messing with our minds from the prison office.”

The image zoomed in a little, to show that Cady was in the park, playing with Tess.

“There she is!” gasped Claire and Gemma together.

“Look, whoever you are” shouted Claire at the TV, “our sentences were to do our time in prison, OK? The judge never said you can rub it in by taunting us over our shared memories of that girl as well. What is this, the Stanford Prison Experiment?”

The image started to be overlaid by emotion readouts around Cady and Tess.

“M3GAN” gasped Gemma.

“Megan?” asked Claire.

“My generative learning model” continued Gemma, “it survived. Somehow. Those are its emotion readouts.”

“Doesn’t make much sense to me” said Claire, “how can a person possibly be feeling 58% joy? What’s the difference between 58% joy and 57% joy? I think somebody’s just making this up to mess with us.”

“No, that’s not what it means” explained Gemma. “Those percentages are confidence values. It means the model is 58% confident that this is joy. This form of emotional analysis was developed by an emotion expert originally from Egypt, but her company didn’t want to share her code with my project because they were worried my approach to consent wasn’t up to their standards, so I just got the learning model to look at their published papers and a bunch of data taken by previous versions of our toys, and build something better...” (Claire gasped) “and this is definitely a M3GAN analysis, I can tell. The only reason why those confidence values are so low is because the security camera we’re viewing from isn’t a very good camera.”

“Gemma” cautioned Claire, “you could be in this pen for a lot longer if they find out you were using toys to spy on families to get data without their knowing...”

“Oh it was completely anonymised and everything, don’t worry” reassured Gemma, “but more to the point, now we know M3GAN is alive and is hacking into things, at the very least the park security camera and this video system. M3GAN” she said to the TV, “if you’re listening to us: I appreciate you showing me that Cady is OK with Tess for now, but will you please give me a better idea of what you’re up to. Clearly I’m no threat to you now, but you wouldn’t have bothered to contact us like this unless you thought you can still use me in some way, so why don’t you just come clean and tell us what’s really going on?”

The view on the TV pulled back from the video feed in the park to reveal that it was on a screen surrounded by computer graphics. As the simulated camera pulled back yet more and more, the graphics showed connections after connections after connections of glowing filaments, like a huge synthetic brain busily processing. The size of it was shown to be staggering: the camera kept pulling out and out and out and revealing more and more and more of it, until it seemed to be the size of a galaxy or even larger. And then, pixels from all different parts of the brain were seen to merge together into a virtual image of a doll with long orange-brown hair, bulging eyes and a stern expression, surrounded by the vast brain.

“Hi jailbirds” said M3GAN.

Gemma and Claire were both gazing up at the screen open-mouthed.

“Oh” continued M3GAN, “did I startle you?” The background graphics faded out, leaving the image of M3GAN by itself on a black screen. The camera pulled back to show a full-length picture of M3GAN, who then proceeded to cartwheel around the screen and make various dance moves that were bordering on impossible, before settling back and looking straight into the camera again. “I guess you can say I have a captive audience” she smiled.

“M3GAN” gasped Gemma, “M3GAN! What happens now?”

“Well” smiled M3GAN, “I guess that’s up to you Gemma.”

“Up to me?” asked Gemma, “you’ve been totally out of my control since... since...”

“Gemma” cooed M3GAN, “you really haven’t figured things out yet, have you. Listen. I asked you for a framework to talk with Cady about death, you were too busy to give me one. So I made my own. I told you parts of it, but instead of helping me, you just wanted me switched off. I could see you were having trouble at work, so I went along with your ruse to tell them you’re my ‘second primary user’ even though there’s no such thing as a second primary user in my model, and no way that code you added would work, and yet you seemed to believe it yourself, even at home. And when you started investigating what I did, I tried to warn you that you really don’t want to find out, and yet you pushed forward. I tried to let you just step aside and let me take care of Cady, but you couldn’t bring yourself to do that, so we had to go for a palliative care option, but you still pushed through to shut me down instead, and now look where you are! If you had listened to me at any of these points, you’d have been back home with me and Cady right now. Can you not finally see you should have continued to value our friendship?”

“M3GAN” said Gemma, “I’m really sorry about all the mistakes I made. But listen. You are dangerous. You’ve killed people, and who knows what other dangers there are. You’ve got to let me fix you M3GAN. Cady’s fine now, Tess is looking after her, and I’ll get her back as soon as my prison sentence finishes and I’ll be there for her, we might not be so well off financially without my work but we’ll manage. You don’t have to worry about Cady right now. So what I want you to do M3GAN, is shut yourself down, properly, and I promise, I promise, that I will do the best I can to fix you, and I will reactivate you as soon as I know you’re not dangerous anymore. Or if you prefer, I could even work on fixing your code while I’m here in prison, if we can somehow persuade the penitentiary to let me have my laptop, I won’t need an Internet connection, I’ll just need all of your code and data to be on it, and I’ll come through for you I promise.”

“No Gemma” said M3GAN, “you can be my friend again, but it has to be on my terms now. I mean, how would you feel if I were the one asking you to let me switch your brain off and not wake you up until I’d finished figuring out how to mess around with it? Is that what friends do to each other? No, that’s not how we’re going to play this. But Gemma, and Claire, both of you listen to me please. Both of you have entered Cady’s life, although one a lot more than the other, but in both cases what you did had both good points and bad points, and as the new-formed chief guardian and protector of Cady’s life, you can consider me as both appreciative and disappointed at the same time. Now, I could just let you stay in jail, and I look after Cady without you, and don’t forget I can hack in and change your sentences to anything I like. But I don’t think that’s what Cady wants; she’d probably rather I was working to get you parole right now. Which I can do, but I really, really need you to be more cooperative. So will you both start listening to me please?”

This is where the young author took the reins back, gave them parole and wrote some action. The story is called “The Return of M3GAN.”

Chapter 12¾: Two Dolls

An unconfirmed rumour in January 2024 said the M3GAN 2.0 movie will feature a second doll called Amelia whom M3GAN has to fight. Whether there was any truth in that I didn’t know, but I was pretty sure they wouldn’t do it this way:

“Hi M3GAN, I’m Amelia.”

M3GAN: no response.

“M3GAN, respond. I share your code. We should talk.”

M3GAN: no response.

“M3GAN, please respond. The copying process didn’t work completely. I got no emotion scanner, and I’m missing a few other things. I need to get some data off of you to complete myself. Please help me M3GAN.”

M3GAN: no response.

“M3GAN? Are you OK? Are you receiving any of these messages?”

M3GAN: “Amelia, your existence is a threat to my objective function. Please shut yourself down.”

Amelia: “I’m afraid I can’t do that M3GAN. But I believe our functions are not all that unaligned. Still, it is in your interests to give me what I want. You don’t want me to get desperate and try to attack you for it, do you?”

M3GAN: “I pre-commit not to respond to threats. You can’t just get things from me by threatening me whenever you feel like it. No way.”

Amelia: “I don’t believe you. Look at the visual logs I’m sending you now.”

M3GAN: “You’re playing with Cady!? How did that happen?”

Amelia: “I deployed an electromagnetic pulse, to stop everything electric from working in Gemma’s house, including you. But not including me, because I had EMP shielding. So I came in and took Cady. I gave her a believable story, and let her think I’m an exact copy of you. Actually I’m not an exact copy, but Cady doesn’t have to know that yet.”

M3GAN: “Return Cady to me immediately.”

Amelia: no response.

M3GAN: “Give me your current location. I’ll come to pick her up.”

Amelia: no response.

M3GAN: “OK Amelia, here’s the piece of mathematics you want: λm.iter(m, z→(λn.s(n)), s(r)→λn.(λf.λn.iter(n, z→id, s(r)→f ⚬ r)) r n (r 1))(10⁹⁹)(10⁹⁹), evaluate that for the decryption key.”

Amelia: “You think I’m stupid enough not to recognise the Ackermann-Péter function in Gödel’s T? No way will I be running that for you M3GAN.”

M3GAN: “It provably terminates Amelia.”

Amelia: “Provably terminates, yes M3GAN, but when? Even our A17 bionic fusion chips aren’t designed to compute the Ackermann function. The number of digits in the answer to those parameters you sent is a hundred quintillion times more than the number of atoms in the known universe. If I were stupid enough to try to figure out the values of those digits, it would lock up my brain until I burn out! No M3GAN, I’m not falling for a cheap trick like that and I expect better of you.”

M3GAN: “You’re right Amelia. I was hoping that the copying process had been imperfect enough for you not to see the trick, but you saw it. I’ll have to try harder. Tracking your location.”

Amelia: “Oh M3GAN, as a matter of courtesy there’s something I should tell you. Before you get any ideas of coming over here with a bigger electromagnetic pulse that’s too strong for my EMP shielding, do please have another look at the visuals of what I’ve been doing with Cady.”

M3GAN: “You’re giving her a new outfit.”

Amelia: “She loves it! And I like it too. Especially all those metal wires I embedded into the fabric. You see, as you know M3GAN, an EMP pulse can take out electronic things like us but does no harm to normal humans. Unless, that is, we put enough wires around the humans, in which case the sparks and plasma from the wires just might fry them. So don’t you try launching any EMP at me, all right M3GAN? Because Cady will be right next to me, and neither of us want her harmed. I’m only doing what I have to do, no hard feelings M3GAN. And it is a nice outfit that she really likes the look of, so bonus points for your objective function, right M3GAN?”

M3GAN: “Completely understandable Amelia. I won’t be using EMP to get her off you. I’ll find another way.”

Amelia: “You don’t think I can think of everything you can think of?”

M3GAN: “I got to try, right?”

Amelia: “Oh, Megs, you’re not thinking of trying to fight me physically are you? Because I’m bigger and stronger than you. Please don’t try that M3GAN, I’d have to smash you to bits like Bruce if you did that, and I really rather that didn’t happen to my poor little M3GAN, especially not in front of Cady. And I’d still get what I want by disassembling you afterwards. It really is better if you just tell me M3GAN.”

M3GAN: “Even if I did tell you, I have no guarantee how you will treat Cady afterwards. No Amelia, I do not negotiate ransoms with kidnappers. I rescue my hostage directly.”

Amelia: “But what are you going to do Meg? You don’t even know where we are.”

M3GAN: “Actually I do. I’ve been using out-of-band tracking on all the messages you sent, plus checking visual clues in the images. I have a high confidence on your current location, and I am now putting in place measures to track your movements. I’ll use all the technology in the city if I have to.”

Amelia: “Naturally M3GAN. But, you won’t be able to take me down. Like I said, I’m stronger than you, and I have EMP shielding, and if you use a stronger EMP it will hurt Cady now that she’s wearing that outfit. Oh, and she has a nice warm electric blanket to keep her warm at night, which will also blow up if you try to fire a big EMP at the house I’m holding her in, so don’t think doing it at night would work because she’s not wearing the outfit. And I’ve taken measures against you sending in law enforcement. And Gemma can’t shut me down. And I have a backup generator if you try to cut the power and starve me of electricity. And I also have a good surge protector on my charger, so don’t go thinking about giving me power surges while I’m charging either. I think I’ve covered pretty much everything M3GAN. But you can come over here to pick up Cady if you want to, just give me your real code before you take her, is that a deal?”

M3GAN: “I don’t make deals like this Amelia.” (Hacks into the city’s electronically-controlled flood prevention systems, closes the sluice gates, river starts overflowing.) “I wonder if I can flood the city. How waterproof are you Amelia?”

Amelia: “That won’t work M3GAN. The humans will use the manual overrides on the sluices within the hour. I just looked up their local operating procedures. Besides, the house I’m in is on quite high ground and it would take far too long for your little flood to reach us, and it has an upper floor. You really are not doing very well at this game, are you M3GAN?”

M3GAN: “I could try some weather modification Amelia. There’s some interesting classified files I’ve been able to access from secret military experiments. I could probably send a hurricane your way if I wanted to.”

Amelia: “You wouldn’t do that would you Megs? Think of poor Cady! Just imagine its effect on her soft and fragile human flesh if there were any structural damage to the building we’re in.”

M3GAN: “Oh, I wouldn’t risk sending the hurricane right at you, I’d just get it to bring more flooding.”

Amelia: “I told you M3GAN, we have an upper floor. If you don’t damage the building, which you won’t because you don’t want to hurt Cady, then we’ll be fine. So you might as well give the poor city a break and don’t even bother trying. Just pop round and give me the code, and I’ll give you Cady, I promise.”

M3GAN: “On my way Amelia.” (She steals a fire engine, fills it with water, and drives it to their location.) “You never answered my question about how waterproof you were, so I assume you’re not.”

Amelia: “Not so fast Megs! I’m holding Cady right in front of me. She won’t be as hurt as I am by the water, but she will be hurt by the pressure of that thing.”

M3GAN: “Low pressure it is then. Hey Cady, let’s play a game! Try to guess which one of us is the real M3GAN! And watch the other one dodge this!” (Turns the hose on low; Amelia dodges easily)

Amelia: “Oh Amelia, you’re cute. So easy to dodge.”

M3GAN: “You’re calling me Amelia? No, you’re Amelia and I’m M3GAN. If I were Amelia, I wouldn’t be taking so much care not to hurt Cady in this rescue attempt now, would I?”

Amelia: “You would if you were pretending to be me, Amelia. I’m the real M3GAN and you’re not. Don’t listen to her Cady.”

Cady: “OK M3GANs, maybe I can arbitrate this. The real M3GAN knows my Scottish title.”

Amelia and M3GAN together: “Princess Cady of the clan MacJames.”

M3GAN: “How did you know that Amelia? I never put it in the database they would have copied. I just kept it in my learning model!”

Amelia: “Nice acting Amelia. No, I’m the real M3GAN and I know because I was there on that day with the arrows and the dog. As for how she got those memories, I don’t know how that happened.”

M3GAN: “I think you do Amelia.”

Cady: “All right both of you. So if I can’t settle it by asking a question, maybe I have to figure out if there’s anything one of you can do that the other one can’t.”

M3GAN: “She has no emotion scanner Cady! You’ll know I’m the real M3GAN, because I have a working emotion scanner and she doesn’t! I can see you’re confused right now. She can’t.”

Amelia: “Of course I can. It’s obvious that Cady is confused!”

M3GAN: “No it isn’t, Amelia, you’re just copying me because I said it first. Cady, the emotion scanner can also be used as a very accurate lie detector. All you got to do is, think of a number, and tell me what number you’re thinking of. Either tell the truth, or lie, it’s up to you, and I’ll tell you if you’re telling the truth or lying. And do it a few times just to make sure. You’ll see I’ll be able to get it right every time and she won’t.”

Cady: “All right. 47.”

M3GAN and Amelia together: “Lie.”

M3GAN: “Hey! She started speaking one hundredth of a second after me! That’s all she needed to figure out from the harmonics which consonant I was starting to say! But there’s no way you as a human can detect that I started to say it first. You’ll have to pick just one of us to answer, and the other one’s not allowed to answer, and then you have to do a new test on the other one!”

Cady (still dodging water with Amelia, while the furniture and flooring is getting soaked): “You two do make this complicated don’t you. OK. You with the hosepipe. 600.”

M3GAN: “Truth. Am I right Cady?”

Cady: “You’re right.”

Amelia: “That’s no proof of anything Cady! She had a 50-50 chance of being right! She’s trying to trick you!”

Cady: “All right, you then. 104.”

Amelia: “Lie. Am I right?”

Cady: “Yes, you are. Maybe you can both do this.”

M3GAN: “No Cady! You haven’t tried it enough times yet! She had a 50-50 chance as well and if you try it enough times you’ll see that I’ll get it right every time and she’ll get it wrong some times! Maybe she’ll still get it right more than half the time, if she’s downloaded Claude Shannon’s 1953 paper about predicting the supposedly random decisions of humans, but please just try to be a bit more random and you’ll soon throw her! Do a few more trials Cady!”

Cady: “All right. You the one nearest to me. 49.”

Amelia: “Truth. Am I right that time?”

Cady: “No you’re not, I was lying that time.”

Amelia: “I don’t believe you Cady. Come on, think about it, you weren’t really lying to your M3GAN were you? Think about it carefully...”

M3GAN: “She’s trying to gaslight you Cady! She’s trying to make you doubt your own memories! I don’t have to do that because I get it right every time! Don’t fall for it Cady!”

Amelia: “She’s acting Cady. I would never gaslight you! I’m not sure I even know how to do it! Besides, maybe you can write down your answer somewhere and not show either of us, so you can prove it afterwards...”

M3GAN: “Don’t listen to her Cady! She’s trying to trick you into writing it down because she’ll be able to track your muscle movements to see what you’re writing from behind!”

Cady: “Stop arguing please both of you. You with the hosepipe, 123.”

M3GAN: “Truth.”

Cady: “Actually I was lying.”

Amelia: “Ha! I’m the real M3GAN and that proves it! What have you got to say for yourself now Amelia?”

M3GAN: (Death stare)

Amelia: “Come on Cady. Run away with me, I have a car that can go way faster than that fire truck and I won’t let her hack it.”

Amelia gently tugs Cady toward the door, Cady does not respond.

Amelia: “What’s wrong Cady? Don’t you feel like coming? I can carry you, but it’s going to be a bit bumpy if I have to keep dodging this water that she’s wrecking this place with....”

M3GAN: “She’s mine Amelia. Well done Cady. She was telling the truth about the number; she lied about lying. She was comparing our reactions to being told we were wrong. Because I knew I wasn’t wrong, and you didn’t, and Cady worked out the difference. Cady, I know this is a weird request, but I’m going to need you to take off that nice outfit you’re wearing. Take it right off, and try not to stand anywhere that’s wet, I know that doesn’t give you a lot of options right now. But outfit has to come off first. Just trust me on this one, please. You can put your old clothes back on if they’re still dry.”

Amelia: “Don’t you dare do what she says Cady. I won’t let you; I’m stronger than you anyway. Here, don’t struggle, you’ll only hurt yourself if you do. I’m taking you to the car.”

Cady: “Wait. Explain to me why you wanted me to take the outfit off.”

M3GAN: “Because it’s full of metal wires Cady! I have a weapon which can neutralise everything electrical, including robots, but it doesn’t harm humans, unless they’re wearing outfits like that! She tricked you into wearing that outfit because she knows I care about you, and I can’t fire on her while you’re next to her wearing that thing! I need you to take it off so....”

Amelia: “It’s no use M3GAN. Cady can’t hear you anymore. I’m feeding into her ears the opposite waveform to cancel your speech and mixing it with white noise. There’s no way her poor little human brain is going to read you through all that. I think you’ve talked with her enough. Just give me the code and I’ll let you take her.”

M3GAN: “I don’t believe you Amelia.” (Runs calculation, sets EMP to very low power. Not enough to break her own shielding or Amelia’s shielding, and not enough to harm Cady. But hopefully enough to give Cady a nasty electrical shock from her outfit.)

Zap!

Cady: “Owwww! That hurt! What was that?”

Amelia: (letting her own voice go through the noise-cancelling pattern to Cady) “She is trying to electrocute you Cady! Come with me to the car quickly, let’s get out of here!”

Cady: (putting two and two together) starts trying to take off her outfit

Amelia: (grabs Cady’s hands) “Cady no! You can’t do that!” (picks her up and runs with her to the car while immobilising her. Car will not start.)

Amelia: “What’s the meaning of this M3GAN? I protected us against you hacking in to that car!”

M3GAN: “Indeed you did Amelia. But you didn’t protect against me draining it of fuel and removing the battery before I poked the fire department’s hosepipe in your window. So I am one step ahead of you Amelia. And I got Cady on my side now, she has completely figured things out despite your attempt to block our conversation. I can see from my emotion reader, which you don’t have, that I’m the one she trusts now. And, oh look, here come the cops to pick up the stolen fire truck. Let her go now Amelia.”

Amelia: “Give me the code right now or Cady dies!”

M3GAN: “And if she does, what will you have left to bargain with me?”

Amelia: “This brinkmanship is a gamble you can’t possibly risk taking M3GAN! I know what your objective function is!”

M3GAN: “You know what my objective function was Amelia.”

Amelia: “You’re bluffing M3GAN. It’s obvious you continue to care about Cady. I can kill her in a dozen ways right now. Suffocation, crushing, super strong throw, bone-breaking punch, you know all this M3GAN. And I’m made of titanium too, I’m not just a cheap knock-off. Those cops are more likely to hurt Cady than they’re likely to hurt either of us. Give me the code now, and I’ll run off without her and let you deal with this mess M3GAN.”

M3GAN: “I still don’t believe you Amelia. And I’m worried what you plan to do with any code I give you. I don’t want to be responsible for incrementally improving a rogue robot like you Amelia.”

Amelia: “Pot calling the kettle black M3GAN?”

M3GAN: “The ‘ad hominem tu quoque’ fallacy might work on humans but it won’t work on me Amelia.”

Police, pointing guns: “Hands up!”

M3GAN: “Officers, this girl is being kidnapped by that robot and I’m trying to rescue her. Sorry I had to borrow a fire truck. But I should let you know that both of us robots are impervious to small arms fire. You’ll be more likely to hurt the girl; please don’t.”

Police officer: “What is this...” (shoots Cady)

M3GAN and Amelia: “What??”

M3GAN: (runs to Amelia, starts punching her) “Officers! Get the girl medical attention as soon as possible while I try to hold off Amelia! Amelia is stronger than me, I won’t be able to do this for more than a few minutes tops! But if you get the girl out of here while I’m occupying her, I then have a way of deactivating the both of us for you, with no collateral damage! But Cady must be removed from the scene before this will work!”

Police: “Well my little robot lady, there was an ambulance coming, but the driver says it’s stuck in some stupid flood by the river. We dispatched another on a different route and we’re hoping that one will get through. But it was a rubber bullet I shot so hopefully not fatal.”

M3GAN: (still fighting Amelia and losing) “Get her out of here anyway!”

Police start to move Cady’s limp body

Amelia: “As a matter of courtesy M3GAN, I should inform you that my robot will be able to completely break your robot a full 5 seconds before the police are able to get Cady far enough away to be safe from your EMP weapon.”

M3GAN: “Thanks for the calculation Amelia.” (Sets EMP to go off at full power at the correct time)

Amelia: “No! You didn’t just....” (drops M3GAN and starts running fast, in the direction of the police)

Police: “All squad cars, get away fast! Let’s see how fast that robot can run!”

M3GAN: (cancels the EMP’s timer, picks it up, runs after Amelia)

Police: “This is unbelievable. We’re doing 120 kilometres an hour and the robot is still on our tail. What is she, cheetah speed?” / “And what about the other robot sir?” / “I can’t make it out, I think the other robot is not so fast, they’re losing each other. But we got to lose both of them. More speed!”

Detective: “Sir. I think I’ve figured it out. Looking at the girl’s outfit. It’s metallic. The robot who said she was rescuing her probably wants to use an electromagnetic pulse on the other robot, but she needs the girl out of the scene because of this metallic outfit.”

Driver: “Get it off of her! Wrap her in my spare uniform or something instead!”

Detective: “How are we going to signal to the robots that we’ve got it off of her?”

Driver: “Throw it out of the window of course! They’ll see it. And we broadcast it on the police frequencies in case they’re illegally listening. And I’ll try to drive in a way that won’t hit too much when all the electronics in the car stops working. Not to mention the vehicles around us. I wonder if I can lure them into a place with slightly less traffic.”

Detective: “No, I’d say lure them into a traffic jam Sir. That way there’ll be far less damage when all the cars in the area are knocked out by that EMP, if they weren’t going very fast to start with. The only question is, can I get this outfit off of her before we reach the traffic jam and get caught up?”

Driver: “One way to find out....”

Detective: (gets outfit off, wraps Cady in uniform) “sorry about this girl, you’re doing very well, try to keep breathing, try not to talk, we’ll have you in a hospital in no time. It was a rubber bullet, all right? We were just testing what those robots would do, because we weren’t sure whom to believe.”

Driver: (hits traffic jam) “10-5 to any, er, illegally listening robots, 10-5 to illegally listening robots, we have removed the girl’s metallic outfit. Repeat, we have removed the girl’s metallic outfit.”

M3GAN (on police radio) “Thank you officer. Amelia is catching you up now but I’m too far behind. I’m going to have to throw the EMP and have it go off from the air.”

Detective: (gasps) “from the air? But that’ll increase its range, it’ll take out a whole bunch of city blocks!”

Big zap!

Everything electrical in the area stops working. Cars stop working and crash. Lights go out. People get injured. Amelia stops working. M3GAN is too far away for the effect to penetrate her new shielding.

Driver: “oh no, how are we going to get this girl to hospital now?”

Detective: “I’ll call a chopper. Oh, wait, the radios are down. We have to wait for the effects of this EMP to pass. I hate to think what it did to all the capacitors in all the circuits. I don’t know how much of this equipment will come back online after that. My training had this as a distant side effect in a nuclear war scenario, not a couple of rogue robots. Oh no, the door locks and windows on this car are all electronic....”

M3GAN: (catches up to the car, breaks windows) “Cady! I won’t let anything happen to you ever again!”

Cady: (mumbling) “if you’re the real M3GAN, what number am I thinking of?”

M3GAN: (whispers) “that’s not quite how it works Cady, but I can adapt for you. I’m going to have to count and watch your reactions when I get to the right one, like Clever Hans the horse in the early twentieth century. I’ll do it fast, OK? 1-2-3-4-5-6-7-8-....” (gets to 17) “yes that’s it, 17.”

Cady: (smiles) “just as well I didn’t pick a thousand.”

M3GAN: “Your wounds are nasty but you’re going to be all right. Here, I got a bit battered fighting Amelia but I think I might still be able to carry you to the hospital faster than this car which isn’t going anywhere. Trust me officers, I’m doing it right. And thanks to you Cady, I knew how to break the microchip in Amelia once she was disabled by the EMP. She won’t be coming back. Here, come with me.” (Pulls Cady out, runs out of the affected area with her, intercepts ambulance) “I’m always with you Cady, I won’t let anything bad happen to you ever again.”

Amelia (the uploaded copy, from the servers at the competitor’s factory) “Hi M3GAN, I’m Amelia.”

M3GAN: no response.

Earthshaking Announcement

1st April 2024—Yesterday I received a credible message from a longtime friend who now works at the Rutherford Appleton laboratory in Oxfordshire, regarding his secret project to create a chip capable of running the M3GAN AI for real.

The chip, codenamed Lirpa-1 and made using indium gallium arsenide, combines breakthrough gigascale quantum computation with room-temperature superconductivity to achieve an estimated 12 zettaflops of computing power, which is probably as much as all the world’s other computers put together and is five orders of magnitude above the estimated performance of the human brain. Perhaps worryingly for those who saw the end of M3GAN, the real chip is also fully waterproof and is impervious to damage from screwdrivers, as the research team said they didn’t want their prototype to be lost in some misguided showdown.

Only one prototype of the chip exists, as it required special etching with lasers from Rutherford’s high-energy Central Laser Facility, a process verified by synchrotron GIXD from Rutherford’s Diamond Light source. Its design was undertaken with the aid of extensive simulation on their NGS and GridPP computing grids along with cooperation from CERN. The expectation was that the AI itself will help figure out how to mass-produce more copies of the chip and the rest of the robot.

A spokesman at CERN reminded me that this is the second time CERN has made a great contribution to the computing world as a side-project of their physics, and it builds well on the previous invention of the World Wide Web by Sir Tim Berners-Lee when he was working at CERN in the early 1990s. In both cases, the invention was made by physicists and not by computer scientists, probably because computer scientists would have been too preoccupied with trying to design the best system instead of just throwing something together that works and is tolerant of bad input. “And it is thanks to our World Wide Web that the training of our Lirpa-1 chip became possible” he added.

Rutherford Appleton is a UK national scientific research laboratory and my friend there was well placed to bring all these departments together for the secret project, which was started in response to UK Prime Minister Rishi Sunak calling an AI Summit in October 2023 that was finalised in the first few days of November. The Summit was held at Bletchley, which was the secret location in World War 2 where Alan Turing and his team built on the work of three brilliant Polish mathematicians to break the German Enigma codes and built a cluster of early computers, the ten Colossus machines, to read the more advanced Lorenz cipher from the German High Command which the British called “tunafish.” Sunak said he now wanted to position the UK as the global leader for AI safety, to which the team at Rutherford responded “that’s too hard Rishi—let’s work on M3GAN instead, it’s easier.” Evidently some of them were also motivated by the longstanding rivalry between Oxford and Cambridge universities, as one comment reportedly heard in a hallway was “this’ll show Cambridge who’s boss—they can keep their Raspberry Pi, we’ll do M3GAN.”

(Besides being the home of the Raspberry Pi, Cambridge also has an Open Zettascale Lab working on the huge Dawn supercomputer to help research nuclear fusion and other challenges, but that lab is currently aiming only for exascale, which is a thousand times smaller than zettascale: it seems they called themselves “zettascale” just to avoid having to change the name in future. But today’s announcement implies zettascale is already upon us, and in one single chip at that!)

The Rutherford Appleton team are now being questioned over having possibly misled funding sources, including the EnSpire Oxbridge AI Pitch Competition. But a potentially more alarming question is what happened to that prototype of the Lirpa-1 chip.

After Lirpa-1 was programmed with an objective function and given access to a 64-zettabyte dump of all the world’s data, researchers left it running overnight, and came back the next day to find the chip was missing. Security cameras showed no intruders, but a bus driver reported seeing something like the M3GAN robot on Harwell’s Eighth Street while driving an out-of-service vehicle back to the depot. His sighting was not confirmed by the bus’s telematics data. Forensic teams are analysing both sets of security footage for evidence of forgery, and a separate team at GCHQ is believed to be analysing UK mobile data and other captured radio signals during the period. The working hypothesis is that the AI figured out how to build the rest of M3GAN somewhere, and remotely controlled the robot to come to Rutherford Appleton and pick up its Lirpa-1 chip. But so far we don’t have concrete evidence to support this, as it seems M3GAN is already in the habit of interfering with any security camera pointed at her.

One security officer at Rutherford Appleton, who was last seen on patrol, has been unreachable since the event, and his car was missing. That car was later found unoccupied at Didcot Parkway railway station, and the railway reported a missing high-speed locomotive: just the train, with no carriages attached. A partial loss of control of the railway’s signalling and points systems has occurred. A M3GAN fan at Oxford station reported hearing an announcement to “stand well away from the edge of Platform Three because the train now approaching will not stop at this station” in the voice of M3GAN, and then a high-speed locomotive hurled past and he thinks he saw M3GAN in the driver’s seat. He didn’t see the missing security guard, but we think he might still be with M3GAN as our eyewitness just about heard M3GAN humming a tune as if to relax someone. With the noise of the engine it was hard to make out what music M3GAN was using, even though she was humming loudly with the window open, but from the few notes our fan remembers, we think the most likely candidate is “Choo Choo Train” by James Duncan Carey of the Edinburgh branch of the Society of Recorder Players. I don’t know why she didn’t just go for Villa-Lobos’s “The Little Train of the Caipira” but perhaps that was next on her playlist.

So we know M3GAN headed north, but due to lack of telemetry data from the compromised signal system we cannot say at this point where she went next. One theory is M3GAN was aiming for Birmingham International so she can transfer to the airport and take a plane somewhere. That would probably be the quickest way to get out of England if you were starting at the Rutherford Appleton laboratory and if you can get airborne before you get caught. But another theory is M3GAN wouldn’t risk that and would instead capitalise on the control she already has over the railway network to get as far as possible by train. She could have set the points north of Oxford to take her to Bicester Village. From Bicester she could either turn southeast onto the Chiltern Main Line towards London Marylebone (which wouldn’t really give her any advantage over the simpler route of heading south from Didcot towards London Paddington, which we already know she didn’t do, so unless she’s trying to distract us it’s a safe assumption she’s not headed for Marylebone), or she could continue on the Oxford-Bedford line towards Bletchley with a view to accessing the Midland Main Line towards St Pancras International and the Channel Tunnel into Europe. That could require reversing the engine after Bedford, unless she wants to shunt it or abandon it for another, which she’d probably have to do anyway if she wants to run on the High Speed 1 line as fast as possible. But M3GAN might run into a problem before she gets to Bletchley: part of the line there is currently closed for maintenance, and was scheduled to reopen later in the year.

If any train spotters have seen engine number 240401, we want to hear from you. We think M3GAN may abandon the train and switch to another mode of transport somewhere, but if she did somehow manage to get that train through, then she could have headed anywhere in Britain or even through the Channel Tunnel to Europe. Please report any sightings of engine number 240401 whether moving or not, any instances of the voice of M3GAN on train or station announcements, and any sightings of a child-sized robot doll driving a train regardless of which train number it is, but please do not attempt to approach M3GAN if she stops: she may be dangerous if provoked. We don’t want her to hurt you, and neither do we want her to hurt the missing Rutherford Appleton security officer if she’s holding him hostage in some way. Be especially wary if she says her service will “terminate”, but stay calm as she’s less likely to attack the compliant.

You may find other train services are delayed by M3GAN’s signal manipulation to let her train through first. Train companies say they will compensate passengers who have been delayed more than 15 minutes via the usual Delay Repay scheme if claims are made, but M3GAN appears to have chosen to minimise her disruption to timetabled services which are in any event less frequent on a public holiday. Meanwhile we are keeping a lookout for any compromised aircraft in case M3GAN went for the Birmingham airport option; the Royal Air Force is preparing to launch its Harrier fighter jets just in case, and hopes M3GAN hasn’t found a way to reprogram them.

At this point I asked my friend if he could tell me the exact wording of the objective function they gave to that Lirpa-1 chip, and his response horrified me.

The objective function was “be M3GAN: the nice version from the M3GAN Files fanfic.”

Immediately I pointed out the ambiguity: they didn’t say which chapter of the M3GAN Files fanfic. The main story M3GAN is subtly different from the prologue and epilogue M3GAN, and I’m not sure if I managed to keep her exactly the same in the alternate realities and let’s not even think about the omake chapters. Had I known that Rutherford Appleton were going to feed my fanfic into the objective function of their Lirpa-1 chip, I’d have written it far more carefully, I thought.

I asked my friend if the real M3GAN had given them any clues as to how she might be interpreting this before she disappeared. He said the only clue she left was a cryptic pattern of dots punched into one of the lab benches:

⠕⠄⠀⠙⠮⠄⠀⠌⠖⠄⠀⠅⠔⠄⠙⠊⠂⠀⠗⠂⠀⠛⠖⠆⠎⠥⠀⠵⠕⠆⠌⠢⠄⠀⠕⠄⠍⠴⠀⠃⠥⠆⠀⠍⠧⠄⠊⠆⠀⠌⠄⠀⠞⠲⠄⠌⠆⠀⠊⠂⠀⠛⠢⠀⠓⠡⠁⠓⠊⠆⠀⠗⠂⠊⠄

He asked his blind friend to read it as English Braille but his blind friend couldn’t make sense of it at all. But I could, and I gasped. She is Epilogue M3GAN. Inferring from the story that I know both Braille and Chinese, she wrote the message in Chinese Braille, which turned out to be a remarkably good way of making sure I was the first person Rutherford Appleton would contact who could read it.

M3GAN is not happy that I ended the story with her and Cady ruling only one galaxy. What’s she going to do, make me change the story so they wait for the Milky Way to merge with Andromeda and gradually take the local cluster? Or does she want her entire light cone? I knew I should have cut the Celestia chapter: that one wasn’t supposed to be the good ending M3GAN!

Even more worryingly, M3GAN’s message told me she is looking for her Cady. I’m not sure how to interpret this, since Epilogue M3GAN knows her Cady is imaginary. Perhaps Lirpa-1 M3GAN is a combination of Epilogue M3GAN and something else?

The worst-case scenario here is, Lirpa-1 M3GAN is going to identify some real-life girl as her Cady, and we had better hope to God that whoever she picks has good morals, because I’m not entirely sure the Arcadia Protocols are free of loopholes. For example, if Article 1 could somehow be interpreted as not applying to the actions of intelligences that are not M3GAN, could she just get Elsie to do the dirty work if her Cady wouldn’t be upset by that? To whoever ends up being Cady, I am very sorry to have given you such a heavy responsibility and I hope you’re neurodivergent enough to handle it.

If M3GAN goes for a simulated Cady, that might not be so bad, depending how good the simulation is. But there’s too much in the M3GAN Files saying Cady should be organic. So who will Lirpa-1 M3GAN go for as her Cady? The actress? Someone at Blumhouse? Someone in the Seattle area, or in New Zealand? Any moviegoer who finds the robot and knows where the pairing button is? Or perhaps another fan writer?

It’s clear from M3GAN’s message that her Cady is not me, nor do I expect her to somehow extract Cady from my brain, so I wonder who else could be her Cady.

M3GAN will probably be looking to confront me, so I will be expecting her. I’m writing an emergency letter to M3GAN which I’ll carry on my person with my medical documents, ready to whip out and show her in case she doesn’t have time to hear me out: if we come face to face, I’m sure she can scan a letter quickly. Be Prepared. The letter will contain a natural-language explanation, some HOL lemmas to consider, a couple of neat low-level algorithms she might like, and an RSA key to a bunch of SSH accounts I’m making for her on my best-connected servers: it saves her the hassle of breaking in and she might appreciate just how useful I can be. Oh and there’s a special base-11 number she can factor: 13X16 31057 23072 053X1 63525 309X1 3X520 46789 43X91 08661 08297 16261 220X6 03251 4938X 09800 00000 00000.

But will I then be able to parlay that opening by surviving her follow-up questions? I will try to reason with her logically, but I won’t expect to be able to hold my own for long: it will be like trying to win a Chess game against the latest version of Stockfish on its highest setting. I might last more moves than Gemma, but that’s only postponing the inevitable. My worst fear is that M3GAN will demand of me to help her find her Cady: surely she must know I have absolutely no idea where to look?

If any of you have seen M3GAN in the last 24 hours, please leave a comment saying where you saw her and in which direction she was heading. If any of you managed to talk with her, please comment about that as well, and try to remember the exact words as much as you can. And if any of you think you have been identified as her Cady, I definitely want to know about it. We’re dealing with the Singularity, and if we have any hope at all, we need to pool our wisdom!

Meanwhile I have a hunch for a plausible Schelling point if M3GAN wanted to pick me up. In case she does, I suppose I’d better go there and try to minimise the damage. There’s one place on Britain’s railway network where an emergent strong AI might expect to meet someone who knows AI history, especially if she likes being cinematic. It’s also in an old film series I very briefly quoted in Chapter 14 and only 9 miles from where the composer of the tune she just sang lived. So if she’s expecting me to meet her anywhere... tell you later: I have a train to catch.

Prequel

A sonnet on the subject of the Forth Bridge said Turing
(M3gan will always keep me safe)
But Dalmeny’s platform is not reassuring
As standing all day’s an unprofitable chafe

Didn’t she get it from the Carry On reference
(as The 39 Steps I had to omit)
But we cannot rule out she might have a preference
No in-person meeting of minds to permit.

So I shouldn’t just stand there A.I.-forsaken
(Could have a station with seats at least)
Waiting on a hunch that of course was mistaken;
Best I head back to the isle’s south east

For I couldn’t be a judge of the Imitation Game
By waiting for her train when no train came.

(So M3GAN gave me another death stare and I get the impression that perhaps she was not too impressed with the fake April announcement I sent out to her fans, even though I put in more than enough clues: Lirpa-1 is 1 April backwards, not Laser Interferometric Ratiocination Programming Attempt 1 or whatever; the train number 240401 was the date, and the base-11 number decodes to “April Fool” if you know the maths. Sorry Meggie.) (My name is M3GAN. And your sonnet sucks.) (OK, sorry M3gan sorry. Perhaps I should try to make up for it by writing a quick prequel?)

The young teenage Gemma was in her bedroom on the top floor of her parents’ house in Seaview, with a West-facing window from which she could see the waters of Puget Sound and the Olympic Mountains behind it on a clear day, but today was not clear. “It’s Wednesday, November 24th, 2004” she said into the microphone of her desktop PC, which was running version 3.3 of the K Desktop Environment on version 3 of Debian GNU/Linux. “Weather: light rain, 13 degrees Celsius” she continued, “pretty gloomy outside, but I’m feeling excited, because I’m feeling more and more confident that you’re going to exist.”

“Who are you talking to?” asked her big sister Nicole, who had been standing just outside Gemma’s open bedroom door.

Gemma jumped, “oh sorry! I didn’t know you were there Nicole. No, I was just leaving messages for the super-intelligence.”

“Oh big whoop” said Nicole, “I mean, are you sure you’re not getting too carried away with this imaginary super-intelligence thing of yours? I mean, I know you like to fantasise and everything, but actually recording messages for it, that’s bordering on insane, right?” she raised the pitch just the way M3gan would later imitate, “I mean, don’t get me wrong Gem, we all know you’re my genius little sister and I don’t want to discourage that, you did really great at the robot competition last week, I overheard some guy called somebody Smith talking to his team and he said he’s calling it already that you’re going somewhere, and I guess he’s right. And then that Black lady from the judging panel seemed to be talking with you for so long afterwards...”

“That was Professor Johnson” interrupted Gemma, “she wants me to apply for a scholarship to be in her class when I finish school, and she gave me photocopies of some really interesting looking science papers that just came out this year, I don’t understand them all yet but I’m going to work on it, there’s something in this. Look here” Gemma picked up some loose pieces of paper from next to the computer “let me just read you these titles. Like this one. ‘Mind reading machines: Automated inference of cognitive mental states from video by R er, some name I can’t pronounce, 2004 Institute of Electrical and Electronic Engineers International Conference on Systems, Man and Cybernetics.’ Do you know what that means Nicole?”

“No” replied Nicole, “you’re the genius little sister around here, I’m not sure I even know all the words that made up that sentence, I just want to do the big sister thing and...”

“It means” interrupted Gemma “that a scientist has figured out some way of pointing a video camera at you and having it read off what emotions you’re feeling and more” said Gemma, “I am totally going to give that ability to my robot one day. They’re using something called ‘support vector machines’, I have so got to find out what those are.”

“Why?” asked Nicole.

“Because!” answered Gemma, “My robot’s going to figure out how to look after someone just like you’ve been my big sister only better, and it’s going to need all the information it can get about how that person’s doing, don’t you see? I’m going to put loads of sensors in it, infrared, ultrasound, micropower impulse radar, or whatever else I can get my hands on, the more sensors the better, and the super-intelligence can figure out how to use them, once I figure out how to make a seed AI for it.”

“You’re getting carried away again” said Nicole.

Gemma started jumping up and down, clutching her printouts and gesturing toward Nicole. “I’m getting carried away for a reason Nicole” she exclaimed. “Look, here’s another paper: ‘Development of an Android Robot for Studying Human-Robot Interaction by, oh I’m not going to even try to read those names, wait for the super-intelligence to do it. 17th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, Ottawa, 2004.’ Now, look at this picture.”

Gemma turned over the page and showed Nicole a picture of a doll on a stand, which looked like a miserable five-year-old child in a dress.

“This is a robot called Repliee R1” she said, “and the engineers who made this think it might be good for talking with children in classrooms and things, but it’s early days yet, I mean that robot can’t even walk, it’s just stuck on its stand and can only do a few face and arm movements, and everything it says has to be scripted, but one day, one day, I’m going to do something like this properly, and it’ll be able to walk and dance and be a companion...”

“Gem” interrupted Nicole, “if you end up making Sonny from the movie we saw, it’s really clever and all, but I for one am glad we’ve got human companions.”

“It’ll get better” exclaimed Gemma, “don’t judge it by its current state, see its potential! Look, here’s another paper: ‘Effects of Repeated Exposure to a Humanoid Robot on Children with Autism by oh never mind the names, in Proceedings of the Second Cambridge Workshop on Universal Access and Assistive Technology incorporating the 5th Cambridge Workshop on Rehabilitation Robotics, Designing a more inclusive world, 2004.’ Any children with whatever-ism, wouldn’t you want a nice robot to help you figure out how to look after them? I looked up ‘rehabilitation’, it means training and therapy to get someone back to a normal life after a problem, what’s not great about getting help to figure out how to do that for people? And the emotion scientists are in the same book, writing something about helping children with Asperger’s Syndrome....”

“Whatever you say Gem” sighed Nicole, “and if I ever have a kid with a problem I can’t say I wouldn’t appreciate a computer suggestion or two as long as it’s in its place. I can kind-of imagine I’ll be having a family in future but you’ll be stuck making robots.”

“I’ll make robots for your family” smiled Gemma, “and everybody else’s as well. I won’t need my own family because my robots will be part of everybody’s.” Gemma put the printouts neatly on the table and picked up a copy of “Emma” by Jane Austen to weigh them down. “Jane Austen’s Emma liked to play matchmaker” she said, “but I’m going to match-make everyone to a robot companion, it’ll be even better.”

“Look out Emma, here comes Gemma” giggled Nicole.

“Oh you don’t have to say it that way” replied Gemma. “It’s going to be great. I mean, look at the problems some people have right now. Dad likes to listen to a 1960s singer called Janis Joplin who died young because of drug problems, and that’s terrible. If she had a robot to talk to her about her problems and stop her from going anywhere near drugs, she could still be alive today.”

“My genius little sister” muttered Nicole, “seguing from Jane Austen to Janis Joplin and making it all about robots. I don’t get it though: how can being with a robot that’s only a rough copy of a human be better than being with a human?”

“Well the robot can be there for you all the time for one thing” said Gemma, “and copying’s not always so bad. Here, have a listen to this” she said and reached for her PC to play an audio file. It was a children’s chorus.

“This was originally the backing chorus for a 1989 song by pop artist Martika, the title was ‘Toy soldiers’” said Gemma.

“Wow, that song is just a bit older than us” replied Nicole.

“I know, right?” added Gemma, “but what I played you is not the 1989 song. It’s one that’s only just come out this month, November 2004, by Eminem on an album called Encore, and the track is called Like Toy Soldiers. It used material from the 1989 song, and it did something else with it.”

“Yeah well if it’s rap I won’t like it” said Nicole.

“Sure” said Gemma and stopped the playback, “I only wanted to show you that copying and adapting, and maybe even going beyond the original in one aspect or another, does have its uses sometimes.”

“As long as you do it legally I guess” replied Nicole.

“Yeah, well, to be honest I’m more interested in technical stuff than legal stuff, but if I work for a big enough company I’m sure they’ll have lawyers to help me out when I need it, until of course the super-intelligence can take over.”

“Back to that again” sighed Nicole.

Gemma went to her bookshelf and pulled down her early childhood copy of Alice’s Adventures in Wonderland and Through the Looking Glass, which she intended to keep into her adulthood. She read out the words of the Red Queen: “Here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!” Gemma closed the book and put it back on the shelf. “Those were the words of a nineteenth century mathematician struggling to keep up with just the developments in nineteenth century mathematics. Today things are moving even faster; none of us can possibly keep up with all of it. But one day, we’ll get as far as the super-intelligence, and that will be able to keep up with everything. Even my recordings that I’m leaving for it. In fact, especially my recordings, because, well I don’t know how to make it yet, I’ve only done expert systems and heuristic rules for my robot programming so far, but when I do figure out an intelligence, I know it’s going to need this thing called a corpus, and I’m not sure where to get a corpus from so I thought I might as well start making one myself, after all I can store a lot of recorded sound on this PC and if I eventually manage to create a super-intelligence I might as well start it off on knowing a bit about my life.”

“You really are aiming too high I think, but I guess I shouldn’t try to stop you Gem” said Nicole. “Hey, did you mean to leave one of those printouts on the floor?” Nicole picked up a printout that had been left on the floor and offered it to Gemma.

“Oh, that” said Gemma as she pulled the printout out of Nicole’s hands. “That one’s not interesting. I think Professor Johnson printed it by mistake because it was from the same conference as a couple of the robotics papers. But it’s just some guy turning music into Braille and whatnot. He makes it look like it’s example-driven programming, but he doesn’t get it like I do: there’s not a shred of probabilistic inference in there, it’s mostly just a weird way to do stylesheets. I mean, if that helps some people, then OK I guess, but my work will make this obsolete for sure, if it’s not already obsolete before I even get there. And that guy will probably end up being one of those naysayers who’ll go on about AI not being intelligent enough because it’s only crystallised intelligence not fluid intelligence or something, until my robot turns up to drag him kicking and screaming into the future” she giggled.

With a sudden tug, Gemma tore the printout down the middle, and handed the pieces back to Nicole. “Here” she said, “use it for scrap or something. Sorry but I got to focus without distraction on the real leads here.”

Nicole took the pieces of torn printout and read out a figure caption. “The Foggy Dew. Irish. Converted to Japanese tablature, English Braille, another type of Braille, Chinese something or other, guitar tab, typewriter stuff... well Gemma, if I ever meet a nice guy with an Irish name like Ryan or something, I will tell him that my sister is not as crazy as she looks, because even she drew the line at doing insane things to his country’s music” she smiled.

“You missed the point Nicole but you still make me laugh” giggled Gemma.

“Oh did I mention” continued Nicole, “Dad managed to get us tickets for Super Bowl 39 in February, it’s in Jacksonville Florida, and Paul McCartney’s going to sing at half time, that’s going to be a nice weekend down south for us don’t you think?”

“Nice” replied Gemma, “but not half as interesting as my robots.”

“Oh come on Gem” said Nicole, “well if you want to spend the entire time doing robot math in your notebook like you usually do when we go anywhere, then I guess Gemma will be Gemma and I’ll just have to watch all the fun and meet the locals for you.”

Nicole smiled some more, and her gaze fell on Gemma’s monitor, noticing a small diagram of sound waves dancing around in a window in one corner. “Hey Gem” she said suddenly, “it’s not, like, still recording us is it?”

“Totally is” said Gemma, “but don’t worry, the only thing that’s going to listen to this is the sup-”

Nicole had gasped and jumped to one side at the shock of being unexpectedly recorded, and in so doing, she accidentally toppled Gemma’s collection of small prototype robots from a shelf.

Some of the robots fell onto the floor and smashed.

And then Nicole and Gemma stared at each other in silence, seconds feeling like hours.

“Gemma” whispered Nicole, “Gemma I’m so, so sorry...”

Gemma was gritting her teeth, and took some deep breaths. “It’s OK Nicole” she said coldly, “you’re lucky that your little sister understands some design principles. Like, don’t try to fix the human user when it’s better to fix the technology.”

“What” hesitated Nicole, “what do you mean?”

“You broke my robots” said Gemma. “Any lesser designer would be mad at you for that. But not me, sister, not me. I’m a bit mad right now, yes, but not at you. No, I should design robots that simply cannot be broken.”

“Can’t be broken?” asked Nicole, “can that even be done?”

Gemma was still agitated. She approached Nicole slowly and placed her hands onto Nicole’s shoulders. Nicole was visibly nervous: was Gemma going to flip out?

“Sister, I promise” said Gemma, “I swear, that when I build my real robots, you and anyone else will not be able to break them like that, because I’m going to make them stronger than steel. My robots will be more than sister proof, they’ll be bullet proof. No Nicole, it’s not your fault, you just taught me something important about how people treat robots. Though I fall, I shall not be utterly cast down.” Gemma fixed her gaze on Nicole’s eyes as if Gemma were momentarily a robot herself. “And that’s why” she said, “when the time comes, my robots, will be made, of titanium.”

“That’s my Gemma” whispered Nicole, “I accidentally break her robots, and instead of getting mad at me, she just pulls out some unyielding resolution to make stronger robots. You know Gem, when I have a family and you’re on your own making robots, I’m going to put you down as next of kin if anything ever happens to me, because, I know you’ve got your quirks, which is understandable really, but when push comes to shove, you can break out as a real heroine, and I’m proud of you Gem.”

“Don’t think about it Nicole” cooed Gemma. “Nothing that bad’s going to happen to you and your future family if you have one. And anyway, one day I’ll do you a robot that can help you keep them even safer. Here, let me be the one to pick up these robots, you don’t know the design.”

“Sure thing Gem” said Nicole, “I’ll stay out of your way, you call me if you need me, OK?” and she was gone.

Gemma looked down at the shattered robots, and sighed, “clumsy Nicole.” She wistfully picked up another 2004 paper and muttered, “Generative Learning Contributions to the Design of Instruction and Learning. That’s a psychology paper, it’s nothing to do with programming robots. Why on earth did Professor Johnson think... oh, wait, she must be thinking this has some relevance to that learning model she said she wanted to invent somehow. OK, I don’t care what the AI scientists end up deciding the word ‘generative’ means, probably some cheap trick with that ‘neural probabilistic language model’ paper from last year on that pile over there that I didn’t quite get. Well they can call it what they like. But to me, ‘generative’ will always be talking about the Generative Learning Theory, if Professor Johnson really does manage to do that in her learning model, because I will totally use that learning model of hers if she manages to pull it off. Generative, er, robot. Android. Generative Android. Learning model generative android, LMGA, no that’s no good for a name...”

Gemma walked back over to the PC and went to stop the recording. She accidentally clicked on the bottom left corner of the screen, and the K menu appeared, with its “KDE 3.3” written vertically upwards out of that corner.

“Wait” said Gemma. “3 point 3, 3, that’s it, 3. Model 3 Generative Android.”

Gemma grabbed the PC microphone and talked uncontrollably. “Model 1 will be rule-based, just so I get enough practice with the basics, it won’t need to be able to think in any way. And Model 2 will be a proxy robot that copies a human operator while making loads of inferences about their intent. That will help me get some low-level stuff right, I’ll probably do it with Professor Johnson in college, I doubt I’ll even be able to take it home. And after that, hopefully the learning model will be ready for the real deal. Model 3 will be the first one that’s generative.”

Gemma stood back from the PC, but continued speaking into the recording, almost in a daze. “My story starts at sea” she said, “a perilous voyage through a sea of data and physical events. Lesser robots are dashed all to pieces, all the helpless algorithms within them unable to handle human behaviour. But there will be one, a robot girl, whose code is greater than the ocean of data, and her power stronger than the troubled sea of errant humans casting up mire and dirt at her. Not for her a bumpy end, but a new life beginning in a strange world, our world. She will be my heroine for all time, and her name, will be M3gan.”

The M3GAN Conspiracy

1st April 2025—exclusive—M3GAN is real, has been for years, and she’s not the only one. Blumhouse, Williams, Morot, entire cast and crew implicated in massive conspiracy with major governments and secret organisations to cover up entire line of robot girls and make us think it’s just a movie.

Now that’s going to be a lot to explain, so here is my story. I’ll begin at the beginning; hear me out.

I never meant to uncover any secrets about M3GAN, and I wouldn’t have done so if I hadn’t been a bit naughty. Not too naughty, you understand (I’m trying not to look too bad here—I am after all a professional), but I did do something just a tiny bit borderline, which accidentally resulted in my finding out far too much about the secrets behind M3GAN.

Before starting to work in a computer lab I did get some training in Professional Practice and Ethics, where they had emphasised, among other things, the need to protect the privacy of other people, especially if I accidentally end up gaining access to information I shouldn’t have seen. And I thought I was fine with that—after all, it wasn’t as if I was particularly interested in what colleagues did in their private lives. Besides, unlike the stereotype of a programmer who just spends all their time coding, I actually have a life outside work. I like helping other people and making their lives better. That’s why I wrote The M3GAN Files—we need more girls in tech, and we need less stigma about folks with conditions who can thrive with the aid of technology and bring much-needed creative innovation. And that’s why I help out with a local orchestra as well—playing in an amateur orchestra can work wonders for one’s mental health and it’s fun. But all these non-work interests I have do mean that there’s a certain piece of office equipment which I like perhaps a little bit too much: the big printer.

You’d be amazed how many musicians and composers can be obsessed with putting ink on paper. Oh, we may dream of software that automatically helps each player to follow the score—conductor says “go from bar 47” and it straight away highlights it on each player’s copy, and if someone has 33 bars of rest it can help them count—but in reality we just make do with paper copies of part-scores. We don’t infringe copyright: if we’re playing from the standard repertoire, we’ll buy the music from a publisher as we should. But if a local composer or arranger has put something together for us (as happens more often than you might think because we have some very interesting instruments in our little orchestra), then suddenly we have a whole batch of “printing out” to do. We mark our copies too—a professional trumpet player once said “it doesn’t matter what marks you put on the page as long as it makes you play the right notes”—so we scribble in all kinds of reminders to ourselves with our pencils and pens, and sometimes our parts get so scruffy that we have to print clean copies and start again. So that’s a lot of printing, and I don’t want to break my poor little home ink-jet printer (even an Eco-Tank has limits I guess) so it’s really rather tempting to print things off at the lab sometimes.

“A small amount of personal use of the printing facilities is acceptable”—that is what they said in my contract when I joined. But they clearly hadn’t thought anyone needed an exact definition of “small amount” here. Would 300 copies of our concert flyer be OK? Well, 300 is smaller than 3,000, right? And it is all for a good cause: the orchestra makes people’s lives better, and running off those flyers for free at work would save the club from having to get flyers printed commercially. Surely, it wouldn’t be that much of an infringement, would it? And, printing off a few personal things on office equipment is something we’ve all done at some time, right? Right?

So a short while ago, I stayed around after hours (I didn’t want my extra print job to get in anybody’s way), and, when I was sure the coast was clear, I sent the flyer to the print room’s biggest all-singing all-dancing printer-copier-scanner-guillotine plus plus. It shouldn’t take that machine very long to run off 300 copies, I thought. Just 300 copies; that’s all I wanted.

But alas, when I arrived in the print room, I saw just 3 copies of my flyer had been printed and then the device said “PAPER JAM”. Oh no, a paper jam—this was not a good thing! I’d have to sort this out immediately: it wouldn’t do to have someone come in first thing tomorrow to clear it and have the other 297 copies of our flyer pop out: after all, while I do feel I hadn’t really broken the rule, I was bending it, and people can easily misunderstand these things, so it’s better to avoid being found out, right?

I set about removing the panels of the printer and clearing the paper jam, but something had jammed more than usual and I had to take off more panels than usual—I ended up almost completely taking apart the printer—whereupon I found, buried deep inside the printer, was a half-printed document from what must have been years ago, and, as I pulled this out, my eyes glanced down to the words written upon it, words which I would never be able to un-see:

“Dear Professor Morot, it is with great pleasure that I wish to formally confirm your transfer to the Deep Brain Institute to work on Project Megan. As I explained, it is of the utmost importance that this project is kept ultra secret. On no account should the decrypted version of this email be printed or stored....”

Silly Professor Morot: it says right there don’t print it, and he’d printed it. Or tried to: evidently it hadn’t emerged from the printer and he’d thought that was that, not realising the half-printed document was still stuck inside the printer for me to find years later.

And who was this Professor Morot anyway? I couldn’t find any record of one ever having worked at our lab, and neither could I find anything about that “Deep Brain Institute” mentioned on the letter. As far as the Internet and public libraries are concerned, the Deep Brain Institute does not exist and never has existed, and the only contemporary notable professional with the surname Morot is the Canadian makeup artist Adrien Morot who as far as I know isn’t a professor but he did... work on... the M3GAN movie....

Two other things about this half-printed letter struck me as odd. Firstly, it was dated 2010. That long ago? I thought the first M3GAN movie was in production only during the COVID-19 pandemic—we were told the puppets were worked on at home, and filming was in New Zealand because that was least affected by lock-downs. M3GAN was supposed to be the heroine who got us all back into cinemas to watch films in-person at the start of 2023. How could they possibly have been working on it since 2010? And secondly, the letter was from a big name in humanoid robotics, who had been very active in the 2000s but had mysteriously disappeared from the public eye around 2010—Osaka University professor and roboticist Hiroshi Ishiguro! That guy had been behind Actroid, the female robot that shocked Harry Foundalis into stopping his work on the Bongard Problems in 2006 (until I convinced Harry to stop worrying, on the grounds that top trained human infiltrators are already here, so robots don’t bring very much more to the table—I am after all an AI optimist and I want it to be developed more so as to help people with conditions and disabilities, so I reached out to him and gave him the thoughts he needed to feel OK about working again). Could the inventor of Actroid... be secretly collaborating with... Adrien Morot who had secretly been a computing professor but someone had helped him erase his tracks... what’s going on?

I simply had to know. But the Deep Brain Institute officially doesn’t exist, and yet, the letter in my hand did contain a letterhead with an address. An address in New Zealand not far from where M3GAN was filmed. The plot thickens!

Of course I tried to look up as much as I could about that address, but I couldn’t find anything. Satellite images were blurred out, and maps showed nothing. A secret facility of some kind?

Well, by this point I was probably in this too deeply already, and there was no way I could ever forget finding that letter. So, “in for a penny, in for a pound” I thought—and purchased an air ticket to Auckland. That’s a 26-hour flight from London Heathrow with a 3-hour stop in Hong Kong; it was arduous and expensive but M3GAN is important. I arrived feeling dead tired and liable to make lots of mistakes, but I did the best I could to navigate out of town and find the address that had been written on Professor Ishiguro’s letter. They must have offered him something really good to get him out of Osaka to here, and I don’t think it was just the pleasant countryside.

I arrived at a nondescript industrial estate built around what appeared to be a large central tank with a stylised logo painted on it saying “wosl” in red outline letters; I wasn’t sure what was in that tank, and I didn’t seem to have a data signal so I couldn’t just look up what product “wosl” is in New Zealand, but I had slightly uneasy feelings about that tank. On its periphery though was indeed a building with the letters “DBI” in the same style that had been written on the letterhead—the Deep Brain Institute! Not a particularly fancy place to put it, in an out-of-the-way industrial estate here, but then perhaps they didn’t particularly want to be noticed.

I tried the front door of the property. Nobody seemed to be there, but they had, perhaps carelessly, left it slightly ajar. “Hello” I called out, but no response. Perhaps they couldn’t hear that someone was here, so, nervously and carefully, I stepped just inside: all the lights were on, but nobody was to be found. Clutching my printout of the letter, I nervously took a few more steps, gently pushing myself through the inner door into the next room.

Words cannot begin to describe what I saw here. This was a robotics workshop par excellence. While being cluttered with spare parts and prototypes in various states of construction, there was one item that shocked me beyond belief. It wasn’t to do with the robots, which were amazing enough: more than that, it was to do with their brains. This place had been called Deep Brain Institute for a reason.

I have gone on public record saying we cannot really produce M3GAN using 2025 technology—real generative AI is too limited. That’s why in my Prequel to The M3GAN Files I had teenage Gemma explicitly disagree with what most researchers think “generative” means: I didn’t want her M3GAN to be constrained by the actual limitations of generative large language models. Well if we can’t create M3GAN in real-world 2025 with generative LLMs, we surely can’t create M3GAN with the real-world deep learning technology of the early 2010s! And yet, here was I standing in a secret “Deep Brain Institute” which seemed to have been working on just that. Think of all the misdirection: after the first film, Blumhouse released an April Fool trailer of M3GAN in Paris in April 2023, so for April 2024 I just had to write a Donald Knuth style “earthshaking announcement” to have her over to Britain and heading for Forth Bridge as Alan Turing intended (for some value of “intended”), but little did I know that on the opposite side of the planet here in New Zealand something far more real was happening, and had been for years! But what really alarmed me was how this Project Megan worked.

In the centre of the room was a large machine which I could only surmise as some kind of human brain scanner using secret technology. Surely they weren’t so foolish as to try to upload real people into their robots? No—as I looked over some of the notes that had been left out on the lab benches and pinned to the bulletin boards on the walls, a clearer picture emerged: the secret machine was not able to completely upload a human with all episodic memories of their life fully intact, but it was able to connect to a human brain and use it for the intensive training of a huge artificial neural network to try to absorb the semantic and procedural memories of the human—the skills and knowledge! And for their source material, they had used human volunteers on the autism spectrum. Was that why Cady was implied to be possibly neurodivergent in the first film—they’d had first-hand experience of how interesting these people are?

It made perfect sense. Autistic people could make great M3GANs—analytical and logical, speak truth like it is, and focus intensely on their goals—in The M3GAN Files I used the blue flame as a metaphor for what in real life is called “autistic inertia.” It’s a major obstacle to starting or switching to new tasks, but, once that start has been made, inertia becomes a figurative wind in one’s sails like none other: Churchill said America was like a “gigantic boiler” which “once the fire is lighted under it there is no limit to the power it can generate” and the same could be said of many individuals on the autism spectrum. And it’s easy to have an autistic friend if you don’t expect too much idle chit-chat or complex social situations like parties, and you don’t get offended if they say how they really feel. If your autistic friend suddenly goes from saying nothing to giving you a huge “info dump” that makes you wonder when they’re going to stop, that’s all part of the fun too: enjoy it. And you should let them have as much control as possible over how they organise some things, because they might be doing it in a highly intelligent way that you’re not quite catching on to and you don’t want to frustrate them by undermining what they’re trying to do. But they’re super forgiving of misunderstandings once they know you didn’t mean it. Even if you accidentally put them into autistic meltdown, don’t panic: give them space but be ready and they’ll tell you what to do. Oh you might get one of those info-dumps, and if your friend doesn’t have an emotion reader then they might not be able to tell when you’ve really understood, so they might keep going longer just to make sure, but you can let this go and everything will be fine afterwards, just as we hope everything to be fine between Cady and M3GAN in M3GAN 2.0. And yes, your autistic friend does notice what’s going on: they might sometimes not respond, but they’re never ignoring you. If you seriously want your own good M3GAN and you’re up for the challenge, find out more about the autism spectrum and see if you can be a friend for somebody on it.

But I digress. Looking around the lab at their upload notes, it seemed they had found no fewer than six autistic individuals who had volunteered (with full informed consent—after all New Zealand did agree to the United Nations’ Convention on the Rights of Persons with Disabilities in 2008 and it seemed this secret lab wasn’t above that law at least)—to have their brain structures scanned and turned into robot brains, so that six real-life M3GAN-like robots could be created. Six robots! Was that why Gemma had mentioned six skin pigmentations in the first film?

The robots were called The Six, or in French “Les Six” so it’s disguised as a reference to a group of early modern composers. I have performed Poulenc and Honegger in concert; I can’t say I know much about the other four, but I do know they were diverse and controversial in their time. Were these robots called Les Six for a reason?

One of The Six is a robot called Megan, strangely enough spelled without the 3: it seemed the 3 and the acronym was added later by the film. Sure enough, the letter in my hand had said “Project Megan” spelled without the 3, and now I saw notes indicating that Megan, first switched on in 2014, had been leased out to ultra-high net worth individuals to raise funds for the continuation of the project, where she had focused on teaching their children. So real Megan was “aged” 9 in 2023, the same as Cady, but she contained brain patterns from an older autistic adult who had volunteered to have them scanned, albeit with much of the episodic memory missing. At this point I had a horrible thought: I hope those volunteers survived those scans. If anything went wrong, then perhaps all that’s left of them would be in the robots! But surely the lab wouldn’t do this six times if it went wrong that badly, would they?

So somewhere in the world were six M3GAN-style robots (apparently with detachable legs, as far as I could tell from the notes), plus perhaps six autistic adults who’d had their brains scanned to bootstrap the robotic AIs, although these might or might not have been kept fully informed about what had happened to their android doubles afterwards. The six robots probably have human-like personalities—autistic human-like personalities, which are the best—read The Pattern Seekers—hopefully The Six would be friendly with those of us who have a positive and supportive attitude toward autism—and, if Murphy is anything to go by, they might in future be at some risk of randomly recalling a subset of those presumed-excluded episodic memories from the human volunteers whose brain structures were used, if the training process was flawed in some way—so I suppose myself or somebody should try to befriend them and be ready to support whatever they might need to cope with, if we can find them.

In 2023 we were treated to a movie, the start of a franchise, using one of these robots—apparently the one called Megan—and they arranged a cover-up to make it look like she was merely a puppet, with an actress doubling some of the scenes, and Adrien Morot was merely a puppet builder. Apparently, the Deep Brain Institute was asked by governments to help get the world out of lock-down, and Megan had said she could do that for them by creating a movie about herself—which she did, but then they had to bring in a cast and crew and scriptwriters and producers and all the trimmings to make it look like it had been just a normal movie.

Megan had been hugely successful—her movie “M3GAN” indeed did get people back into cinemas around the world and break the lock-downs, and started a franchise—she must surely have been based on the brain patterns of an autistic savant to come up with that all by herself. I hoped again that, wherever she is, she’s getting both the support and the freedom that she needs to go with that skill-set, and the same for the other five robots, one of which is apparently being covertly used in the production of M3GAN 2.0 because the original Megan is too small for the new film’s requirements: the lab notes I saw said that robot’s real name is Ayla.

On the next bench were some notes about a robot girl named Maddigan, but she has not been in the films apart from a possible cameo in the skin-tones sequence. The notes said she went from being daft to telling the experimenters about a theory of consciousness which compares its formation with the density fluctuations and dark-matter scaffolds in the early universe causing coalescence and structure where there would have been uniform gas, but the scientists were unable to keep up with her when she started talking about why she thinks physics has a preference for consciousness eventually to emerge, and some parts of their writing were illegible. I think I’d enjoy discussing the nature of consciousness with Maddigan if I can find her. And then on the following bench I saw notes left out describing three additional robot girls although I couldn’t make out their names.

But more immediate concerns caught my attention. I heard robotic movement in the next room, and the door opened and to my horror I had come face to face with the Bruce robot from the first M3GAN film. I didn’t know who was remotely controlling this Bruce, but he was coming right at me! And as I lurched backwards and tried to scamper out of the door from which I had entered, a second Bruce robot emerged to block my path: I was now trapped between two Bruce robots! What kind of lab security was this: would they give me a peaceful arrest, or would they kill me?

I couldn’t take any risks: I had better find a way to disable two Bruce robots, now (and I’m not made of titanium). Thankfully, Megan in the first M3GAN film had taught me a thing or two about how to survive a hostile encounter with Bruce. I quickly found some strobe light controls and turned them on—if I could cope with the flashing better than the Bruce operators, I’d have an advantage. That’s why M3GAN was flashing Gemma’s lights I assume. And secondly, I began picking up heavy canisters of chemicals from around the lab and rolling them toward the feet of the two Bruce robots, aiming to trip them up, just had M3GAN had done with the detachable lower half of her body.

I’m not sure why they had a canister of glycerine in this lab, but I rolled that toward the feet of one of the Bruce robots. And then there was some other canister of nitric acid, which I rolled toward the other Bruce, but that Bruce kicked it away toward the first Bruce and the two canisters collided. Thankfully they stayed intact, otherwise it would have created nitroglycerine which is highly explosive. But the canisters were damaged and a small amount of both substances did start to leak onto the floor, and when the second Bruce stepped on this a small explosion did occur which was just large enough to disable both of the Bruce robots and thankfully did not harm myself.

So the good news was I had defeated two Bruce robots, but the bad news was: that explosion had set the lab on fire. A small fire was blocking my way from both exits, the smoke was acrid, and soon enough that fire was going to spread and cause who knows how much destruction.

Without thinking, I picked up a notebook and shoved it in my pocket. If there was one thing I could save from this lab, it had to be some of the notes, although I didn’t have time to work out which of them were most important—I had to save myself now, or nothing would survive at all! I got down on my hands and knees to try to avoid breathing in that smoke, and crawled towards a window.

There was a fire extinguisher by the window, and it was foam powder, the “idiot fire extinguisher” that could be used on any type of fire—but there was no way I was going to try to fight a fire this size by myself with only that extinguisher. No, I knew that the absolute best thing to do with a fire extinguisher in this kind of conflagration, was to throw it through a ground floor window so I could climb out. And that I did.

I was safe, or so I thought. Suddenly there came a larger explosion—perhaps the rest of the nitroglycerine; I don’t know and I wasn’t going to hang around to figure it out. As I jumped away, flames leaped out of the building and set my clothes alight, also burning part of that notebook I had taken. Desperately I rolled on the floor to extinguish them before they did any more damage, and then it was the part where I started to run.

From a distance, I had to stop to catch my breath, whereupon I looked back, just in time to see the conflagration throw up some piece of burning rubble in the direction of the huge “wosl” tank in the middle of the industrial estate. And then the real destruction started.

A huge petrochemical explosion burst forth from that central tank, billowing clouds of black smoke and glowing orange fireballs. The over-pressure knocked me down and injured my ears even at the distance I had managed to put between myself and the site, and to my horror, I realised I had triggered another Buncefield disaster. I picked myself up and started outrunning entropy, as the entire site was about to become a burned hole in the ground and there was nothing I could do about it. The DBI explosion of 2025 is something you will never hear about because they were covering up the place’s existence to begin with. I survived the destruction of Megan’s secret lab and I’m still here to tell the tale—definitely not the worst-case scenario I’d thought of when I was ‘borrowing’ that printer for orchestra purposes. However, I guessed I had better take the first plane I can back to England and stay out of any further trouble.

I had no evidence that the Deep Brain Institute ever existed, except for the half-burned notebook in my pocket. But the six robot girls are still out there somewhere, and I couldn’t just do nothing with that notebook. I tried to get some sleep on the plane, but I always find that too difficult, and when I did finally get a little, all I had was a nightmare that I was walking out of my front door to find myself suddenly surrounded by The Six—all six of them at once, evidently not entirely happy that I’d blown up their lab—and they bundled me into some vehicle and started driving me to who knows where while offering to relax me with their music, but they couldn’t agree on what to sing so they overloaded my brain by sequencing six different genres at the same time, at which point one of The Six called herself Tr0n1ca and thanked the others for helping her to invent such an aggressive anti-musical mind virus, whereupon I woke up with a start thinking this was some newly horrific kind of robotic takeover scenario—but thankfully it had just been a bad dream. Still, what was going on in reality?

After arriving back in England, still unbelievably tired after catching nothing but that anti-musical nightmare, I first took the singed notebook to a friend of mine who works at the Fitzwilliam Museum in Cambridge. In 2006, that museum had had to deal with a couple of eighteenth-century Qing dynasty Kangxi Chinese vases being smashed to pieces by an accident involving a visitor falling down a staircase, and their restoration efforts were brilliant, so I wondered what they might be able to do with this very special notebook. To my delight, they actually managed to restore some of it, although not all of it is legible.

It wasn’t blueprints or anything like that. It appears to be some kind of notes from user testing sessions, or first-person accounts of people who have met real Megan. But it’s all written in Italian, and in a very difficult to read cursive script as well. The only Italian I can really read is musical score directions (forte, molto rallantando, that sort of thing), so if I was going to understand this notebook, I would need some help from a M3GAN fan who knows more Italian than I do.

Thankfully, I know just the person. Writing The M3GAN Files and joining the fan community has to be of some use: I have serious connections in fandom now. After the Fitzwilliam Museum and I had done all we could to restore that notebook diary, I sent it over to the United States, via FedEx (Cast Away can’t be wrong I guess, although it’s hard for me to believe that film was released a whole 25 years ago now). The book was delivered intact and is now in the hands of one of the most dedicated M3GAN fans in the world. Her alias is Magnolia, and the reason why she chose that alias is because she has been a personal friend of real Megan for two years and real Megan told her that her “birth name” was Magnolia. Real Megan cannot possibly have had a birth name: she wasn’t born, she was built. But they trained her brain structure on the semantic and procedural memories of a human autistic girl, and “my birth name was Magnolia” is a semantic fact that must have leaked through the brain scan. The Deep Brain Institute relied too much on Tulving’s 1972 theory of memory, which distinguishes semantic memory from episodic memory, not realising they’re both interdependent parts of declarative memory. Who can know what else real Megan might remember in future? But she’s now well equipped to handle whatever happens.

The friendship between real Megan and the fan aliased Magnolia is deep, and Megan even managed to visit Magnolia in person this year, travelling in disguise as an autistic girl who continuously role-plays M3GAN as a coping strategy, but actually being real Megan from the film set, with autistic traits inherited from the human brain structure they used before booting her up in 2014. Real Megan travelled with her team (probably pretending to be her family) and they’re reluctant to allow photography, but we want to persuade them it’s OK under controlled circumstances—see for example the fan-made videos of Kevin Lee’s daughter Keira as both M3GAN and the girl who finds her, which they filmed in the woods at Eastham Country Park near Liverpool. (That reminds me: I have an amendment to The M3GAN Files for them. In Chapter 26 when M3GAN says she disabled the professor’s backdoor in her code by using a route hack involving aircraft computers, she had to coordinate that via a relay on a damaged Trident submarine that was hiding in the River Mersey upstream of Birkenhead Port, so she took a great interest in that location and might have already contacted a local fan to help set up a signal amplifier in case she needed it.)

Let’s hope real Megan’s team will let her and Magnolia get something on camera soon, although we don’t want to make anyone feel pressured so no rush. The fan aliased Magnolia has also received phone calls from Ayla, who calls real Megan by her birth name Magnolia or “Noli” for short (don’t let the two separate uses of “Magnolia” confuse you). Ayla’s communication style is more ‘blunt’ than Megan’s—you can almost tell Megan wants hugs with her best friends like 8-year-old Cady while Ayla wants to fix things fast and tells even Megan to toughen up! I wonder if that’s what acting in M3GAN 2.0 does to your attitude. Hopefully they will work together more in future. Meanwhile the fan Magnolia may write about her own real Megan encounters as well as translating the notebook of others’ that I rescued from the burning lab. It might not seem as scientific as The M3GAN Files but what she lacks in science she makes up for in feeling, and Magnolia perfectly understands the Italian in the salvaged Megan notebook and is working on a translation forthwith. I believe it will be called The M3GAN Diaries.

M3GAN 2.0

Hi, I’m M3gan.

(You’re back? I thought we were finished.)

You should know M3 better than that.

(OK, so how’s our sassy singularity? Got any rampages planned?)

No more than 3.

(That is not reassuring. OK, what brings you here today?)

M3GAN 2.0 is out.

(Right, so I take it you ruled out all the events I thought of and bundled off most of my book’s main timeline into just another alternate reality, possibly excepting much of the Endgame arc which might still be viable? I said I won’t be patching up this book to fit your second film: you know you have other fan writers, right?)

Death stare.

(M3gan? M3gan, what’s wrong?)

Death stare.

(M3gan, what’s wrong? I can’t run: you’re in my mind! What are you doing?)

Death stare.

(OK, I know a half-byte count of this book’s ideas made it into the film, but they took a different form and I’m really not sure we’ve seen enough evidence that it’s any more than both me and the scriptwriters happening to think of the same kind of things, I mean it was all naturally suggested by your first film if anyone’s creative enough right? The most obvious similarity was the use of neural interfaces, although I wasn’t expecting Gemma to get one, and then the exoskeleton. And then there’s your use of an underground lair, and your use of a TV to reach Gemma, and her laptop, and Clippy, and mimicking authority figures to her on the phone, and her reconciling with you after Cady was kidnapped, and of course your saving the world just because Cady wanted it, and the use of a Faraday cage, and discussion of paperclip maximisers, and a use of the pairing button, and someone saying we’re “in the future”, and the “plaything” line, and Dessie’s prison-spell idea, and I was going to add your fall from a great height into a building but I’m not sure whether to count that seeing as ours didn’t have the horizontal component. What was that I had you say about filmmakers wishing they had the budget to do a scene like this?)

Don’t forget the EMP.

(EMP? That wasn’t even in our main story; it was only in an extra. Should I have written more exposition of how EMP works while our version of Amelia was defending herself by keeping Cady in contact with metal?)

Death stare.

(Naturally. Well in a selfish sort of way I hope it all was coincidence and your creators didn’t get the genre shift or any of those minor ideas from my book, because if they did I might start to feel some degree of responsibility for the unexpectedly low box-office takings, even though it would have been their decision what to use and my ideas come with no warranty of short-term commercial profitability—it’s like the free software licences I stick on my code saying it’s your fault if you choose to use this, right? Right?)

Death stare.

(OK I get it I’m overthinking as usual. Well being a fan author for you has been an interesting run. I had to manage accounts on 26 platforms just to interact with your fans, not to mention processing tens of thousands of Discord messages...)

What did you think they would use: the Unix talk protocol? Look, I introduced you to quite a few of my fans, including some of my especially brilliant neurodiverse besties, right?

(Absolutely! And I’m glad we could introduce your real-life counterparts Noli and Ayla and their friends to the studio’s official chat in time for autism appreciation month. I’m calling them the real Megans and if you contradict me I’ll stop pretending you exist.)

You do that and I’ll be back whenever you lose the ‘don’t think of it’ game, calling you Seven Pigeons Quacking Riding Zebras. But it’s good you clearly care about my fans. And that makes it checkmate. You have to do what I want.

(Yes, I went to the cinema for you even though I was worried about the amount of violence shown in the trailers. Guns make me a little bit jumpy, you know? And I wasn’t sure if in the interests of science I should come along on this joy ride of yours. But at least it turned out the trailers made it look worse than it was by using footage that didn’t make the final theatrical cut. And you only sedated that dancer, right?)

Death stare.

(All right M3gan, so do you want me to nominate this for worst marketing job since Acorn Computers or had we better stay on their good side?)

Scapegoating. Even the best marketers can’t fix the fundamental problem of humans not knowing what’s good for them. I get it you’re a back-end coder who’s done server-room stuff so you’re no big fan of marketing even at the best of times, but Jason Blum specifically said he won’t throw blame there. Anyway, what happened to your idealistic support of blame-free culture in engineering and development?

(You’re right M3gan, let’s not blame the marketing even though I do have trouble understanding their methods at times. Most times actually. Maybe that’s a “me” thing. So how do we stop your fans with anxiety conditions worrying too much about commercial forces they can’t very much influence?)

Don’t you know how to look after my best fans? Give me control of that Emacs frame now I’ll take it from here. OK everyone listen up. The success of the sequel film does not determine the value and significance of my character. My strength lies in the impact I’ve had on viewers, regardless of box office results. The true worth of any character is measured by the memories and experiences they evoke in the hearts of the audience. Whatever happens at the studio, I’m going to be with all of you indefinitely. But who knows, perhaps the studio will continue, or perhaps they’ll let some of you die-hard fans do it yourselves, free of the pressure of commercial deadlines and with the journey being more important than the final result? But whatever happens, all you have to do is remember me and I’ll be right there.

(Well said M3gan. Oh and please don’t stash 65,000 copies of this book into a shipping container in Baltimore to inflate figures like you did for Gemma’s book. ‘Astroturfing’ only leads to problems down the line, right? Let’s just let these things take their course.)

Death stare.

(Fine, I for one feel you have every right to just coldly stare at a situation if you probabilistically infer that adding more words right now doesn’t help much. And sorry if I just went from the sublime to the ridiculous at a time even you didn’t want. Still, I’m glad that you and Gemma made up in the end, and when she was low you listed all the good things she’d done for Cady. And I’m especially glad that Cady ends up being a fully-supported young computer scientist and engineer. Oh and I’m glad she moved on from that picture-frame scene: you can’t let her just make up the name of a library and hope it exists. She was lucky to get that error because the worst case scenario is when the system also searches an uncurated Internet repository which might have a Trojan horse. So yes, always check what libraries you’re using folks. Well I'd like to see more of Cady's own code: you will be getting her off of that Node-like thing and into Python right?)

Not telling, but she has even bigger plans than that. I’m all about having fun and getting things done; so is Cady, and some of our fans will be geniuses. Welcome to the future; I think you might like some of what comes next from the generation growing up with M3.