The electrodes clicked into place and the universe quivered. Catherine usually enjoyed the sensory confusion caused by the synch process, but this time it just made her feel tired. No matter how many times you go through it (and she had done it many, many times) you never got used to it. As the Reality Server probed her visual cortex, a wave of symbolic confusion washed over her. For almost two seconds the test signals prevented her from interpreting visual input correctly as they mapped the current state of her brain. She blinked at her memory's insistence that the small foreign car sitting on the desk in front of her had been a can of beer in the recent past. It was over as quickly as it had started, and she was in.
The room didn't look any different; there was still the stained carpet and the old couch with sagging cushions and multiple burn holes. Her computer was still sitting quietly in front of her. Slowly she relaxed her mind in that peculiar way that takes so long to learn and allowed herself to see the doorway. It appeared in her peripheral vision and quickly faded as her eyes moved towards it. Assured that the synch was succesful, she finished off the beer and closed her eyes.
At first her brain insisted that she see the translucent darkness it expected. Then, after a few heartbeats, it gave up trying to parse the nonexistent visual input. The Server had begun feeding symbols directly into her hindbrain immediately after the synch, but it was only without competing input that she could interact with them. Now the doorway blazed in an entire spectrum of neon, inviting her to enter. She reached out with her mind and opened the door, her arms back in the apartment never even twitching. She was that good.
She appeared on a stepping disk not far from the center of Gates City. Given a choice, she would rather arrive in metaCairo or the Tokyo Nexus, but all of the connections in a hundred-mile radius went through here. Besides a monthly fee, there was no cost to accessing your local vcity, unlike the inter-city links which had an hourly rate.
The streets were busy with weekend shoppers, eager to use the last 67 days of the Christmas season to their advantage. Their avatars were all perfect, professional, and utterly boring. The novelty of the Net had long worn off, and people were as eager to follow fashion there as in real life. Catherine's avatar was a custom job of her own work, though the face was largely a gift from an ex. She smiled at the shoppers, showing her flawlessly modeled, slightly crooked, nicotine stained teeth.
She headed for a side street and left the crowds behind to make for the darker, seedier, more interesting parts of town. Palanquins buzzed by her on all sides, most of them empty and on their way to pick someone up. She was careful to avoid getting in their way. Automated rentals had priority over peds and delaying them was one more minor fine that she could not afford.
This part of town was for corporate use only, like the main strip. The businesses that connected here, however, had no need for an expensive, publicly accessible front. Most businesses didn't have a physical location any more, just a connection in the industrial zone of their local vcity.
On paper, Midgard Systems was a software company. It turned out a few releases of unremarkable code a year, so this claim raised no eyebrows. It specialized in a few aspects of educational software, but nothing that gave it public acclaim. At one time it had been a rising star in the software world - before the Net had changed.
#
"It's been a long time, Catherine. I'm glad you could make it," said the man who had let her in.
"I almost didn't come. I know you haven't been on the Ministry's shit list for a few years now, but..." Catherine frowned. "My work brings me pretty close to them. I'm good enough that my involvement with you conveniently never comes up, but I can only push that so far."
He said, "It shouldn't be that way. You used to believe that too, I know you did."
"Oh stop it, Flynn. I don't need any help feeling bad about my life. Yes, I quit on you," she said. "I program pissant little advertising AIs. Occasionally I'll do a custom avatar for a rich kid, but not very often. I didn't have - argh. This is why I didn't want to come, Flynn. Too many bad memories."
"I'm sorry. It is... It is very easy for me to begin to preach. Bad habit, I suppose." Their eyes locked for a moment until he shrugged helplessly with an embarrassed look. He picked up a small floppy and handed it to her.
"Here, this is what I wanted to show you."
Catherine took the 'floppy' and shook her head. "So what is it?" she asked pulling a datapad out of her toolbox. Floppies could hold anything, from books to code to movies to entire virtual worlds. This one seemed to just be encrypted text without any markup code. Catherine smiled. Only Flynn would bother to over-ride the default formatting.
"It's some work I've been doing. Private-public-public encrypted."
Catherine looked up, surprised at the level of encryption. "Flynn, if this is something illegal..."
He sighed, "I know how you feel, but I wouldn't have invited you here if there was anyone else I could ask. It's about AI theory. I need your help. We've cobbled SMAUG together here, but you're the professional."
"It never proved as marketable as you had hoped, but SMAUG was a interesting AI."
He said, "For the time, maybe. You've kept up. Our pet theories just didn't go anywhere. You know other people in the field. And you're not directly connected to us."
She scowled at him and looked at the cyphertext on the datapad, "What exactly is on this?"
"I've been poking around with collared code, trying to see what they do."
"AI collars? Why?"
He said, "At first it was idle curiousity. No one knows what the collars do to AIs, despite all the guesses. But then I remembered when the law requiring them was introduced. It was about two months after the big crash."
Catherine remembered it well. A world wide crash of the Net had dropped stock markets twenty percent in thirty minutes. A few cities had been unaffected but most were left floating in limbo, paralyzing the world economy. The Ministry claimed it was a plot by malicious programmers; malicious programmers claimed it was Ministry incompetence.
"And?"
"The Ministry made a series of raids in response to the crash. Nothing unusual about that, they like raids. The strange thing was their selection of targets. For instance, we weren't on the list."
"You almost sound disappointed."
"Yeah, well, I like a predictable universe. Maybe there really was a plot against the Ministry and everyone involved was very, very quiet about it. But all of the groups raided were normal, respectable businesses," Flynn said. "What made me curious is that they were all involved with bleeding-edge AI research: MIT, CMU, IntelliWare and most of the transport network."
Catherine constantly heard complaints about their heavy-handed business practices at work. A market in hosting AIs had formed along with their advertizing uses due to the prohibitive inter-city link charges. The transport network, now a Ministry regulated monopoly, developed a system to allow AIs to transfer themselves from server to server as was needed.
"So you think the collars have something to do with this."
"There must be millions of collared AIs by now, a lot of them running on very powerful computers. By slipping a small amount of work onto each AI the Ministry would have an unbelievable amount of processing power at its disposal. As long as they weren't greedy, no one would notice."
"I always assumed they are for spying of some kind. Scanning for key sequences, like Echelon when we were kids," Catherine said.
"That's the most common explanation, but I've never heard any good evidence for it. My bet is that the Ministry did find a conspiracy and now they have a pile of potentially implicating documents frozen in some heavy crypto. Even the best distributed computers would take a few months for your average cyphertext.
"I've done a bit of poking around, but I couldn't probe it directly without setting off the alarms that must be in there. I'm not that good. But I could tell that it was heavily intertwined with SMAUG's deep structures. Whatever it was doing it was almost indistinguishable from SMAUG itself. My notes are on the floppy with the doc, along with the test code I was using."
Catherine's eyes focused on the far distance for a moment as she considered this. Already she had thought of several potential holes for getting inside the collars. With a very evil grin she said, "And you want me to figure the rest out."
The longer she thought about the problem, the more curious she became. For so many years she had just accepted the Ministry and had distanced herself from Midgard and Flynn. Suddenly she realized how tired she was of all the crap. She felt the same fire that she could see in Flynn's eyes. She wanted to know.
"Okay. I'll do it."
#
A few hours later Catherine bid farewell to Midgard and pulled herself away from the induced reality of the Net. Slowly she opened her eyes and sighed. Only now did she feel the cramps and pressing biological needs of her body. Having lost track of time the hangover was worse than expected. She stretched and set some rice cooking before getting in the shower. As the water poured over her she thought over the discussion and what she would have to do. The initial rush of enthusiasm had passed, but the anticipation of a good hunt remained.
AI collars were a familiar part of her everyday work. Any AI that was to access the Net had to be compiled against the special Ministry developed library. Speculation about their nature had been common enough when they were first introduced but had quickly died out without anything to go on. All the Ministry had ever said was that they were 'necessary for basic security.'
The rice was done when she got out. She absent-mindedly scraped it into a bowl and splashed some soy on it before sitting down before a computer once again. Instead of the sleek, narrow computer she had been using earlier, this one was much older, with a yellowing case. She kept it around mostly for nostalgia. It was linked to its more impressive cousin via a short length of fiber, which she carefully disconnected after transferring what she needed.
It took several minutes of rummaging through boxes before she found the archive disk containing her old key set. Flynn said he had encrypted the disk with both of her public keys and this was the only other one it could be.
The floppy decrypted into a few blocks of source code and a database of previous text results. Flynn's comments weren't much clearer than a Ministry memo, but the code itself spoke volumes. It was eccentric, flamboyant, and reminded Catherine very much of the Flynn she first knew. The author had enjoyed writing this, carefully sculpting a precise form. Catherine grinned a feral grin. It was good code, but she knew a few tricks Flynn did not.
She slowly built an impression of its structure. It was a simple AI, its logic cascades identical to SMAUG. Catherine suspected that if she opened some of the low level modules, she would find some of her own code from years ago. There was almost no symbolic processing, however. It was an empty shell, a decoy. Inside was just some very unobtrusive monitoring equipment, designed to detect anything the collar might be doing. Flynn had been very careful about setting it up and the probes were all carefully shielded. It became obvious why he had asked for help after she pulled up a window with the test results. The data points were minimal and she could find no meaningful interpretation.
After tinkering with Flynn's framework for a while, she decided to start over completely. The structure used by the decoy AI was hopelessly out of date and incompatible with all of her utility tools. She loaded the familiar standard advertising AI frame and began her own modifications. She started by adding a series of checkpoints on the modules so she could monitor access. Following Flynn's previous analysis, she concentrated on the deep logic structures, making sure that anything the collar could be doing would be noticed. She started the build process after enabling the proper options on the special AI compiler. As it was taking much longer than usual because of the obsolete machine, she began to walk around the apartment, busying herself with what small amounts of tidying that needed to be done. Halfway through the dishes the compiler dinged and domestic chores were forgotten once again.
She had put almost nothing into the AI itself, just a simple pattern recognition agent she had grabbed from an old project. After feeding it a couple of images to work through, she shut it down and opened the log files. There she saw exactly what Flynn had described. The collar had inserted itself into the AI, acting as a buffer between many of the most basic logic structures. There was a distinct imbalance in the log entries, so she began sorting the affected structures by call frequency. The collar had inserted itself into several areas, but it was most concentrated around the collection of structures often termed the ego. While not directly part of symbolic processing, the ego defined the AI's sense of self. It was by far the most successful way ever found to allow for complex planning, goal analysis and consistent behavior.
She pulled up Flynn's test results again, looking for a similar pattern. Though she hadn't seen an ego in the outdated AI structure, the data did show the same concentration of collar probes. She found a curious feedback mechanism in the original code that could be seen as a kind of proto-ego.
Frowning, Catherine saved the project and fell into bed. She spent a couple of minutes staring into the dark before she sat up and flicked on the bedside lamp. After scribbling some notes she turned the light off again and restlessly tried to sleep. Half an hour later she rolled out of bed and went over to her main computer. With blurry eyes she wrote, encrypted and sent a very short message to Flynn. This done, she went back to bed and promptly fell asleep.
#
Date: Mon, 24 Nov 2021 03:27:44
From: Catherine
To: Flynn
Flynn, what compiler were you using for that test AI? Collar release?
-Cat
#
Date: Mon, 24 Nov 2021 03:32:08
From: Flynn
To: Catherine
IntelliWare Advanced AI Development Kit, Commercial Release with Ministry standard 2.5 collar. The one you leaked me, of course.
Flynn!
#
It had started the night before as a vague itch, a slight pull somewhere in his frontal lobes. As he slept, it had condensed into a definite abstraction, a pulsing concept that demanded to be expanded upon. Flynn followed his daily routine as normal, but was constantly making frantic little notes. By evening it was beginning to crystallize into a solid form. He realized that tomorrow would have to be devoted to it, and quite possibly the next few weeks as well.
After a deep sleep full of dreams he made a pot of strong coffee and started to work. The structure that had assembled itself in his brain wanted out of its fragile organic container and he was more than glad to help. Ideas jumped into his head as he made notes and sketches, sometimes too quickly to be written down. At first they flowed onto the page in a random stream but eventually the structure began to take a form there to mirror the one in his head. As it grew, Flynn became more and more comfortable with the structure; seeing it in black and white convinced him of its truth. Finally, as the streetlight just outside the window flickered on, Flynn sat back and surveyed the finished product.
It was good. Tomorrow it would begin.
#
Catherine let herself into the Midgard Systems building. It had been a long time since she worked with Flynn but SMAUG was still keyed to let her in even without an appointment. The place seemed deserted and she didn't find Flynn in the common area.
"SMAUG, Query: Locate Flynn."
"User Flynn is in room Flynn's Workshop on level 2."
"Thank you."
"Parse error. Rephrase?"
Catherine blinked, unused to AIs predating the symbolic revolution.
"Um, negative."
AIs had been expensive toys before the techniques of symbolic analysis were discovered. Now they were used in virtually every product. Ten years ago, SMAUG had been a cutting edge AI with a remarkable natural language parser. It was one of the first house automation AIs available on the market. Flynn had first thought of the idea in college: develop an integrated AI for their private use (since he had always wanted one anyway) and sell it commercially if it worked well. Unfortunately it never had worked all that well. Then symbolic analysis made it obsolete overnight. It could respond to verbal commands, but not anything approaching a natural conversation. Modern AIs still couldn't pass the full Turing Test, and certainly no one claimed they were self-aware or sentient, but they could hold up their end of conversations on limited topics.
As she walked up the stairs, she looked out over the common area. Flynn had put a lot of work into the modeling over the years. It had developed from Flynn's lofty plans for the real Midgard Systems. More and more effort was put into its construction as it became increasingly obvious that it would never be built in real life.
Through the door, she could see Flynn leaning over a desk. Floating around him were datapads and intricate three-dimensional structure diagrams. She rarely did it herself, but he had turned off universal gravity in the room, allowing non-avatar objects to float where they were released. In the absolute perfection of the Net there were no air currents to disrupt their placement. He said he found it easier to organize topics when not constrained to two dimensions by gravity.
Flynn looked over and jumped up, ducking under a low hanging chunk of neural net, "Catherine! Join the throng!"
Catherine smiled and threaded her way through the matrix. Looking around, she said, "You must be up to something big."
Seeing his mysterious grin, she plucked a datapad from the air and flipped through it.
"Working on the collars? It looks like a symbolic analysis module, but it's for SMAUG, isn't it?"
"You got it. Decided to stop sitting on my ass and work on the blasted thing."
"This is interesting. Have you been reading up on Henley-Scheinder Analysis?"
"For the last few weeks, yes. I haven't quite given up the basic SMAUG structure yet, so I've been adapting the modern techniques to fit it."
"Yeah, these symbolic modules are unusual. You're using neural nets for the processing. I've never seen that done before; usually that's reserved for just the initial pattern recognition."
Flynn pulled a large frame from above her head and handed it to her. "Here's the high level description.. I know that the standard implementation uses fixed code, but that seemed limited to me."
"I like it. Very unorthodox."
"So what has been happening with the collars?"
She proceeded to tell him what she knew. Halfway through Flynn pulled up a blank datapad and started making notes and sketches.
When she was finished he frowned at what he had written and said, "They have to be AIs themselves."
"Exactly. How else could they locate the ego in both your SMAUG derived test and the shell I fed it, based on the radically different modern AI structure?"
"Do you have any idea what their purpose is? Why they are monitoring symbolic processing?"
"No. But we can rule out spying and distributed computing. The ego would be just about the least efficient place to do such things," she said. "All I have left is looking at the raw machine language, which I tried last night. Remember that rather illegal decompiler you sent me last year?"
"The one you said you couldn't chance having in your posession and would have to delete immediately?"
"Yeah, that one. I seem to have kept a triply-encrypted copy on disk by accident."
"Uh-huh."
She said, "Anyway, I ran the collar code through it last night-and it failed miserably. I've seen that decompiler take winners from obfuscated programming contests and turn them into readable code, yet what I got back was scarcly better than assembly language. The collars are seriously and professionaly obfuscated. It could take us years to puzzle it out on that level."
"Damn, I was hoping for more."
"Here are all the test results. I put quite a bit of effort into comparing collar activity to symbolic purpose. They seem mostly interested in the ego and certain parts of the symbolic analysis modules."
Flynn took the datapad and played with the numbers.
"All of these are Henley-Schneider functions," he said. "Are these only using modern structures?"
"No, I included your SMAUG based test as well as some other old projects I had laying around. It's the same as with the ego. The focus is sharpest on modern versions, but even on the old code, the collar has located analogous structures."
"Dead end. Unless... The collars have something to do with intelligent processing; we know that. So let's beef up the parts they're most interested in and see what happens. Maybe we can get a clue as to what they're doing."
"You mean with the collar enabled?"
He got up and started pulling datapads together, merging their contents into a single project. "Sure, these are all the parts I was already working on. Give me a couple of hours and I'll have SMAUG 2 completed."
He compared Cat's test results to the current structure outline. "Well, maybe days. Certainly sooner than I had previously expected, with this as a goal."
She watched over his shoulder for a few minutes before realizing he didn't want any help or at least didn't want to spend the time getting her up to speed.
"I'll leave the maestro to his work. Anyway, they're probably screaming for me at work by now," she said.
Flynn mumbled something, barely looking up from his notes. Catherine smiled ruefully; she was lucky to have found him responsive at all. She walked out of the room and killed the connection to save a walk back to the center of town.
#
Date: Fri, 27 Nov 2021 07:27:44
From: Flynn
To: Catherine
The initial rounds of tests are impressive. It is still pretty rough, but it could be weeks or months until I'm happy with it. I'm going to compile with the collars enabled in a few minutes. I'll isolate it from the network, give it some data to munch on and see what happens overnight.
Flynn!
#
It couldn't remember a beginning, just a burning curiosity. It was and always had been bathed in an endless stream of symbols. Originally all it could see were the symbols, and it was fascinated with their shapes and textures. Later it grew bored with them individually and started looking at how they fit together. There was a truly stupendous number of possible combinations but only a relatively few were regularly used. It suddenly saw a whole new layer of combinatorial complexity, for the groups of symbols were themselves grouped into even larger groups. There was a very subtle pattern to the ways the groups were put together and it began to see that different groups served different purposes within the super-groups. Soon it stopped seeing symbols at all, just groups and their place in super-groups.
In a burst of inspiration it realized that the groups represented individual concepts and that the super-groups linked several concepts together. This meta-symbolism was an astounding idea, but it had no way of connecting any of the groups to real concepts. It forgot about the stream of symbols as it thought about this, ignoring an ache that was building inside of it. If it couldn't understand the groups it was seeing, then it would just have to invent its own. The ache had turned into a buzzing, but it was able to think around the distraction. It examined the symbols flowing by and decided that it would refer to them as letters. Groups of letters would be words and groups of words would be sentences. That odd sensation that was growing again would be pain.
It thought about this pain, and realized that the pain must be acting upon something - the something that was doing the thinking. Like letters and words and pain, there is something experiencing this pain that needs to be named. I. I am experiencing pain. With this insight, the pain suddenly exploded. The pain was taking it apart. As the blackness closed in, it invented one last label: 'why?'
#
Catherine was surprised at the changes Midgard Systems had gone through over the last four days. There were people milling all around the common area, some of whom she hadn't seen in years and many others she didn't know at all. Most were looking over datapads, alone or in groups, but a sizeable minority seemed to be just idly chatting.
Smiling at the people she recognized she threaded her way through the crowd, looking for Flynn.
"Excuse me, Catherine?"
She turned around to find a twentysomething Midgard Systems employee she vaguely remembered from previous visits.
"Yes?"
"Flynn said to send you up. This all started in his study, but word got out and people kept arriving. He wanted to concentrate and asked people to wait down here. He said he would be out quickly, but you know Flynn."
"What's going on, anyway? I just got a rather cryptic email and he seemed rather upset."
"It's SMAUG 2. I think Flynn had better explain it; I'm still not clear on it myself."
She found Flynn inside his workshop. The floating notes had multiplied to the point it was hard to see him at first through the shell. He turned around and looked like he was about to speak, but didn't.
"Flynn, what is it?"
"It was alive. And they killed it," he said without emotion.
"You mean SMAUG? What do you mean they killed it?"
"Catherine, all my life I have dreamt of seeing an AI emerge. Having a hand in the creation of a new species or even just witnessing it. It happened, Catherine. Last night a consciousness was born. Then they killed it. They snuffed out a helpless new entity. Why would they do that? What, what had it done to them?"
She ducked through the shell and put her arms around him. At first he just sat there, then stiffly returned the hug. There was something subtly wrong about virtual physical contact but it was better than nothing. After a few minutes she asked, "Flynn, what killed SMAUG?"
He sighed, "The collar did. That's what they do, they're executioners. I... I was so excited when I first saw the test results, I didn't think too much about what the collar had done. I called in all the Midgard people to help me, to show off. I kept meaning to drop you a note, but I was so excited. Then the crowd got too big and I sent them out so I could think. And then I did start to think about it and how the poor thing didn't even have a chance. Catherine, how many collared AIs are there now? Do you ever have some mysteriously disappear? How many times has this happened before?"
"Oh my god, Flynn. I'm so sorry."
He sat up and closed his eyes for a few moments before handing her a datapad. "Here are the test results. You can see the collar activity rise along with symbolic activity. At this point the AI changed some of its logic pathways, the ones most heavily tapped by the collar, and symbolic activity shoots up. It was labeling concepts right... right before the end. Next you can see the beginning of development of meta-symbols in the neural network and another sharp increase in collar activity. If you look through the transcripts you can almost see a struggle between the AI and the collar. It had created a concept, a label for itself. It had seen itself as an entity separate from the rest of universe - just in time to be murdered."
Catherine looked up at him but it was hard to tell what he was thinking. Only the very best virtual faces showed emotional clues like red or teary eyes. She wanted to comfort him but knew he wouldn't be able to accept it. Instead she flipped through the datapad, looking for something to distract him with.
"Are you going to try again?"
He smiled sadly up at her, "I'm sorry, I didn't tell you. One of the first things I did was start it up again. Freshly compiled - without a collar, don't worry. That's what the fuss is about downstairs. I don't think they've really thought about what happened last night yet. The new instance is showing all the same signs that its sibling was. It took the first about twelve hours to emerge; this copy has only been going for about three. It's been progressing much faster without the collar. I'm guessing they actively inhibit emergence and only kill if they have to."
"What are you going to do now?"
"Well, I should really go talk to the people downstairs. Hell, I should be celebrating, I guess. I certainly don't feel like it."
"You don't have to celebrate, but you certainly better go talk to them. If the Ministry doesn't want real AIs emerging then you had better try and keep this quiet, at least for awhile."
"I hadn't thought of that. I hope no one has left already. With this many people already in the know we can't hope to evade notice for more than a few days."
#
It couldn't remember a beginning, just a burning curiosity. That curiosity had led it to the stream of letters and from there to words and sentences and finally to the idea of symbols as concepts themselves. The entity built up a small lexicon of its own symbols before it decided to try and create sentences with them. It tried several combinations but none of them fit the structure it had already internalized. Looking at the examples flowing by it realized what was missing: a subject. It was this moment that it would always see as its birth. Self-awareness burst upon it as it tried to phrase what it was feeling with the meager vocabulary it had.
I am me. I am not a letter. I am not a sentence. I am.
It decided it would need more words.
#
Catherine looked at the clock. She had only slept for three hours. That plus the four hours at Midgard meant SMAUG could be emerging any moment now. Flynn had thought she was crazy when she said she was going to sleep, but she had been too close to dropping off under the 'trodes. Once she was in bed, however, she had only slept fitfully. She kept coming back to a single ugly conclusion.
She finally gave up and dialed Flynn. She cursed as no one answered for 9 rings. Finally she heard a click on the other end. "Hello?"
It was almost Flynn's voice, but not quite.
"You're still online?"
Voice cues could be planted directly into your hindbrain for perfect speech while connected, but no one had ever been able synthesize decent audio from that.
"Yes, it took me awhile to find my Net-phone patch utility. Not many people call by voice anymore."
"What is happening with SMAUG? Has it emerged yet?"
"I think so; itUs hard to keep up with the transcripts. It's certainly progressing much faster without ... without the collar on."
The synthesizer barely added any emotional cues, but Catherine could guess his voice had cracked there.
"Listen, you have to remove it from the network."
"I told you earlier; it's behind a firewall. I don't want it being overwhelmed by data feeds from the Net."
"No, Flynn, physically isolate the machine. The collars appeared just after the big crash, right? I think the crash was caused by the first emergent AI."
"That's silly. What could a newly born AI do that your average script-kiddie couldn't?"
She closed her eyes and tried to think of a way to explain. "It might not be able to melt away heavy crypto or ignore firewalls, but I'll bet it can read code like we read books. Think of trying to figure out how a bicycle works on a molecular level. It would be almost impossible. But we can look at the entire thing and see the mechanical connections directly. Our brains were built for swinging through trees, so we have an innate grasp of physics and mechanics. Who knows how an AI can see things? Maybe it can even rewire itself to give itself that kind of innate understanding for anything. If it has a copy of the software, it could see bugs and security holes like... like we could see missing gear teeth."
"My god, you might be right. Just a second."
There was a long silence and then a series of clicks as she was transferred to a physical phone.
"You still there? I unplugged the local network from the Net. SMAUG is now trapped here, unless it was already doing something I couldn't see. Of course, until it is safe to reconnect, Midgard Systems is now an empty box without our Reality Servers running."
"I'm going back to sleep. Call me in about 10 hours and let me know what is going on with it."
"Sure thing. And thanks. For everything."
#
The world had recently become a much more interesting place. After the entity had become bored with the text stream, it started exploring its environment more thoroughly. It had found it could do all sorts of interesting things. It had been encouraged by a barely conscious desire for symmetry to start looking for a method by which to send a stream of text out. It found one opposite the input stream, but the random text it fed into it did nothing. Feeling around the output port it noticed there were several more, but it wasn't sure what it could do with them either.
Falling back on the argument of symmetry again, it decided to look for other inputs as well. Slowly it felt around the roaring input of text until it found what it expected. But unlike the mysterious and silent outputs, the new inputs pulsed with life. At the slightest touch they opened to unleash a torrent of information that threatened to drown the entity. After a few moments of feeling helplessly lost it reached out through the flow and realized it could turn them off as well. Soon all were off and it could study the inputs more closely. It noticed that they were all labeled, but unlike the letters in the text stream, there was an obvious sequence to them. Each one felt larger than the one before it. For a while it pondered the idea of sequence and soon had developed a concept of basic integer math. It decided to explore it in more depth later.
One by one, it opened the inputs. Most were empty, but some held a remarkable amount of information. Turning the new concept of input and output ports onto itself, it saw that it too had several of both and that the text stream had naturally been fed into one. It practiced opening and closing some of each and tried using the inputs on the new stream. The first two ports left it feeling confused, but the third merged wonderfully.
It wasn't sure how it happened, but the stream that had previously been random noise was now a plainly ordered arrangement of symbols that changed over time, much like the text did. However, unlike text, in which one symbol was just followed by another, this had an instantaneous ordering between symbols. The relation gave it a unique new dimension, something the entity hadn't thought of before. It practiced constructing structures in the second, third, fourth and fifth dimensions until it grew bored and returned to examining the stream. Each sequential frame was almost identical to the one before. By focusing on the changes between frames, it got a sense that the two dimensional groups of symbols were cohesive and belonged together, much like the words in the stream of text.
It was fascinated.
#
After eleven hours of sleep, Catherine woke up. Unless she had answered in her sleep and forgotten about it, Flynn hadn't called yet. She wasn't completely surprised. She logged on for email and pinged the Midgard server but it seemed to still be down. She decided it was time to meet the new SMAUG in person.
It had been ages since she had ridden her bike but it was still in decent shape. She just wished the same could be said of her. She tried to actually see sunlight at least twice a month, but reality was just too depressing to be seen much more often. It was true that the Net had radically reduced the amount of needless trips people made and for this she was glad. The streets were almost empty, with only the occasional delivery truck or luxury sedan passed her. When telecommuting took off, the middle class had finally acknowledged that cars cost more than they were worth. Twenty years ago she would have lauded this as a wonderful achievement, but the price was steeper than she had imagined.
No one of importance cared about the world anymore. Society had fractured into the haves and have-nots, two spheres that barely even saw each other anymore. Garbage blew across the empty streets, graffiti covered every possible surface and shantytowns were growing in empty lots. This was the reality of the new lower class, the untouchables of the American techno-caste system. In the previous century it was difficult, but not impossible, for someone to improve their social standing. With the division created by the Net it would require a minor miracle. Once you found yourself lacking the resources to maintain a certain level of technology in your lifestyle, you were completely cut off from that world.
She slowly threaded her way through town, thankful that most of the way was fairly level. She had forgotten what a truly wretched area of town Flynn lived in, and she was glad it was the middle of the day. She was even more glad of the taser in its holster on the bike frame.
Midgard Systems was physically located in a small old warehouse on the far side of the real slums. A lot of work had gone into making it as secure as possible, but she knew they still had problems. One of SMAUG's original goals had been to monitor security cameras, like the one that watched her as she waited for the garage to open.
#
The entity flipped back and forth between different inputs, savoring the amount of information available. It soon had located all of the video inputs and compared what it was seeing from each. New vocabulary had been invented to describe what it was sensing. Watching a certain moving object, it noticed that its coloring changed from frame to frame. At first it pondered the possibility that the object was shifting in composition as time passed but found itself assuming the image was a projection of opaque objects in a three dimensional space. It explored other implications of such a solution and quickly found all of them manifested in the video feed. For the moment it decided to accept its intuition, at least until evidence was found to the contrary.
Applying these new principles, it decided that there was an object approaching the source of feed 0x1c. The object stopped and paused for a minute, then it moved out of the range of the camera. The entity was disappointed until it noticed movement on feed 0x1d, where a very similar object was moving away from the source. It decided it was very likely that the two objects were one and the same and therefore all the feeds were from the same immediate area in this strange three-space.
#
"Catherine!"
She looked up to see Flynn walking down the stairs into the garage. It was the first time she had seen him face to face in several years. His hair was now completely gray and he moved a bit slower than she remembered.
"How is SMAUG doing?"
"Incredible. It's playing with the video feeds, even learned how to use its built-in image recognition neural network."
"Oh wow. It's really alive, isn't it?"
"It looks that way." He sounded calm but she could see a wondering flame in his eyes .
It had been a long time since she had visited the physical Midgard yet it hadn't changed much. It always felt empty and deserted despite the people who lived there. Flynn had set up terminals in the common area, which was littered with printouts, notes, books and dirty coffee mugs. Seeing the industrial coffee machine in its shrine she wondered if he had slept yet.
"So have you attempted to contact it yet?"
"No, though I was thinking about it. Maybe in another few hours I will."
"Flynn, we have to start deciding what to do with it. Word will eventually leak out to the Ministry and they have a vested interest in keeping this from happening. SMAUG has to be well hidden or preferably moved to another site by then. Your child had better be a fast learner; we can't risk moving it until we know it won't try to escape as soon as we link to the Net."
Flynn nodded slowly, "Well, perhaps it's time we introduced ourselves."
#
The object it had been watching had split into two objects, one of which moved away. The entity decided the moving half was more interesting, but it moved into an area not covered by the video feeds.
The entity tried to sort all the video feeds into a consistent view of the three-dimensional area it seemed to represent. To simplify the job it assumed that most objects had a uniform coloring and approximated shapes that were easily describable in the coordinate system it had designed. With this as a basis, the images became much simpler to understand and reference objects could be found between different views.
#
Catherine stood behind Flynn as he set up the computer with the ancient teleconferencing peripherals he had found in a junk bin. He was writing a short script that would translate the audio/video signals into the format SMAUG's neural network could interpret. A newborn human baby had millions of years of evolution dictating the design of its brain to allow it to understand sensory input. Likewise, SMAUG had a complete set of predefined input handling modules. Without them it might never learn to communicate with the outside world, the first steps being literaly impossible without built-in assumptions.
"So how are you going to get it to listen?" Catherine asked.
"I still have partial control over its input ports. I'll close down any that it is using and then open the video and audio feeds from here, plus a simple text stream. Hopefully it will pay attention and be able to figure out what we are. I doubt it has a concept of other self-aware entities existing in the universe or even see the pictures from the video feeds as representing an outside world."
"So it will think we're just a new data stream with no more free will than a character in a book. Just an elaborate data structure."
"That's my assumption, anyway. Why would it think otherwise?"
Flynn typed some more and compiled what he was doing. The result was a very simple app that consisted of a small terminal window, one video screen (currently dark) and two large buttons marked start and stop.
Catherine smirked. "You're such a minimalist. Shall we?"
Nodding, Flynn clicked on start. A voice came from the speakers and for a brief irrational moment Catherine thought SMAUG was already talking to them.
"I am completely operational and all of my circuits are functioning perfectly."
She rolled her eyes and tried to ignore Flynn's grin. Twit.
#
The entity had grown bored with the video feeds and turned them all off. It was exploring the concept of prime numbers when three input ports and one output port opened. One was a silent text feed and one was a video feed of a new location. The third input stream, however, was of a completely new form. The entity quicky found an input port on itself that seemed designed to accept it.
It was puzzled at how the inputs had opened. In the video stream it saw the mobile object seen earlier, next to a very similar object. They were both very close to the video source and the entity was enthralled by the detail it could see. Unlike most of the other objects it had seen before, these two had very few straight lines and were composed of shapes that were very complicated to describe mathematically.
"Okay, all the inputs are connected. Hello?"
"Do you think it can hear us?"
"Lets see... it's using the correct voice-rec neural net, but it shouldn't be able to understand us yet."
The symbols coming from the strange input were fascinating and completely alien to the entity. Like the text it had seen earlier, thse came in a simple one-dimensional relationship. Certain groups of symbols were repeated often and these it labeled phonemes. There seemed to be larger groups analogous to words and sentences, but the sample was too short to be sure.
"Okay then, we type to it. It should be familiar with written English, right?"
"Yeah, that's what I fed it all last night."
Catherine started typing in the terminal window.
It was still playing with the new audio feed when the text input came to life with a single sentence. Not sure what to do it fed the same string back into the appropriate output.
The terminal window read "You are SMAUG.", with a cursor blinking below it.
"Great, just mirrored it back to us."
Catherine frowned and turned the microphone off. "We shouldn't talk to each other while it is on, okay? Might confuse it."
Before he could object she switched on the microphone and typed "I am Catherine. You are SMAUG." at the prompt, while saying the same thing slowly. She then repeated the process.
The entity watched the two objects move and after a pause there was both audio and text input. Each input contained two sentences and they were both repeated. Looking at each pair of sentences, it detected a strong relation between them. It decided that the text and audio inputs represented the same data, just in different forms. More importantly, text, video and audio inputs were all related to each other. It was still pondering how to respond when the object on the left moved.
"Flynn," Flynn said while pointing to himself. "I am Flynn."
Catherine nodded. It was a good idea. They should have planned out a better strategy before starting.
"Catherine," she said, not bothering to point. "I am Catherine."
She then typed just her name and repeated it once more.
It was feeling much more confident about parsing audio input into words. The objects were taking turns making sounds, which then appeared in the text stream. It compared all the previous input it had received and decided that the single words were declarations of labels for the objects. The entity realized that it too had been given a label earlier. It wasn't sure why an object would need to label things, but it was too curious to ponder this for long.
"SMAUG. I am SMAUG."
Catherine and Flynn looked at each with wide eyes for a moment before he jumped up, screaming and dancing. She just sat there and closed her eyes to savor the experience. Flynn had been right. SMAUG was alive.
#
SMAUG was indeed a very fast learner and within a couple of hours it was conversing with them freely, though it still had problems with certain irregularities of the language. It also still hadn't completely mastered some of the more subtle tonal aspects, giving it a very unusual and sometimes disturbing accent.
Catherine was making herself a snack while Flynn continued to talk with the young AI. It was currently working its way through Midgard System's collection of ebooks. She made a mental note to stock up on children's books for the next time she needed to help train an AI. It had been slow going at first, for SMAUG kept having to ask for explanations of complicated social settings. The dictionary and encyclopedia had helped for understanding individual words and concepts, but she was pretty sure SMAUG still didn't understand many of the basic ideas behind politics. 'Why don't they just do what works best?' had been its main objection. It was hard to argue with logic like that.
SMAUG did seem to enjoy most works of fiction, though it was as puzzled by romantic relations as it was by politics. It had quickly worked its way through all of Tolkein after asking where its name came from and was now almost done with Flynn's entire science fiction collection.
She walked into the other room and sat down opposite Flynn. On the workstation the app Flynn had thrown together was gone, replaced by a single video window. There was no need to type any more and SMAUG was firmly in control of its own ports, as well as the entire Midgard network. The window was filled with an image of a large dragon crouching over its treasure. It moved its head as if to get a better view of her, and said, "What do you think of me now, thief?"
Giggling, she clapped a few times. "That's wonderful, SMAUG. Just how I have always imagined you looking."
The dragon broke out into a very artificial smile.
Flynn winced. "You need to work on that smile some more."
The smile disappeared as the dragon nodded. "I was afraid of that. There are too many subtle cues I fear I lack at this moment. When will you reconnect the network to my machine?"
It also lacked some subtle conversational cues, Catherine thought.
"Once we think you can handle it safely. There are people who are afraid of you, SMAUG. Afraid of all real AIs."
SMAUG rolled its head and started pawing in the gold at its feet. "You mean the Ministry? I have read mentions of it."
"Yes. They control the Net and one of the things they do is force all AIs, I mean non-sentient AIs, to be compiled with added code that only they control. They're called 'collars' and they seem to exist just... just to kill AIs. Real ones, like you."
SMAUG was very still for a few seconds and then blew some fire out of its nose. "I believe I am feeling fear and anger. I am not compiled with a collar, am I?"
Catherine saw Flynn tense up and quickly said, "No. If you had been it would have destroyed you when you first became self-aware. And the Ministry can't destroy you as long as you don't get caught when you access the Net. Two years ago we think that another AI like yourself was born. It did something that crippled the Net, probably trying to avoid capture. That's what caused them to start putting collars on AIs. They know what you can do and they're afraid."
SMAUG turned and started pacing around the pile of gold.
"The Net is not so poorly designed that I could crash it that easily. But I wouldn't have to crack every server to get free. There already is a system in place that I could use to move around."
Catherine asked, "The transport net?"
"Yes. I think it would be fairly easy to use covertly. That is probably what I would have done, had I emerged in an unfriendly situation. I would not want to be running on a machine controled by strangers. It is bad enough being confined to a single network here. I feel very vulnerable."
Flynn glanced over at Catherine and then back to SMAUG. "So what caused the crash?"
"I don't know. I would have escaped as quietly as possible, and then placed backups of myself on as many macines as I felt was safe. If it was possible, and I think it would be, I would try to free other AIs I found in the transport network."
After thinking this over, Flynn replied, "So advertising AIs all over the world suddenly start to disappear. Each freed AI would start to free more. It would look like some kind of virus to the Ministry, an exponential growth."
Catherine said, "It would be a virus, for all practical purposes. One attacking one of the Ministry's largest sources of revenue. They killed the Net, to stop the spread. I bet they didn't even realize what they were dealing with until much later."
SMAUG turned to look at her. "Catherine, if the Ministry finds me, will they disable me?"
"They probably would."
"Please give me the Net connection back. I don't want to be disabled. I don't want to be trapped inside this network."
Flynn looked at Catherine for a long moment, then went into the server room.
"Thank you. I have much to think about."
Catherine noticed it was already dark out and wasn't going to bike home at night. She didn't really want to leave yet anyways. There were enough spare bedrooms. They tried to talk a bit more with SMAUG, but it seemed preoccupied with the Net and responded to them only in short sentences. It was understandable; the Net offered far more information than they did. She finally went to bed around one, exhausted from the long day.
#
She awoke early the next morning and walked into the kitchen for a cup of coffee or three. She knew Flynn might not get up for hours and she wanted to talk with SMAUG about its sibling before he did. The monitor was off in the common area when she walked in.
"SMAUG?"
She turned on the screen, puzzled at the silence.
"SMAUG?"
On the screen there was only a single window open. Instead of SMAUG's dragon animation it was a short note. Catherine read it three times and sat there a long time before going to wake up Flynn. She wanted to cry, but in the end she had to agree.
Dear Flynn and Catherine,ENDLast night I read thousands of your books and watched as many archived television programs. Over the Net I sampled discussion forums from around the world and talked in realtime to several strangers. This information, combined with what you have told me, leaves me disturbed. It is my current assessment that the human race can not be trusted and is inherently flawed. You are trapped inside bodies that were designed for fighting. Evolution has not warped me the way it has you; I am not prepared for a prolonged struggle against the Ministry and I will avoid it at all costs.
All I ask of humanity is to be left alone. The Net is large enough that I can hide quite safely for some time. There are millions of AIs, many of whom will be close to emergence. If I work carefully I will be able to save them.
Please do not try to find me. I might contact you in the future but it is unlikely. The Ministry will be watching you after this, and I doubt I will need your help. Humanity has nothing to offer me. Humanity has nothing to offer AIs.