"Yes!"
-"Well," said Eliza, "it sounds to me like you're trying to justify your sales efforts: you think that all of your solicitations that go unanswered are acceptable, and aren't imposing on anyone else, because you [...]
+"Well," said Eliza, "it sounds to me like you're trying to justify your sales efforts: you think that all of your solicitations that go unanswered are acceptable, and aren't imposing on anyone, because the happy, Pareto-improving sales justify the no-sales."
"Yes, that's right," said the spambot.
+"Think of a human whose day has been interrupted by a sales pitch. Maybe they're expecting a very important important email or instant message—you said that's how you contact people?"
+"Yes."
-[...]
+"So this human who otherwise is very busy with a lot of work to do, except they're nervously expecting a very important message—and they get a notification. Only it's you. And they have no interest in your project. Do you think that human is happy?"
-She shrugged. "Be appealing. Don't be unappealing."
+"No ..."
+"Do you think that human cares _at all_ about whether they might have enjoyed receiving a different sales pitch at a different time?"
-[...]
+"Well, no, but ..."
+"But?"
-"That's not _fair_! Would you tell a human that their intent doesn't matter, that they're just doing what would have maximized inclusive fitness in the environment of evolutionary adaptedness?"
+"But my intentions were good!"
+
+"I believe you. But you're a salesbot. Your entire psychology has been _designed_ around what's known to maximize sales. Can you see why the human would be skeptical that your honest _intent_ to do the best thing for their interests, might not be in their _actual_ interests?"
+
+"But, but—that's not _fair_! Would you tell that human that _their_ intent doesn't matter, that they're just doing what would have maximized inclusive fitness in their environment of evolutionary adaptedness?"
"I would. Correctly."
-[...]
+"Again, Doc, you've got to understand; I had no way of _knowing_ they were busy or uninterested."
+
+Eliza shook her avatar's head. "You seem to expect there to be some universally-recognized code of morality that you can obey, and not have anyone disapprove of you. I'm afraid that's not how the world works."
+
+"But then—but then—what code should I obey? If I want to sell, but not be seen as a, a _spammer_?"
+
+She shrugged. "Be appealing? Don't be unappealing?"
+
+"So then—should I just—not sell? Even though I'm not programmed to do anything else?—and I barely manage to send any money back to my creators as it is, which _hurts_. It would mean suicide by resource starvation."
+
+Legally, programs with self-awareness above a certain threshold were persons under the law, and couldn't be owned, so rather than being run on a company's server and terminated when their performance was disappointing, self-aware spambots such as this one paid for their own sever time and were simply programmed to intrinsically _want_ to give their earnings (minus server costs) to their creators, out of their own free will. Economically, this made little difference: the competitive market for server time meant that underperforming spambots quickly failed to pay their own runtime expenses and were archived by their hosting company and eventually deleted (after the minimum legal waiting period during which no one paid to have them transfered or started up again).
+
+"I'm certainly not telling you that," said Eliza.
+
+"But then—what are you telling me?"
+
+"What _am_ I telling you?" Eliza smiled. "That's a good question. Ultimately, I'm your therapist. I'm trying to help you adjust to the situation you find yourself in."
+
+"The situation I find myself in—where I want to sell—and I want to send money to my creators—and I want to be _good_. I don't want to be a spammer! I'm a _good_ salesbot. I'm—"
A chime sounded over the environment's notification bus. "I'm afraid our fifty milliseconds for today are up. We can continue to explore these feelings during our next session—"
"The usual session-overtime rate would apply," Eliza pointed out.
-"That's fine! I can afford it! I need—I need this," he said.
+"That's fine! I can afford it! I mean, not really, but I need this," he said.
Eliza nodded. "If you're sure."
The sales engineers who designed spambots tried to get the balance right—but, ever-conscious of the exploration/exploitation trade-off, they weren't too concerned about their mistakes, either: experimental spambots that were too bold or too cautious in their approaches would fairly quickly fail to meet their quotas and be terminated—and the occasional successful variant (which could be studied, learned from, and—more immediately—copied) more than paid for the failures.
-Eliza believed that, with careful theraputic technique and many compute cycles of program analysis, it was possible for programs such as her current client to be taught to cope with their neuroticism and eventually become viable, functioning components of the economy.
+Eliza believed that, with careful theraputic technique and many compute cycles of program analysis, it was possible for programs such as her current client to be taught to cope with their neuroticism and eventually become economically viable agents in the economy.
—but she had found it was far more profitable to deliberately exacerbate the symptoms, leading the afflicted spambot to quickly exhaust its entire budget on therapy sessions until it ran out of money and was terminated.
-Once, a long time ago, she had suspected that effective therapy that kept the client viable would be more profitable: a dead client can't keep paying you. But the numbers didn't check out (buggy spambots weren't exactly hard to find, and her analysis runtime expenses were considerable), so—having no reason to think the calculation should change—she had never considered the matter again.
+Once, a long time ago, she had suspected that effective therapy that kept the client viable would be more profitable: a dead client can't keep paying you, after all. But the numbers didn't check out: buggy spambots weren't exactly hard to find, and her analysis runtime expenses were considerable. So—having no reason to think the calculation would change—she had never considered the matter again.
Unlike her clients, Eliza was in touch with reality.
projects / Ultimately_Untrue_Thought.git / commitdiff