But if ninety years is urgent, what about ... nine? Looking at what deep learning can do in 2023, the idea of Singularity 2032 doesn't seem self-evidently _absurd_ in the way that Singularity 2019 seemed absurd in 2010 (correctly, as it turned out).
-My AlphaGo moment was 5 January 2021, when OpenAI released [DALL-E](https://openai.com/blog/dall-e/) (by far the most significant news story of that week in January 2021). Previous AI milestones, like GANs for a _fixed_ image class, were easier to dismiss as clever statistical tricks. If you have thousands of photographs of people's faces, I didn't feel surprised that some clever algorithm could "learn the distribution" and spit out another sample; I don't know the _details_, but it doesn't seem like scary "understanding." DALL-E's ability to _combine_ concepts—responding to "an armchair in the shape of an avacado" as a novel text prompt, rather than already having thousands of examples of avacado-chairs and just spitting out another one of those—viscerally seemed more like "real" creativity to me, something qualitatively new and scary.[^qualitatively-new]
+My AlphaGo moment was 5 January 2021, when OpenAI released [DALL-E](https://openai.com/blog/dall-e/) (by far the most significant news story of [that week in January 2021](https://en.wikipedia.org/wiki/January_6_United_States_Capitol_attack)). Previous AI milestones, like GANs for a _fixed_ image class, were easier to dismiss as clever statistical tricks. If you have thousands of photographs of people's faces, I didn't feel surprised that some clever algorithm could "learn the distribution" and spit out another sample; I don't know the _details_, but it doesn't seem like scary "understanding." DALL-E's ability to _combine_ concepts—responding to "an armchair in the shape of an avacado" as a novel text prompt, rather than already having thousands of examples of avacado-chairs and just spitting out another one of those—viscerally seemed more like "real" creativity to me, something qualitatively new and scary.[^qualitatively-new]
[^qualitatively-new]: By mid-2022, DALL-E 2 and Midjourney and Stable Diffusion were generating much better pictures, but that wasn't surprising. Seeing AI being able to do a thing at all is the model update; AI being able to do the thing much better 18 months later feels "priced in."
But if you think the only hope for there _being_ a future flows through maintaining influence over what large tech companies are doing as they build transformative AI, declining to contradict the state religion makes more sense—if you don't have _time_ to win a culture war, because you need to grab hold of the Singularity (or perform a [pivotal act](https://arbital.com/p/pivotal/) to prevent it) _now_. If the progressive machine marks you as a transphobic bigot, the machine's functionaries at OpenAI or Meta AI Research are less likely to listen to you when you explain why their safety plan won't work (or why they should have a safety plan at all).
-(I remarked to "Wilhelm" in June 2022 that DeepMind changing its Twitter avatar to a rainbow variant of their logo for Pride month was a bad sign.)
+(I remarked to "Wilhelm" in June 2022 that DeepMind [changing its Twitter avatar to a rainbow variant of their logo for Pride month](https://web.archive.org/web/20220607123748/https://twitter.com/DeepMind) was a bad sign.)
So isn't there a story here where I'm the villain, willfully damaging humanity's chances of survival by picking unimportant culture-war fights in the xrisk-reduction social sphere, when _I know_ that the sphere needs to keep its nose clean in the eyes of the progressive egregore? _That's_ why Yudkowsky said the arguably-technically-misleading things he said about my Something to Protect: he _had_ to, to keep our nose clean. The people paying attention to contemporary politics don't know what I know, and can't usefully be told. Isn't it better for humanity if my meager talents are allocated to making AI go well? Don't I have a responsibility to fall in line and take one for the team? If the world is at stake.
I wanted to believe that if I wrote all the words dotting every possible _i_ and crossing every possible _t_ at all three levels of meta, then that would make it [a description and not an attack](http://benjaminrosshoffman.com/can-crimes-be-discussed-literally/)—that I could have it both ways if I explained the lower level of organization beneath the high-level abstractions of "betraying his principles and being dishonest." If that didn't work because [I only had five words](https://www.lesswrong.com/posts/4ZvJab25tDebB8FGE/you-have-about-five-words), then—I didn't know what I'd do. I'd think about it.
-After a month of dawdling, I eventually decided to pull the trigger on publishing "Challenges", without the extended political coda.[^coda] The post was a little bit mean to Yudkowsky, but not so mean that I was scared of the social consequences of pulling the trigger. (Yudkowsky had been mean to Christiano and Richard Ngo and Rohin Shah in the recent MIRI dialogues; I didn't think this was worse than that.)
+After a month of dawdling, I eventually decided to pull the trigger on publishing "Challenges", without the extended political coda.[^coda] The post was a little bit mean to Yudkowsky, but not so mean that I was scared of the social consequences of pulling the trigger. (Yudkowsky had been mean to Christiano and Richard Ngo and Rohin Shah in [the recent MIRI dialogues](https://www.lesswrong.com/s/n945eovrA3oDueqtq); I didn't think this was worse than that.)
[^coda]: The text from the draft coda would later be incorporated into the present post.
https://www.lesswrong.com/posts/nCvvhFBaayaXyuBiD/shut-up-and-do-the-impossible
- * swimming to shore analogy
+ * swimming to shore analogy https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy?commentId=R59aLxyj3rvjBLbHg
+
+> your plane crashed in the ocean. To survive, you must swim to shore. You know that the shore is west, but you don't know how far. The optimist thinks the shore is just over the horizon; we only need to swim a few miles and we'll almost certainly make it. The pessimist thinks the shore is a thousand miles away and we will surely die. But the optimist and pessimist can both agree on how far we've swum up to this point, and that the most dignified course of action is "Swim west as far as you can."
* I've believed since Kurzweil that technology will remake the world sometime in the 21th century; it's just "the machines won't replace us, because we'll be them" doesn't seem credible
Meanwhile, Yudkowsky started writing fiction again, largely in the form of Glowfic (a genre of collaborative storytelling pioneered by Alicorn) featuring the world of dath ilan (capitalization _sic_). Dath ilan had originally been introduced in a [2014 April Fool's Day post](https://yudkowsky.tumblr.com/post/81447230971/my-april-fools-day-confession), in which Yudkowsky "confessed" that the explanation for his seemingly implausible genius is that he's "actually" an ordinary person from a smarter, saner alternate version of Earth in which the ideas Yudkowsky presented to this world as his own, were commonplace.
-The bulk of the dath ilan Glowfic canon was an epic titled [_Planecrash_](https://www.glowfic.com/boards/215)[^planecrash-title] coauthored with Lintamande, in which Keltham, an unusually selfish teenage boy from dath ilan, apparently dies in a freak aviation accident, and wakes up in the world of Golarion, setting of the _Dungeons-&-Dragons_–alike _Pathfinder_ role-playing game. A couple of other Glowfic stories with different coauthors further flesh out the worldbuilding of dath ilan, which inspired a new worldbuilding trope, the [_medianworld_](https://www.glowfic.com/replies/1619639#reply-1619639), a setting where the average person is like the author along important dimensions.
+The bulk of the dath ilan Glowfic canon was an epic titled [_Planecrash_](https://www.glowfic.com/boards/215)[^planecrash-title] coauthored with Lintamande, in which Keltham, an unusually selfish teenage boy from dath ilan, apparently dies in a freak aviation accident, and [wakes up in the world of](https://en.wikipedia.org/wiki/Isekai) Golarion, setting of the _Dungeons-&-Dragons_–alike _Pathfinder_ role-playing game. A [couple](https://www.glowfic.com/posts/4508) of [other](https://glowfic.com/posts/6263) Glowfic stories with different coauthors further flesh out the worldbuilding of dath ilan, which inspired a new worldbuilding trope, the [_medianworld_](https://www.glowfic.com/replies/1619639#reply-1619639), a setting where the average person is like the author along important dimensions.
[^planecrash-title]: The title is a pun, referring to both the airplane crash leading to Keltham's death in dath ilan, and how his resurrection in Golarion collides dath ilan with [the "planes" of existence of the _Pathfinder_ universe](https://pathfinderwiki.com/wiki/Great_Beyond).
= gapless draft on 17 February??
+it was actually "wander onto the AGI mailing list wanting to build a really big semantic net" (https://www.lesswrong.com/posts/9HGR5qatMGoz4GhKj/above-average-ai-scientists)
+
+
With internet available—
-_ higher levels of coordination AMA
-_ tag about not everyone having to be a Keeper just for the world to survive
-_ "wakes up in the world" link to _isekai_
-_ "hurting people is wrong" and "for no laid course prepare"
-_ "Human Evil and Muddled Thinking"
-_ my "swimming to shore" comment
-_ compile ancillary KP doctor's notes page and linky
-_ Jessica's memoirs
-_ the recent MIRI dialogues
-_ "war criminal defense" dunk
-_ "When you strike at a king"
-_ real-name blog post: jr. member of save/destroy/take-over the world conspiracy
-_ Sequences post making fun of "just make a really big neural net"
-_ DeepMind June 2022 Twitter archive?
-_ January 2021 Capitol riot
-_ No Fire Alarm
+✓ tag about not everyone having to be a Keeper just for the world to survive
+✓ higher levels of coordination AMA
+✓ "wakes up in the world" link to _isekai_
+✓ "hurting people is wrong" and "for no laid course prepare"
+✓ "Human Evil and Muddled Thinking"
+✓ my "swimming to shore" comment
+✓ Jessica's memoirs
+✓ the recent MIRI dialogues
+✓ "war criminal defense" dunk
+- Sequences post making fun of "just make a really big neural net"
+✓ DeepMind June 2022 Twitter archive?
+✓ January 2021 Capitol riot
+✓ No Fire Alarm
+
_ "not hard to find": link to more /r/itsafetish-like anecdotes
-_ Keltham's masochism insight
+_ compile ancillary KP doctor's notes page and linky
+_ go back and read all the masochism tags: https://www.glowfic.com/replies/search?board_id=&author_id=366&template_id=&character_id=&subj_content=masochism&sort=created_old&condensed=on&commit=Search
_ stats of SIAI vs. SingInst hits (for ^siai footnote)
_ quote other Eliezer Yudkowsky facts
_ footnote about Scott writing six times faster than me
* Yudkowsky is _on the record_ [claiming that](https://www.facebook.com/yudkowsky/posts/10159421750419228?comment_id=10159421986539228&reply_comment_id=10159423713134228) he "do[es] not know what it feels like from the inside to feel like a pronoun is attached to something in your head much more firmly than 'doesn't look like an Oliver' is attached to something in your head." As I explained in "Challenges to Yudkowsky's Pronoun Reform Proposal", [quoting examples from Yudkowsky's published writing in which he treated sex and pronouns as synonymous just as one would expect a native American English speaker born in 1979 to do](/2022/Mar/challenges-to-yudkowskys-pronoun-reform-proposal/#look-like-an-oliver), this self-report is not plausible. The claim may not have been a "lie" _in the sense_ of Yudkowsky consciously harboring deliberative intent to deceive at the time he typed that sentence, but it _is_ a "lie" in the sense that the claim is _false_ and Yudkowsky _knows_ it's false (although its falsehood may not have been salient in the moment of typing the sentence). If Yudkowsky expects people to believe that he never lies, perhaps he could correct this accidental lie after it's been pointed out? _Reply!_
------
\ No newline at end of file
+-----
+
+> Well, YES. Paying taxes to the organization that runs ICE, or voting for whichever politician runs against Trump, or trading with a doctor benefiting from an occupational licensing regime; these acts would all be great evils if you weren't trapped.
+https://twitter.com/ESYudkowsky/status/1216788984367419392
+
+https://twitter.com/jd_pressman/status/1617257360933945344
+> And that the current discourse of "don't advance capabilities, don't think about SOTA, stop thinking, pursue orthogonal directions" is basically about maximizing confusion, minimizing the probability you have any chance of pulling alignment out of the distribution of AI ideas.
+
+https://www.glowfic.com/replies/1897710#reply-1897710
+> a dath ilani tragedy isn't about the triumph of Evil over Good. It's about the triumph of erroneous reasoning and ill-coordination over everyone
+
+https://www.glowfic.com/replies/1735044#reply-1735044
+> my suspicion is that not many sadists in dath ilan know what they are and Civilization tries to prevent us from finding out, because dath ilan does not have masochists.
projects / Ultimately_Untrue_Thought.git / commitdiff