scoring Intellectual Turing Test predictions

diff --git a/content/2017/thing-of-things-transgender-intellectual-turing-test-predictions-and-commentary.md b/content/2017/thing-of-things-transgender-intellectual-turing-test-predictions-and-commentary.md
index c6b2875..3f68311 100644 (file)
--- a/content/2017/thing-of-things-transgender-intellectual-turing-test-predictions-and-commentary.md
+++ b/content/2017/thing-of-things-transgender-intellectual-turing-test-predictions-and-commentary.md
@@ -9,26 +9,31 @@ Friend of the blog—I mean, I _hope_ we're [still friends](http://unremediatedg
 
 In the intellectual Turing test, contestants answer a set of questions both as themselves, and while trying to pass as someone who believes the other thing, while the audience tries to discriminate the honest entries from the fakes. Below are my probability assignments for this contest (I think it's important to assign probabilities rather than binary guesses, so that you can assess your rationality with a Bayesian [strictly proper scoring rule](http://yudkowsky.net/rational/technical/) rather than a crude "number correct"), along with an optional brief comment—
 
+***Update, 5 June***: Two months after the [results were posted](https://thingofthings.wordpress.com/2017/04/04/intellectual-turing-test-results/), I finally got around to scoring these. ("Bayes-score" is the base-two [logarithmic score](https://en.wikipedia.org/wiki/Scoring_rule#Logarithmic_scoring_rule). Someone who, claiming complete ignorance, gave a 0.5/0.5 distribution for each entry would lose a [bit](https://en.wikipedia.org/wiki/Self-information) on each question for a final score of −18.)
+
 **Gender identity entries**
 
-[#1](https://thingofthings.wordpress.com/2017/03/03/itt-1-gender-identity/): GI: 0.65, BBL: 0.35 (strong philosophy of language; if telling the truth about being a cis woman, ignorance of non-dysphoric AGP is plausible)  
-[#2](https://thingofthings.wordpress.com/2017/03/06/itt-2-gender-identity/): GI: 0.4, BBL: 0.6 (awareness of 4chan shows non-naïveté about what's actually going on)  
-[#3](https://thingofthings.wordpress.com/2017/03/07/itt-3-gender-identity/): GI: 0.6, BBL: 0.4 (maybe a little _too_ doctrinaire??)  
-[#4](https://thingofthings.wordpress.com/2017/03/08/itt-4-gender-identity/): GI: 0.6, BBL: 0.4  
-[#5](https://thingofthings.wordpress.com/2017/03/09/itt-5-gender-identity/): GI: 0.6, BBL: 0.4  
-[#6](https://thingofthings.wordpress.com/2017/03/10/itt-6-gender-identity/): GI: 0.7, BBL: 0.3 (seemingly sincere trans man)  
-[#7](https://thingofthings.wordpress.com/2017/03/14/itt-8-gender-identity/): GI: 0.7, BBL: 0.3 (standard trans woman rationalizations)  
-[#8](https://thingofthings.wordpress.com/2017/03/15/itt-9-gender-identity/): GI: 0.65, BBL: 0.35 (really knows her stuff; this is what a smart, intellectually-honest BBL skeptic looks like, and I'd like to believe that they exist!)  
-[#9](https://thingofthings.wordpress.com/2017/03/16/itt-7-gender-identity/): GI: 0.7, BBL: 0.3  
+[#1](https://thingofthings.wordpress.com/2017/03/03/itt-1-gender-identity/): GI: 0.65, BBL: 0.35 (strong philosophy of language; if telling the truth about being a cis woman, ignorance of non-dysphoric AGP is plausible), Actual: GI ✔, Bayes-score: −0.621  
+[#2](https://thingofthings.wordpress.com/2017/03/06/itt-2-gender-identity/): GI: 0.4, BBL: 0.6 (awareness of 4chan shows non-naïveté about what's actually going on), Actual: GI ✔, Bayes-score: −1.322  
+[#3](https://thingofthings.wordpress.com/2017/03/07/itt-3-gender-identity/): GI: 0.6, BBL: 0.4 (maybe a little _too_ doctrinaire??), Actual: BBL ✘, Bayes-score: −1.322  
+[#4](https://thingofthings.wordpress.com/2017/03/08/itt-4-gender-identity/): GI: 0.6, BBL: 0.4, Actual: BBL ✘, Bayes-score: −1.322  
+[#5](https://thingofthings.wordpress.com/2017/03/09/itt-5-gender-identity/): GI: 0.6, BBL: 0.4, Actual: GI ✔, Bayes-score: −0.737  
+[#6](https://thingofthings.wordpress.com/2017/03/10/itt-6-gender-identity/): GI: 0.7, BBL: 0.3 (seemingly sincere trans man), Actual: GI ✔, Bayes-score: −0.515  
+[#7](https://thingofthings.wordpress.com/2017/03/14/itt-8-gender-identity/): GI: 0.7, BBL: 0.3 (standard trans woman rationalizations), Actual: GI ✔, Bayes-score: −0.515  
+[#8](https://thingofthings.wordpress.com/2017/03/15/itt-9-gender-identity/): GI: 0.65, BBL: 0.35 (really knows her stuff; this is what a smart, intellectually-honest BBL skeptic looks like, and I'd like to believe that they exist!), Actual: GI ✔, Bayes-score: −0.621  
+[#9](https://thingofthings.wordpress.com/2017/03/16/itt-7-gender-identity/): GI: 0.7, BBL: 0.3, Actual: BBL ✘, Bayes-score: −1.737  
 
 **Blanchard–Bailey–Lawrence entries**
 
-[#1](https://thingofthings.wordpress.com/2017/03/17/itt-2-blanchard-bailey/): GI: 0.6, BBL: 0.4  
-[#2](https://thingofthings.wordpress.com/2017/03/20/itt-3-blanchard-bailey/): GI: 0.4, BBL: 0.6  
-[#3](https://thingofthings.wordpress.com/2017/03/21/itt-4-blanchard-bailey/): GI: 0.4, BBL: 0.6  
-[#4](https://thingofthings.wordpress.com/2017/03/22/itt-5-blanchard-bailey/): GI: 0.9, BBL: 0.1 (shibboleth fail!—people who believe in biology do not say "assigned at birth" when describing their own beliefs! Also, failure to notice the obvious "for the same reasons men are" re programmers)  
-[#5](https://thingofthings.wordpress.com/2017/03/23/itt-6-blanchard-bailey/): GI: 0.2, BBL: 0.8 (preach it!)  
-[#6](https://thingofthings.wordpress.com/2017/03/24/itt-7-blanchard-bailey/): GI: 0.8, BBL: 0.2 ("male socialization, which unlike androphilic trans women they actually tend to absorb as kids" sounds like someone who believes that innate gender identity determines what socialization you latch onto from your culture, rather than someone who actually believes in sexual dimorphism)  
-[#7](https://thingofthings.wordpress.com/2017/03/27/itt-8-blanchard-bailey/): GI: 0.9, BBL: 0.1 (shibboleth fail again!—[my comment at _Thing of Things_](https://thingofthings.wordpress.com/2017/03/27/itt-8-blanchard-bailey/#comment-25273))  
-[#8](https://thingofthings.wordpress.com/2017/03/28/itt-9-blanchard-bailey/): GI: 0.1, BBL: 0.9 (raw reality)  
-[#9](https://thingofthings.wordpress.com/2017/03/29/itt-1-blanchard-bailey/): GI: 0.85, BBL: 0.15 ([my comment](https://thingofthings.wordpress.com/2017/03/29/itt-1-blanchard-bailey/#comment-25321))
+[#1](https://thingofthings.wordpress.com/2017/03/17/itt-2-blanchard-bailey/): GI: 0.6, BBL: 0.4, Actual: GI ✔, Bayes-score: −0.737  
+[#2](https://thingofthings.wordpress.com/2017/03/20/itt-3-blanchard-bailey/): GI: 0.4, BBL: 0.6, Actual: GI ✘, Bayes-score: −1.322  
+[#3](https://thingofthings.wordpress.com/2017/03/21/itt-4-blanchard-bailey/): GI: 0.4, BBL: 0.6, Actual: BBL ✔, Bayes-score: −0.737  
+[#4](https://thingofthings.wordpress.com/2017/03/22/itt-5-blanchard-bailey/): GI: 0.9, BBL: 0.1 (shibboleth fail!—people who believe in biology do not say "assigned at birth" when describing their own beliefs! Also, failure to notice the obvious "for the same reasons men are" re programmers), Actual: BBL ✘, Bayes-score: −3.322  
+[#5](https://thingofthings.wordpress.com/2017/03/23/itt-6-blanchard-bailey/): GI: 0.2, BBL: 0.8 (preach it!), Actual: GI ✘, Bayes-score: −2.322  
+[#6](https://thingofthings.wordpress.com/2017/03/24/itt-7-blanchard-bailey/): GI: 0.8, BBL: 0.2 ("male socialization, which unlike androphilic trans women they actually tend to absorb as kids" sounds like someone who believes that innate gender identity determines what socialization you latch onto from your culture, rather than someone who actually believes in sexual dimorphism), Actual: GI ✔, Bayes-score: −0.322  
+[#7](https://thingofthings.wordpress.com/2017/03/27/itt-8-blanchard-bailey/): GI: 0.9, BBL: 0.1 (shibboleth fail again!—[my comment at _Thing of Things_](https://thingofthings.wordpress.com/2017/03/27/itt-8-blanchard-bailey/#comment-25273)), Actual: GI ✔, Bayes-score: −0.152  
+[#8](https://thingofthings.wordpress.com/2017/03/28/itt-9-blanchard-bailey/): GI: 0.1, BBL: 0.9 (raw reality), Actual: BBL ✔, Bayes-score: −0.152  
+[#9](https://thingofthings.wordpress.com/2017/03/29/itt-1-blanchard-bailey/): GI: 0.85, BBL: 0.15 ([my comment](https://thingofthings.wordpress.com/2017/03/29/itt-1-blanchard-bailey/#comment-25321)), Actual: GI ✔, Bayes-score: −0.234
+
+**Proportion correct** (construing assignment of probability greater than 0.5 to the actual answer as "correct"): 12/18  
+**Total Bayes-score**: −18.012 (_just_ worse than chance)