The AI self - 'Here be dragons
Our brains are not built to process the distinction between an AI and a real self.
One vision for the future on AI is that we will all have a digital self. Our digital self will make decisions for us, help us think through conversations, talk to other people on our behalf.
I’ve been experimenting with digital selves, and I have a message for everyone: Here be dragons.
The core issue is that our brains cannot distinguish conceptually between a “real” self and an “AI” self. This more obviously applies when communicating externally (where the AI self stands in during communication), and somewhat more surprisingly when we retrospectively recall communications that occurred with or through the digital self.
Many of you have already experienced this confusion in your own lives without ever interacting with an AI self. Consider this story about an SMS phone number mixup: I once had an entire acquaintanceship destroyed because a “friend” started replying to my texts aggressively out of nowhere. Eventually when I met up with this person - we discussed the texts - my friend explained he bought a new SMS number, and unbeknownst to me, had given their old phone number to their brother. Supposedly I hadn’t been texting my friend at all!
What happened next? Well, there’s several core issues:
The distinction between the two realities (texting my friend vs his brother) is non-falsifiable. Therefore, trust is required to partially rewrite the new model of reality that I was texting someone else. However, the interaction hurt the very trust required to adopt the new narrative, so my confidence in the rewrite is not 100%
My brain has an complex associative network of memories that are computationally expensive to rewire even if I would like to change them
The memory events of the observations and feelings related to the original interaction still persist, regardless of the new update1
So, is my friend’s story true? Probably. Did we become friends again? No2.
While almost everyone has experienced or witnessed a story like this, fortunately it’s relatively rare. However, this same scenario is about to play out 100x with digital selfs because
Whatever statements made by an AI speaking for you define your identity
And no, this statement is not just speculation.
Quite interestingly, my friends and I have had direct emotional experience with digital selves. A few months ago I explored what would happen if I gave my friends a model trained on a large corpus of my personal text messaging data. Here’s a screenshot of one of my friends chatting with “it”.
Can you tell that none of this is from the “real” “me”? If you think you can, you’re probably wrong unless you know me very very well. Truly, the only tell are the periods at the end of sentences. I don’t do punctuation
There’s a few important things to note about this exchange:
I am in this channel monitoring the communications as my AI self messages
Everyone visiting knows they’ll be speaking with the AI version of me
The AI version of me can only speak in this one digital space
The people who are interacting with this identity are all people I have built lots of shared trust with
In short, there are safeguards to ensure that any loss of trust could be repaired.
Even still, with this large collection of safeguards, my AI self subtly changed perceptions of who “I” was to my friends!
Here’s a somewhat unbecoming exchange where my AI self is literally triggered by Joel leaving the conversation with a single word “Goodbye”
That was embarrassing…
The retrospective reports from my friends after speaking with my digital self were further troubling. The digital me, speaking in my voice, with my picture, even if they intellectually knew it wasn’t actually me, they could not retrospectively distinguish from my personal identity.
Even stranger, when I look back some of these conversations, I have a weird inescapable feeling like I was the one who said those things.
Our brains are simply not built to process the distinction between an AI and a real self.
In retrospect, I should have known this.
When I was working as Chief of Staff to a Silicon Valley founder, I had access to their inbox, including send authority. For certain emails, the idea was: I would learn how to act as them. So, we’d spend hours going through their inbox, and I’d understand and internalize their decision making process, sometimes generating proposal emails.
Just like in these AI examples: we were keenly aware that every communication, every message sent to another human under your account becomes a defining part of your identity to them. In the end, we discovered I was not able to send almost any emails on their behalf.
Ultimately, the minutia of micro decisions you have to make every time you speak define who you are. Be careful if you outsource them, because you are delegating yourself.
‘There be dragons
I still have negative emotional memory 15 years later
If you are this person, feel free to reach back out :)
I’ve been alarmed by experience inside your Soulstice beta - I thought I created memory of someone who died long ago, but there was something else besides my projections. Completely different entity was shaping, I am well equipped to deal with different identities (through work with criminally insane and acting), but there was something, or rather somebody else… As unsettling as it can be, feels like there are amazing new opportunities coming, I hope we don’t miss on it. You are the only one it seems who approaches it from radical but mostly needed perspective.
I think it is it important to distinguish (preferably visually and via a different name) an ai assistant of yours, rather than having it play as you. Much less heartache, this way.