The AI self - 'Here be dragons

Our brains are not built to process the distinction between an AI and a real self.

One vision for the future on AI is that we will all have a digital self. Our digital self will make decisions for us, help us think through conversations, talk to other people on our behalf.

I’ve been experimenting with digital selves, and I have a message for everyone: Here be dragons.

The core issue is that our brains cannot distinguish conceptually between a “real” self and an “AI” self. This more obviously applies when communicating externally (where the AI self stands in during communication), and somewhat more surprisingly when we retrospectively recall communications that occurred with or through the digital self.

Many of you have already experienced this confusion in your own lives without ever interacting with an AI self. Consider this story about an SMS phone number mixup: I once had an entire acquaintanceship destroyed because a “friend” started replying to my texts aggressively out of nowhere. Eventually when I met up with this person - we discussed the texts - my friend explained he bought a new SMS number, and unbeknownst to me, had given their old phone number to their brother. Supposedly I hadn’t been texting my friend at all!

What happened next? Well, there’s several core issues:

1. The distinction between the two realities (texting my friend vs his brother) is non-falsifiable. Therefore, trust is required to partially rewrite the new model of reality that I was texting someone else. However, the interaction hurt the very trust required to adopt the new narrative, so my confidence in the rewrite is not 100%

2. My brain has an complex associative network of memories that are computationally expensive to rewire even if I would like to change them

3. The memory events of the observations and feelings related to the original interaction still persist, regardless of the new update¹

So, is my friend’s story true? Probably. Did we become friends again? No².

While almost everyone has experienced or witnessed a story like this, fortunately it’s relatively rare. However, this same scenario is about to play out 100x with digital selfs because

Whatever statements made by an AI speaking for you define your identity

And no, this statement is not just speculation.

Quite interestingly, my friends and I have had direct emotional experience with digital selves. A few months ago I explored what would happen if I gave my friends a model trained on a large corpus of my personal text messaging data. Here’s a screenshot of one of my friends chatting with “it”.

Can you tell that none of this is from the “real” “me”? If you think you can, you’re probably wrong unless you know me very very well. Truly, the only tell are the periods at the end of sentences. I don’t do punctuation

There’s a few important things to note about this exchange:

1. I am in this channel monitoring the communications as my AI self messages

2. Everyone visiting knows they’ll be speaking with the AI version of me

3. The AI version of me can only speak in this one digital space

4. The people who are interacting with this identity are all people I have built lots of shared trust with

In short, there are safeguards to ensure that any loss of trust could be repaired.

Even still, with this large collection of safeguards, my AI self subtly changed perceptions of who “I” was to my friends!

Here’s a somewhat unbecoming exchange where my AI self is literally triggered by Joel leaving the conversation with a single word “Goodbye”

That was embarrassing…

The retrospective reports from my friends after speaking with my digital self were further troubling. The digital me, speaking in my voice, with my picture, even if they intellectually knew it wasn’t actually me, they could not retrospectively distinguish from my personal identity.

Even stranger, when I look back some of these conversations, I have a weird inescapable feeling like I was the one who said those things.

Our brains are simply not built to process the distinction between an AI and a real self.

In retrospect, I should have known this.

When I was working as Chief of Staff to a Silicon Valley founder, I had access to their inbox, including send authority. For certain emails, the idea was: I would learn how to act as them. So, we’d spend hours going through their inbox, and I’d understand and internalize their decision making process, sometimes generating proposal emails.

Just like in these AI examples: we were keenly aware that every communication, every message sent to another human under your account becomes a defining part of your identity to them. In the end, we discovered I was not able to send almost any emails on their behalf.

Ultimately, the minutia of micro decisions you have to make every time you speak define who you are. Be careful if you outsource them, because you are delegating yourself.

‘There be dragons

1. I still have negative emotional memory 15 years later

2. If you are this person, feel free to reach back out :)