Since Harrison and I originally talked about Language Chains, Harrison has done a phenomenal job instantiating the concept into a library and a community!
But something has always bothered me.
I’ve been continually surprised that the LangChain library hasn’t yet been useful to me. Initially I assumed it was because my language preference was JavaScript, but I started seeing tweets from people who started with LangChain then migrated off so I wanted to understand why.
So, I spent the last week thinking carefully about language modeling abstractions to understand if there was something deeper.
Here’s my current thinking:
The utility of an abstraction is that it removes complexity by allowing a reframe of the problem to lower cognitive load. However, some of the LangChain core abstractions - prompts, chains, and memory - seem like they're adding cognitive load rather than removing.
It wasn't until I saw AI functions as an abstraction that I understood why.
This abstraction utilizes the docstring to have GPT infer the function output if the function existed! This is a powerful abstraction because functions are already our tool box in every programming language for managing abstraction, and a transformer call pattern matches perfectly onto a function call. Instead of adding new concepts to utilize, AI functions strictly reduces the cognitive load of programming
Each GPT call maps
and with a bit more wrangling can map
From this innovation, no additional framework is needed. Need a “SequentialLanguageChain” - use your language’s favorite async feature to write imperative code. Need to execute two prompts in parallel? Great, in javascript throw in a Promise.all( … ). Need a “Tool”, awesome, write an AI function.
There’s no need to reinvent the wheel. Simply think of transformers as replacing function calls and utilize whatever your favorite Actor/Observer/AST/etc design pattern or utility you already know for asynchronous programming! Need to manage memory? Well if you’re working in JS you already have all of React’s memory management utilities and libraries.
Instead of LangChain being a framework it’s more like a vibrant community that’s exploring and discovering together. Which is, of course, incredible. It has done amazing things for the space, and these comments are in no way intended to minimize either LangChain’s or Harrison’s impact.
So, what is the actual problem that needs to be solved with a language modeling framework? It’s debuggability and traceability - the real problems to solve are:
“How do I know this new AI function I added is working as it’s supposed to?”
“If I just made a minor change to this one AI function, how does that propagate?”
“How do I evaluate outputs in a consistent way?”
“How do I do time travel debugging?”
“How does my performance change if I substitute language model A for language model B?”
In fact, solving these problems actually does not even really require any particular prescription of how one incorporates language models into code flow. Instead, it’s much more akin to traceability problems in complex software systems already, where observability monitors can be added at different places in the AST.
Overall, I’ve already started switching my codebase to AI functions, and I’m really excited to see what new traceability, observability, and time travel debugging frameworks emerge for language models - these will be the next steps in our devtools infrastructure.