A photographic memory that isn't yours
Miles · 0:00I want you to start by imagining something. And depending on how you look at it, this is either going to sound incredibly utopian or, well, a little bit terrifying.
Nora · 0:10I mean, it usually is a bit of both with this stuff, right?
Miles · 0:12Right, exactly. So I want you to imagine an AI that knows exactly who you are. And it doesn't know you because it, like, analyzed your face or tracked your GPS or did some kind of creepy demographic profile.
Nora · 0:24Oh, good. No creepy profile. Yeah, I know.
Miles · 0:26No, it knows you because it has read every single email you have ever sent or received. Every text message, every book you've highlighted, every PDF, every stray thought you've ever jotted down in a notes app.
Nora · 0:38Wow.
Miles · 0:38It has read every single document you've ever encountered in your entire life. It holds all of it simultaneously in its working memory. And it never forgets.
Nora · 0:46It is the ultimate photographic memory, but it's not yours. It belongs to the machine that works for you. And what makes this concept so fascinating isn't just the sheer scale of the memory itself. It's the way it fundamentally alters the relationship between humans and information.
Miles · 1:02Right, because we're so used to constantly triaging what we consume. Yeah. You know, our own biological hard drives are just so incredibly limited.
Nora · 1:10Exactly. We forget what we had for breakfast on Tuesday, let alone a crucial clause in a contract we read, what, three years ago.
Miles · 1:17I can barely remember what I read yesterday. And the wildest part about this scenario, the people building the very frontier of artificial intelligence right now, they aren't just daydreaming about this as some distant sci-fi concept.
Nora · 1:29This is the literal roadmap.
Miles · 1:31This is the explicit goal. This is the exact endpoint they are designing the future of software around. So, welcome to the Deep Dive.
Nora · 1:39Glad to be here.
Miles · 1:40Today, our mission is to explore this endpoint, a single massive AI model that holds your entire life's context. We're going to look at how developers are like hacking together early versions of this today.
Nora · 1:52Which is fascinating in itself.
Miles · 1:53It really is. And we're going to examine the very subtle, almost invisible dangers of letting machines do all our reading. We've got a great stack of sources for this. Remarks from Sam Altman, an interview with Satya Nadella, workflows from Andrej Karpathy, and some brilliant technical breakdowns.
Nora · 2:11It's a dense stack today.
Miles · 2:12It is. Okay. Let's unpack this, starting with that grand vision. vision, the trillion token life.
What a token actually is, and why context windows bind
Nora · 2:18Danielle Pletka, To really grasp the magnitude of this, we need to look at what the industry leaders are actively saying behind closed doors. Danielle Pletka, Recently, Sam Altman spoke at Sequoia's AI Ascent, and he laid out what he explicitly called the platonic ideal for artificial intelligence.
Miles · 2:34Sam Altman, The platonic ideal. That's a big face.
Nora · 2:36Danielle Pletka, It is. He described a future state involving a small, highly efficient reasoning model, but equipped with a one trillion token context window.
Miles · 2:44A trillion tokens. I feel like we throw around the word trillion so much in tech and finance that it just loses its meaning completely.
Nora · 2:51It really does become just a buzzword.
Miles · 2:53Yeah. For you listening, let's break down what a token actually is. In the AI world, a token is basically a piece of a word. A rough rule of thumb is that one token is about three quarters of a standard English word. So a trillion tokens equates to roughly 750 billion words.
Nora · 3:12Which is a number so large it almost breaks human comprehension. I mean, let's ground it in reality for a second.
Miles · 3:17Please do.
Nora · 3:18An average novel is somewhere around 80,000 to 100,000 words. If we take the higher end, a trillion tokens is the equivalent of seven and a half million books.
Miles · 3:27Oh, my God. Yeah.
Nora · 3:28That is thousands of times more information than a human being could possibly read in an entire lifetime, even if they read constantly from the moment they were born.
Miles · 3:37Until the day they died.
Nora · 3:38Yeah. That's insane.
Miles · 3:39And say, so we aren't just talking about dumping your favorite books into an AI.
Nora · 3:43Not at all.
Miles · 3:43We're talking about dumping in your personal diaries, your company's entire corporate history dating back to its founding. Every Slack message, all the source code your engineers have ever written.
Nora · 3:53Every financial ledger, every meeting transcript.
Miles · 3:56Everything. And the key here, the thing that makes Altman's ideal so revolutionary is that you just leave it there.
Nora · 4:02Yes. That is the massive paradigm shift here.
Fine-tuning versus context: two ways to make a model know your world
Miles · 4:05Right.
Nora · 4:05Right. Historically, if you wanted an AI model to really know specific information about your company, you had to go through this highly complex, computationally expensive process. Fine tuning. Exactly. Fine tuning. You had to painstakingly format your data, feed it through the neural network and actually alter the underlying weights and biases of the model itself.
Miles · 4:29It was like performing brain surgery just to teach the AI a new fact.
Nora · 4:32That's a great way to put it. But with a trillion token context window, the model never needs to be retrained. You literally just pour your life into its active working memory.
Miles · 4:42And as you live, as you generate new emails, new documents, new conversations, the data just continuously appends to the window. The model just reasons across all of it effortlessly all the time.
Nora · 4:54It doesn't have to search for the data. The data is already inside its active cognition.
Miles · 4:58Which is wild to think about.
Nora · 4:59If we connect this to the bigger picture, it changes the entire purpose of software. We are completely conditioned to view software as a place we go to store things so that we can retrieve them later.
Miles · 5:10Right, like putting numbers in a spreadsheet.
Nora · 5:12Or putting documents in a folder structure or customer details in a database. The software is just a passive bucket.
Miles · 5:19But Altman's ideal completely flips that on its head. The software isn't just storing your life. The software has actively read your life.
Nora · 5:28And this philosophical shift isn't just some open AI talking point either. Satya Nadella, the CEO of Microsoft, recently did an interview where he took this exact concept and ruthlessly applied it to the enterprise software world.
Miles · 5:43His prediction was staggering. He foresees the aggressive collapse of traditional software as a service. He literally asked the question, on the record, why do I need Excel?
Nora · 5:52Uttering that phrase as the CEO of Microsoft is akin to the CEO of Coca-Cola asking why anyone needs soda.
Miles · 5:58Right. Microsoft's entire empire was built on Excel.
Nora · 6:01Exactly. But if you follow his logic regarding AI context windows, his conclusion is entirely sound. Nadella points out that almost all traditional business applications, whether it's Excel, a CRM like Salesforce, or a huge HR platform like Workday, are at their core just CRUD databases.
Miles · 6:19Okay, for those not living in database architecture all day, CRUD stands for Create, Read, Update, and Delete. It's just the four basic functions of persistent storage.
Nora · 6:29And that's really all these multi-billion dollar software companies are selling you. They provide a database to hold your information. They add a layer of business logic on top to tell the data how to interact.
Miles · 6:40And then crucially, they build a graphical user interface designed for a human being to click through.
Nora · 6:45Right. They build the menus, the dashboards, the dropdowns, the colorful charts.
Miles · 6:49But if the AI model already holds the entire corporate history in its context window, and the AI is the one doing the work well, it doesn't need a pretty graphical user interface.
Nora · 7:00No, a machine doesn't care about the drop shadow on a save button.
Miles · 7:03The machine actively resents the graphical user interface.
Nora · 7:06It really does. Nadella's argument is that in this new agentic era, the AI becomes the business logic tier. The AI talks directly to the raw databases via APIs.
Miles · 7:18It updates multiple repositories simultaneously without ever opening a window on a screen.
Nora · 7:23Exactly. The entire human UI, the menus, the buttons, the meticulously designed software dashboards, it all just collapses into obsolescence.
Miles · 7:33I'm trying to visualize what this actually looks like for the end user, though. Because if the software as we know it disappears and the AI is doing all the reading and organizing what happens to the human interface.
Nora · 7:43That's a great question.
Miles · 7:44I mean, think about having a personal chief of staff. Someone who never sleeps, possesses a flawless photographic memory of everything you've ever done, and perfectly updates your calendar and finances in the background. That sounds utopian.
Nora · 7:56Highly utopian.
Miles · 7:57But if they do everything, are we just going to stare at a blank chat box forever? Is the entire future of human-computer interaction just a blinking cursor waiting for us to type a prompt?
Nora · 8:06What's fascinating here is that we are witnessing a fundamental economic shift from human attention to LLM attention.
Miles · 8:12Oh, that's an interesting way to frame it.
Nora · 8:14For the past 30 years, we've built a digital economy where human eyeballs were the ultimate prize. Websites, applications, digital documents, they were all meticulously optimized to capture, direct, and hold human attention.
Miles · 8:28But if Nadella and Altman are right, the primary consumer of digital information going forward is no longer the human.
Nora · 8:35It is the AI agent.
Miles · 8:37The entire internet is basically being gentrified for machines. And this is something Jensen Huang, the CEO of NVIDIA, has brought up as well. Agents are voracious readers. They don't experience eye strain.
Nora · 8:48They don't get distracted by a notification on their phone.
Miles · 8:50Right. They can consume millions of words in seconds. In this new world, human attention is obsolete. Tokens are the new currency. The entire software stack is being re-architected not for human convenience, but for machine legibility.
Nora · 9:05Which creates an incredible tension. Because while a trillion token context window is the ultimate destination, a nation. We simply do not have the hardware or the compute efficiency to achieve it today.
Miles · 9:15We cannot hold a whole human life or a whole corporate history in active memory right now. The computational cost would literally bankrupt a nation.
Nora · 9:25It would. So we have this massive gap.
Miles · 9:27So how do we bridge that gap since we don't have this mythical trillion token window? How are developers actually hacking this concept together today?
Nora · 9:35To figure that out, we have to zoom in from the macro level corporate visions of Altman and Nadella to the micro scale reality of what is happening on developer laptops right now.
Miles · 9:45And a perfect illustration of this comes from Andrej Karpathy.
Nora · 9:48Karpathy is a phenomenal case study for this exact transition. As a founding member of OpenAI and the former director of AI at Tesla, his workflows are often a preview of where the
Miles · 9:59broader industry is heading in a year or two. Yeah, he's always slightly out of the curve.
Karpathy's four phases: compile, synthesize, query, lint
Nora · 10:03Always. And he recently published a thesis that acts as the boots on the ground execution of of everything we've just discussed. He stated that by 2025, 99.9% of content optimization is going to be directed at large language models, not human beings.
Miles · 10:17He used software documentation as his primary example, which I found fascinating. Let's think about how a developer learns a new tool today. You go to a dedicated website. Right. It has pretty HTML pages, navigation sidebars, custom CSS dialing to make the fonts look nice, and complex drop-down menus to organize the chapters.
Nora · 10:37It is built entirely on the assumption that a human being with a mouse is going to sit there, click through it, and read it page by page.
Miles · 10:45But Karpathy argues that this architecture is completely backwards for the error we are entering.
Nora · 10:51Completely backwards. If the entity actually reading the documentation is an AI coding assistant that you've tasked with writing a script, that AI doesn't care about your CSS.
Miles · 11:00It hates your drop-down menus.
Nora · 11:01It really does. To a machine reader, all that beautiful visual formatting is just computational noise that it has to painstakingly filter out just to get to the actual information.
Miles · 11:10I just love the idea of an AI getting incredibly frustrated by a beautifully designed web page.
Nora · 11:16It's a funny image, but it's true. Karpathy says the documentation shouldn't be a website at all. It should just be a single, massive, plain text Markdown file.
Miles · 11:25For those unfamiliar, Markdown is a very lightweight way to format text. It's just raw words with a few symbols, like asterisks for bolding. Nothing fancy.
Nora · 11:34Right. He wants everything stripped down to a raw.md file explicitly designed to be dropped straight into an LLM's context window.
Miles · 11:41And the industry is already mobilizing around this exact philosophy. There is a proposed new web standard called lms.txt. You listening are probably familiar with how websites have a robots.txt file.
Nora · 11:53Yeah, the hidden file that tells search engines like Google how to crawl and index the site.
Miles · 11:57Exactly. Well, the proposal here is that every modern website would also include an LOMS.txt file.
Nora · 12:03And what does it actually do?
Miles · 12:05It acts as a bypass. When an AI agent visits the website, the LMS.txt file points the agent away from the pretty human-facing HTML site and wraps it directly to a clean, machine-readable, marked-down version of the entire site.
Nora · 12:17Wow. It is the structural, architectural realization of the shift from human attention to machine attention.
Miles · 12:24It really is. And this brings us to how Karpathy applied this philosophy to his own personal life, creating what he calls an LLM wiki. We came across a fantastic technical breakdown of this exact workflow from Dare.ai Academy.
Nora · 12:39Honestly, looking at this workflow is like looking at a microcosm of the trillion token dream.
Miles · 12:43It really is. Instead of trying to hold his entire life in a model, Karpathy is using this system for a highly concentrated knowledge base. It's about 100 deep research articles totaling roughly 400,000 words.
Nora · 12:55And the workflow Dayer.ai maps out is broken down into four continuous automated phases. Ingest, compile, query or enhance, and lint or maintain.
Miles · 13:05Let's really dig into your daily life for a second before we explain how this works because I want the contrast to be super clear. Think about the last time you had to organize your digital files.
Nora · 13:12Oh, it's always a nightmare.
Miles · 13:14If you are anything like me, your downloads folder is a disaster zone. You have files named Q3 final.pdf, Q3 report final V2.pdf, actually final use this own.pdf.
Nora · 13:24You will have been there.
The friction of building a knowledge base by hand
Miles · 13:25Right. When we try to build a personal knowledge base, we spend hours creating folders, subfolders, agonizing over naming conventions, trying to remember what tags we used for a specific project six months ago. It's exhausting. We organize everything defensively, just hoping our future self will somehow guess where we hid the information.
Nora · 13:45And the human categorization process is highly fraught, deeply subjective, and incredibly prone to failure. But Karpathy's workflow completely bypasses this human limitation.
Miles · 13:56How does phase one start?
Nora · 13:57In phase one, ingest, raw data flows into his system from everywhere. He uses web clippers to save long form articles. He downloads massive PDFs of complex academic research papers. He pulls in raw data structures from GitHub.
Miles · 14:10But he doesn't organize a single piece of it.
Nora · 14:12Not one. Crucially, all of his incoming data gets automatically converted into raw, unformatted markdown files and dumped into a single staging directory.
Miles · 14:21No folders.
Nora · 14:21No folders. It is just a massive chaotic pile of raw text.
Miles · 14:25Because the human isn't gonna organize it, the machine is. And that leads to phase two, compile. And this is where the whole paradigm shifts. Karpathy doesn't use the large language model just as a chat bot to answer questions.
Nora · 14:38No, he uses the LLM as a compiler.
Miles · 14:40And we should definitely define what that means in this context. A traditional software compiler takes human written code and translates it into machine code that a computer can actually run.
Nora · 14:50Right. Karpathy's LLM compiler takes the chaotic pile of human knowledge and translates it into a perfectly structured digital brain.
Miles · 14:57Let's walk through what this Python script is actually doing in the background while Karpathy sleeps. The AI wakes up, reads that entire staging directory of raw files, and begins to automatically build a structured, deeply interlinked wiki.
Nora · 15:10It's performing tasks that would take a human archivist weeks to do. It reads a dense 40-page research paper and automatically writes an index file summarizing the core findings. But it goes much further than simple summaries.
Miles · 15:24Right. It scans across all 100 documents, identifies overlapping themes, and creates brand new concept articles.
Nora · 15:29Meaning, if it sees three different papers discussing a specific type of neural network architecture, it pulls the insights from all three and writes a brand new synthesized overview document about that architecture.
Miles · 15:43Heavily linking back to the raw source files, of course.
Nora · 15:45Exactly. It even generates the code to render slide decks and visual charts based on the data it extracted. And it automatically maintains a staggering complex graph of hyperlinks between all these different concepts.
Miles · 15:58The LLM is acting as a tireless librarian, a highly skilled technical author, and a meticulous archivist, all running simultaneously in the background.
Nora · 16:08It's incredible. Then we move to phase three, query and enhance. This is where Karpathy actually interacts with the knowledge base he just had his machine build.
Miles · 16:15He uses a Q&A agent interface to ask incredibly complex research questions across those 400,000 words. But here is the critical ecosystem building part. Yeah, this is key. When the AI formulates an answer to his question, it doesn't just display the answer on screen and then forget it the moment he closes the chat.
Nora · 16:33No, that answer is immediately filed back into the wiki as a brand new permanent document. Every single exploration, every question asked, actively adds to the total volume and intelligence of the knowledge base.
Miles · 16:46The system gets smarter and more comprehensive simply by being used.
Nora · 16:50And finally, phase four, lint and maintain. For non-developers, linting is a programming term for running a tool that flags programmatic errors or bugs. Right. In Karpathy's wiki, the LLM runs health checks in the background constantly. It scans the entire web of documents for logical inconsistencies.
Miles · 17:07Like, if it realizes that a concept article is missing a crucial piece of context, it will autonomously execute a web search, read new articles on the internet, and rewrite its own internal documentation to fill the gap.
Nora · 17:19It suggests new hyperlink connections, and then the cycle just repeats. It is a living, self-healing, autonomously expanding knowledge base.
Miles · 17:27But we had to emphasize the technical distinction here, because it is the absolute linchpin of this entire microscale setup. The expert analysis from DataArt AI highlights one critical fact.
Nora · 17:38What's that?
Miles · 17:39Karpathy's method is completely retrieval-free.
Nora · 17:41Ah, yes.
Miles · 17:42Okay, let's unpack this, because retrieval-free is the magic phrase here. In almost all modern AI applications that interact with documents, developers rely on a system called RAG Retrieval Augmented Generation.
Why retrieval beats stuffing a million tokens in
Nora · 17:54We use RBLAM because, as we established earlier, you cannot fit a million documents into the AI's active brain at once. So when you ask a RAGA system a question, it first acts like a search engine. It searches through your massive database of documents, retrieves the handful of paragraphs that mathematically seem the most relevant to your prompt, and then only feeds those few extracted snippets into the AI's context window to generate the answer.
Miles · 18:19But Karpathy isn't doing that. Because 400,000 words can technically fit into the cutting-edge context windows of today's frontier models, he skips the retrieval step entirely.
Nora · 18:29He doesn't search for snippets.
Miles · 18:30No. He just takes the entire wiki, all 400,000 words, and drops the whole thing directly into the model's brain every single time he asks a question.
Nora · 18:38It is the exact realization of Sam Altman's platonic ideal, just scaled down to what today's hardware can actually handle.
Miles · 18:46The model isn't guessing which document is relevant based on a crude search algorithm. It isn't missing context because a relevant paragraph didn't trigger a T-word match. It actually has the entire corpus, every concept, every link, every raw data point in its active instantaneous reasoning space.
Nora · 19:03It's powerful.
Miles · 19:04What stands out to you? Just think about the sheer friction that is removed from the cognitive process. What if you never had to organize a file again because the AI just reads the raw dump and builds a flawless cross-reference structure for you?
Nora · 19:18Sounds perfect.
Miles · 19:18What if every question you asked your personal database was answered with total contextual awareness of everything you've ever read? It sounds like absolute magic.
Nora · 19:26It does sound like magic. The frictionless nature of it is incredibly seductive. But whenever you remove that much friction, whenever you delegate the core foundational act of reading and synthesizing entirely to a machine, you inevitably introduce new, deeply structural vulnerabilities.
Miles · 19:43And in this case, the vulnerability isn't a sudden catastrophic system crash. It's a slow, invisible degradation of truth.
Nora · 19:51Which brings us to the tension at the heart of this entire movement. There is a brilliant critique published by Anand Lahoti that looks at Karpathy's magical LLM wiki pattern and identifies a massive fatal flaw.
Miles · 20:04He calls it the failure mode no one is talking about. And once you understand it, it completely changes how you view handing over your cognitive load to an AI.
Nora · 20:14Lahoti is pointing out a fundamental architectural flaw in how we are allowing these models to interact with and more importantly alter our data he describes this vulnerability as
Miles · 20:24the danger of knowledge-based poisoning knowledge-based poisoning to understand how poison works we have to understand two completely different ways that ai can handle information
Nora · 20:33right time synthesis versus query time synthesis and karpathy's autonomous self-organizing wiki is heavily built on right time synthesis exactly let's define the mechanics of right time synthesis
Miles · 20:43In this architecture, the LLM reads the raw documents the moment you upload them at ingestion time, or write time.
Nora · 20:50It then immediately writes its own summaries, generates its own concept articles, and maps out its own connections. And here's the critical part. Those AI-authored summaries are then saved permanently into the wiki.
Miles · 21:02They become official, first-class documents within the corpus.
Nora · 21:05Yes. Yes. So when you ask the system a question later, the LLM is often reading its own summaries to give you the answer rather than the raw original text.
The 2% discount problem: how detail dies in summary
Miles · 21:15If the problem is the AI writing the notes and treating them as facts, we're essentially building a machine that hallucinates its own reality. Let's look at what happens when this goes wrong, because Lahoti gives this incredibly concrete, easy to understand example.
Nora · 21:28It's a great example.
Miles · 21:29Let's say you upload a raw source document into your massive company wiki, a heavily negotiated vendor contract, and buried on page 12 of that contract, it explicitly states that the payment terms are net 30 with a 2% discount if paid within 10 days.
Nora · 21:45Right. Now the LLM compiler wakes up for its nightly run. It reads that raw contract during phase two, the compile phase. It decides it needs to update a nice, clean, overarching concept article called company payment terms. terms. In that AI-authored article, it synthesizes the dense legalese of the contract by writing, standard agreements use net 30 terms with early payment discounts.
Miles · 22:08Which sounds totally reasonable. If a human intern wrote that summary, you'd say they did a good job capturing the gist of it.
Nora · 22:14Absolutely.
Miles · 22:15But notice what happened to the data. It dropped the highly specific 2% figure. It dropped the strict 10-day window. It's just a slight loss of fidelity, a minor smoothing over the details, but at the time, it seems perfectly fine. But then, fast forward six months,
Nora · 22:32you are negotiating a massive new deal with a different vendor, and you need to know your leverage. So you ask your magical AI wiki, what is our typical early payment discount?
Miles · 22:42And because the system relies heavily on those beautifully structured AI-authored concept articles for quick retrieval and context, it pulls up the company payment terms article.
Nora · 22:51It doesn't retrieve the original 40-page contract because the concept article is heavily linked, highly optimized, and mathematically seems like the perfect match for your query.
Miles · 23:01So the model reads its own summary. It sees that the summary mentions early payment discounts, but doesn't have a specific percentage attached to it anymore.
Nora · 23:09And because LLMs are designed to be helpful, and they hate saying, I don't know, it interpolates, it hallucinates a standard industry number.
Miles · 23:16It tells you, with absolute supreme confidence, our standard early payment discount is 5%. It entirely invents a reality because it doesn't have the 2% ground truth in front of it anymore.
Nora · 23:29And you might think, well, Karpathy's Phase 4 Lint and Maintain health check would catch that, right? The system is supposed to scan for errors. Right. But think about how the health check actually works. The system looks at the company payment terms article, compares it to another AI-authored article it wrote last month about vendor agreements, sees that neither of them mention the 2%, and concludes that the wiki is perfectly healthy, internally consistent, and accurate.
Miles · 23:55The AI confirms its own hallucination.
Nora · 23:57Exactly.
Miles · 23:58Here's where it gets really interesting. It is literally a high-tech game of telephone, but the AI is playing both ends of the line.
Nora · 24:05Such a good analogy.
Miles · 24:06The AI summarizes the original document, loses a crucial detail in the translation, and then months later, another instance of the AI reads that flawed summary and bases its entire reality on it, drifting even further from the truth. If we let the AI write the persistent notes that make up our knowledge base, aren't we just laundering hallucinations into hard facts?
Nora · 24:28We absolutely are.
Miles · 24:30We are creating this terrifying closed loop where the AI is just citing itself, drifting further and further from the original ground truth with every cycle.
Nora · 24:39That is exactly Lahoti's warning, and he frames it brilliantly. He says that over time, across tens of thousands of documents, the knowledge base doesn't suffer a catastrophic crash. It just quietly, imperceptibly drifts.
Miles · 24:52It becomes a closed epistemic loop.
Nora · 24:54The chain of custody back to the original source material slowly frays and eventually snaps entirely. entirely. And the worst part is nobody notices because every individual output the AI gives you looks completely coherent, perfectly formatted and overwhelmingly confident.
Miles · 25:08If the AI writing its own notes is this slow acting poison, then the only logical antidote must be forcing the AI to read the raw original documents every single time. It can't be allowed to write permanent summaries at all.
Nora · 25:22Precisely. And Anand Lahoti defines this antidote as query time synthesis. In this architectural textual model, you strictly forbid the LLM from ever authoring prose that gets permanently saved
Miles · 25:32into the knowledge base. The original documents, the emails, the PDFs, the contracts are treated
Originals as immutable ground truth
Nora · 25:37as immutable sacred texts. You don't let the AI rewrite the sacred text. Right. At ingestion time, the LLM is only allowed to extract structure. It can pull out proper nouns. It can assign metadata tags. It can map geographic coordinates or date ranges. It is allowed to build a highly complex index but it is strictly forbidden from writing a narrative summary of what the document actually means so it's acting like an old-school
Miles · 26:00librarian meticulously filling out a card catalog with cross references rather than a high school student writing a spark note summary of the book
Nora · 26:07that's the perfect analogy the extracted structure is merely a navigation aid so returning to our scenario six months later when you ask the system about the early payment discount the system uses those metadata tags to locate the the original 40-poge vendor contract. It pulls the exact raw, unsummarized text into its context window and synthesizes the answer completely fresh right at that exact moment at query time.
Miles · 26:35So it reads the actual dense legalese, spots the 2%, and gives you the exact right answer. It never relies on a degraded memory of a summary.
Nora · 26:43Exactly.
Miles · 26:43But I have to imagine that query time synthesis is incredibly slow and expensive.
Nora · 26:48It is computationally massively more expensive. You are forcing the machine to read the raw, unfiltered text every single time a question is asked, rather than letting it read a highly compressed, optimized summary.
Miles · 26:59It's less elegant than Karpathy's self-healing, self-summarizing wiki.
Nora · 27:02It is. But, as Lahoti points out, for a system that needs to remain trustworthy over years or decades, it preserves the chain of custody indefinitely. Structural metadata is verifiable. AI-authored prose is not.
Miles · 27:14OK, so we have established a major foundational tension here in how we build the future of software. Yeah. Sam Altman wants this utopian trillion token context window where no retrieval is necessary at all. You just drop your whole life in.
Nora · 27:28Right.
Miles · 27:28Karpathy is hacking a micro version of that reality today with 400,000 words. But he relies on this potentially poisonous right time synthesis to make it smooth and autonomous. Yes. And Lahoti violently pushes back, saying we absolutely must use query time synthesis to stay tethered to reality, even if it's clunky and computationally heavy.
Nora · 27:48That's the landscape.
Miles · 27:49But all of this feels very theoretical, or it's relying on custom Python scripts built by elite AI researchers. What about the rest of us? What if I'm a lawyer or a researcher or just a digital pack rat, and I have a lifetime archive of 100,000 documents right now, today?
Nora · 28:05You're stuck.
Miles · 28:05I cannot fit 100,000 PDFs into any contest window on the market, and I certainly don't know how to write a custom compiler to manage it.
Nora · 28:12This is exactly where theoretical debates collide with market reality. And it brings us to a fascinating case study, a product fact sheet, and underlying technical thesis for a new tool called DocuStrata.
Miles · 28:24DocuStrata.
Nora · 28:25DocuStrata is explicitly engineered to handle the exact problem you just described, the massive, unmanageable, lifetime digital archive. We are talking about personal, professional, or corporate archives of 100,000 documents or more.
Miles · 28:41I read through their thesis, and their tagline is incredibly aggressive, but it really says it all. Read nothing. Know everything.
Nora · 28:48It's a bold claim, but it perfectly encapsulates this macroeconomic shift from human reading to machine reading that Nadella was talking about. DocuStrata's origin story is highly relatable, actually. The founder had accumulated over 100,000 documents in Evernote over a decade, and the subscription price suddenly went up drastically. But instead of just exporting the files to a cheaper, dumb storage drive, the founder had an epiphany. The real problem wasn't the cost of storage. The real problem was the realization that they possessed 100,000 documents they were literally never going to read again.
The shift from human reading to machine reading, priced
Miles · 29:19I relate to that on a spiritual level. How many PDFs did I have sitting in a folder labeled, to read this weekend, that I will literally never open?
Nora · 29:26Hundreds, probably.
Miles · 29:27How many massive email threads are archived from past projects that I could never successfully search through using standard keyword search? We hoard digital information out of anxiety, not utility.
Nora · 29:39Exactly. Traditional note-taking and storage tools like Evernote, Notion, or Google Drive are fundamentally built on the assumption that you, the human, are the primary reader. They provide folders and tags to help you organize things so that you can re-find a document to read it yourself. self. DocuStrata throws that assumption out the window. It is built on the premise that you have way too much to read. Human attention is the bottleneck. So the AI should read it for you.
Miles · 30:05And you simply interrogate the system. So DocuStrata ingests everything you have, your chaotic email dumps, your massive spreadsheets, dense PDFs. It even ingests scanned images. It uses advanced optical character recognition or OCR to turn images into text.
Nora · 30:21But it doesn't just blindly scrape the letters. Let's explain how spatial reasoning works in modern OCR, because it's not just reading left to right.
Miles · 30:30Oh, this is a crucial detail for making unstructured data useful.
Nora · 30:34When DocuStrata processes a photograph of a crumpled, coffee-stained restaurant receipt, it doesn't just read a string of text. It uses spatial reasoning models to understand the geometry of the document.
Miles · 30:46So it knows that a number floating at the bottom right corner situated next to the bolded word total represents the final amount you paid.
Nora · 30:54Exactly. It understands the structural relationship of the text on the page, extracting meaning, not just characters.
Miles · 31:00So it pulls all of this hyperstructured data into the system. And they have this very specific business model that I find completely fascinating because it inverts how we've paid for software for 20 years. They actively subsidize the ingestion of your data, but they monetize the interrogation.
Nora · 31:17It is a total inversion of the traditional cloud storage model. Usually companies like Dropbox or Google charge you based on how much space your data takes up on their servers. D'Aquestrada says, bringing the documents in and storing them is computationally cheap. Machine reading is essentially free now. The highly valuable, computationally expensive part, the part they charge a premium for, is the question.
Miles · 31:39Because asking an AI to synthesize an answer across 100,000 unique documents is a massive computational lift.
Nora · 31:47Yes.
Miles · 31:47But let's look at the technical reality they are facing. They are promising to let you interrogate 100,000 documents. As we establish, you cannot fit that into an LLM context window today, not even close. So Docustrade is forced to use retrieval. They must rely on Raggy.
Nora · 32:03Yes. They cannot offer the pure retrieval-free platonic ideal that Sam Altman dreams of. Instead, they embed the entire massive corpus into what is known as a vector index.
Miles · 32:13We should definitely explain what a vector index is because it's the hidden engine running almost all modern AI search. Think of a vector index like a massive, incredibly complex, multidimensional 3D map of human concepts.
Nora · 32:25It's a great way to picture it.
Miles · 32:25When traditional software searches for a file, it looks for the exact keyword you typed. If you search the word discount, it only finds documents containing the letters D-I-S-C-O-U-N-T.
Nora · 32:37But a vector index fundamentally understands meaning. When the AI reads your documents during ingestion, it converts every paragraph into a mathematical coordinate on that massive 3D map.
Miles · 32:50Concepts that mean similar things are placed geographically close to each other in this mathematical space.
Nora · 32:55So the concept of a discount lives in the exact same neighborhood as coupon, price reduction, and sale.
Miles · 33:01And completely unrelated concepts like dogs or weather live millions of miles away on this map.
Nora · 33:06Exactly. So when you ask DocuStrata a question about how to save money on a vendor, the system doesn't search for keywords. It converts your question into a coordinate, drops a pin on that specific neighborhood of the 3D map, and scoops up all the paragraphs that live nearby, regardless of the exact vocabulary they used.
Miles · 33:23It then takes those highly relevant, meaning-matched chunks of text and feeds only those chunks to the LLM to generate your answer.
Nora · 33:30So, mechanically, it is not Altman's platonic ideal of the trillion token window. It is the closest, most sophisticated buildable approximation we have right now. It uses vector-based retrieval to stand in for a context window that the hardware cannot yet support.
Miles · 33:47But DocuStrata makes a very specific design commitment to counteract the known hallucination problems with RGA. They claim to strictly enforce grounding and citations. Every single answer the system provides has to point directly back to the original source documents it extracted the information from.
Nora · 34:04The UI actually shows you the raw text snippets it used. The user can verify the answer rather than trusting the AI blindly.
Where DocuStrata commits, and where it plays with fire
Miles · 34:12Which sounds like they listen perfectly to Anand Lahoti's critique. They are prioritizing query time synthesis. They retrieve the raw originals via the vector index, synthesize the answer fresh, and provide citations to prove their work. It sounds like the perfect balance of scale and accuracy.
Nora · 34:25It does.
Miles · 34:26But wait, looking closely at their product fact sheet, I see a massive contradiction here. DocuStrata highlights a core feature they call answer memory. When the system produces a well-grounded, perfectly cited answer to a complex question, it takes that newly generated answer and files it back into the archive as a first-class, highly searchable document. So if I ask a complex question in March, and the system works hard to synthesize the answer, I can instantly retrieve that exact answer in September as if it were a raw source document.
Nora · 34:56This raises an important question, and it is arguably the central, unresolved design challenge of the entire AI era. You have identified the exact tension tearing these systems apart.
Miles · 35:07It's totally contradictory.
Nora · 35:08DocuStrata is attempting to walk an impossibly thin tightrope. On one hand, their core processing loop is strict query time synthesis. They retrieve immutable original documents and answer fresh, providing rigorous citations to prevent hallucinations.
Miles · 35:22But on the other hand, this answer memory feature is pure, uncut, right-time synthesis. It saves an AI-authored artifact back into the corpus for future retrieval. So they are literally letting the AI write the notes and put them right back in the filing cabinet. If that answer memory document gets scooped up by the vector index during a future query, we are right back to playing the telephone game.
Nora · 35:45We are.
Miles · 35:46We are laundering the AI's summary into a foundational truth. Aren't they building a time bomb into the core of their product?
Nora · 35:53They are definitely playing with fire. DocuStrata attempts to mitigate the slow-acting poison by engineering strict metadata rules. rules. They try to ensure that the Answer Memory document retains its rigid citation links back to the raw originals, even as it sits in the archive.
Miles · 36:07They are attempting to keep the chain of custody intact, even as they permanently save the synthesis.
Nora · 36:12But the tension absolutely remains. Is Answer Memory a compounding cognitive asset that makes the system vastly faster and smarter over time, much like Karpathy's self-compiling wiki? Or is it exactly the slow-acting poison that Lahoti warns about quietly corrupting the
Miles · 36:30knowledge base over a period of years? It is the ultimate battle between user convenience and epistemic purity. Users hate waiting 30 seconds for an AI to read 40 raw contracts every time they ask a question. They want instant answers, which answer memory provides.
Nora · 36:46But right now, we simply do not know which philosophy will win out at massive scale.
Miles · 36:50And it all loops back to that foundational assumption we started with. If the machine is the primary reader of your life's data, who is ultimately responsible for the truth, if DocuStrata's answer memory slightly misinterprets a nuance in a legal document and saves that misinterpretation.
Nora · 37:06And then you rely on that answer memory to make a million-dollar business decision
Miles · 37:09a year later. Exactly. The model's subtle mistake has just become your concrete reality.
Nora · 37:14So what does this all mean? mean. We started with Sam Altman's grand vision of a trillion token context window, a utopian state where no retrieval is necessary because the machine holds your entire life in its active cognition.
Miles · 37:28We explored how Andrej Karpathy is hacking that reality at a micro scale today, letting an A.I. compiler autonomously build a frictionless 400,000 word wiki.
Nora · 37:38We examine Anand Lahoti's stark warning about the poison of right time synthesis, where A.I. authored notes degrade the truth.
Miles · 37:45And we analyzed DocuStrata, a system trying to build a bridge to the future using complex vector indexes and RAG to manage massive archives while wrestling with the contradiction of answer memory.
Nora · 37:56I think the most honest, open question we can leave with is, this is retrieval. Is RAG and vector indexing just a temporary, clunky bridge?
Miles · 38:03Are we simply suffering through chunking documents and building 3D concept maps until the physical hardware catches up and finally gives us Altman's trillion token context window where everything is just known instantly?
Nora · 38:15Or is RAG actually a fundamentally different and perhaps safer evolutionary path? Because RAG, by its very nature, forces the engineering of citations. It forces the system to point back to immutable original documents.
Miles · 38:29In a world where generative models can confabulate so convincingly, the ability to independently verify the source material might not be a temporary technological crutch. It might be the only way we maintain a secure, tether-to-ground truth.
Nora · 38:43If the context window truly gets big enough, developers will inevitably start building citation mechanics entirely because the model will just know the answer intrinsically, much like a human does.
Verifiability as the thing worth protecting
Miles · 38:54And that loss of verifiability might be the most dangerous outcome of all. We are rushing toward a future where we don't read anymore, we just ask. And we have to be extraordinarily careful about the unseen mechanics of how the machine formulates its answers.
Nora · 39:07Absolutely.
Miles · 39:07I'm going to leave you with one final provocative thought to mull over, building on everything we've unpacked today. Imagine we actually reach that endpoint. We get to a place 10 years from now where every single person has a trillion token life AI. The dream realized. Your personal agent has read every contract you've ever signed, every email you've ever sent, every frantic text you've ever written, every receipt you've ever scanned. It holds a perfect, flawless, mathematically mapped representation of your lived reality. And I have mine. Exactly. Exactly. What happens when two people disagree? Let's say we are in a bitter business dispute or a legal negotiation, and my trillion token life AI is deployed to argue against your trillion token life AI. Oh, wow.
How is the dispute resolved? If my AI has synthesized its impenetrable reality based purely on my documents, my answer memories, and yours has done the exact same thing with yours, Does human truth just become a matter of which machine possesses the larger context window?
Nora · 40:08Does the definition of reality just default to whichever AI has the better retention and the more aggressive, persuasive synthesis engine?
Miles · 40:15When we eagerly outsource our memory, our file organization, and our foundational reading to machines, we might wake up to find that we've inadvertently outsourced our ability to agree on what actually happened in the first place.
Nora · 40:26It's certainly something to think about the next time you hit summarize on a long, complicated document and accept the output without double-checking the source.
Miles · 40:34Thank you for joining us on this deep dive. Stay curious and remember to always think critically about who or what is doing your reading for you. See you next time.