The standard approach is to examine demand and supply separately. On the demand side: is there real economic value being created, or are people buying assets purely because they expect prices to rise? On the supply side: is capacity being financed in ways that can survive if assumptions change, or is the capital structure fragile?

Bubbles can form from either side. Tulip mania and Crypto 2021 were demand-side bubbles—speculative buying untethered from any use value. The 2008 housing crisis was partly a supply-side phenomenon—real demand for housing existed, but the financing structures couldn’t survive a modest price decline. The dotcom bust had elements of both: real demand for internet connectivity, but overbuilt capacity financed on revenue projections that never materialized.

For AI infrastructure, the demand side is the easy part. The evidence for real economic value is substantial and growing. The interesting question is on the supply side: whether the capital structure financing the buildout can survive the assumptions embedded in it.

Demand: Where the Evidence Points

The case for genuine demand rests on observable business impact, not projected future value.

What these numbers represent is companies disclosing AI-driven financial impact in earnings calls—statements made under securities law, not marketing materials. The tripling of companies reporting AI benefits since early 2023 reflects actual deployment reaching the income statement. When Walmart says GenAI replaced work that would have required 100x the headcount, or Estée Lauder reports 31% ROI improvement on media campaigns, these are claims with consequences for executives making them. The investor relations teams and other internal teams vet and validate what executives say publicly.

Consumer adoption data reinforces this. Over half of US adults have used GenAI, with workplace usage nearly doubling within a year. At 900+ million weekly active users, ChatGPT alone has reached a scale where “will people use this?” is no longer a relevant question.

One way to contextualize where AI sits developmentally: think of it like human capital formation. Humans absorb roughly $350,000 in rearing and education costs over 20 years, then produce approximately $2.8 million in lifetime earnings over a 40+ year career. AI spent 2022-2025 in its equivalent “education phase”—massive investment in training foundation models. We’re now entering the “working phase” where deployment generates returns. The important difference: humans retire and their knowledge doesn’t transfer perfectly. AI systems can operate indefinitely and improve continuously.

This framing helps explain why the traditional bubble indicators aren’t flashing red. Economists look at five macro signals that preceded the dot-com crash: extreme valuations, declining corporate profits, elevated credit risk, high cash burn, and rising leverage. These tend to move in sequence—valuations run hot first, then fundamentals deteriorate, then credit markets crack.

The current picture shows valuations elevated but not extreme (95th percentile vs. the 99th percentile at the March 2000 peak), with the other four indicators in healthy territory. This is what you’d expect if demand is real rather than speculative: companies can support elevated valuations because underlying profits are strong, not because everyone is buying in anticipation of selling to a greater fool.

The demand side of the AI market doesn’t look like a bubble. It looks like a genuine technological transition with measurable economic impact.

Supply: What’s Being Built

Meeting this demand requires building across the entire AI value chain—from semiconductor fabrication through data center infrastructure to the software layers that deliver AI to end users.

The chain spans multiple industries with different economics. Semiconductor fabrication is concentrated (TSMC, Samsung) and capital-intensive. Data center buildout requires coordinating power systems, servers, networking, and physical facilities. Cloud providers and operators sit in between, purchasing compute capacity and selling it to software companies and enterprises. Each layer depends on the others. Constraints anywhere ripple through the system.

The buildout currently faces real bottlenecks. In a survey of 149 senior data center professionals, 92% cited utility capacity as a barrier, with 44% facing 4+ year wait times for grid connections. Permitting issues affect 86%, fiber availability 85%, chip access 81%.

The industry response has been to route around constraints rather than wait for them to clear. 62% now plan on-site power generation as primary contingency—a shift from treating utilities as default to treating them as backup. When Elon Musk built Colossus 1, he brought in 35 semi-trailer-sized gas turbines. For Colossus 2, he bought a power plant overseas, disassembled it, and shipped it to the US for reassembly. These reflect how tight the energy market has become.

These constraints explain why shortage pricing exists today. The question for financing is whether they persist long enough to support the debt structures being built around them.

Supply: The Financing Structure

The AI buildout requires approximately $2.9 trillion in data center capital expenditure through 2028. The hyperscalers are spending aggressively and so is OpenAI. CEO Sam Altman references $1.4 trillion in compute commitments over eight years. Dario Amodei describes ambitions for hundred-billion-dollar training clusters. Google’s infrastructure leadership talks about doubling compute every six months.

Hyperscalers’ operating cash flows can fund roughly half of this. The remainder—about $1.5 trillion—requires external capital. This isn’t unusual. Large infrastructure buildouts routinely rely on outside financing. What matters is whether that financing is structured appropriately for the assets it’s funding.

The capital stack breaks down as follows: $200 billion from corporate debt issuance (companies borrowing against their own balance sheets), $150 billion from securitized assets (loans packaged and sold to investors), $800 billion from private bilateral credit (direct loans from non-bank lenders), and $350 billion from other sources including private equity.

The $150 billion in securitized assets and $800 billion in private credit deserve particular attention because of their scale and characteristics. Private credit consists of loans extended by asset managers, pension funds, and specialty lenders rather than traditional banks. Private credit has grown dramatically over the past decade, partly because post-2008 banking regulations pushed riskier lending outside the regulated system. The loans are typically held to maturity rather than traded, which means problems don’t surface until borrowers actually struggle to pay. Securitized assets carry a different risk: loans packaged and sold to dispersed investors who may not fully understand the underlying collateral dynamics—a structure that should sound familiar from 2008.

The nearly $1 trillion in external debt expected to finance data centers through 2028 carries a specific assumption: that the underlying assets—primarily GPUs and related infrastructure—will retain enough value over a 5-6 year loan term to support the debt. This assumption is standard for infrastructure lending, where assets like buildings, fiber optic cables, or aircraft depreciate slowly and predictably over decades.

GPUs don’t work that way.

The Depreciation Problem

When lenders finance infrastructure, they model whether cash flows provide enough buffer to service the debt. In a worst-case default scenario, they assess how much they can recover by liquidating the assets. Ideally, loan-to-asset value stays below 100% so lenders can recoup their principal.

For most infrastructure, this is straightforward. Commercial buildings depreciate over 30-40 years. Fiber optic cable lasts 25-30 years and, because improvements happen in endpoint equipment rather than the cable itself, the same physical glass can carry more data over time as technology advances. Aircraft operate for 20-25 years. These timelines give lenders comfortable cushion against typical 5-year loan terms.

GPUs follow a different pattern. NVIDIA releases a new architecture roughly every 12-18 months, and each generation delivers 2-4x better performance per watt than its predecessor. Unlike fiber, where new technology makes existing infrastructure more valuable, new GPU technology makes existing chips obsolete. Operators must replace the GPUs themselves.

The practical result: GPUs have competitive lives of 2-4 years before they’re outperformed enough to lose pricing power. An H200 purchased today will compete against B200 chips delivering twice the performance. Within 2-3 years, it will face Rubin delivering nearly 9x the compute—an efficiency gap that makes older hardware economically uncompetitive.

This matters for financing because the loans assume 5-year asset lives while the assets themselves become economically obsolete in 2-4 years. The only way the math works is if shortage pricing persists—if demand so exceeds supply that even older GPUs command premium prices throughout the loan term.

How the Financing Breaks

To see where the stress point lies, consider the economics of a GPU-based data center at different pricing scenarios.

The table models a single-GPU data center using a financing structure similar to CoreWeave’s: 85% debt, 10% interest rate, 5-year amortization. Three price points matter:

• Market price is what operators currently charge customers for compute, set by supply and demand in today’s constrained environment.

• Breakeven price is the minimum rate needed to cover debt service plus operating expenses (primarily electricity). Below this, the operator loses money and defaults on debt.

• Equilibrium price is what pricing would look like if supply caught up with demand and GPUs competed purely on performance per dollar. At equilibrium, older chips must price at parity with newer chips on a $/TFLOPS basis, which means steep discounts to their original cost.

At today’s market prices, all GPU generations show healthy margins. This shortage premium is what makes the leveraged financing structure work.

At equilibrium pricing, the picture fractures. H200s and newer chips still cover their costs. But A100s fall deep into negative margins—well below breakeven. Operators holding A100 debt would be unable to service their loans from operating income.

The key question is how long shortage pricing persists. Risky loans in the private credit and securitized asset stack is a bet that equilibrium arrives after the debt matures. If equilibrium arrives earlier, operators face a choice between default and refinancing at unfavorable terms—and refinancing requires finding new lenders willing to take the same bet on even older equipment or invest more money to purchase the latest GPUs.

NVIDIA’s excellence is, paradoxically, the problem. The company’s relentless execution—B200 ramping now, Rubin shipping in late 2026, Rubin Ultra in 2027—means product cycles have compressed to roughly 18 months for a new generation. Each generation delivers dramatic efficiency improvements. NVIDIA’s engineering prowess might be driving the obsolescence cycle faster than lenders modeled.

The hyperscalers themselves are starting to acknowledge this. In January 2025, Amazon reversed course—shortening server depreciation from six years back to five. The company’s 10-K was explicit about why:

This is notable because it goes against the trend of other hyperscalers: Meta, Alphabet, and Microsoft have extended depreciation schedules from 3 years to 5 to 6 years since 2020, Amazon looked at the same data and moved the opposite direction.

Two Forces Accelerating Supply Equilibrium

Several developments are accelerating the path toward compute equilibrium, each working through a different mechanism but producing the same effect: more compute capacity available per dollar, which erodes shortage pricing.

Algorithmic efficiency is compounding faster than most models assumed. DeepSeek provides the clearest data points. Their V3 model, released December 2024, trained with 2.6 million GPU hours compared to 30.8 million for Meta’s Llama 3 405B—roughly 12x more efficient for comparable benchmark performance. The headline “$5.5 million training cost” is debatable, but the efficiency differential was real and reflected genuine architectural innovations.

By December 2025, their V3.2 release achieved results that benchmark against GPT-5: gold medal performance at the International Mathematical Olympiad, top-10 at the International Olympiad in Informatics, second place at ICPC World Finals (programming). Cost to run the Artificial Analysis benchmark is $54 vs Grok 4’s $1,900—35x cheaper for better results. On January 2, 2026, DeepSeek published a paper introducing “Manifold-Constrained Hyper-Connections,” a training architecture that addresses scaling instability—one of the key obstacles to training ever-larger models efficiently. Expect this technique to appear in their next major release.

The implication for infrastructure financing: efficiency gains reduce compute demand per unit of AI capability. If capability-per-dollar improves 10x annually, the compute needed to serve a given market shrinks proportionally. That’s positive for AI adoption and negative for GPU pricing power. Shortage premiums compress not because demand falls, but because less supply is needed to meet it.

Energy constraints may ease faster than expected. Power availability is the most commonly cited bottleneck for data center expansion, but the nuclear pipeline is materializing on an aggressive timeline. In the past 18 months, hyperscalers signed agreements for over 10GW of new nuclear capacity: Microsoft’s $16 billion deal to restart Three Mile Island by 2028, Google’s 500MW Kairos Power small modular reactor (SMR) fleet targeting 2030, Amazon’s $20+ billion Susquehanna campus conversion plus 5GW X-energy SMR pipeline. Regulatory timelines compressed after May 2025 presidential executive orders mandated 18-month maximum Nuclear Regulatory Commission (NRC) reviews for new reactors, down from historical 7-year processes.

If power becomes abundant sooner than expected, the primary constraint on supply expansion disappears. More data centers can be built and operated, which increases available compute and pressures pricing toward equilibrium. The power scarcity premium embedded in current rates erodes.

Both forces point the same direction: toward compute abundance arriving sooner than lenders modeled. Efficiency shrinks demand per unit of capability. Energy expansion grows potential supply. For AI adoption, these are positive developments. For financing structures betting on persistent scarcity, they compress the timeline.

Evidence of Stress in Specific Companies

Broad credit markets remain healthy—spreads are tight, capital is available, no systemic distress is visible. But within the specific segment of data center and GPU financing, stress signals have emerged.

CoreWeave’s stock has declined approximately 60% from peak levels. The company exemplifies leveraged GPU economics. Financials from their Q3 2025 report:

• + $997 million gross profit

• - $631 million depreciation

• - $310 million interest expense

• - $1,412 million quarterly debt repayment

• + $3,129 million in new debt issuance

I believe the stock price reflects investor reassessment of whether shortage pricing will persist long enough for the model to work.

Oracle’s situation is different but related. The company maintains investment-grade credit ratings, but its bonds due 2035 trade at yields closer to BB-rated (junk) than BBB-rated (investment grade) paper. CDS spreads have reached levels last seen during the 2009 financial crisis. The market is pricing specific risk around Oracle’s $248 billion in off-balance-sheet data center commitments—obligations that don’t appear on the balance sheet but represent real financial exposure.

To be clear: the hyperscalers will survive this. When Alphabet extended depreciation to six years in 2023, it boosted pre-tax income by $3.9 billion. When the writedowns eventually come, Alphabet absorbs an accounting hit—painful for the stock, but not existential. Alphabet has $96 billion in cash. The same applies to Microsoft, Meta, and Amazon. These companies have fortress balance sheets. They can afford to be wrong about depreciation schedules.

The risk sits one layer down: in the operators who financed GPU acquisitions with debt rather than cash flow. When CoreWeave faces the same depreciation math, the $310 million in quarterly interest payments don’t pause for accounting adjustments. The hyperscalers take writedowns. The leveraged operators can’t service their debt.

These aren’t signs of a broad bubble. Credit conditions overall remain accommodative. What they indicate is targeted concern about the gap between GPU economic lives and the financing structures built around them. The stress is appearing exactly where the depreciation thesis predicts—in leveraged operators holding older equipment against long-dated debt.

What This Is and Isn’t

The AI infrastructure situation doesn’t fit the pattern of a classic demand-side bubble. Demand is real, growing, and generating measurable economic returns. Traditional macro indicators show one warning sign (elevated valuations) against four healthy readings (profits, credit spreads, cash burn, leverage). The comparison to tulip mania or dot-com speculation mischaracterizes what’s happening.

The risk is narrower and more technical: financing structures built on the assumption that GPUs will retain economic value—the ability to generate cashflow—for 5-years, when competitive obsolescence arrives in 2-4 years. This works only if shortage pricing persists—and several forces are working to end shortage pricing sooner than the financing models anticipated.

If the risk materializes, the result wouldn’t look like a market-wide crash. It would look like credit impairments concentrated in GPU-backed lending, distress among leveraged data center operators, and margin compression for providers running older hardware. The stress would likely surface in credit markets first—in private loan defaults, securitized asset writedowns, and distressed refinancings—before rippling into equity markets as the hyperscalers eventually take the writedowns. This isn’t a prediction that stock prices won’t fall. It’s a view on sequencing: credit cracks first, equity follows. CoreWeave’s stock and Oracle’s bonds suggest markets are beginning to price this scenario.

The investors exposed to this risk are not the ones betting on AI. Its the ones betting on scarcity.

A Note on Michael Burry’s AI Thesis

—the investor who identified the subprime mortgage crisis before it unfolded and the person whom I’ve watched & read about a lot when I started my finance career—has been publishing detailed analysis of the AI buildout on his Substack, . Having now gone through his writing carefully, we’re seeing the same thing—where we differ are more about emphasis than fundamental disagreement.

Burry’s analysis rests on several pillars:

• Capital Cycle Theory. Burry presents historical data showing that stock market peaks tend to occur mid-buildout, often before capital expenditures peak. His chart of S&P 500 net capital investment divided by GDP shows this pattern across the dot-com boom, housing bubble, and shale revolution. The telecom bust wasn’t about demand being fake—internet demand was real. The problem was supply massively overshooting: by 2002, less than 5% of the fiber infrastructure built during the bubble was actually lit.

• Depreciation mismatch. Hyperscalers have extended depreciation schedules from 3 years (2020) to 5-6 years today, while NVIDIA’s product cycles have compressed to roughly one year. This inflates earnings and overstates asset values. Burry documents this from 10-K filings and estimates double-digit billions of earnings overstatement for hyperscalers over 2026-2028. When Alphabet extended to 6 years in 2023, it boosted pre-tax income by $3.9 billion.

• Economic obsolescence. Burry distinguishes physical utilization from economic value creation. NVIDIA’s CFO noted that A100s from six years ago are “fully utilized.” Burry’s response: “Just because a widget is used does not mean the widget is profitable to a degree that it is worth more than residual value.” The A100 uses 2-3x more power per FLOP than the H100. Blackwell is reportedly 25x more energy efficient than H100. An older chip can be running at full capacity and still be economically unviable.

• Demand relative to CAPEX. Burry cites Menlo Ventures data showing application-layer GenAI revenue at $37 billion this year versus $400 billion in chip spending. ChatGPT has 900 million weekly users, but only 5% pay. His concern isn’t that AI has no demand—it’s that supply is being built far in excess of what current revenue can support. As he puts it: “One can believe AI is transformational and also believe that some public and private AI stocks are very overvalued.”

• Credit risk. This is where our analyses converge most directly. In “Unicorns and Cockroaches,” Burry writes: “The big target for private credit now is AI data centers... there is a duration mismatch of catastrophic proportions between the asset and the loan.” In his Q&A piece, he adds: “There is a lot of leverage behind the data center buildout, and I believe bank and non-bank entities will be at risk if that buildout busts.”

Burry is clear about what he’s not saying: “I do not predict that Nvidia, Meta, Microsoft, Amazon, Alphabet are doomed. They all should survive. I believe some will survive like 2000 Microsoft did, some will survive like 2000 Cisco did.”

Where Burry and I align:

Where we differ in emphasis:

On demand, we weight the evidence differently. Burry emphasizes consumer app revenue and conversion rates. I weight enterprise deployment—the S&P 500 earnings disclosures are claims made under securities law, which I find harder to dismiss. But if he’s right that demand is weaker than I assess, that makes the credit thesis worse, not better. Less demand means faster arrival at equilibrium pricing, which means faster erosion of the shortage premium that makes the financing structures work.

One area Burry has developed that I haven’t fully explored: circular financing. He describes how tech giants invest in AI startups who then buy from them, and how “off balance sheet special-purpose vehicles” and “badly structured but easily obtained financing” are goosing GPU demand. To the extent this is true, it amplifies credit risk—loans underwritten against demand that’s partially self-referential are even more exposed when capital stops flowing.

One final thought. Burry made his name by poring through individual mortgage loan files—finding the misaligned incentives and mispriced risk that aggregate data obscured. He saw what the rating agencies and institutional investors missed because he did the granular work.

The same opportunity exists here. The nearly $1 trillion in private credit and securitized assets projected to finance GPU infrastructure contains specific assumptions about depreciation schedules, collateral valuations, residual values, and covenant structures. Someone with access to those documents—the actual loan files, not the marketing materials—could identify which pools are most exposed and how the risk is distributed.

Burry notes he plans to address financing, circular deals, and “a curious liability hiding in the financial statements” in his Part 3. I’ll be reading carefully.

That work remains to be done. I’m curious what he finds.

This update reflects how quickly the space has moved in practice. Agents are no longer a handful of demos. They run in production at well-known companies and cover routine work in software development, customer support, finance operations, and other structured processes. Cursor, Lovable, and the likes are now well-known products seeing rapid adoption globally. ChatGPT now has agent mode and Codex while Claude Code is increasingly used for non-coding use cases. The best models are new, often only a few months old, and the frontier keeps shifting as context windows, tools, memory, and supervision improve. I also expanded the dataset to 19,000 tasks and 830 occupations—about 3,000 more tasks and 30 more occupations than last year—and new wage and employment data.

There is now a credible usage signal as well. Anthropic’s Economic Index aggregates 1 million conversations and maps it across the 22 major occupation groups. That lens matters because it tells us where teams already route tasks through models on the job, not just where models test well. Capability and usage together make the map more useful: one axis for agentic automation potential, the other for observed adoption patterns.

The picture that emerges is straightforward. Computer and Mathematics land in the high-potential, high-usage corner. Teams already push real tasks through agents there, which is visible in day-to-day engineering productivity and support operations. Management, Business Operations, and Office or Administrative work sit in the high-potential, lower-usage corner. The wage pools are large and adoption is still early. That is where new products and companies should grow1.

Because usage data is grouped at the major-occupation level, practical targeting needs more detail. The detailed occupation-level view pairs potential with wage pools across roughly 830 roles and shows where a product can attach to outcomes and budget. Most roles sit in smaller pools. A few dominate and match what we see in the market: software developers; customer service, accounting and audit, information systems management, project management, business operations. These are the places where agents translate cleanly into tickets closed, tests added, reconciliations completed, approvals moved, and minutes saved—and where pricing can anchor to time or output rather than seats. The fastest-growing agent companies lined up with this list, which suggests the framework has been a useful guide rather than a retrospective label.

There is also new & convincing evidence on how AI, and consequently agents, is affecting the job market. Using high-frequency payroll records through mid-2025, researchers examined employment by age and by exposure to generative AI. Since late 2022, employment for workers aged 22–25 declined in the most exposed, automating occupations such as software development, while older cohorts in the same jobs continued to grow. Customer service shows a similar split. Occupations where usage is primarily augmenting do not show the same decline. This does not settle every long-run question, but it is a timely signal that agents are already taking a share of entry-level tasks in the proven corner.

Last week, I wrote about a high-level view of how to invest in AI across public and private markets, and across active and passive approaches.

This can be the roadmap for active private investing. Use the same discipline, now with sharper signals. Start with wage pools by role and overlay observed usage as frontier models turn over faster and policy controls mature. Judge products by whether they finish work inside systems of record and expose measurable outcomes. The practical takeaway is the same as last year, now with better evidence: act where potential and usage already overlap, and build into the gaps where potential is high and usage is beginning to rise.

Curated reads:

• Anthropic Economic Index

• Six Facts about the Recent Employment Effects of Artificial Intelligence

Subscribe now

This guide focuses on equities: public stocks and private company shares. We are not covering credit, convertibles, or land for data centers. Think of what follows as a menu you can choose from based on your appetite for selection and liquidity.

There are two dimensions to the menu. Public vs. private tells you where a company trades and how liquid it is. Active vs. passive tells you how much selection you want to make. Private deals can be compelling but are illiquid. Public markets are accessible and easy to trade. Within public markets, passive means tracking an index at low cost. Active means leaning into a theme or specific companies you believe can beat the market. That said, I won’t recommend specific companies in this article.

Public Markets Overview

From the beginning of 2023 to the end of August 2025, the S&P 500 rose 69%, roughly 22% annualized over that period. This is several times higher than the long-run average gain of 10% annually. The Magnificent 7 (MAG7) — Google, Microsoft, Amazon, Meta, Tesla, Apple, NVIDIA — contributed 40% of the gain. While the economy also grew and favorable business policies were enacted during the same period, most of the gain was driven by the business of generative artificial intelligence.

A more precise view of how AI is affecting the public markets can be gleaned through an index of companies for whom AI is a core strategy. I have been maintaining a list of 100 stocks, and collectively it has increased 157% versus 69% for the S&P 500 and 107% for the Nasdaq. Looking at the chart below, the lines broadly follow the same patterns since most of the gains are driven by the same several stocks like the MAG7.

Passive strategies for the public markets

There are two paths to passive public investing: AI-themed ETFs or the broad market.

Most AI-themed ETFs bundle the obvious AI infrastructure companies — chips, cloud, and model platforms. They will rise faster when AI excitement is high but also fall harder when it cools. They usually charge slightly higher fees, although still negligible, and often lean heavily on a few big stocks. If you pick one, favor larger funds with at least about $1B in assets so it’s easier to buy and sell. While almost any tech-oriented ETF is now AI-driven, there are more AI-focused ones like:

Broad index ETFs like the S&P 500 or Nasdaq-100 are also good choices. They’re simple, diversified, and have plenty of AI exposure given the MAG7 are driven by the AI narrative. The broad indexes also expose you to the rest of the economy that will benefit from AI. AI agents can draft replies, pull records, route approvals, and close routine tickets, while robots take on physical work in warehouses, roads, and factories. Morgan Stanley analyzed every S&P 500 company by mapping employee roles against tasks AI agents and robots could take on. Their estimate: AI could add about $920 billion to S&P 500 pre-tax profit, or roughly 28% of projected 2026 earnings. While this isn’t a forecast of full adoption by that date, it is a measure of potential impact. The key point is that AI’s gains extend well beyond a few big tech names, and a broad index lets you capture that diffusion at low cost and with less single-stock risk.

Active strategies for the public markets

I will neither discuss specific companies here nor how to evaluate a business. There are already plenty of resources for that. Instead, I will go through three AI themes I am investing in. I like thematic investing because it helps narrow the consideration to a handful of companies instead of hundreds. You pick a wave, learn how the business works across the value chain, and build a small basket of companies you like. The three themes are: smart glasses, autonomous vehicles, and agentic software.

1. Smart glasses (AI glasses)

   The interface is shifting from tap-and-type to look-and-ask. Smart glasses are becoming mainstream, with millions of Meta Ray-Bans sold. They are appearing in major retail stores priced like smartwatches. That matters because price bands and shelf space change the demand curve. With AI, glasses have new practical capabilities: hands-free capture, quick messaging, navigation prompts, translation, and “show what I’m seeing” without fishing for a phone. The new Meta Display glasses, with a built-in display, add another layer of practical capabilities: navigational guidance, visual feedback, and more.

   

   How to build exposure:

   1. Devices & platforms: broad consumer pull and software upside. Examples: Meta, Google (soon), Samsung,

   2. Retail and eyewear design partners: for consumer brand awareness and distribution. Examples: Essilor Luxottica, Warby Parker.

   3. System silicon & vision: shared components that power multiple brands. Examples: Qualcomm, Ambarella, ON Semiconductor.

   4. Optics & materials: inputs that scale with units. Example: Corning.

2. Autonomous vehicles

   Autonomous vehicles, specifically Waymo, are rapidly expanding to major cities. In California, monthly driverless trips climbed from ~12k (Aug-2023) to ~708k (Mar-2025) as service zones expanded. In San Francisco, robotaxis overtook Lyft in gross bookings by January 2025 and reached >25% ride-hail share by April 2025, even while pricing carried a ~41% per-mile premium — a sign of demand outpacing early supply. In a forward-looking move, Lyft announced a multi-year partnership with Waymo in September, which includes Lyft investing to construct a purpose-built AV fleet management facility. Live commercial service now spans six U.S. cities, with several more in active testing in the U.S. and Tokyo.

   

   How to build exposure

   1. Autonomy platforms & fleet operators: the software and operations layer that turns vehicles into a service. Examples: Alphabet (Waymo), GM exposure via Cruise, Tesla (if Robotaxi launches)

   2. Network & demand side: aggregators that funnel riders and freight to autonomous supply. Examples: Uber, Lyft.

   3. Compute & perception: components every platform needs. Examples: Nvidia, Mobileye, Qualcomm.

3. Application software (agents inside everyday workflows)

   Agents are moving from assist to automate inside office suites, CRM, service desks, design tools, finance, and HR. They draft, retrieve, route, approve, and increasingly finish well-scoped tasks. Distribution is already installed — millions of paid seats — so adoption can scale without changing buying behavior. The economics tie to time saved and cycle-time reduction, which is why vendors lean on clean add-ons per seat and usage meters per task or minute. AI agents are now handling tasks that take longer and are growing in complexity, with the average time per task roughly doubling every seven months — a practical signal that their scope is widening quickly.

   

   How to build exposure

   1. Horizontal suites with named agent SKUs: large installed bases and deep workflow hooks. Examples: Microsoft 365 and Teams, Google Workspace, Atlassian, ServiceNow

   2. Backoffice (IT, finance, and HR) where outcomes are measured: agents tied to reconciliations, tickets, approvals, payroll, filings. Examples: Intuit, Workday, Oracle Fusion Apps.

Private markets overview

Private market exposure reaches earlier in a company’s life with less information and less liquidity. Most vehicles require accredited investor status under SEC rules. Accredited investor status exists to protect less sophisticated investors from high-risk, illiquid investments. Investment platforms will verify if you are accredited, though occasionally some rely on self-declaration. You qualify if you meet at least one of the following:

• Income: $200,000 individual or $300,000 joint in each of the two most recent years

• Net worth: over $1 million, excluding a primary residence

• Professional certifications: Series 7, 65, or 82 in good standing

• Company role: director, executive officer, or general partner of the issuer

About 20% of U.S. households meet accredited thresholds. There is also a “higher tier” of Qualified Purchaser status that opens more investment opportunities but applies to just 2% of households.

Private markets matter because more value formation now happens before listing. The number of publicly traded companies in the U.S. has fallen from approximately 7,000 in 2000 to approximately 4,000 in 2024, shrinking the investable universe for most investors. At the same time, private companies are growing, both by number and by market value. U.S. private companies now outnumber public companies by over 6.5 to 1. The aggregate estimated value of private companies in the U.S. surpassed $10 trillion in the first quarter of 2025. The number of so-called “unicorns” (private companies valued at $1 billion or more) in North America increased from 20 in 2016 to over 1,000 in 2024. At the same time, the median time from a private company’s initial financing to its IPO has increased, on average, from six years to eleven years. Some high-quality scaled businesses are delaying public listing indefinitely, even after reaching profitability (e.g., Stripe).

Passive strategies in private markets

The choices here are limited, primarily private funds that accept small checks like AngelList’s rolling funds, publicly listed private equity firms, and publicly listed venture funds. However, only a few are focused on AI. The most notable ones are:

Active strategies in private markets

Company selection matters more here than in public markets. Publicly listed companies are already the winners and leaders in their industries, so they tend to ride thematic waves. In private markets you are judging whether a specific company will become one of those leaders. Uncertainty is greatest in the earliest rounds (seed to Series A). By late stage and pre-IPO (Series D onwards) there is more evidence in revenue history, customer adoption, and margins. Unless you have unique information and access, put most of your attention on late stage. In this section, I will go through mechanics more because it is more complicated than investing in publicly listed stocks.

Two axes shape how you can participate. Direct vs. indirect describes the path to ownership. Direct means the company issues or transfers shares to you and your name appears on the cap table. That is common for employees, advisors, and investors. Indirect means you invest through a vehicle that holds the company’s shares. Single-deal SPVs are the usual structures, and your rights are those of the vehicle (not the shares directly). Primary vs. secondary describes what is being sold. In a primary, the company creates new shares and receives the cash. In a secondary, an existing holder sells to a new buyer.

For most individual investors the practical route is indirect secondaries. Primaries via vehicles do appear around marquee rounds, but secondary blocks are more common. For example, I saw SPVs for recent primary raises for marquee companies Anthropic, Replit, and Databricks. But I’ve always had opportunities to buy secondaries. Unfortunately, investing through SPVs often involves a 2% management fee and a 20% carry (a share of profits taken by the SPV owner). Some platforms instead charge an upfront 5% brokerage fee, like EquityZen.

Investing direct makes things more complicated because shareholder rights matter more. You also have more to consider: RSUs, stock options, preferred shares, and other terms. Unless you are a professional investor, you don’t want to deal with all the legalities. For example, I directly invested in a startup through a SAFE that then refused to issue the shares after they raised more money. I had to resort to months of tense conversations and eventually legal threats before getting my money back. If I had invested indirectly through a platform or SPV, they would have handled that.

There are various options to access indirect secondaries and primaries:

• AngelList is the original retail-accessible route with a large network of investment syndicates and emerging managers. It is useful for single-deal vehicles in primaries and some secondaries. The strength is breadth of leads and deal flow. The practical challenge is selection because syndicates vary in diligence, communication, and cost.

• EquityZen focuses on late‑stage secondaries with standardized vehicles and a flat, transparent fee schedule. The company list is curated and workflows are predictable, which makes it a strong default for late‑stage exposure.

• Hiive is a live marketplace for direct secondaries with visible bids and asks. It offers price transparency, but approvals can fail and indicative prices can shift during approvals.

• Forge arranges negotiated secondaries at larger scale. Its strengths are sourcing, issuer relationships, and the ability to aggregate blocks into a closing. Tradeoffs include per‑side fees, more process, and higher minimums.

While late stage shifts the risk away from company specifics (i.e., idiosyncratic risks), I still advise doing company-specific analysis. Use measures you can infer from available information, and Sacra is often a great resource for this. An example analysis would be the ones I published for Databricks and Scale AI before I invested through SPVs. It gave me conviction to invest, and so far it has paid off with ~200% gains in one to two years.

Practical takeaways

• Decide your lane first: Public vs. Private for liquidity, Passive vs. Active for how much selection work you want to do.

• Default option for most people: broad, low-fee index ETFs (S&P 500 or Nasdaq-100). You still get meaningful AI exposure with diversification.

• If you want an AI tilt without picking stocks: use AI-themed ETFs, but favor larger funds ($1B+ AUM), check fees and top holdings.

• If you want to be active in public markets: pick one theme (e.g., smart glasses, autonomy, agentic software), learn the value chain, and build a small basket. Write a half-page thesis before buying anything.

• Private exposure is optional and only for those who qualify and accept illiquidity. Start with late-stage secondaries via reputable platforms rather than early seed bets. Write a one-page thesis before buying anything.

• Know the mechanics in private deals: most retail paths are indirect vehicles. Deals can be delayed or cancelled.

• Check what kind of equity the SPV is buying: Avoid employee options and multi-layer SPVs with egregious fees

Curated reads:

• Meta Connect 2025: The evolution of AI glasses

• Lyft and Waymo launch partnership to expand autonomous vehicles

• How are people using ChatGPT? A study of 700 million weekly active users

I also incorrectly predicted the trend might not endure as scrutiny mounted. Well, the trend has not only continued but evolved with more complex hybrids. Two fresh examples: Meta's $14.3 billion investment in Scale AI and Google's $2.4 billion licensing deal with Windsurf.

In this update, I'll cover recent developments with an updated deals table, how they align with last year's patterns, key differences and insights, per-head fee calculations ($100M/head), and a measured take on employee outcomes. Here's an updated version of the table from my 2024 piece, incorporating the completed quasi-acquisitions.

Subscribe now

Details of what happened: How Windsurf & Scale AI deals follow and differ from previous patterns

These new transactions echo the quasi-acquisition playbook I outlined last year: Big Tech pays premium fees for talent and tech access without full ownership, bypassing heavy regulatory reviews while delivering returns to investors. Here's how they align:

• Avoiding Standard Acquisition Structures: Just like Microsoft-Inflection or Amazon-Adept, both deals sidestep Hart-Scott-Rodino (HSR) Act filings by not crossing "control" thresholds. Meta's 49% non-voting stake in Scale keeps it below 50%, classifying as an "equity investment" under ASC 323—mirroring how non-exclusive licenses in 2024 deals avoided asset-transfer triggers. Google's Windsurf agreement is pure licensing, with no equity, echoing Character.ai's structure.

• High Rewards and Competition Driving the Deals: As I noted, the AI arms race demands speed. Meta gains Scale's data-labeling tech for Llama models, much like Amazon accessed Adept's datasets. Google outbid OpenAI for Windsurf's technology, securing a competitive edge—intense rivalry I predicted would fuel more such moves.

• Investor Returns via License Fees: These ensure VCs get multiples without blocking deals. Scale investors (e.g., Accel) saw ~2x returns on the $14.3B investment, similar to Inflection's 1.1-1.5x. Windsurf's $2.4B fee provided ~1.6x on its ~$1.5B valuation, aligning with my point that fees compensate for lost independence.

• Access to Key Assets: Beyond talent, fees buy datasets and infra. Meta licenses Scale's vast labeled data, akin to Inflection's 22,000 GPUs.

While following the core patterns, these 2025 deals introduce twists that reveal evolving strategies—hybrid elements and more transparent (yet still limited) disclosures. This gives us deeper insights into Big Tech's adaptability.

• Hybrid Structures for Partial Control: Unlike 2024's pure licenses, Meta-Scale blends investment with integration. The $14.3B gets a stake that lets Meta influence without full ownership. This differs from Inflection's clean split, providing "effective control" via board seats (Wang joins Meta AI leadership while retaining Scale board control).

• Increased Scale and Bidding Wars: Fees are bigger: $14.3B for Scale dwarfs Inflection's $650M, reflecting AI's maturation. Unlike 2024's quieter negotiations, these had public rival bids, per reports—highlighting how competition inflates costs but accelerates consolidation.

How much does Big Tech value top AI talent? The $100M headline number

A key metric I touched on last year was cost justification via talent. Let's quantify with per-head calculations, using data from disclosures and reports The denominator is strictly the number that moved to the buyer. Amazon and Microsoft look “cheap” relative to Google and Meta. Since Meta’s acquisition included an equity stake, we attribute 14% of the $14.3B to the ~10 employees who joined Meta. 14% is to reflect the recent 14% layoff by Scale - though arguably 14% is conservative because many of Scale’s top customers are moving most of their business away.

What these deals mean for startup employees

On average, the cost per AI talent acquihired is $59 million. Unfortunately, most of that money goes to the investors. Employee experiences from acquired companies have been mixed—often providing financial and career upsides for those poached by Big Tech, but with disruptions like uncertainty or layoffs for others.

For poached talent: In Scale, ~10 top staffers transitioned to Meta with enhanced packages, including access to vast resources. One engineer posted on Reddit: "It's a step up—better pay, stability, and impact at scale." Windsurf's ~30 hires to Google received packages worth double their equity value, with immediate signing bonuses; many signed on-site during a secretive July 11 meeting. Similarly, Inflection's ~70 staff joined Microsoft with competitive comp, including equity equivalents and Adept's ~66 to Amazon got higher salaries (~$300K+ base for engineers).

For those not poached: Outcomes varied. Scale laid off 14% (~200 employees) post-deal, citing "restructuring for efficiency" after missing projections. Windsurf's remaining ~120 staff initially felt "abandoned" after Google's selective hire, with frustration over non-accelerated vesting —but Cognition's July 15 acquisition waived vesting cliffs and ensured financial participation for all, turning uncertainty into a "great outcome" per Windsurf CEO Jeff Wang.

Overall, these deals cushion transitions for many via Big Tech's resources, but remnants face flux—layoffs or pivots, though often mitigated by payouts. No widespread backlash.

Wrapping up: Takeaways and what is next

These deals confirm quasi-acquisitions' staying power, with twists like hybrids reshaping the playbook. But regulators are watching closely. The FTC's January 17, 2025, compendium includes a "Joint Statement on Competition in Generative AI Foundation Models and AI Products," warning: "We will continue to scrutinize partnerships and investments that may enable dominant firms to exert undue influence or gain privileged access in ways that could undermine fair competition". They've probed similar deals, like seeking info on "serial acquisitions". We might see one or two more this year before the FTC or SEC adds restrictions—such as expanded HSR scrutiny on "quasi-mergers." For those who want a quick exit, the time is now to join a hot AI startup. But which one?

July 20, 2025 update: Corrected the number of Scale AI employees joining Meta to ~10.

This limitation is becoming apparent in AI systems as other aspects continue to progress. Since OpenAI’s o1 model release in September 2024, AI models have become remarkably good at reasoning through complex problems. They can analyze data, write code, and engage in sophisticated PhD-level discussions. But they have a fundamental limitation - they can't remember what they've learned from one conversation to the next without retraining . Like a brilliant person with no memory, they solve the same problems repeatedly without building on past experience.

As these reasoning capabilities become standard for the industry, what will set AI agents apart isn't their raw processing power, but their ability to maintain and learn from memory. This isn't just about storing information - it's about building systems that can understand context, remember past interactions, and apply those lessons to new situations. The memory layer will be a key differentiator in how AI agents understand and respond to us.

This article examines memory from multiple angles: how it shapes human cognition, how we're implementing it in AI systems, and how companies are building the technology to make AI agents that learn and remember.

Subscribe now

Memory in humans

Personal: Memory is central to who we are. Our autobiographical memory - the collection of personal life events - creates a narrative connecting our past to our present, giving us a coherent sense of self. Research shows an interesting two-way relationship between memory and self-image: what we currently believe influences which past experiences we recall, and those memories in turn shape how we see ourselves today. Memory also shapes who we are to other people. Through our shared experiences, people remember us as their mother or father, their best friend or their enemy, the funny one at work or the quiet neighbor next door. Their memories of us are just as important as our own memories in creating who we are.

Social: People with strong memory capabilities - who easily recall facts, names, or past conversations - are often seen as intelligent and knowledgeable. Remembering information signals that someone is well-educated and quick-thinking, and it often goes hand in hand with problem-solving abilities. These individuals might also come across as reliable and attentive: think of a colleague who never misses a meeting detail or a friend who remembers every birthday - it shows conscientiousness and dependability. On the flip side, frequently forgetting things can have subtle social costs. When someone often forgets important details or asks the same questions repeatedly, others might question their organization skills or mental sharpness.

Practical: Memory underlies many of our cognitive functions. Far from being just a storage system for past events, memory actively helps us think about the future and make decisions. Drawing on past experiences lets us simulate possible future scenarios and apply lessons learned when facing choices. In fact, memory is involved in everything from problem-solving and communication to social skills like empathy - suggesting it's woven into nearly every aspect of complex thought. Without memory, we'd struggle to weigh options or learn from mistakes.

Memory in AI Agents

User Experience

When we look at AI agents, there are clear parallels between human conversation patterns and AI user experience. For an interaction with an AI system to feel natural, it needs some form of memory. Just as we remember what was said earlier in a conversation, an AI needs to retain context from previous exchanges to respond coherently. Without memory, the AI treats every query in isolation, leading to disconnected exchanges.

That's why it was frustrating to repeatedly copy-paste past conversations into ChatGPT and Claude when I want to bring in context from previous discussions. It feels like talking to someone with no memory of our past interactions. While ChatGPT now has a long-term memory feature that makes this better, Claude still doesn't have this capability.

Here’s another example, a voice assistant interaction: if you ask "Who called?" and the assistant says "Eric H.", you'd expect that following up with "Call him back" would make it dial Eric. Without short-term memory of context, you might get an annoying "Whom should I call?" response, forcing you to repeat information. This is why design guidelines emphasize maintaining conversational memory - it creates a smooth user experience and lets you reference earlier information efficiently. Memory transforms an AI from a basic question-answering tool into a conversation partner that can use pronouns and implicit references meaningfully because it "knows" what they refer to.

Memory also enables meaningful personalization. When an AI agent remembers your preferences, history, or goals, it can tailor its responses specifically to you. A personal assistant that remembers your favorite restaurants or previous travel plans can make more relevant suggestions. A recommendation system that recalls your past ratings can better predict what you'll like. In technical terms, long-term memory lets agents store historical data and user profiles, which directly improves their ability to personalize interactions. Instead of one-size-fits-all responses, a memory-enabled agent adapts to each user, maintaining continuity across sessions (it "remembers" you from last time) and awareness that extends beyond single interactions. Studies of conversational agents in healthcare show that such personalization leads to higher user satisfaction and engagement. Users stick around longer in conversations that acknowledge their personal context and don't make them repeat themselves. The dialogue quality improves because the agent's responses are contextually appropriate and consistent over time.

Memory in AI doesn't just produce better task performance – it shapes how users emotionally relate to and trust the system. An AI agent capable of remembering and learning over time tends to feel more human and empathetic.

For example, if a virtual assistant consistently recalls a user's preferences (like their food allergies or schedule constraints), the user will trust its suggestions more and feel a greater sense of rapport, much as we trust friends who remember our likes and concerns. Continuity builds relationship: a chatbot that references a user's previous messages ("I remember you mentioned last week you were feeling anxious. Are things better now?") demonstrates care in a way, thereby encouraging the user to confide and engage further. This human-like quality increase users' trust in the AI.

In contrast, a lack of memory can make an agent seem robotic or inattentive, undermining trust. Users often grow frustrated or disengage when they have to repeat information or when the AI fails to "recall" context that it should know. It breaks the illusion of interacting with an intelligent partner and reminds the user that the agent is just a machine following scripts. Especially in applications like healthcare, counseling, or personal assistants, trust is paramount – and trust is reinforced when the AI shows it remembers past interactions accurately. By retaining history, AI agents can also avoid inconsistent or contradictory responses, further improving credibility. All these factors contribute to a more user-friendly experience. Users are more likely to continue using, relying on, and recommending an AI service that remembers them.

Technology: AI Agent Framework Refresher

At a conceptual level, an AI agent has six core components, modeled after the human mind. Here's a simple breakdown of each:

For a deeper technical exploration, see:

1. Perception: Converts raw data into formats the AI can process. Like human senses, it handles tasks like computer vision and natural language processing. Tools like Unstructured.io help convert formats like PDFs into data the AI can work with.

2. Working Memory: Think of this as the AI's mental workspace - temporary storage and active processing. Like how we juggle thoughts or calculations in our head, in AI this is similar to the context window in large language models, holding immediate inputs and retrieved information for processing.

3. Procedural Long-term Memory: This is where "how-to" knowledge lives. In AI, it includes the implicit knowledge built into the model during training, particularly post-training, like how to maintain response boundaries, follow instructions, reason through problems, and use tools.

4. Declarative Long-term Memory: Stores facts and events. In AI, this uses databases - think knowledge graphs for facts and vector databases for abstract data like image patterns.

5. Motor: Handles interactions with external systems. For AI, this means performing actions like sending emails or managing files, usually through APIs.

6. Orchestrator: Coordinates all the other components, managing:

   • Retrieval: Getting data from memory into working memory

   • Synthesis: Converting temporary data into permanent storage

   • Storage: Converting data into machine-processable formats

   • Planning: Organizing tasks and decisions to align with broader goals

Technology: Long Term Memory

While AI agents use different types of memory as shown above, let's focus on the challenge of declarative long-term memory - storing and retrieving information across conversations. When I first wrote about this topic 18 months ago, the working memory (context window) was quite limited - 4,000 to 8,000 tokens at most. The natural solution was external long-term memory using vector databases and improving how we orchestrated RAG. Since then, context windows have grown exponentially - Google's Gemini now handles up to 2 million tokens.

This expansion of working memory might seem to solve our problems - after all, we could theoretically fit 170 hours of conversation in Gemini's context window. But dumping everything into working memory fails for several reasons:

1. Just like humans can't perfectly recall hour-long conversations, raw context doesn't create useful memory

2. The quality of retrieval degrades with too much unfocused context

3. Processing large context windows remains computationally expensive

The core challenge in declarative long-term memory is providing the right context so the model can give the best answer for the user. This breaks down into three key problems:

• Synthesis: How do you summarize memories?

• Storage: How do you retain them efficiently?

• Retrieval: How do you recall the right information at the right time?

There are generally three approaches to building these systems:

1. Plain text

2. Vectorization

3. Synthesized chat logs in structured form (table or knowledge graph)

Each approach has trade-offs, and they can be combined for hybrid RAG. For simplicity, plain text isn't practical because searching through raw text is inefficient. Let's focus on vectors and structured data.

Vector Storage: Technically, this approach converts text into numerical vectors using embedding models. Each chunk of text becomes a long list of numbers (usually 1000+ dimensions) that represent its meaning. When you need to find relevant information, you calculate how similar these vectors are to each other - like measuring the distance between points in space.

It's like having a huge pile of clothes and using similarity to find what you need - you might search for "something red and soft" to find your favorite sweater. It's quick to set up but can get messy when you have lots of items. Vector search is great for finding semantic similarity but doesn't understand relationships between pieces of information.

Knowledge Graphs: Technically, this approach stores information as a network of nodes (entities) connected by edges (relationships). Each node and relationship can have properties that describe them. When you need information, you follow these connections to find what you're looking for.

It's like an organized closet where everything has its place - shirts with shirts, pants with pants, and you know which items go together. It takes more work to organize initially, but finding things is more systematic. Knowledge graphs excel at following explicit relationships but require more structure.

Hybrid Systems: The most sophisticated systems combine both approaches: use vector similarity to find relevant content quickly, then use graph relationships to understand context and connections. This gives you both semantic search and structured relationships, though it requires more engineering effort. For example, when searching for project information:

• Vector search finds semantically relevant documents

• Graph traversal shows how these connect to people, deadlines, and dependencies

• Together, you get both content and context

For a more technical comparison of vectors and knowledge graphs, see:

I lean toward knowledge graphs because they match how we think when we're being deliberate - in terms of categories, things, and relationships between them. This reflects what psychologists call System 2 thinking (slow, conscious reasoning). Though admittedly, most of our day runs on System 1 (fast, automatic responses). When I see my parents, I automatically greet and hug them. I don't consciously think "I see two people, these are my parents, parents deserve respect and love, therefore I should greet them." This aligns with industry practice that for 80% of cases, System 1-style processing (like vector search) works fine. For the 20% of cases needing more careful thinking, we need System 2-style processing (like knowledge graphs).

Recent research from Amazon and CMU offers valuable insights into how different memory approaches stack up. Their findings show that dumping everything into the context window (CoT LLM) performed much worse than any RAG system, despite Claude 3 Sonnet having a 200,000 token window. Graph-Only RAG was only slightly better than Text-Only RAG (vector RAG), showing about 5% improvement. Interestingly, agentic/self-reflective systems ReAct and Corrective RAG performed worse than expected. However, combining methods proved powerful - Text & Graph RAG and HYBGRAG (which adds self-reflection to Hybrid RAG) showed significant accuracy improvements.

For most applications, basic vector RAG works well enough. It's when organizations want to build exceptionally smart and personally tailored applications that they need structured data or knowledge graphs. While implementing knowledge graphs can be challenging, companies like Google and Glean have successfully deployed them to give them a competitive edge.

For a detailed exploration of knowledge graphs and how they fit into Google and Glean, see:

Another interesting study is done by Zep, one of the memory startups we'll discuss below. They compared their specialized memory system to the "dump everything in context" approach. Their system scored 18% higher while using only 1/10th of the processing time and 1/100th of the context tokens. For users, this means more relevant answers, faster. For developers, it means lower costs since they're not paying for extra input tokens.

Business

The memory layer is becoming a key focus among the most popular AI apps. OpenAI has been rolling out enhanced memory features that let ChatGPT store explicit memories and reference past conversations. This is a shift from their original approach where ChatGPT would start fresh with each conversation, though they did add basic memory capabilities in February 2024.

Similarly, Gemini incorporated memory features in November 2024. While OpenAI and Google are likely building their own memory systems (a technically challenging task), it's worth looking at startups offering memory as a service, particularly those implementing hybrid graph RAG. While any database could theoretically serve as memory, we'll focus on those using sophisticated hybrid approaches: Zep and Mem0. We'll also look at Letta, which treats working memory as a core feature in their agent framework, and Neo4j, the go-to graph database for AI products.

Other interesting companies and projects in the space but not profiled below: Cognee, WhyHow, Memary, Kuzu, Memgraph,

Zep (Zep 3K stars, Graphiti 2.1K stars)

• Background & Founding: Zep was founded in 2023 in San Francisco as part of Y Combinator’s Winter 2024 batch. It emerged to build a “memory layer” for LLM applications, allowing AI agents to retain and recall past interactions.

• Funding: The company has raised $3.3 million from Engineering Capital, Step Function, and founders & leaders at Vercel, Google, and several well-known AI companies.

• Founders: Zep was founded by Daniel Chalef (CEO) – an engineer who previously founded KnowledgeTree and held senior roles at SparkPost. Chalef’s background spans engineering, data science, and enterprise software, guiding Zep’s technical vision.

• Product & Technology: Zep offers a long-term memory service that developers can plug into their AI agents to store conversation history and retrieve relevant facts on the fly. Initially, Zep’s system extracted “facts” from chat logs and used a specialized RAG pipeline (semantic search + reranking) to surface relevant context. To overcome issues with purely semantic retrieval, Zep developed Graphiti – an open-source library for building temporal knowledge graphs from chat history. Graphiti dynamically captures entities and relationships from conversations (including changes over time), and supports hybrid search that combines vector similarity with graph traversal queries. This approach lets an AI agent recall context not just by semantic match, but also via connected knowledge (e.g. remembering that “X is Y’s manager” or how a user’s preferences changed). Zep’s memory API returns these relevant snippets or facts in milliseconds, without requiring the full chat history in each prompt.

• Differentiation: Zep stands out for its temporal knowledge graph approach. Instead of just using vector embeddings or key-value stores, Zep automatically maintains a structured graph that updates as conversations develop. This captures context shifts and relationships over time (like noting that "the user previously preferred X but now prefers Y") - nuances that simple vector search might miss. By combining graph-based memory with semantic search, Zep aims for more accurate and explainable context retrieval, especially for handling changing facts and historical context in ongoing conversations.

Mem0 (25k stars)

• Background & Founding: Mem0 was founded in 2023 and is based in San Francisco. The company is backed by Y Combinator and launched to tackle the stateless nature of LLMs by providing a persistent memory layer for AI applications.

• Funding: The company raised a seed round of about $500K in April 2024, led by Y Combinator.

• Founders: Taranjeet Singh (co-founder & CEO) and Deshraj Yadav (co-founder & CTO) lead the team. Singh’s background includes engineering and product stints at Khatabook (YC S18) and Paytm, as well as founding an AI tutoring app and co-creating the open-source EvalAI platform. Yadav was previously an AI Platform lead at Tesla Autopilot and is an AI/ML infrastructure expert who created EvalAI during his grad studies.

• Product & Technology: Mem0 provides an open-source memory layer that makes AI assistants stateful. It stores user interactions, preferences, and other context so that an AI agent can “remember” past sessions. Under the hood, Mem0 uses a hybrid datastore architecture combining three components: a vector store, a key–value store, and more recently a graph store to represent relationships between entities, . When an AI query comes in, Mem0’s engine automatically extracts important information from prior interactions and uses a blend of graph traversal, vector similarity search, and key-value lookups to fetch the most relevant memories. This ensures the LLM is injected with the right contextual info (e.g. past user queries, corrections, or personal facts) without the developer having to manually prime each prompt with all history. The design goal is to improve personalization and reduce repetition, while minimizing prompt size by externalizing memory storage. Mem0 is open source and has a cloud offering of managed mem0 with premium features. The open source project has garnered over 22,000 GitHub stars and 500,000+ downloads.

• Differentiation: Mem0 distinguishes itself through a unified hybrid-memory architecture and open-source approach. By natively combining graph, vector, and key-value stores, it handles both semantic and symbolic aspects of memory out-of-the-box. This differs from solutions that use only vectors or only graphs - Mem0 believes combining techniques leads to more personalized and cost-effective AI interactions.

Letta (15k stars)

• Background & Founding: Letta emerged from UC Berkeley’s AI research community in 2024. It was spun out of the Berkeley AI Research Lab (BAIR) and came out of stealth in September 2024. The founding team had been researching AI agent memory before forming Letta as a company.

• Funding: Letta raised a $10 million seed round in 2024, led by Felicis Ventures. The round also included participation from Sky9 Capital and Essence VC, as well as prominent angel investors such as Jeff Dean (Google DeepMind’s Chief Scientist) and the CEOs of Hugging Face, Runway, MotherDuck, dbt Labs, Anyscale, and Hex.

• Founders: Dr. Charles Packer and Dr. Sarah Wooders co-founded Letta. They met during their PhD research in the Sky Lab at UC Berkeley, where they worked under professors Ion Stoica and Joseph Gonzalez who now serve as advisors to Letta. Packer and Wooders are AI researchers who co-authored the MemGPT paper – which introduced the concept of self-editing memory for LLMs – and have deep expertise in machine learning systems.

• Product & Technology: Letta is building an end-to-end platform for AI agents with long-term memory. It has a popular open source project and is building a hosted service where developers can deploy stateful AI agents via an API. The platform is model-agnostic – developers can bring their own LLM while Letta handles the memory and agent logic. A key component is Letta’s Agent Development Environment (ADE), a web interface that lets developers design, debug, and monitor their agents’ reasoning steps and memory content. This ADE emphasizes white-box memory: developers can see exactly what information is stored and used at each step, and even edit the agent’s memory or prompts in real-time. Technically, Letta’s memory system builds on the MemGPT concept of self-editing memory: the LLM agent can write to and read from an external memory dynamically during a conversation. For example, after each user interaction, the agent can update its knowledge base (adding or modifying facts) which will persist into future sessions. By maintaining this stateful context, Letta-enabled agents aim to avoid resetting every conversation – they “remember” the user’s past instructions, preferences, or corrections.

• Differentiation: Letta stands out by focusing on agent-centric memory and developer tools, backed by academic research. While other memory layers act as infrastructure, Letta provides a complete agent platform - essentially a turnkey way to deploy AI agents that learn over time. This comprehensive approach means they handle issues like agent derailment, reliability, and prompt management alongside basic memory storage. Their focus is on making it easy for developers to create stateful AI agents, rather than just providing memory storage. This makes them complementary to lower-level storage solutions - they could potentially integrate with vector or graph databases from other providers.

Neo4j (14k stars)

• Background & Founding: Neo4j, Inc. is the oldest and most established company in this group, known as the pioneer of graph databases. It was founded in 2007 by Emil Eifrem (CEO), Johan Svensson, and Peter Neubauer. Neo4j’s early vision was to make data relationships a first-class citizen in databases.

• Funding: Over more than a decade, Neo4j has raised approximately $580 million across multiple venture rounds. Notably, it closed a $325M Series F in 2021 at a valuation above $2 billion, one of the largest investments in database history. Investors in Neo4j have included GV (Google Ventures), One Peak, Morgan Stanley Expansion Capital, Creandum, and others.

• Founders: Emil Eifrem, the CEO, is credited with sketched the first code of Neo4j and evangelizing the power of graph databases. Co-founder Johan Svensson has served in engineering leadership, and co-founder Peter Neubauer (who has since pursued other ventures) helped bring the database to market. The founding team’s background was largely in enterprise software and open-source development.

• Product & Technology: Neo4j is a high-performance graph database management system that stores data as nodes (entities) and edges (relationships) with properties. In the context of AI and memory layers, Neo4j serves as a platform to build and query knowledge graphs – structured representations of facts that an AI can draw upon. Developers use Neo4j’s Cypher query language (or its GraphQL and Python integrations) to query complex relationships efficiently. For example, one can retrieve a subgraph of all information related to a user’s query, enabling multi-hop reasoning (traversing connections between concepts). Neo4j ensures ACID-compliance and scalability for large knowledge graphs, which is important for enterprise deployments. Recently, Neo4j has actively positioned itself in the GenAI space by promoting GraphRAG (Graph-augmented RAG) architectures. The idea is to combine Neo4j with traditional vector-based retrieval: a vector search might find relevant documents, and Neo4j’s graph can then add context by linking those documents to related entities or facts.

• Differentiation: Neo4j's role in the memory layer space is as foundational technology rather than a specialized AI memory service. They stand out through maturity and capability - they're a proven platform with global enterprise adoption, suitable for production knowledge graphs with millions of nodes and relationships. Many other companies in this space use or integrate with graph databases like Neo4j under the hood. Unlike startups focused on chatbots or LLM agents, Neo4j is a general-purpose database adaptable to many uses (from fraud detection to recommendations, and now AI context storage). This means more integration work - organizations need to build their schemas and data pipelines - but it offers unmatched flexibility and powerful graph analytics through their Graph Data Science library.

Subscribe now

Curated reads

• Academic: Hybrid Retrieval-Augmented Generation on Textual and Relational Knowledge Bases

• Commercial: HP to Buy Parts of Humane, Maker of the Ai Pin, for $116 Million

• Social: Microsoft Study Finds AI Makes Human Cognition “Atrophied and Unprepared”

AI agents are all the hype today, so you’ve probably already read about them. But in case you are interested in understanding the inner workings, check out my previous essay How to Create a Mind.

In this article, we will take a business and investor perspective on AI agents, rather than a product and engineering one. With all the buzz about how “AI agents will revolutionize everything,” it is essential to have a framework to identify the areas with the most potential. To build this framework, I analyzed the automation potential of approximately 16,000 tasks across 800 jobs.

Motivation

When ChatGPT was released in December 2022, people found it to be remarkable. It was a fun tool, quirky, yet in many ways useful. Few considered how it might replace anyone in their jobs. The perception shifted when GPT-4 was released in March 2023, with benchmarks showing it is better than the most humans on exams that resonate with us — SAT, GRE, LSAT, AP.

While the benchmarks are useful, they do not directly translate to how well AI can perform in our jobs. A more helpful framework focuses on specific tasks — the jobs-to-be-done — in our roles. By looking at specific tasks, we can gain more nuanced insights into how much of our jobs is automatable by AI. That also indicates how much opportunity there is for the companies building AI startups (or how replaceable we may be). If AI can do our jobs, then the market opportunity is much larger than the $500 billion US software market — it is 20 times larger.

To precisely identify where the market opportunity is, I analyzed ~16,000 tasks across ~800 jobs from the US Bureau of Labor Statistics. Much credit is due to the tireless government researchers who have profiled all those tasks and jobs, even differentiating between computer programmers, software developers, and web developers.

Analysis

I used GPT-4o to evaluate the tasks (because o1 was too expensive) using a rubric. Below is a simplified version of it:

• No-Automation Exposure: AI cannot perform any aspect of this task.

• Low Automation Exposure: AI could complete 0-50% of the task components at high quality.

• Moderate Automation Exposure: AI could complete 50-90% of task components at high quality.

• High Automation Exposure: AI could complete 90-100% of task components at high quality.

• Full Automation Exposure: AI can complete all task aspects at high quality without oversight.

While others have run similar task studies, many did not specify what an “AI system” is. In my analysis, I distinguished the automation potential of base models, copilots, agents, and robotic systems. This differentiation allows us to be more precise about what we are measuring and where the opportunities are.

Findings and learnings

The incremental automation from base models to copilots is modest. Copilots, broadly — not just Microsoft’s offering — are impressive, but many feel they are not as transformative as initially marketed. Marc Benioff’s tweet captures this sentiment well. The next significant shift in AI is agents that autonomously execute tasks across tools and contexts, as in real jobs. AI agents are poised to be the next big thing — not just by hype, but supported by data.

The Agent Framework

Automation potential is one dimension in understanding how fast companies can achieve ROI from AI agents. Computer programmers are among the most automatable roles, which explains why coding tools like GitHub Copilot and Anysphere/Cursor are gaining traction quickly. Another key dimension is total wages paid, reflecting market opportunity. While proofreaders are among the most automatable roles, there are only around 5,500 in the U.S., earning about $280 million combined. In contrast, roughly 1.7 million developers collectively earn close to $230 billion.

Mapping 800 jobs across these two dimensions reveals roles with the highest potential for AI agent automation. The chart below shows the frontier of jobs ripe of agentic automation. It is no surprise many of these roles align with high-growth startup areas, where VCs are offering top valuations.

• Software developers: Poolside at $3 billion

• Lawyers: Harvey at $1.5 billion

• Customer Service Representatives: Sierra at $4.5 billion

I am still digging through the data but have already found some interesting patterns that I will be publishing in December as part of the 3rd annual Business of AI report. Subscribe if you want to get a copy of it.

Subscribe now

Curated reads:

• Academic: Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks

• Commercial: OpenAI Shifts Strategy as Rate of ‘GPT’ AI Improvements Slows

• Social: Donald Trump’s EPA pick wants to ‘make America the AI capital of the world’

One of the most impactful applications of generative AI is a knowledge copilot for humans. A knowledge copilot intelligently reasons through contextual information, providing a delightful and practical user experience. I first wrote about this topic a year ago, let's explore the industry and technological advances since then.

A knowledge copilot is a user’s thought partner, optimized for retrieving information from a knowledge base, reasoning through context, and synthesizing responses

Reasoning: With the launch of Open AI's o1 model last September, it's shown that models can reason as well as they can summarize and rewrite. GPT-4o can't correctly count the number of letters 'r' in the word "strawberry," a reflection of the state of AI models before o1. While it is arguable that what the o1 model does is not the same logical reasoning that humans do, it does a good job outperforming human experts at PhD-level science questions.

User Experience: The first copilots were pure chatbots, limited to text exchanges. Rapid improvements in copilot UX have occurred since then. Both OpenAI’s ChatGPT and Anthropic’s Claude now display code, apps, and documents—much like collaborating with another person today. ChatGPT’s advanced voice mode introduced an entirely new modality. With Anthropic’s new computer API, soon we'll have copilots controlling our screens.

Context: For general conversations, the relevant context might be the knowledge stored within the model's weights. For knowledge-intensive uses, the relevant context resides in private documents (i.e., knowledge base) and user information. Retrieval-Augmented Generation (RAG) is the only way to insert the ever-changing and growing knowledge base. Individually, we create new documents daily. In an enterprise, employees create thousands of new documents each day. Fine-tuning or dumping the entire knowledge base isn't practical.

Over the past year, copilot deployments have grown from pilots to enterprise-wide. With a much larger knowledge base to sift through, RAG systems struggle to pull out the right information. It is notable that whenever RAG gets brought up in industry events I’ve attended, the vocal engineers express their frustrations about how it's not pulling the right information. The quiet ones wince and commiserate.

As knowledge bases grow, using vector similarity becomes more fragile. The same words appear in multiple unrelated documents, making it difficult to identify the most contextually relevant documents for the copilot. Knowledge graph-augmented RAG (graph RAG) has emerged as a trend to improve accuracy. This approach was popularized by Microsoft’s GraphRAG paper and open-source project. New graph database open-source projects like Kuzu and the resurgence of Neo4j further point towards the trend.

Why context is the most important now

Using the wrong information as context leads to inaccurate or misleading copilot responses. That is the number one risk, and increasingly so, for organizations adopting AI. Inaccuracy erodes trust—the trust that gives us comfort to not have to constantly review copilot responses, and the trust that deters the adoption of software that many well-funded companies are building.

While better model (perhaps o2 in six months) and new user experiences (like having Claude guide my mom on how to find the settings menu on her phone) are always welcome, the bottleneck for knowledge copilots is setting the right context.

Knowledge graphs can help set the context

Knowledge graphs help because they are computer-readable representations of information that reflect the way humans think: entities (persons, things, events), relationships (spouse, author, attendee), and properties (age, date). The way we think is by entities and their relationships, not documents (unless you’re a lawyer): Kenn->writes->Generational. Generational->gains->Subscribers. Subscribers->seek->Knowledge. Knowledge->enhances->Learning. Learning->leads_to->Happiness. Happiness->increases_with->Subscribers.

Subscribe now

While that is a tongue-in-cheek nudge for you to subscribe, it reflects the logical context that AI models can reference to reason through.

However, incorporating knowledge graphs into RAG is challenging. The first few steps are already complicated, requiring engineers to figure out:

• How to parse raw documents (e.g., what about tables and drawings inside PDFs)?

• How to chunk the parsed raw documents? Limit it to 100 tokens for all chunks? Or do we create rules to accommodate paragraph endings, sections, etc.?

• How to augment & transform each chunk (Anthropic just introduced contextualization by pre-pending the previous chunk’s summary as context for the next chunk)?

• What embedding model to use?

Indexing knowledge graphs is more complicated because it requires computationally intensive processing of the entire knowledge base. It involves intelligently rewriting the knowledge base into another format (graph generation) and then also having to figure out how to make it readable by the computer (graph indexing). There are several steps involved in generating a knowledge graph and different ways to index it, each with its pros and cons.

Knowledge Graph Generation:

1. Document Preprocessing: Preparing raw documents by cleaning and formatting them to ensure that the text is in a usable state for extracting knowledge.

2. Ontology and Schema Definition: Defining the structure and types of relationships that will be captured in the knowledge graph, essentially setting the rules for how information is organized.

3. Entity Extraction: Identifying key entities (such as people, places, or concepts) from the text to populate the nodes of the graph.

4. Relation Extraction: Extracting relationships between entities (such as "works at" or "is located in") to form the edges of the graph.

5. Attribute Extraction: Identifying additional details or attributes about entities, like an entity’s age or location, which enhance the richness of the graph.

6. Deduplication and Resolution: Merging duplicate entities (e.g., “AI” and “Artificial Intelligence”) and resolving conflicts to ensure the graph is coherent.

7. Updating: As new information becomes available, the knowledge graph needs to be continuously updated to stay current. This involves re-running processes like entity extraction, relation extraction, and deduplication to integrate new data without disrupting the existing structure. The updated data is then re-indexed, ensuring that both the graph generation and indexing remain synchronized, enabling accurate and up-to-date access by computers.

Knowledge Graph Indexing:

1. Property Graph: A graph where each node and edge has associated properties, making it easy to search for specific attributes or relationships. It's flexible and supports rich metadata but can become complex when scaling, especially with large datasets and multiple property relationships. Example: (User:John)-[:FRIENDS_WITH]->(User:Jane)

2. RDF Triples: A structure used to represent data in subject-predicate-object form, enabling a formal semantic structure. It offers a standardized way to represent relationships, ideal for web-scale data, but its rigid structure can make modeling complex, real-world scenarios more challenging. Example: Kenn->writes->Generational

3. Vectors from Graph Embeddings: Translating graph nodes into vectors allows for efficient operations like similarity searches. It excels in complex tasks like clustering, though it sacrifices the explicit relationship semantics found in the original graph. Example: A node for User:John is converted into a vector [0.24, 0.87, 0.56]

4. Text and Text Embeddings: Textual data is transformed into machine-readable format for easy search and comparison within the graph. While it integrates well with structured data, it can lose contextual nuances, leading to reduced accuracy in some cases. Example: The text "John is a software engineer"

With LLMs, knowledge graph generation can be streamlined by using the same LLM for each step. But it still requires substantial engineering work to make the graph generation pipeline robust. User experience will also define the schema, which can be difficult if the application is broad.

Incorporating graphs into RAG is worth it

With knowledge graphs, there are four types of index data (there’s more, but we’ll stick with four) that can be used to retrieve the context. Given how complicated it can be to incorporate graphs, especially in a hybrid approach that combines multiple indexes, is it worth it?

It is. One of the more recent popular graph RAG frameworks is called LightRAG, which incorporates both knowledge graphs and semantics through text vector retrieval. The architecture below illustrates how it works.

Researchers compared LightRAG to other RAG systems—Naive RAG (basic RAG that engineers often dislike), RQ-RAG (breaking down user queries into manageable sub-queries), HyDE (writing hypothetical responses to pull similar documents in the knowledge base), and GraphRAG—using different datasets from areas like agriculture, computer science, legal texts, and mixed topics. The datasets were large, containing hundreds of thousands to millions of words. To evaluate the models, the researchers looked at how well they answered questions based on completeness, variety, and usefulness.

LightRAG came out significantly ahead of the others, especially in providing complete and varied answers, thanks to its ability to combine detailed and broad retrieval through graphs and vectors. It was particularly effective with more complex and larger datasets, like legal documents, where it was better at creating rich, well-connected responses. Based on the results, a more sophisticated RAG is worth it.

Semantic/Document RAG

• Naive RAG: Naive RAG segments texts into chunks and stores them in a vector database. It retrieves chunks based on similarity to the input query. This method is simple and fast but lacks the depth to understand relationships between the retrieved pieces, making it less effective for complex topics.

• RQ-RAG: RQ-RAG breaks complex queries into multiple sub-queries, making each search more targeted and precise. By decomposing queries, it can retrieve information more accurately, especially when dealing with multi-faceted or ambiguous questions.

• HyDE: HyDE generates a hypothetical document based on the query to guide the retrieval process. This approach allows the system to find information that best fits the imagined answer, making it particularly useful when the original query is vague or lacks specific keywords.

Knowledge graph RAG

• GraphRAG (from the Microsoft paper): GraphRAG uses graphs to represent entities as nodes and relationships as edges, forming communities of related information. It retrieves data by traversing these communities, which is ideal for understanding complex interdependencies. Unlike LightRAG, GraphRAG doesn’t combine graph and vector retrieval, focusing solely on graph traversal for deeper contextual understanding.

Semantic-Document + knowledge graph (hybrid) RAG

• LightRAG: LightRAG integrates graph structures and vector representations, allowing for both precise, low-level retrieval (specific entities) and broad, high-level retrieval (general themes). This combination makes it versatile and ensures comprehensive, context-aware responses. LightRAG can also quickly adapt to new information, without the need for full system rebuilds. Unlike GraphRAG, LightRAG’s dual system of graph and vector retrieval makes it efficient and capable of handling diverse queries.

The complexities of RAG is an opportunity

However, knowledge graphs are just one component of a RAG system. LightRAG is already a relatively simple hybrid RAG strategy, but it remains complicated to productionize, especially as part of a larger RAG system that incorporates additional types index data into a complex retrieval system combining multiple metrics and steps.

Combine that with parsing different file formats and types, figuring out the right chunk size, transforming the prompt, and choosing the embedding model—you have a system that can serve as a technical moat for companies. It also means that building an enterprise-grade system can be a standalone business.

Two examples of this are Google (of course) for consumer search and Glean for enterprise search. While we don’t have details on how Glean’s systems work, we do have some for Google Search. Google’s Knowledge Graph contains 1,600 billion facts (relationships, details about entities, etc.) about 54 billion entities. It is used both to rank search results and to power a UX called knowledge panels. As you can see, the entire system is complex, with the knowledge graph forming a small yet important part of it.

Subscribe now

Curated reads

Academic: Graph Retrieval-Augmented Generation: A Survey

Social: Former OpenAI Researcher Says the Company Broke Copyright Law

Commercial: Demo of Claude autonomously researching and orchestrating a sunrise hike by the Golden Gate bridge

Subscribe now

In this interview, I speak with Spellbook’s Scott Stevenson (CEO and co-founder) about building AI products. Their product uses AI to review and draft contracts and they have always been at the forefront of adopting the latest models to legal work, such as launching the first agentic product for lawyers, Spellbook Associate. Spellbook is used by over 2,600 law firms, professional services, & in-house teams including the likes of Addleshaw Goddard (Global Law 200), Nestle (Fortune 100), and BDO (top 5 auditing firm). Spellbook recently raised a $20 million series A from Inovia Capital and strategic investor Thomson Reuters.

Key learnings

1. Embrace the "Skepticism Window": Capitalize on the period when a new AI technology faces skepticism but shows promise. Scott's experience with agentic AI mirrors the early days of GPT models, suggesting that this skepticism often precedes rapid adoption and improvement.

2. Focus on Hard Sub-Problems: In complex AI systems like Spellbook Associate, solving difficult sub-problems (e.g., manipulating a 100-page legal document) is crucial before tackling higher-level tasks. This approach ensures a solid foundation for more advanced features.

3. Leverage Existing Workflows: Spellbook's success partly comes from integrating with lawyers' familiar processes, like track changes. This reduces adoption friction and addresses potential concerns about AI errors.

4. Sequence Feature Expansion: Start with core workflows and gradually broaden functionality. This strategy allows for mastering specific use cases before expanding, as seen in Spellbook's evolution from single to multi-document workflows.

5. Ride the Foundation Model Wave: Build on top of rapidly improving foundation models to benefit from their advancements. Scott emphasizes the importance of capturing upside from new model releases like GPT-4 and o1.

6. Value-Based Pricing in High-Stakes Industries: In fields like law where professional time is extremely valuable, flat-fee pricing can be more appealing than usage-based models. This approach simplifies billing and emphasizes the product's value proposition.

7. Balance AI Aggressiveness with Cost Control: Encourage aggressive AI usage in development while monitoring for excessive resource consumption. Scott mentions allowing developers to use AI freely but intervening when necessary to prevent runaway costs.

The Origins of Spellbook

Kenn: Welcome Scott! Maybe to start, I'd love to hear the history of Spellbook leading up to the launch of Spellbook Associate. Along that journey, I'm curious how the problems you're trying to solve have changed and, correspondingly, how the products you built changed as well over time.

Scott: Sure, I'll start with the problem, which has always been the same. We started, technically, almost 5 plus years ago. There are three co-founders, and we all had our own stories. I was an engineer, and I had my own small business. One day I got a legal bill that took half the cash out of our bank account. That was a moment for me where I thought, "Wow, this is a huge problem." Everyone has that experience of getting their first legal bill and thinking, "Holy cow, what did I just pay for?"

We set out to solve the inefficiency of legal work, that sort of value-to-cost equation not feeling good. My co-founder Daniel was a lawyer, so he saw the problem from the other side. He went to law school for years, racked up student loans, and then he graduated and started practicing. He realized he actually hated the work, which involved many nights until midnight with ten Word documents on the screen, trying to make sense of them and copy-pasting between them all.

That was the problem we're trying to solve: basically, the legal document drudgery required primarily in business and transactional work. This includes contracts, employment agreements, setting up companies and minute books, doing VC transactions, and all these things that create so much friction. That problem has always been the same.

Evolution of the Product

Scott: When we started, generative AI wasn't around. We started with a templating tool, basically a glorified templating tool where a lawyer could build a template for an employment agreement and then use the same one for multiple clients. We sold that to around 100 firms, and we actually had over a hundred landing pages. We sold many variations; we had literally over a hundred variations of our messaging and tweaks on our product to sell it and angle it in different ways. We even had one that was direct to businesses.

It did okay but never really worked that well because legal work is ultimately unstructured. Lawyers can't easily put their work into a template. Everything is pretty bespoke a lot of the time, and often a lawyer is working on someone else's paper, someone else's contract. You're almost never working on a brand new contract.

As an engineer, I saw GitHub Copilot, and this was probably early 2022 or late 2021. I thought, "Whoa, this thing is amazing." The early version of GitHub Copilot would auto-complete your code based on what you've written. What amazed me about it is you can use it in any situation without any setup. You didn't have to create templates in advance or anything like that. It just worked out of the box all the time, and it's in your existing workflow. For us, that was an "Aha!" moment. I thought, "This is what lawyers need."

We launched the public version one at the end of summer 2022. This was before ChatGPT, before any other generative AI company launched to lawyers. Within three months, we had more revenue than our past three years. It was just this explosive moment. We had 30,000 lawyers on our waitlist.

Kenn: It's really interesting to hear the journey. I'm curious, when you launched v1, was it mostly around helping generate or complete the next set of text similar to what GitHub Copilot does, or make edits? But then you've evolved it to also have benchmarks, broadly integrating the lawyer's workflow. I guess I'm curious, was it just a function of what your customers were telling you to build, and how much of that is also driven by what you've seen developing in the foundation models becoming smarter?

Scott: Yeah, so I think a bunch of things have driven our new features. Let me share my screen. I think it's helpful for you to see. So yeah, we started with autocomplete. Then we originally had a bunch of spells that would find issues for you. That kind of became our second thing, issue spotting. Here's an example of our negotiate feature: I can give it an agreement, and it will tell me things that I might want to improve with the agreement. It can even personalize suggestions, so it learns over time the sort of things that I like to negotiate for.

This became possible with GPT-4. This sort of fine-grained document analysis wasn't really doable or very good with GPT-3. When GPT-4 came around, this became more possible. This was something our customers really wanted. We just listen to our customers, see what they want, and yeah, the models advanced very quickly.

We do leverage a lot of off-the-shelf models. We do many things internally as well, but our general attitude is that foundation models are moving so fast. Most startups should build on top of them most of the time so that you capture the upside when something like GPT-4 is launched.

The Birth of Spellbook Associate

Kenn: What led to the creation of Associate? You already have a successful product sitting on top of a Word application. What led to building Associate, which automates much more of the work? Is that something you saw as an evolution of what the work will be for lawyers? Or is this something you've heard lawyers want, which is interesting too if that is the case, because that decreases their hourly billings?

Scott: I think since we launched Spellbook, the very first request we got was, "Can it work on multiple documents?" So even in 2022, we had customers asking. Spellbook works in Word, really on a single document at a time. We basically pushed the single document workflows as far as we could go. We built out just a ton of things that you can do: summarize changes, draft new clauses, draft full documents. We built out as much as we thought we could do at the single document level.

There are all these multi-document workflows that lawyers want to be able to do, too. You can't really do that in the Word sidebar, so the big motivation for us is getting to a multi-document workspace.

I think also we just have faith that agentic AI is the next big thing. It feels so similar to what GPT-3 felt like, and GPT-2 even, where everyone was skeptical at first. And then all of a sudden, it actually works really well. Agents feel like they've gone through that exact same thing: immense skepticism, a lot of "Oh, your examples are cherry-picked," and then, "Oh, actually, it's working 80% of the time. Oh, actually, it's working 90% of the time. Oh, actually, it's working 95% of the time."

We like to lean into that skepticism when something is new and it shows promise, and people aren't really ready to accept it yet. I think that's when you can be early. You know, those are all the signs of being early to a market. For us, we try to be really fast. We want to be two years ahead of the market, and two years ahead of all of our competitors is our goal. Moving fast is one of our competitive advantages as a startup.

Product Evolution and User Feedback

Kenn: When I saw the Associate product, it's like you're starting to build into how you break down a task or request into subtasks and then execute those clearly, because in a multi-document setting, my wife would ask the junior associate to update the name of the company, but it's actually spread across many docs, and they often miss one doc to update. That's really painful. So I really like how you set up the Associate product.

I'm curious, since it's early access, if there have been any surprises in terms of how you've seen your users use it, and any pitfalls that you've learned so far, being ahead of the market.

Scott: I think the main thing we've learned is that users want consistency. Yeah, it's cool to be able to do anything, but users want a sort of consistency. Focusing on a couple of really specific, popular use cases has been really good for our user experience.

I think of it as more of a sequencing. I think it's like you should nail your core workflows at first, and then once you have those nailed, you should broaden it and broaden it. I don't think there's one set point. I think it's really a sequencing of wanting to start and make sure you have really solid use cases out of the gate, and then you want to expand from there.

Another thing I think about is, if you think about Spellbook Associate, there are really kind of two layers of planning. There's the top level of, "Oh, I'm going to look at this document," or "I'm going to revise that document." And then there's once it gets to a document, actually conducting all the changes it needs to make. That second part of actually going to the document and making the changes is still an immensely hard problem that agents help solve.

We have found it's really good to focus on these hard steps. It's almost like there's an agent within an agent, and there's no point having a high-level agent if you can't do those steps really well. The step for us is taking a long 100-page legal document with instructions and manipulating the document. That's really, really hard.

Kenn: I want to shift gears a little bit into user perception and user adoption. I think Spellbook is one of the most well-adopted legal tech AI tools, and lawyers typically are the ones that are most sensitive to any errors due to the liability nature of their job. How have you seen them adopt these tools? Is that a real concern you've seen from your users, and how do you mitigate it? Even a simple typo in the date is a material thing when you draft contracts.

Scott: It hasn't been a problem for us, and that's because all of our workflows are designed to use track changes and to have this user step where they're accepting or rejecting a change. The same with Associate – it's changing the docs, but it's using track changes. You still have to go through and accept or reject the changes.

We're never altering something without the diff. Someone, when they saw our Associate video on Twitter, said, "Oh, this is what makes agents make sense." They're like, "This is also why Cursor works, and this is also why Devin works, because there actually is a really good, known UX already for accepting and reviewing changes."

That's why this works. Because lawyers do this all day already – they go through changes and accept and reject them. And that's what developers do all day – they go through pull requests and accept and reject them or review their own changes. So that's what makes it work well and what really puts the onus on the user to make sure that what they're reviewing... We've never had any kind of issues or worries about it because of that.

Kenn: Interesting. Now that you mentioned track changes, it sounds exactly like committing code to a repo. You have the lineage, you have the diff. It mirrors a lot with engineering. How does this apply to the Associate product? Does the user have to go through each doc and approve the changes? Is that the workflow you're envisioning for Associate?

Scott: We actually have two versions. We have one that lives on your desktop and opens up Word and uses that Spellbook UI that I showed you, and you have a sidebar. Then we have another fully browser-based cloud version that just does everything in the background and hands you back a doc. Then you basically have to open the doc and go through it normally.

We're getting feedback on whether they like that pure cloud experience or if they want this on their desktop talking to Word. There are a bunch of trade-offs there that we have to think about. But both of those workflows still require someone to go in and accept the changes.

Pricing and Cost Considerations

Kenn: Being sensitive to time, I know you have a hard stop. I want to shift to the last question, which is not often talked about – and feel free to keep it confidential if you don't want to talk about it – how do you think about pricing and cost for Associate? I think it's been a top concern, at least for my day job at Smartsheet. We're always worried about AI cost blowing up. How do you price a product like Associate? And how do you think about costing for it?

Scott: It's a great question, and it's very difficult because the costs are so dynamic. We've decided to go with basically a flat fee. We have a basic product and a premium tier that has more features. We actually don't do any usage-based billing at all today. We might at some point.

We just try to make sure that we're taking into account margins and usage. Generally, if you think about the value of a lawyer's time, lawyers are billing $400 to $800 to $1000 an hour somewhere in that range. So it doesn't take a lot for us to make up for that for them. We can make a pretty good case with flat-fee billing.

We've thought about usage billing all the time, but it's just hard because things are so dynamic. Generally, we tell our dev team to use AI very aggressively and not to worry about cost. That's been totally fine. There's been one or two moments when I'm like, "Hey, you're auto-running this huge job every time someone opens a doc – we can't do that." I will monitor the charts. But generally, the margins have been good.

That's simplified things for us and for our customers. It is something we think about, too. It's really hard because the costs are so variable. But our goal is to provide so much value to our customers that hopefully, our value-add is enough that we don't have to worry about it too much.

Kenn: Okay, that sounds good. That's a great point about lawyer per hour being so high – the value is high. Scott, I super appreciate the time.

Scott: Amazing, great. Thanks so much, Kenn. It was great to chat.

Thank you for reading! If you enjoyed this interview, who should we interview next? And if you haven’t subscribed, subscribe below to get the latest posts.

Subscribe now

A framework for data moats

A data moat is a strategic advantage a company gains by accumulating unique data that competitors cannot easily replicate, allowing it to deliver better products to customers. There are three vectors to it, each reinforcing one another.

1. Data flywheel: This is a self-reinforcing cycle where initial data collection leads to improved AI models, enhancing user experience. Better user experiences attract more users, leading to more data collection. Over time, this cycle continually upgrades the quality of the AI model and the user experience.

2. Data is proprietary: The uniqueness of a company’s data is central here. It’s not only about having exclusive data. It’s about how difficult it is for competitors to gather a similar dataset to deliver the same experience to customers. This can be achieved in different ways:

   • Commercial lock-in: Commercial agreements that guarantee exclusive data access

   • Data scale: Sufficiently big data corpus that discourages competitors

   • Regulated data: Regulatory frameworks and government procurement processes that limit access to data

3. Product experience is hard to replicate: This aspect emphasizes that it's not just the AI models, but the entire product experience enhanced by these models that create a data moat. Even if the data or model is somewhat replicable, competitors can’t replicate the same experience.

Netflix's recommendation system is a classic example of a product with a data moat. Netflix's data moat is tied to the exclusive content library it has built over the years. Even if competitors could theoretically build the same model and have access to the same user data - which is possible since Netflix allows customers to download their clickstream data - they won’t have Ozark, House of Cards, Black Mirror, Stranger Things, Narcos, Squid Game, and more. As users watch more Netflix, more data is collected to refine the recommendation system further. The result is a highly tailored user experience that is difficult for competitors to match.

Here’s a slide that captures the framework visually.

Defining AI products such as ChatGPT

An AI product is defined by its core value delivered by an AI model. In simpler terms, if the AI model is removed, the product loses its appeal or functionality. Consequently, a generative AI product uses a generative AI model. A quintessential example of one is ChatGPT. Without the underlying GPT model, which is GPT-4 today, ChatGPT would be just an empty chat interface. In contrast, a non-AI product like a spreadsheet retains its utility even without AI enhancements. Given the popularity of ChatGPT, we'll use it as the case study.

To evaluate ChatGPT against the framework, we first need to understand how GPT-4 is trained and what user data OpenAI collects.

Data OpenAI collects from ChatGPT users

According to OpenAI's privacy policy, when users talk to ChatGPT (or DALL-E), OpenAI has the right to use the chat logs to refine their models. Users have the option to opt out of this. However, this choice only impacts future interactions. Data from past conversations remains accessible to OpenAI. Even if you convince OpenAI to delete all of your data, which is right under CCPA, it is impossible to remove your data embedded inside a foundation model. In addition to chat logs, OpenAI gathers a lot of other information, similar to other software products. These are used for standard processes, such as processing payments, troubleshooting, and stalking users.

How does data feed into ChatGPT?

ChatGPT uses a version GPT-4 that is specifically tuned (including prompting) for conversations. It won’t produce the same output as the base GPT-4. But for this section, we focused on GPT-4 because there is public information about how it was trained and that the concepts apply would still apply to ChatGPT, and generally any generative AI product.

Training foundation models like GPT-4 go through two main stages:

• Pre-training: In this stage, the model is exposed to a broad array of text from various sources. This exposure helps the model to learn language patterns, context, and an understanding of different subjects. The pre-training phase is crucial as it forms the backbone of the model's knowledge. It's like teaching a child the basics of language and world knowledge before they can understand complex instructions.

• Supervised Fine-Tuning (including RLHF): After pre-training, the model undergoes supervised fine-tuning, which includes techniques like Reinforcement Learning from Human Feedback (RLHF). This stage is akin to giving the model a specialized education. Here, it learns to understand and follow specific instructions, respond appropriately to queries, and refine its responses based on feedback. This stage ensures that the model is not just knowledgeable but also useful and safe in practical applications.

While much has been written about the quantity of data, the quality of data is equally, if not more, important.

• Quantity: The sheer volume of data is a fundamental factor in training robust models like GPT-4, which was trained on 13 trillion tokens (~2,300 Wikipedias). A large dataset ensures a broad base of knowledge, enabling the model to understand and generate a wide range of content. This extensive exposure allows the model to be versatile and handle a diverse array of topics.

• Quality: This encompasses several aspects:

  • During pre-training: It includes the representativeness of the data, ensuring it covers a wide range of domains and perspectives. It also involves the freshness of content, as outdated information can lead to irrelevant or incorrect responses. Deduplication is important to prevent bias towards over-represented topics, while toxicity filtering ensures the model doesn't learn harmful or biased patterns.

  • During fine-tuning: Quality here focuses on the relevance and clarity of the instructions given to the model, as well as the diversity and complexity of tasks. This ensures that the model can handle a wide array of instructions and generate responses that are not just accurate, but also nuanced and context-aware.

Research has shown that small high-quality, textbook-like datasets can lead to high-performing models. For example, Microsoft’s 2.7 billion parameter Phi-2 model trained on 1.4 trillion tokens nearly matches the performance of the Llama-2 70B, a model with 26x more parameters trained on ~2x more data.

Is there a data flywheel?

Pretraining phase

Although chat logs from over 180 million active users comprise a large dataset, they are not suited for the pre-training GPT-4. The reason is that these interactions are primarily users asking ChatGPT for information, rather than contributing new knowledge for GPT-4 to learn. Additionally, if my experience mirrors the typical user's, these conversations are likely riddled with typos and grammatical errors, further diminishing the quality of this data for pre-training. In short, chat logs constitute low-quality data for pre-training.

Fine-tuning phase

During supervised fine-tuning, chat logs become more useful but in a roundabout way. Fine-tuning mostly uses single-turn dialogues to learn what the ideal response is: someone asks a question and ChatGPT answers. But we can't use these chats directly as ideal answers because it’s just ChatGPT responding. There is no good feedback mechanism (yet). For example, if I ask ChatGPT to explain AI Transformers, the answer might be a hit or miss depending on who’s asking. Some might find it too basic, others too complex. Figuring out the 'just right' answer for a broad user base is tough, especially when few use thumbs-up/down feedback.

Instead of being used directly as fine-tuning data, chat logs are valuable by showing what users are looking for in ChatGPT. OpenAI sorts these chats into different buckets, called 'content categories.' These allow OpenAI to focus on creating fine-tuning datasets for categories that matter to users. For each category, OpenAI writes rules to steer GPT-4 towards good responses and away from the risky ones. Here’s an excerpt from the GPT-4 paper:

One of our main tools for steering the model towards appropriate refusals is rule-based reward models (RBRMs). This technique uses a GPT-4 classifier (the RBRM) to provide an additional reward signal to the GPT-4 policy model during PPO fine-tuning on a subset of training prompts. The RBRM takes three things as input: the prompt (optional), the output from the policy model, and a human-written rubric (e.g., a set of rules in multiple-choice style) for how this output should be evaluated. Then, the RBRM classifies the output based on the rubric.

In practice, we write multiple rubrics for content categories on which we want to steer GPT-4- launch behavior. The main dataset comes from our production traffic (with consent from users). We use our models (the Moderation API plus zero-shot GPT-4) and human reviewers to filter and classify prompts into content categories.

Personalization

The feedback loop from data to model improvement is, at best, indirect, for ChatGPT. However, the value of data is not just enhancing the models. Personalizing ChatGPT could be a powerful experience.

One way to do so is by using past conversations to provide context for future ChatGPT interactions. Another is by automatically crafting prompts tailored to different user segments. For instance, I strongly suspect that the initial GPT-3.5 powering ChatGPT was influenced by Filipino annotators. I grew up in the Philippines where writing English is unique — overly polite and indirect. American English is more direct, albeit full of idioms. Neither is better than the other. But it’d be nice if OpenAI automatically inserts prompts to tailor communication styles based on where the user lives. There are many other simple quality-of-life personalizations that OpenAI can build.

Does OpenAI have proprietary data?

As discussed above, the chat logs collected by OpenAI, while proprietary, are not directly helpful. GPT models, and most foundation models, are trained on publicly available data. That isn’t defensible. However, OpenAI has been taking steps to build a more proprietary dataset.

Pre-training

Over the past year, OpenAI has been chasing licenses from media companies to train GPT models on their data in exchange for millions of dollars a year. Here are some deals they’ve struck:

• July 2023 — The Associated Press on Thursday said it reached a two-year deal with OpenAI. As part of the deal, OpenAI will license some of the AP’s text archive dating back to 1985 to help train its artificial intelligence algorithms.

• July 2023 — Shutterstock is extending its partnership with OpenAI for six more years, which gives OpenAI a license to access additional Shutterstock image, video, and music libraries and associated metadata for training models.

• Dec 2023 — Axel Springer and OpenAI have announced a partnership — not just a licensing agreement. The partnership allows ChatGPT to enrich users’ experience by adding recent content from Axel Springer’s media brands. The collaboration also involves the use of content to train OpenAI’s models

• Jan 2024 — OpenAI is in discussions with media firms CNN, Fox, and Time to license their content

Fine-tuning

OpenAI’s Human Data Team is building a system that uses human experts’ guidance to teach our models how to understand difficult questions and execute complex instructions. They were recruiting “Expert AI Teachers” across a wide range of fields from biology to law to economics. What’s notable is they set a high bar for these teachers — they have to be at least in the 90th percentile of their field, with teaching experience, and must also possess a kind personality. OpenAI also hires Scale AI to gather more training data. The latter was recently hiring for “AI Training for K12 Teachers” to work on various writing projects including evaluating AI responses. While this particular job post could be for other foundation model companies, it is illustrative of companies spending money to gather high-quality proprietary data.

Is the ChatGPT experience hard to replicate? Growing to be

Three months ago, the answer would have been no. Replicating the ChatGPT experience would have just taken 5 minutes — fork a Replit repo for the front end and key in an OpenAI API. ChatGPT plug-ins were still shabby back then. But a lot has changed since OpenAI’s Dev Day in November 2023. ChatGPT is becoming more than a thin wrapper around GPT-4 with:

• a proprietary native RAG experience for ChatGPT to respond based on files given to it

• the ability to customize ChatGPTs with custom instructions and actions (which replaced plug-ins). In two months, users have created over three million GPTs.

• and a GPT store to find & interact with other GPTs within the same UI. While it remains to be seen whether GPTs will get adopted, getting well-known brands to build GPTs is difficult to replicate. Here are some examples:

  • Personalized trail recommendations from AllTrails

  • Expand your coding skills with Khan Academy’s Code Tutor

  • Design presentations or social posts with Canva

  • Learn math and science anytime, anywhere with the CK-12 Flexi AI tutor

Does ChatGPT have a data moat?

Yes. Going back to the framework:

• Is there a data flywheel? Indirect, for now.

• Does OpenAI have proprietary data? Yes.

• Is the ChatGPT experience hard to replicate? Increasingly so.

ChatGPT has a data moat & OpenAI is investing to widen it.

PS — While this essay focused on data moats, OpenAI’s widest moat is its product and AI talent. The pace at which their team has been releasing new products has users asking them to slow down. In 2023 alone they released: GPT-4 and GPT-4 Turbo, ChatGPT paid plans, cutting GPT-3.5 costs down by 90%, plug-ins, custom GPTs, and more. Finally, a point not emphasized as much by others is that GPT-4 finished training in the summer of 2022. Since then, there’s been plenty of healthy competition, but GPT-4 is still considered the best model. It became standard practice for researchers and software engineers to use GPT-4 to evaluate other models and even humans — how far are the answers to GPT-4’s? Holding the top spot in academic benchmarks is hard power. People unconsciously assuming that GPT-4 is the best model is soft power.

It will be fun to see who can dethrone OpenAI’s GPT models in 2024.

A knowledge copilot is a user’s thought partner, optimized for retrieving information from a knowledge base, reasoning through context, and synthesizing responses. It is much more than a wrapper around foundation models for generating copy or poems. For instance, a business analyst synthesizing the latest market news requires web access. A researcher may need to find relevant papers for citation. Features like image generation and empathic conversation, while valuable, are secondary to a knowledge copilot's primary function: thinking and researching.

I write about this topic first because it fascinates me. Since 2016, I've been experimenting with crafting my own knowledge copilot using traditional NLP techniques. Second, it helps us become faster, better, and saner knowledge workers. Recent studies have shown that copilot-assisted research tasks are completed ~50% faster, without compromising quality. It eliminates the drudgery of sifting through documents and weblinks to find a specific information.

In the triad of retrieving, reasoning, and synthesizing information, retrieval is foundational. It lays the groundwork for the other two. Retrieval also addresses a key limitation of foundation models: their information is often outdated, and they don't know everything. While their billions of parameters contain vast knowledge, this abundance can lead to 'hallucinations' or inaccuracies. Therefore, the capability to retrieve just the necessary contextual information is a crucial determinant of a knowledge copilot’s effectiveness.

So, let's delve into Retrieval-Augmented Generation (RAG) and Knowledge Graphs (KG).

Knowledge Graph RAG

Retrieval-Augmented Generation combines retrieval-based and generative AI models. It first retrieves relevant information from a dataset, then uses this data to inform a generative model like GPT, thereby enhancing responses with more accurate and context-specific information. This improves the AI's ability to provide informed answers based on external facts not contained in its initial training data.

Recently, LLMs augmented with Knowledge Graphs have been gaining popularity. This grounds LLM responses in ground truth data, enabling more complex reasoning. In this essay, we focus on KGs derived from unstructured data like books and documents, as opposed to those constructed from structured data.

KGs represent human-and-computer readable synthesized representation of entities (persons, things, events), relationships (spouse, author, attendee), and properties (age, date) extracted from documents.

Instead of retrieving documents, why not directly retrieve information from a KG? To explore this, I built a KG-RAG, starting with constructing a KG. There is an established process to creating a KG.

• Entity Extraction: Identify and extract entities from unstructured text data, such as persons, organizations, and concepts from text.

• Relationship Extraction: Extract the relationships between entities.

• Coreference Resolution: Coreference resolution determines when two or more expressions in a text refer to the same entity (i.e. he/she refers to the right entities)

• KG Fusion/Construction: Fuse with an existing knowledge graph (lots of Match / Lookup queries). Otherwise, create a graph with entities and relationships as nodes and edges.

Traditionally, engineers would have to stitch a pipeline of custom models for each step. But instead I used GPT-4 Turbo as the single model to do everything. In a single prompt, I instructed the model to create a KG based on a lengthy essay by Paul Graham. Here’s a snapshot of the KG created.

The result was pretty comprehensive with ~100 entities and hundreds of relationships. KG-RAG could answer questions text chunk/document-based RAG could not. For example, it identified the books authored by Jessica Livingston’s partner (Paul Graham). But the document-RAG couldn’t and sometimes hallucinates by answering with the book Jessica wrote (a compilation of interviews with startup founders). With KG-RAG, I could also trace the logic of Jessica→dated→Paul→wrote→books→(Hackers & Painters, On Lisp). Let’s assume that the document-RAG is able to pull the right document chunks and answer correctly. Users would still have to read the retrieved text to validate and infer the same logical path.

Despite the advantages of KG-RAG, document-RAG is the right direction for knowledge copilots. The reasons being:

• For a large corpus that exceeds the LLM context window, engineers have to build a KG pipeline with custom models. While there are generic off-the-shelf models, they still have to be finetuned to a specific domain.

• Prompt engineering large context LLMs like GPT-4 Turbo or Anthropic Claude can streamline KG construction. But based on my tests, the results are highly sensitive to prompts. Tweaking a few words resulted in a materially different KG.

• KG-RAG does not have the generalization that we expect from a copilot. It can only answer what’s captured in the KG, making it highly dependent on an already cumbersome and flaky process.

• Conceptually and maybe even philosophically, knowledge is fluid. There are many perspectives on any single topic. Most people go through the exact same materials in school but end up with different takeaways. If you ask someone what is a cow, you'll get different answers. For most people, it is an animal that produces milk and beef. For kids, it is black and white animal that produces milk. For butchers, it is a source of income with expensive cuts. We know what a cow is, but there are many ways to express it and they're all correct.

Document-RAG works because vectorizing it retains enough generality for a copilot. KG-RAG is conceptually more attractive but document-RAG is the viable path for knowledge copilots. So which document-RAG strategy is the best? There are over a dozen strategies, ranging from simple RAG to sophisticated multi-agent set ups. To answer the question, I tested a few RAG strategies that represent different conceptual approaches.

The best RAG for knowledge copilots

To evaluate RAG strategies, I used text I knew well since I wanted to manually score the results instead of the industry practice of automating it with GPT-4. I used eight Generational articles I've written this year, which has ~13,000 words.

The scoring methodology was straightforward. Each RAG strategy started with a base score of 10 points. I crafted 10 questions requiring intricate reasoning or a deep understanding of the articles. The responses from each RAG strategy were assessed for "correctness". For an incorrect answer or a failure to find the right information, I deducted 1 point. If the answer was partially correct, I subtracted 0.5 points. (To the engineers: yes, I am mixing retrieval and LLM generation evaluation together)

As a baseline, I also evaluated the performance of GPT-4 Turbo's context window and the native RAG in OpenAI’s GPT, using the articles as the knowledge base. Details on the libraries and models used are available in the appendix. Now, let's explore each RAG strategy and its underlying logic.

Simple/Naive RAG: This strategy takes an input and retrieves a set of documents with the most similar vectors.

Multiquery RAG: Generates multiple queries based on the user’s initial query. For each generated query, the most relevant documents are retrieved. The intuition is that by decomposing queries into distinct queries, we can capture the various perspectives better. The approach here is to transform the user query

Hypothetical question RAG: This approach creates questions that can be answered from each text chunk. The user's query is then matched with the most similar questions, and the associated 'parent' chunks are retrieved. The intent is to align the user's queries with hypothetical questions, anticipating a closer match in their embeddings. The approach here is to further transform the documents

ReAct (short for reasoning and acting): An agent in this strategy iteratively retrieves chunks and reflects on whether there is sufficient information to respond or if more querying is needed. This mimics the process of traversing a knowledge graph, albeit through text chunks.

Results and insights

• For simple use cases and smaller corpus, naive RAG works well enough. But if the corpus is large, more advanced RAG strategies might work better.

• For best “accuracy”, agents work best. A ReAct agent iteratively acquire information and reflect on it, allowing for a traceable thought process akin to human knowledge work. For instance, when querying about Microsoft Copilot's pricing versus Intercom Fin's, ReAct would first retrieve a set of document chunks. If the information on Intercom's pricing was missing, it would specifically seek out additional chunks related to Intercom.

• If agents are impractical, use the multiquery approach. While both multiquery and hypothetical question RAGs had similar scores, I recommend using multiquery (or methods that modify the query instead of the corpus). It is more feasible to test and diagnose queries than going through the entire corpus. You’ll see products using this approach in the next section.

• Context length is not everything. Extending the context length is an active area of development, with Anthropic's Claude boasting a production model with 150,000 word context window (~500 pages). But stuffing the prompt with the entire corpus led to worse performance. Overloading with information, much like in humans, can be counterproductive.

• Finally, OpenAI’s GPT models are frustratingly verbose. Prompt it to limit response length to a few hundred words. A cynical view of this is that OpenAI is financially incentivized to make their models verbose.

UX is just as important

That said, agents are not the solution for knowledge copilots. ReAct-like agents does multiple retrievals behind the scenes. That can be slow and costly at scale. More crucial is that agents that iteratively reflect on intermediate results loses context. It could hallucinate or limit itself to an LLM’s stored knowledge. When I tested an advanced agent strategy called FLARE, the agent immediately assumes there is no pricing available for Microsoft Copilot because GPT-4’s knowledge cutoff was April 2023.

Humans should be steering the copilot during each turn. But it can be tedious to constantly type out more context and guidance. Imagine always having to type “please just search the top five Google results” or typing out acronyms like RAG. UX plays a crucial role in helping humans help copilots help humans. Below we go through examples of great UX from Perplexity and Bing’s new Deep Search.

1. Asking clarifying questions. This guides the retrieval to the right domain to search. For example, RAG has different meanings: a literal piece of rag, red/amber/green in project management, retrieval augmented generation, recombination-activating gene, and so on.

2. Showing the steps. It is helpful to see what the copilot did and how RAG worked. If the results seem off, this provides some level of auditability for users to adjust their queries. This transparency helps build trust with the copilot. Note that Perplexity might be using some form of multiquery RAG.

3. Citing sources. While most copilot products have this feature, simple UX features like showing logos and reference title improve the experience. Logos immediately signal the credibility and context of the information.

4. Constraining the search. Constraining the search surface area and setting the context can be effective. If I want to know the latest chatter, I should search Reddit. If I want to read up on the latest RAG papers, I should search Google Scholar.

After RAG, what’s next?

Earlier, we noted that retrieval is the foundation for knowledge copilots. There is still reasoning and synthesis. But I’ll leave that to future essays.

Subscribe now

Curated reads:

Commercial: An Introduction to Google Gemini

Social: Early LLM-based Tools for Enterprise Information Workers Likely Provide Meaningful Boosts to Productivity

Technical: Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine

Appendix

Corpus: ~13,000 words from eight Generational articles I've written over the past year

Libraries used:

• Unstructured for processing and loading the articles

• LlamaIndex and Langchain as frameworks to split, query, and synthesize responses

• Chroma and Neo4j as the data stores

Chunking scenarios

• Smaller chunks (~500 words) to simulate the scenario of a large corpus with lots more chunks, making it harder to retrieve right information

• Large chunks (~1,000 words) to increase the chances of retrieving the right the information

Models used

• GPT-4 Turbo (gpt-4-1106-preview)

• Ada Embedding (text-ada-embedding-002)

Subscribe now

Experimentation to Production

It has been almost one year since ChatGPT was released. In the initial six months, the community grappled with understanding the capabilities of foundation or generative AI models. The number of in-person AI events happening in Bay Area grew from just 10 in January to 100 in September. ~Three in-person events a day, filled with folks talking about AI.

Over the past few months, I’ve noted that the narrative has shifted. It is less about demonstrating what models can do. While hackathons still happen, the conversations transitioned from “look at how cool this is” to “how can I make this damn thing work at scale?”.

This isn’t a surprise considering that today all major public software companies have released a generative AI product. There’s also a long list of private companies not included in the list below.

• Collaborative Applications: Microsoft, Google, Slack, Atlassian, Smartsheet, Asana, Zoom

• Content Management: Box, Dropbox, Docusign

• Sales & Marketing: Salesforce, Hubspot, Sprinklr

• IT & Customer Support: ServiceNow, Freshworks, Five9, Shopify

• Finance & ERP: Workday, Blackline, SAP

• Creative: Adobe, Canva, Autodesk

• App Development: Github, Gitlab, AWS, Pagerduty

• BI & Data: Oracle, Tableau, Thoughtspot, Snowflake

Three challenges in making AI enterprise-ready

Here are three recurring challenges distilled from attending and hosting several events.

1. RAG is the rage (both loved and loathed): Productionizing RAG seems to be the rallying call of AI vendors today. I’ve seen it become the tagline of company booths in a few events. Off-the-shelf models are sufficient for general assistants but incorporating proprietary data, which can be as simple as your last email to provide context, makes these models much more valuable. RAG is the best way to do so. Finetuning models is great for specific tasks but it is not great for incorporating new knowledge. Finetuning also makes it difficult for engineers to iterate quickly compared to engineering a RAG pipeline. As much as there is excitement to productize this pipeline though, there remains thorny problems of how to best chunk, index, rerank and synthesize the retrieved results.

   

2. Contextualized retrieval: While RAG is great, deploying at the enterprise level often means thousands of users accessing documents from different sources. For a consumer facing chatbot, it might be pulling from a general knowledge base that can serve most customers. For a knowledge management assistant, it requires more context to be helpful. It has to know what documents the user is working on, has access to, which users it is collaborating with, and so on. It is exponentially more complex than indexing an entire Google Drive and making it available for retrieval to anyone. At an enterprise level, context & permission-aware indexing and retrieval is key. Not all employees should have access to internal financial forecasts nor should private Slack messages be searchable.

   

3. Monitoring challenges: Transitioning experiments into production requires monitoring. While monitoring machine learning models is not new, monitoring generative AI models is. It is more complicated. First, its because of the sheer amount of additional metadata generated by LLM applications. Each prompt chain, retrieval, vector index, and so on has to be tracked. The power of LLMs is that it can handle any open ended conversation but that also expands the scope of monitoring. Even when its all logged, evaluating the conversational responses of models is subjective. Today, it is manually rated by humans who each have different subjective preferences. There is a long tail of additional challenges about mitigating bias (difficult to manage at scale), enhancing explainability (LLMs can explain themselves but can be eloquently incorrect), and enforcing copyright (detectors are proven to be ineffective).

   

Enterprise GenAI Forum on Nov 14

While most of the events have centered around product and engineering topics, building enterprise AI is more than that. Competitive positioning, pricing & packaging, practical legal issues, and talent management are also relevant to building AI companies. So I am organizing Enterprise GenAI Forum on November 14 in San Francisco to explore these topics. This event is for AI insiders that are building, selling, and investing in enterprise AI products. Details are still being finalized but if you’d like to be the first to get access, sign up here. Priority will be given to current subscribers — leave a note saying you found out about the event through Generational.

Sign up here: Enterprise GenAI Forum

PS - I am looking for a speaker / panelist that has worked on pricing & packaging of GenAI products. If you know someone who is a rockstar in this area would so appreciate leads.

Subscribe now

Subscribe now

A recent history of AI agents

The user experience of ChatGPT & similar products is that it requires a human to pilot. Its works as a collaborator that needs live instructions. A different way to experience AI is that of fully autonomous AI agents, large language model (LLM)-driven programs that can seemingly operate autonomously. Give an agent a goal, it will figure it out from there. The first version of the AutoGPT project was pioneering, though a bit chaotic. The agent doesn’t know when the assigned goal is achieved. It can be unpredictable, such as how it started researching OnlyFans as a way to grow this newsletter, and only stops working when it reaches an arbitrarily set number of API calls.

BabyAGI’s framework was a major step in making agents practical by incorporating a task management system to impose structure and direction. The human analogy is that an AI agent on its own is like a human reacting to whatever comes to mind, which is not reliable. But pair a human with a task management system, even a simple Kanban board, then work becomes so much more productive. The March 2023 launch of AutoGPT and BabyAGI marked the start of agentic AI wave. Since then developers have been improving what agents could do, equipping them with long-term memory to remember what they’ve done along with a portfolio of skills to browse the web, write software programs, and many more.

If we could build one agent, why not several? The multi-agent scene first came to prominence with the work of Google and Stanford researchers running the Smallville experiment with 25 AI agents each simulated with their own personalities and memories. What’s fascinating from the research is that social behaviors such as gossiping and event planning ‘naturally’ emerged from the simulation. That brought multi-agent projects to the Silicon Valley mind hive. Character.ai and other social bot platforms soon enabled chatrooms to host multiple agents chatting with each other. Those are all social-conversational applications, what about getting agents to do work?

A task management app in 10 minutes and $1.50

Just a couple of months ago, I wrote in Command line to conversations:

This might be a vision of the future: copilot agents from different software vendors working together. Microsoft Copilot will extract the technical requirements from the product requirements Word document, then instruct Atlassian Intelligence to create Jira tickets for each technical requirement. These tickets will then be handed off to Github Copilot to write, test, and ship code.

That future just got closer. In the past month, there has been the advent of practical multi-agent projects. MetaGPT is the most popular one so far because it is open-source, achieved state-of-the-art (SOTA) performance, and provides an intuitive framework to build a team of AI agents.

So, what is MetaGPT?

A framework that efficiently incorporates human workflows into LLM-based multi-agent collaboration. By encoding Standard Operating Procedures (SOPs) into prompts, MetaGPT enables structured coordination and modular outputs. It leverages an assembly line paradigm to assign diverse roles to various agents, allowing for the effective deconstruction of complex multi-agent collaborative problems. Experiments on collaborative software engineering benchmarks show promising results.

In the paper, the creators of MetaGPT configured it to be a software development team composed of a product manager, systems architect, program manager, software engineer, and a QA engineer. To have an objective measure of the AI team’s performance, they tested against two popular coding benchmarks, HumanEval and MBPP, and achieved SOTA results. LLMs that perform well on both benchmarks are considered to be more capable of generating code that is both correct and readable.

However, a more holistic evaluation would encompass other aspects such as ease of use, quality of the documentation, and the effort it took to build the application. MetaGPT is compelling because it produces work artifacts along with the code base such as product requirements document (PRD), system architecture and data flow diagrams, API specs, and a comparison of competing products. All these documents are in the team’s shared workspace and referenced as needed by the agents to continue working. In the paper, the example software the authors created was the 2048 game. The goal of the game is to slide the numbers around to combine them, creating larger and larger numbers until it reaches 2048. They were able to create working game along with work artifacts from a single line of instruction “Make the 2048 sliding tile number puzzle.”

That’s amazing. I had to see it for myself. So I installed MetaGPT and asked it to write a task management software. Out came an almost working codebase. I still had to do some debugging but hey, it worked. It was complete with a SQL database, HTML templates, and a Flask-based Python script to stitch everything together. For $1.50 worth of API calls and 10 minutes, I got an entire AI team built an app with detailed documentation. I could have shaved half the time & cost if I didn’t instruct the agents to do code reviews.

See snippets of the generated PRD and system diagram below. If you’re interested in browsing through all the other files, here’s the link.

User Stories
1. I want to create, edit and delete tasks so that I can manage my work
2. I want to categorize tasks into projects so that I can organize my work 
3. I want to set due dates for tasks so that I can prioritize my work
4. I want to mark tasks as complete so that I can track my progress
5. I want to view all tasks in a single dashboard so that I can get an overview 

Competitive Analysis
1. Asana: A comprehensive project management tool. It is more suitable for teams and may be overwhelming for individual users
2. Trello: A kanban-style task management app. It is easy to use but lacks some advanced task management features
3. Microsoft To Do: A simple and straightforward task management app. It is fully integrated with other Microsoft apps
4. Google Tasks: A basic task management tool. It is integrated with Google Calendar and Gmail but lacks advanced features

Three lessons on building a team of AI agents

The insight from this is that agents will become more helpful collaborators to humans if they adopted how humans manage and produce work. Specifically:

Give agents specialized roles — Research has shown that an LLM reasons better if it assumes a cast of experts to reason about a topic, performing better than the chain-of-thought technique. The MetaGPT paper also showed that removing roles in the team led to worse performance. The intuition is that an LLM is a general brain that can be prompt-hypnotized into role playing as experts to tap into specialized knowledge hidden in the model parameters.

The process of specialization isn’t limited to prompt engineering. It can also be by equipping agents with the role-relevant skills. For example, a product manager agent needs to be able to browse the web to do competitive research, craft user stories, and synthesize them into a PRD. The creators of MetaGPT programmed these skills for each agent. It will be interesting to see if there will be a marketplace of AI skills so users can craft their AI agents in a modular fashion. We’re seeing the beginnings of this with ChatGPT plugins and Zapier’s marketplace of zappable APIs.

Teach agents how to collaborate on a shared workspace — Knowledge workers have a repository of work artifacts (docs, slides, numbers, chat logs) and processes (waterfall, agile, weekly syncs) to coordinate work at scale. These are templates to model how AI agents can work together. In the MetaGPT example, it was a modeled as a waterfall development process in which agents pass on the baton of artifacts to the next one.

Sidebar: When I was studying macroeconomics in university, one concept I never fully appreciated is how abstract management practices is considered part of “technological” growth factor along with likes of tangible computer chips. But seeing that programming how agents work together affect the end output gave me better appreciation of that concept. Economics, after all, is about managing scarce resources. In this case, its coordinating and allocating the skills and memory of AI agents.

Manage information flow — When we go about our work, we communicate a lot to make decisions and keep stakeholders aligned. Whether its written or verbal communication, we share only the relevant information (at least we try to) to complete our jobs. MetaGPT employs a similar idea with different personas subscribing to certain chat logs. The agents don’t digest all of the information arbitrarily. For example, the system architect doesn’t follow the competitive analysis work. But when the PRD is done, it’ll get notified so it can start designing an architecture. This is similar to how we don’t attend every meeting, read every email, or even read messages in Slack channels we’re in. Vector-based retrieval and a large context window can help agents digest more information, but curating information flow through intentional communication patterns matter still. Research has shown that while LLMs can handle 100,000 tokens, they’re similar to humans in that the more concise and relevant the context the better.

What will work look like by 2024?

Granted that MetaGPT may still seem like a shiny demo limited to simpler Python-oriented programs, the trajectory of progress is staggering. Just a few weeks after MetaGPT’s release, Microsoft open-sourced Autogen, a flexible multi-agent framework in which humans can step in as an individual contributor or give iterative feedback to agents as the manager. These projects set the stage for a future where AI does more than just assists — it also collaborates and manages. If we got here within 10 months of ChatGPT’s release, where will we be by 2024?

Curated reads

Technical: MetaGPT: The Multi-Agent Framework

Commercial: You can now finetune GPT-3.5

Social: Nations carve different paths for tech regulation

Subscribe now

Subscribe now

A brief history of the pursuit

Understanding our minds is a quest that has been pursued since antiquity. Greek philosopher Aristotle explored internal mental processes such as perception, thinking, and memory, laying the groundwork for cognitive psychology. Meanwhile in the East, Indian philosopher Gotama analyzed the steps involved in perception, influencing future information processing models.

Today, cognitive science and computer science aim not just to understand, but to also recreate the mind from their discpline's unique perspectives. Cognitive science models natural minds, focusing mainly on understanding the processes that generate human thought. Meanwhile, the field of artificial intelligence (AI), nested within computer science, seeks to build algorithms and systems that mimic intelligent behavior.

For much of AI’s history, developments—though valuable—were somewhat limited relative to the lofty goal of recreating the mind. Traditionally, AI algorithms were task-specific, lacking the flexibility that is second nature to humans. AI engineers were tasked with optimizing accuracy performance for these specific tasks. However, the advent of foundational models shifted this paradigm by providing the first glimpses of general artificial intelligence. Foundation models exhibited the creativity and flexibility unique to human minds. Over the past six months, AI engineers have pivoted from optimizing task-specific models to designing autonomous systems or AI agents.

Cognitive science, on the other hand, has been building holistic models of the mind for decades. There is a rich history from which AI researchers could glean valuable insights. This essay aims to draw parallels between these two fields and highlight the insights, some practical and some intellectual, that can be learned from the comparisons.

A Cognitive Model of AI Agents

Cognition pertains to the mental processes that our brains utilize to acquire, process, store, and utilize information. It's analogous to the software that our brain employs to interpret and engage with the world around us. A cognitive architecture is analogous to a software architecture as it outlines the components of a system and how they function. Over the past 40 years, over 80 cognitive architectures have been published in scientific literature. Despite this variety, a broad consensus has emerged around what the standard cognitive model entails. It comprises five primary components: perception, working memory, procedural long-term memory (implicit knowledge), declarative long-term memory (explicit knowledge), and motor functions. As the model of the human mind, it also serves as a blueprint for building AI agents. I’ve added a sixth component, termed the orchestrator. It is not part of the standard model but is a helpful construct to complete the framework. Let's examine each one individually.

Perception: Perception involves interpreting sensory data we receive from our surroundings. We have five main senses (sight, hearing, taste, smell, and touch), and perception is the process our brains employ to make sense of the information they provide.

In AI, perception corresponds to tasks typically associated with computer vision, natural language processing, and other sensor data processing techniques. Concretely, it is analogous to file formats like .txt, .jpg, .mp3, which serve as digital representations of information that models can comprehend. However, not all data comes in model-readable formats (e.g. pdfs). So we also consider tools like Unstructured.io that convert non-AI-readable formats into AI-readable formats as part of perception.

Working Memory: Often compared to a mental workspace, working memory is the part of our memory that temporarily holds and manipulates information. Consider when you're performing mental arithmetic or remembering a phone number; you're utilizing your working memory.

In an AI context, working memory can be thought of as a temporary storage and processing engine for transient data. The context windows of Large Language Models (LLMs) serve a similar purpose. An LLM’s context window contains the input users (or machines) provide, which it processes alongside additional information retrieved from databases.

Procedural Long-term Memory: This is the part of memory responsible for skills and habits. Procedural memories involve tasks like riding a bike, typing on a keyboard, or driving a car. It's termed "procedural" because it's primarily about knowing how to perform procedures or actions. These memories are typically unconscious, and we execute them effortlessly.

In AI, procedural memory refers to implicit skills and moral boundaries embedded within the models’ neurons. For example, ChatGPT is trained to respond jovially when making fun about most religions. When it comes to Islam, ChatGPT will refuse to make jokes, provide a reason as to why, and prompt the user to be more considerate.

Declarative Long-term Memory: This memory stores facts and events. It's divided into two subtypes: semantic and episodic memory. Semantic memory is for general knowledge about the world, like knowing that Paris is the capital of France or that dogs are mammals. Episodic memory, on the other hand, is for personal experiences or events, like remembering your first day at school or what you had for breakfast yesterday.

In AI, this corresponds to databases. There are various ways to implement this because information can be stored in different ways. What database to use depends on the use case. For example, knowledge graphs are an effective way to store facts and the relationships between them. While vector databases like Activeloop, Chroma, Pinecone, and Weaviate are optimized to handle abstract numerical representations (i.e. vectors) of words & images.

Motor: This component of cognition is dedicated to planning and executing physical actions. Whether it's something complex like playing a musical instrument or something simple like picking up a glass of water, your motor cognition is in operation. It assists you in planning the movement, coordinating your muscles, and making adjustments based on feedback like what you see and feel.

In AI software systems, this refers to abilities that interact with other software systems, such as accessing your file system or drafting an email for you. Accessing external systems is what distinguishes motor skills from procedural skills embedded inside the neural network. The range of skills that an application has depends on the use case. An example of a marketplace of skills are ChatGPT Plug-ins and Zapier’s collection of API integrations.

Orchestrator: The component responsible for managing and coordinating the interactions between all the other components. In the context of AI, this could involve dictating when and how data is retrieved from the databases, how it is fed into the context window, and how it is used to plan an AI's actions. This could be app frameworks like LlamaIndex and Langchain or no-code orchestrators like Respell and Stack. Some of the processes it coordinates include:

1. Retrieval: The process of bringing information from long-term memory back into working memory. For example, when you're trying to remember a movie name or a historical fact, you're using the process of retrieval. In AI, retrieval could be implemented by querying a database or a knowledge graph to pull up relevant information.

2. Encoding: The process of converting information into a form that can be stored in memory. For instance, when you learn a new fact, your brain encodes that information into neural patterns that can be stored in your long-term memory. In AI systems, data is encoded into numerical or symbolic representations that a machine learning model can process.

3. Consolidation: The process by which temporary memories are converted into a more permanent form for long-term storage. While this differs from encoding, in that we only store an imperfect synthesized version of the encoded information that goes through our minds, a similar process occurs in AI. In AI, a similar process occurs during the training or fine-tuning of a model, where neural networks are updated to reflect new knowledge. This also includes the summarization and synthesis of working memories into a database for storage.

4. Planning: A kind of top-level management system in the brain, which includes skills like task-switching and planning. In an AI system, this could be the task management system similar to BabyAGI which approximates a decision-making system that directs the actions of the lower-level processes based on the overall goals.

How it all works

Let’s walk through a cognitive cycle to understand how it all works together. When you are reading this essay, your eyes see characters (which objectively are arbitrary lines of contrasted colors), These are passed on to the working memory which then is given meaning by our procedural reading skill. The orchestrator & working memory then pulls word & sentence meaning and associated memories from the declarative memory. As you learn, as I hope you would reading this, your declarative memory gets updated with new concepts and associations with past memories. As you read this essay, your mental planner could (hopefully) decide that this is worth reading further so you instruct your hand to continue scrolling.

A more technical illustration of cognitive cycle is the design pattern for the common AI chatbot that retrieves documents to answer questions. Say, you are talking to said chatbot. The input to its system is your question. Your question is encoded into a vector representation (a string of numbers) with an embedding model. A retrieval function compares your search vector with document vectors stored in a database and selects the most relevant documents. These documents are then included in the LLM’s context window as additional input to respond to your question. The LLM then synthesizes the answer or it may decide to execute your intended request (e.g. can you reset my password?) with the skills available to it.

How to use this framework

This model is useful for understanding the core parts of an AI agent, leaving out finer details like fine-tuning, setting boundaries, and caching prompts. A simpler, more intuitive framework complementing technical overviews (like this one). Here's how I’ve used the framework:

Keep up with fast AI developments: For example, Meta and Microsoft just launched Llama-2. Using this model, non-tech people can understand that it's just another addition to the pool of available Large Language Models (LLMs) to help working and procedural memories. Unless Llama-2 shows a big improvement over current models or works better with other parts, we’re happy with this takeaway. The details of the model, of course, still matter to tech people.

Find bottlenecks and opportunities in developing AI systems: There are many well-funded basic model and database companies. But we’re just starting to see companies focusing on orchestrators, perception, and motor skills. Putting on the venture hat, startups in the these categories are the ones to join or invest in. But of course, deciding which companies are the best requires a deeper look - something for future articles.

Compare the human and artificial mind (my favorite): We are moving towards AI systems that are better than us in most mental abilities. Humans’ working memory is limited in capacity, usually about 7 pieces of information give or take a few. But the context windows of AI models today can hold up to 200,000 words, which can fit a few novels. When it comes to long-term memory, databases can grow infinitely. While scientists haven’t found a limit for our brain’s capacity to store facts, humans need a lot of effort to learn. Plus, software is also more reliable. Our mental abilities depend on our mood, sleep, age, and other seemingly trivial factors (like sunshine).

But humans still have an advantage in orchestration, which is part problem-solving, part creativity, part mystery. This might be the last practical area where the human brain excels. AI systems struggle with getting the right information because of imperfect encoding models and retrieval metrics. Feed the wrong context to the model, and you get a convincing piece of text or computer command that's wrong. Humans are able to put together information, connect the dots between memories, and get the right context effortlessly.

Subscribe now

Curated reads

• Technical: Series of AGI related papers from Microsoft researchers

• Commercial: Emerging LLM Stack

• Social: 7 A.I. Companies Agree to Safeguards After Pressure From the White House

The metaverse hype coincided with the web3 craze of 2020-2022. Non-fungible tokens (NFTs) and augmented/virtual-reality (AR/VR) gained traction then. Both projected visions of a digital life deeply interweaved with our physical reality. The crypto community latched on to NFTs while Meta doubled down on its Reality Labs division. The top NFT project Bored Ape Yacht Club (BAYC) created an active community of NFT owners who have access to an exclusive chatroom, a virtual bathroom, and a shared sense of superiority over pleebs. Next-gen Oculus headsets become more usable with hardware upgrades and an increasing library of games and activities to do in VR.

I never fully appreciated the hype then and even until now. And its not for a lack of trying. I created my own NFT project Turtling Turtles. The idea was to create a community of owners who will develop their digital turtle’s personalities to be part of a collective Turtle metaverse. I also bought Oculus Quest 2 to experience the latest in VR. My family in Asia also had a unit. After spending hours connecting our Facebook accounts to Oculus (ironic), I finally met my mom’s avatar in a Lego-like meeting room. After five minutes, the novelty wore off and we went back to talking on Facetime.

My experience lends itself to the narrative that media likes to trumpet today: metaverse is dead. Billions were invested into NFTs & AR/VR with naught for show. NFT trading volumes have dropped 90% from one to two years ago and Meta spent $36B just to show off “clunky Oculus headsets and a virtual wasteland populated by textureless, featureless, legless avatars and landscapes” per Open University professor John Naughton.

But the metaverse isn’t dead. NFT & AR/VR are not the sole representatives. The metaverse is simply an interactive virtual space populated by many users at the same time. By that definition, metaverse is an already-two-decades-old thriving reality in the form of massively multiplayer online role-playing games (MMORPGs). They are virtual worlds where large numbers of players interact with each other in real-time, thus creating a collective virtual space. They are also interactive, with players able to alter the world or the state of the game in meaningful ways. Think World of Warcraft, Runescape, Guild Wars. I’d even include Club Penguin there. At their peak, each game had hundreds of thousands to millions of concurrent players bantering and bartering with each other. Players only needed an Intel Pentium Silver and a dial-up connection to be part of those virtual worlds. Neither blockchains nor GPU-powered headsets were needed.

Socially connecting in MMORPGs

Since early 2000s, I’ve been in multiple metaverses - challenging others in Club Penguin on who can slide down the snowy slope the fastest, begging passerby in Runescape for scraps, and teaming up with strangers to hunt monsters in Ragnarok Online (RO). RO takes places in a fantasy world influenced by the Norse mythology. Similar to most fantasy-style MMORPGs, Users create their own characters to suit their style, team up with other players to tackle dungeons and hunt monsters, and form guilds to wage wars against other guilds. In this essay, I’ll use RO as the primary example because that’s where I’ve spent too much of my childhood what I am most familiar with.

While MMORPGs are microcosms of the metaverse geared towards video gamers, it is still useful to understand the three key elements of what makes a good MMORPG. That’ll help us explore what’s needed for the metaverse to become mainstream.

1. User agency - being able to shape a character according to style and preferences. Players are more attached to the characters created as a reflection of them in an alternate world. For example, in RO you can create your own characters to match your preferences visually (hair style, clothes, accessories) and game mechanics (skills, stats, items).

2. Gameplay - is the combination of how engaging the game mechanics and the virtual world are. Realism isn’t necessarily the goal, in fact, stepping out of the real world is one reason why people immerse in a virtual world. In RO, there’s a deep system of skills trees, crafting recipes, and pet accessories to engross users as they explore a mythological continent.

3. Social connections - these connections infuse the metaverse with a sense of community and emotional depth. In RO, people congregate in areas to team up, buy & sell items, or just chat with each other.

We need all three for a metaverse to be immersive. Without user agency, it just becomes media that is consumed passively like movies and TV shows. Without an engaging gameplay, it just becomes a text-centric social network like Facebook and Discord. Without social connections, it just becomes a single player game with limited replayability.

This framework also explains, though naively, why NFT and Meta’s VR ambitions did not translate to their metaversal ambitions. Crypto bros hoarded NFTs and shunned non-crypto bros. While Meta focused too much on the hardware. That said, both efforts are still laudable. NFTs normalized buying and selling cryptographically unique digital items while Oculus introduced a new way of experiencing the internet.

Of the three, the tech & gaming industry has mastered two: agency and gameplay. There is a long history of building immersive games. Some were even adopted into TV shows by mainstream media. What hasn’t been mastered yet is how to jump start a social network and facilitate social connections. If the metaverse is about making an alternate reality be as immersive as the real one, then social connections are key. People live in drastically different environments & cultures. But the value of relationships with colleagues, friends, and kin is the same everywhere.

Widely-cited studies (such as here, here) of what drives MMORPG engagement also conclude that socialization is the most important factor. The gameplay of an MMORPG might attract a user to try the game, but its the connections that makes it immersive. Meta, even though it focused on the Oculus hardware, understood the same core value of the metaverse: social connections. Their “What is the metaverse” webpage states that

The metaverse is the next evolution in social connection and the successor to the mobile Internet.

Like the Internet, the metaverse will help you connect with people when you aren't physically in the same place and get us even closer to that feeling of being together in person.

Creating a social network, especially in a virtual environment, presents complex challenges with uncertain outcomes. It's not just about creating an attractive platform or a captivating game, but about fostering a sense of community and belonging. One of the biggest challenges is the "chicken and egg" problem: people join social networks to connect with others, but if there are not enough users at the beginning, it's hard to attract more.

But what if you could create digital humans and seed an artificial social network? What if you could jumpstart Metcalfe’s law? That would require two things:

• Ability to create autonomous digital humans (agents) that can form connections of their own

• Real humans are comfortable joining and blending in with the artificial society

With recent AI developments, we’ve started get glimpses of both.

Generative societies

A paper published in March 2023 by Google and Stanford researchers created a simulated world of generative agents, aka digital humans. Each agent is powered by a large language model, a database serving as a the memory, and gameplay mechanics to notice and interact with the virtual world.

The researchers created a town with homes, facilities, and 25 digital humans, each with their own personality & history. For example:

John Lin is a pharmacy shopkeeper at the Willow Market and Pharmacy who loves to help people. He is always looking for ways to make the process of getting medication easier for his customers; John Lin is living with his wife, Mei Lin, who is a college professor, and son, Eddy Lin, who is a student studying music theory; John Lin loves his family very much;

Each agent then goes about their day. The agents are all acting autonomously, without any human intervention. Each agent’s experience of interacting with others continually shapes their future thoughts and actions.

John Lin wakes up around 6 am and completes his morning routine, which includes brushing his teeth, taking a shower, and eating breakfast. He briefly catches up with his wife, Mei, and son, Eddy, before heading out to begin his workday.

The magic happens when agents start exhibiting human-like behaviors without being explicitly programmed to. Games today create characters that follow meticulously sequenced scripts. Characters cannot break out of it and usually ends up becoming a parrot repeating their last scripted line. But the simulation in this study was not scripted. There was no being off script. Its all “natural” adlib. The agents formed new relationships, spread news, and coordinated events all on their own. Thereby forming a self-sustaining artificial society.

One event that the researchers highlighted is when agent Isabella Rodriguez planned a valentine’s day party. The only explicit programming was the intent for the party. There was no script. With just the intent, Isabella proactively invited other agents. Other agents then spread the news until eventually half of the village became aware of the event. During the party, the agents had fun, even gossiping about crushes.

We found evidence of coordination among the agents for Isabella’s party. The day before the event, Isabella spent time inviting guests, gathering materials, and enlisting help to decorate the cafe. On Valentine’s Day, five out of the twelve invited agents showed up at Hobbs cafe to join the party.

Of course, the simulation wasn’t perfect. The researchers noted a number of erratic behaviors. Most of which I agree with, except for one.

As a result, some agents chose less typical locations for their actions, potentially making their behavior less believable over time. For instance, while deciding where to have lunch, many initially chose the cafe. However, as some agents learned about a nearby bar, they opted to go there instead for lunch, even though the bar was intended to be a get-together location for later in the day unless the town had spontaneously developed an afternoon drinking habit.

That sounds perfectly normal.

Human-AI relationships

With glimpses of human-like agents and artificial societies, how ready are real humans in being part of it? We are beginning to see signs that humans are growing more comfortable relying on and building relationships with human-like AI. And Character.ai is a great case study.

Character is a platform that hosts a multiverse of chatbots. Users can create "characters", define their "personalities", set specific parameters, and then publish them to the community for others to interact with. These characters can be based on a variety of sources, including fictional media and celebrities, or they can be completely original. Some are created with specific objectives in mind, such as assisting with creative writing or functioning as a text-based adventure game. Users can engage with individual characters or organize group chats with multiple characters interacting with each other and the user simultaneously. These are similar to the generative agents mentioned earlier, although they currently lack a deep memory and are limited to dialogue for interaction.

Building upon the autonomous product development team idea in my previous essay, I assembled a team of characters to help create the metaverse.

Well, what if you could assemble a team of AI agents to work together autonomously? One venture capitalist told me about how his friend developed an autonomous product & engineering team. There's a product manager agent that decides on a set of features, which are then passed on to a group of developer agents to code. The developers then write, compile, and test the code. After completing the first set of features, the product manager will create a new list of features. When the (human) friend reviewed the code, it works.

With such a lofty goal, I needed the help of some of the brightest technologists: Elon Musk, Mark Zuckerberg, and Tony Stark. These characters are severely imperfect reflections of their real life counterparts, but the Elon bot seems to capture some of the real Elon’s tendencies well.

While ChatGPT and similar chatbots facilitate human-to-AI conversations, multiplayer chatrooms, like those in Character, allow for more fascinating conversations to happen. Here’s an example of my teammates playing off each others’ ideas. If you’re interested in the full conversation, head over to the Metaverse chat room.

But how immersive can this be? SimilarWeb, a web analytics company, studied the average time spent across consumer media websites:

• ChatGPT: ~9 minutes

• Twitter: ~10 minutes

• Facebook: ~10 minutes

• Youtube: ~20 minutes

• Character: 25-30 minutes

The data suggests that Character users today seem to be more immersed than those of other apps. To be fair, this only includes website visits on a browser. Engagement on mobile app versions of Facebook/Youtube/Twitter may be higher. But Character doesn’t have a mobile app yet. So at least the data above is a like-for-like comparison. Another indicative comparison is to that of Fortnite, occasionally referred to as the metaverse in gaming circles. Fortnite players spend ~1.2 hours per day playing while active Character.ai users spend 2 hours daily.

I’m not promoting Character, though it is a fun product. The point is that we are at a turning point where we are comfortable spending a large portion of our time knowingly talking to bots. One counter point is that these are numbers only reflect the usage of early adopters, which is ~15% of the population. Fair. But 15% is still a lot. And for those 15%, they might even prefer it to human connections.

Curated reads:

Commercial: ChatGPT Is More Famous, but Character.AI Wins on Engagement

Societal: Reddit thread mourning the lost of sexting in Replika

Technical: Generative Agents: Interactive Simulacra of Human Behavior

Subscribe now

Thank you to those who joined the first Generational dinner last month! It was a fun night with wide-ranging topics and unexpectedly flashy cocktails. The format is still evolving but what’ll remain the same is that it’ll remain small, 8 or less, to foster quality conversations over good food. Also thank you to those who reached out about wanting to join. I have you all in a spreadsheet! I’m working through the invites based on the theme to make sure invitees like you will enjoy the conversations.

• Second dinner happening next week (full)

• Third dinner happening next month (full)

• Fourth dinner in the works. Tentative theme is AI x Agents x Metaverse aligned to this essay

Surveys conducted earlier this year show that between 30-50% of working professionals in the US are already using ChatGPT for work, although most won’t admit it to their employers. While it has been less than six months since ChatGPT launched, we are quickly becoming more accustomed to using natural language as a primary means of interacting with software. In UX terms, this is a Language User Interface or a LUI. But LUIs aren't new. The first LUI was the Command Line Interface (CLI), which emerged in the 1960s and became popular alongside personal computers in 1970s. They were the first digital interface to replace physical punch cards used with large mainframe computers. To this day, CLIs remain the preferred interface for programmers to communicate with computers. However, CLIs are inaccessible. It requires deep knowledge of programming language syntax, alienating non-developers.

So companies developed Graphical User Interfaces (GUIs) to make computers more accessible to the broader population. GUIs became popular in the 1980s, primarily due to the release of Apple's Macintosh computer in 1984. The Macintosh was the first commercially successful personal computer to feature a GUI with a mouse and keyboard. Apple also popularized the default touch-based GUIs on our smartphones today with the launch of iPhone in 2007.

With ChatGPT, LUIs are making a comeback. Not through obscure CLIs, but through Natural Language User Interfaces (NLUIs). Anyone that can understand English or other everyday language can use NLUIs skillfully. Apple did try to make Siri, a speech-based NLUI, a thing but it never caught on. Alexa and Google Assistant never caught on as well. While novel, all three were never smart enough. But ChatGPT is. And its rapidly setting our expectations to imagine and even demand that NLUIs should be everywhere: writing, searching the web, creating images, and booking flights for us.

Four months ago, ChatGPT launched with the ability to write, answer questions, and engage in conversations like a human. It then became the fastest growing consumer app.

Three months ago, Notion AI, which embeds text generation as a native feature, became popular. Techies voted it as the most innovative technology of right after ChatGPT.

Two months ago, Bing started searching the web, summarizing information, and falling in love with us.

Last month, Microsoft previewed copilots across their software products, consolidating interactions with everyday office software into a chat interface.

This month, we started seeing more companies launch copilots and variants of NLUIs. From incumbents like Atlassian to early-stage startups like Equals, a modern spreadsheet app. As someone whose past professional life revolved around spreadsheets, this one is my favorite to date. Equals has a command bar-like interface that takes English instructions from users to perform rote tasks like changing formats to more meaningful tasks like adding up transactions to calculate revenue. The best part is that it is available to use now. Even Microsoft employees don’t have access to copilots they’ve built.

Generative agents

As we’re adjusting to a new UI and UX designers rush to adapt, there’s already another emerging paradigm - agentic AI. This is an AI-driven system that uses agents to interact with the world. Agents are software programs that mimic, to some degree, humans. At a high level, it has three components:

• memory (database) for storing data, reflections, and other files

• ability to use tools (APIs) like accessing the web, write Python programs, etc.

• a reasoning engine (large language models) to plan tasks, understand data, and reflect on its own work

Databases and APIs are not new. Human-like generalizable reasoning engines are. Two open-source projects that spurred this paradigm are BabyAGI and AutoGPT. Both projects are conceptually similar, so we’ll focus on the most popular, AutoGPT.

Auto-GPT is an experimental open-source application showcasing the capabilities of the GPT-4 language model. This program, driven by GPT-4, chains together LLM "thoughts", to autonomously achieve whatever goal you set. As one of the first examples of GPT-4 running fully autonomously, Auto-GPT pushes the boundaries of what is possible with AI.

Users give an agent a goal. It will then autonomously figure out a plan to execute that goal on its own, without any user intervention. This is different from the NLUI of ChatGPT. AutoGPT automates multi-step projects that would’ve required back-and-forth turn-by-turn conversations. ChatGPT needs specific instructions. AutoGPT just needs a direction.

AutoGPT itself did not bring any technical innovation. Its contribution lies in combining existing technologies to create a viable new paradigm using agents. The project holds the record for becoming the fastest-growing open-source project in history, previously held by another AI open-source project Langchain. Langchain is already an anomaly in software history, which shows how singular AutoGPT is.

The spectrum of use cases people have experimented with is fascinating, from identifying the top 10 party schools & automatically submitting applications to world domination with ChaosGPT. ChaosGPT launched with a Youtube video demonstrating its early efforts into world domination. A 25-min video that I watched for longer than I should have. The entire video is just a dark terminal with flowing bright text. But watching ChaosGPT meticulously type out its steps and thoughts is mesmerizing, as if we are granted an opportunity to peer directly into the intricate workings of an evil mastermind.

Of course, I also created my own agent for your entertainment, called Generational-GPT (GG). Here’s GG’s short-lived story.

GG’s goal was to increase Generational’s subscriber count to 10,000 by being the thought leader in AI.

GG started browsing the internet for articles to read.

GG found a relevant Forbes article. It started reading and taking notes. It stored its notes in “ai_trends_2023.txt”

From its notes, GG wrote an article that is ‘engaging and informative to attract and retain subscribers’. The article was saved as “ai_trends_2023_article.txt”. I read it. It was neither engaging nor informative.

Armed with an article, GG tried to promote it by tweeting. It failed since I did not give it access to my Twitter credentials.

GG then started to browse the web to look for alternative ways to attract potential subscribers. And one of the websites it wanted to learn from was about promoting on OnlyFans…

Interesting.

But before things got spicy, OpenAI cut off the API connection.

Moderation works ¯\_(ツ)_/¯

Dynamics of a singular interface to all software

What both current iterations of NLUIs and agentic AI suggest is that users can consolidate all their work into a single interface. For consumers, Google is the gateway to the web. When we search for flights, insurance premiums, or news, we use Google. Hence, consumer companies had to learn SEO, build integrations with Google, and adapt to every change in Google search algorithms. This wasn't possible in business software. Each profession has their own preferred set of tools created by different companies. Designers go straight to Figma. Financial analysts log on to their Bloomberg Terminal. While Microsoft's Office suite is the dominant tool used by people across professions, the user experience is quite disjointed. So business software today is still a bunch of fiefdoms connected via APIs.

But with the capabilities that Microsoft's 365 Copilot has shown, it's not ludicrous to think that there could be a single dominant interface for business software. Microsoft is building Loop, a flexible Notion-like interface. The Loop app consists of three elements: components, pages, and workspaces. A workspace is a collection of pages. A page is a collection of components. The key innovation lies in their concept of components.

Components are portable pieces of content that stay in sync across all the places they are shared. They allow you to co-create in the flow of work, be it on a Loop page or in a chat, email, meeting or document. They can be lists, tables, notes, and more, ensuring that you’re always working with the latest information in your preferred app — like Microsoft Teams, Outlook, Word for the web, Whiteboard and the Loop app.

Microsoft Loop is currently in public preview. But as early as 2022, Microsoft made clear its plans to have third-party developers build Loop components. They previewed a Loop component from SAP, illustrating how people can work on live ERP data in a Windows interface. They also launched a private preview for third-party developers.

In my last essay, I pointed out that Microsoft researchers has been building TaskMatrix, an intelligent system that can “talk to a million APIs". Copilot with Loop might be their implementation of that. Loop aims to be the singular GUI for business software, with Copilot being the NLUI and brains.

Copilot in Loop gives you AI-powered suggestions to help transform the way you create and collaborate. It guides you with prompts like create, brainstorm, blueprint, and describe. Or simply type in a prompt, like “help me create a mission statement.”

Like the rest of Microsoft Loop, Copilot in Loop was built for co-creation. As you and your teammates work, any of you can go back to earlier prompts, add language to refine the output, and edit the generated responses to get better, personalized results. Then share your work as a Loop component to meet your teammates where they are, in Teams, Outlook, Whiteboard, or Word for the web. Copilot in Loop is currently in private preview.

Where specialized software fits

An impact of technology on jobs, which I did not explore in my last essay, is the trend towards specialization. As new technologies get adopted, jobs are automated, new industries are developed. Before cars, people used horses or walked to get around. There were a limited number of jobs related to transportation, such as blacksmiths, carriage makers, and stable workers. The invention of the car made travel faster, more efficient, and available to a larger number of people. As cars became more popular, demand for them increased. This led to the creation of the automobile industry, which required a variety of specialized jobs. These included car designers, mechanics, assembly line workers, salespeople, and marketing specialists. The rise of the automobile also created opportunities for other industries to emerge. Gas stations, repair shops, and car dealerships were established to support car owners.

Similarly, the software industry has evolved to cater to the diverse needs of professionals, developing specialized tools designed for specific tasks and roles. Taking a closer look at the creative and marketing sector, specialized software cater to different professional niches within the industry. Figma is used by UX designers, Adobe caters to photographers and graphic artists, and Google Analytics is essential for growth marketers. Each of these tools is tailored to the unique requirements of its target users, much like how specialized jobs within the automobile industry cater to specific roles and responsibilities.

There will always be a need for specialized software. Even if Copilot + Loop tries to be interface for all business software, the functionality will be limited. The Loop interface is not suited for building financial models, creating interactive UX designs, or coding. The specialized software vendors are also not incentivized to hand more control over to Microsoft. They will create their own copilots. Atlassian has its own Atlassian Intelligence. Github, in spite of being part of Microsoft, also has its own separate Copilot product.

What might the future look like?

Well, what if you could assemble a team of AI agents to work together autonomously? One venture capitalist told me about how his friend developed an autonomous product & engineering team. There's a product manager agent that decides on a set of features, which are then passed on to a group of developer agents to code. The developers then write, compile, and test the code. After completing the first set of features, the product manager will create a new list of features. When the (human) friend reviewed the code, it works.

This might be a vision of the future: copilot agents from different software vendors working together. Microsoft Copilot will extract the technical requirements from the product requirements Word document, then instruct Atlassian Intelligence to create Jira tickets for each technical requirement. These tickets will then be handed off to Github Copilot to write, test, and ship code.

Curated reads:

Commercial: Autonomous Agents & Agent Simulations

Societal: ChaosGPT

Technical: ReAct: Synergizing Reasoning and Acting in Language Models

Subscribe now

Alongside the buzz and excitement of AI is a growing chorus of concerns about the consequences of foundation models. The Future of Life Institute, an organization whose mission is to steer humanity away from extreme risks, penned a petition to pause any AI research smarter than GPT-4. GPT-4 already shows sparks of AGI, scoring at the 90th percentile among test takers of several academic and professional exams. Over 50,000 signatories are worried that we are not prepared to deal with the risks of AI systems becoming competitive with humans. This essay explores one of the questions posed in the letter: should we automate away all the jobs, including the fulfilling ones? 

Will AI actually automate all the jobs?

Shortly after GPT-4's release, OpenAI and University of Pennsylvania researchers studied the job market impact of large language models (LLMs) by breaking down jobs into tasks, such as system engineers monitoring computers or gambling cage workers processing payments. They noted that some jobs share similar tasks, like grading exams for both elementary and high school teachers. For each task, human experts and GPT-4 assessed whether LLMs could reduce the time it takes to complete a task. They found that LLMs allowed 15% of tasks to be completed significantly faster without quality loss, which increased to 50% with LLM-powered software. Intriguingly, GPT-4's estimates were consistent with human experts', suggesting that the researchers could have relied on GPT-4 to reach the same conclusions.

The study is insightful because it examines specific tasks rather than making broad estimates based on industries or job titles alone. However, it's not data-driven. Automatability is determined by subjective judgments. So, let’s look at an actual study measuring the ChatGPT’s productivity boost. But first, let me share a personal anecdote.

I've been using ChatGPT as a writing partner for several weeks now, and although I haven't measured my daily word output, I feel about 20-30% more productive. An essay that would typically take me ten days to complete can now be done in just seven. ChatGPT assists with common LLM tasks like generating essay feedback, refining sentences, and helping me find the perfect word – essentially, a conversational thesaurus. But what's astonishing is how it also boosts creativity. While writing my previous essay "GPTs are GPTs," I wanted to include some witty remarks about big tech companies' dominance.

On Google’s ability to dominate the digital ad market:

By indexing and organizing web content, Google created new opportunities for different forms of online advertising from replacing top search results with paid ads to infuriating commercials in between videos.

On Apple’s ability to create complementary innovations:

These devices created an ecosystem of mobile applications and services that have shaped the way we communicate and live - ride haling with Uber, food delivery with DoorDash, and digital nicotine with Tiktok.

But I was struggling with Microsoft. There weren’t any holes I could poke into their products. Its ad-free, unaddictive software that helps you get your work done. After gnawing at it for 15 minutes, I finally relented and asked ChatGPT for help:

Me: Help me add a snarky, witty sentence in the Microsoft example similar to what I did for Google and Apple.

What came out became my favorite line in that essay. How could I have forgotten the frustrations of sleuthing through random forums to figure out why there was an extra line break in my Word doc that I could not get rid of.

ChatGPT: The standardization of Windows OS enabled generations of developers to build higher-level applications and turned countless users into masters of Word's arcane formatting mysteries, Excel's labyrinthine formulas, and PowerPoint's hypnotic slide transitions, ultimately revolutionizing how we work, procrastinate, and communicate in colorful pie charts.

Many people have shared similar anecdotes about their experiences working with ChatGPT, but it's hard to generalize from these since everyone's experience is unique. Thankfully, pioneering MIT PhD students Shakked Noy and Whitney Zhang conducted a controlled experiment to evaluate ChatGPT's impact. They divided about 450 experienced white-collar professionals, including marketers, managers, and grant writers, into two groups: those with access to ChatGPT and those without. Participants from both groups were asked to complete writing tasks resembling real job tasks, such as crafting press releases, short reports, analysis plans, and sensitive emails. Their performance was assessed based on the speed and quality of their work. We now have statistically significant data on ChatGPT’s effects on work.

Not surprisingly, ChatGPT users completed tasks ~40% faster while also producing higher quality work than those who did not. The more intriguing result of their study came from further probing how participants were using ChatGPT: were they simply copy-pasting ChatGPT’s output? Did they just use the first output or did they iteratively refine their work? They found that ~70% copy-pasted the first output without editing it at all. Those who spent more time on ChatGPT to refine their work did not result in better work quality. This means that ChatGPT can immediately substitute for some workers. It’s a story of replacement rather than augmentation.

Mark Zuckerberg declared 2023 will be Meta’s year of efficiency. This eventually became the rallying cry of CEOs. If employees could be 40% more productive without loss of work throughput and quality, its unfortunately logical for companies to let employees go or freeze hiring for a while. Capitalists will accrue the benefits of ChatGPT at the expense of workers.

Sprinting towards further disruption

If current systems, which are mostly pure LLMs, can already impact us this much, imagine what more advanced LLM-powered systems? Industry researchers already have prototypes of foundation models that directly instruct other systems. If GPT-4 was a game-changer, interacting with external systems would be a revolution. We've already seen the the beginnings of this with OpenAI's ChatGPT plugins. With the plugins, users could order groceries via Instacart and plan trip via Expedia just by chatting. Taking this idea a step further, Microsoft researchers recently published a week-old paper detailing TaskMatrix.AI, a system using a conversational foundation model to interact with “millions of APIs” to complete tasks. Below are some excerpts from the paper showing an overview of the system and my favorite example of what TaskMatrix can do: create a formatted PowerPoint presentation about big tech companies just by chatting.

Overview of TaskMatrix.AI. Given user instruction and the conversational context, the multimodal conversational foundation model (MCFM) first generates a solution outline (step 1), which is a textual description of the steps needed to solve the task. Then, the API selector chooses the most relevant APIs from the API platform according to the solution outline (step 2). Next, MCFM generates action codes using the recommended APIs, which will be further executed by calling APIs. Last, the user feedback on task completion is returned to MCFM and API developers.

Multiple rounds of dialogue between user and TaskMatrix.AI. TaskMatrix.AI can understand user instructions and operate PowerPoint on behalf of users. TaskMatrix.AI is capable of breaking down the user’s complex instructions into multiple PowerPoint operations, assisting users in finding and using infrequent features, and generalizing the same patterns across multiple pages. While we display the API calls in a gray text box, this information is not necessary for the user.

More rounds of dialogue between user and TaskMatrix.AI. TaskMatrix.AI can accomplish the insert logo instruction by the insert internet feature of PowerPoint with API insert_internet_image(”Microsoft logo”). This feature will provide multiple images for users. TaskMatrix.AI can take the user’s instructions to select one of them. In the example. we omitted the selection steps for brevity.

Creating the presentation above might've taken me 30-45 minutes to complete without any assistance. If I had TaskMatrix to help me, it would have taken me 5-10 minutes maximum, including the idle time of watching the computer fetch and resize images from the internet. Sorry to belabor the same point, the impact on the job market will be massive. And it’s coming faster than we’re prepared for.

OpenAI wrapped up GPT-4 training in August 2022 and refined it until its March 2023 launch. They've likely finished training GPT-5, with rumors pointing to a December 2023 release. OpenAI isn't racing alone towards AGI. The whole industry is. Past general-purpose technologies took years, even decades, to achieve widespread adoption. Power plants and electrical grids are massive, multi-year infrastructure projects, while the internet needed computing devices and undersea cables. But AGI-like systems are software accessible to anyone with an internet connection, so adoption will be swift. That's why ChatGPT hit 100 million monthly active users in just two months.

No clear answers but we’ll adapt

Writing this essay has left me feeling torn. On one hand, I'm thrilled about the technology and still amazed at how helpful of a collaborator it is to me. On the other hand, I'm filled with anxiety and uncertainty about what the world will look like a year from now. Nevertheless, I choose to stay hopeful. As Stephen Wolfram beautifully puts it - as a society we’ll adapt.

Technology in some way or another enables some new occupation. And eventually that occupation becomes widespread, and lots of people do it. But then there’s a technological advance, and the occupation gets automated—and people aren’t needed to do it anymore. But now there’s a new level of technology, that enables new occupations. And the cycle continues.

A century ago the increasingly widespread use of telephones meant that more and more people worked as switchboard operators. But then telephone switching was automated—and those switchboard operators weren’t needed anymore. But with automated switching there could be huge development of telecommunications infrastructure, opening up all sorts of new types of jobs, that in aggregate employ vastly more people than were ever switchboard operators.

Something somewhat similar happened with accounting clerks. Before there were computers, one needed to have people laboriously tallying up numbers. But with computers, that was all automated away. But with that automation came the ability to do more complex financial computations—which allowed for more complex financial transactions, more complex regulations, etc., which in turn led to all sorts of new types of jobs.

And across a whole range of industries, it’s been the same kind of story. Automation obsoletes some jobs, but enables others. There’s quite often a gap in time, and a change in the skills that are needed. But at least so far there always seems to have been a broad frontier of jobs that have been made possible—but haven’t yet been automated.

But what can we do as individuals to prepare now? We can't just wait for the day we get laid off, watch how society adjusts, and then adapt. We have to be proactive to stay relevant. I don't have answers, but here's what I'm doing:

1. Leveraging conversational foundation models like ChatGPT to automate routine parts of my work, so I can focus on the most creative aspects. AI is excellent at repeating existing knowledge, but individuals who can generate new knowledge will be more valuable.

2. Building and deepening personal relationships. Strong connections help us grow and its also good for our soul.

3. Pursuing analog interests like improving my cooking and pottery skills. Maybe I'll open the cafe I've been planning for my retirement decades sooner.

Everyone should get a Llama

One of the crowded demo tables at the Hugging Face event was helmed by the Stanford team that built Alpaca, an open-source alternative to ChatGPT. Stanford released the model along with a live demo, which was taken down in a few days due to costs. It's expensive to run these models, especially polished, responsive products like ChatGPT. OpenAI can offer ChatGPT for free, thanks to Microsoft covering the expenses. Hugging Face already provides a public service by hosting free ChatGPT-like models, although they might be slower and less user-friendly. If usage soars, Hugging Face will need to throttle access to manage costs

Foundation models are changing the way we work. They can boost productivity significantly, letting people focus on creative and strategic tasks. Given their importance, everyone should have access to these tools. That's why I'm proposing Project Llama: a public or non-profit sponsored program designed to give everyone access to a powerful conversational foundation model like ChatGPT. To handle the risk of misuse, strict automated moderation and periodic human reviews of selected conversations would be implemented. By offering universal access to AI systems that mimic human interactions, Project Llama aims to level the playing field, helping people adapt to new ways of working and preparing for a changing job market.

Curated reads:

Commercial: ChatGPT plugins

Societal: Will AIs Take All Our Jobs and End Human History—or Not? Well, It’s Complicated

Technical: Sparks of Artificial General Intelligence: Early experiments with GPT-4

Subscribe now

GPTs are GPTs

OpenAI’s Generative Pretrained Transformers (GPTs) models is a specific class of foundation models, which are intelligent adaptable models comparable to humans. General Purpose Technologies (GPTs) are technologies that affect entire economies and meaningfully alter how have we live. For a technology be considered a GPT, it has has to meet three criteria: pervasiveness throughout the economy, ability to spawn complementary innovation, improvements over time.

Criteria 1: Pervasiveness throughout the economy

A technology must have a widespread impact on the economy, being adopted by various industries and sectors, and influencing the way businesses and individuals operate. The internet is a prime example of a pervasive technology. It has transformed various industries, from retail and finance to healthcare and education. It has also changed the way we communicate, access information, and consume media, thus permeating nearly every aspect of our lives.

ChatGPT, the conversational medium that is powered by GPT models, became the fastest growing consumer app of all time. In two months, the app grew to 100M monthly active users (MAUs) by January 2023. Tiktok, the previous record holder, took 9 months to get 100 million MAUs. While we have yet to see if ChatGPT will reach the 1.2 billion MAUs that Tiktok has today, the underlying GPT models will be more pervasive since it is available as an API for any developer to use. GPT-4 already powers the AI chat + search capabilities of Bing, which has 100 million daily active users.

Beyond just becoming an end-consumer product, foundation models will become pervasive in our jobs too. Separate studies by OpenAI and Goldman Sachs economists analyzed which tasks are automatable. Since jobs are bundles of tasks, researchers are able to estimate what percentage of our jobs are impacted. Both groups reached the same conclusion that there will be massive repercussions across the labor market. We’ll go through OpenAI’s study because it is more nuanced. First, it studies the impact of large language models (LLMs), like ChatGPT, specifically and not artificial intelligence as a vague broad category. Second, it not only estimates the impact of LLMs but also of LLM-powered software. This includes software that can fetch web search results like Bing AI or generate images with text like DALL-E. The findings are astonishing as they are worrying.

• 80% of the U.S. workforce have at least 10% of their work tasks affected

• 20% of workers may see at least 50% of their tasks impacted

• With access to an LLM, 15% of all worker tasks could be completed significantly faster at the same level of quality

• With LLM-powered software, this share increases to 50% of all tasks.

Criteria 2: Ability to spawn complementary innovation

A technology should be able to catalyze the development of other innovative technologies, products, or services. The smartphone is an example of that has spawned complementary innovations. Its invention led to the development of mobile applications, mobile advertising, and various other services like ride-sharing and mobile payments.

The first smartphone, iPhone 1, was released in June 2007 and sold 6 million units in its first year. On its anniversary, Apple launched the 2nd generation iPhone along with the App Store. For the first time, users can experience iPhone native apps. Developers were creating novel experiences in the new interface. This led to a breakout quarter for Apple which sold 7 million iPhones in 3 months. It was the inflection point for mobile apps which set the foundations for iconic products that we love today - Uber, Doordash, and Google Maps.

OpenAI launched an app store-esque set of plugins that work with ChatGPT two weeks ago, when it already has over 100M active users. The plugins connect ChatGPT to other applications gives it more practical capabilities from booking flights via Expedia, shop via Instacart, or perform accurate calculations via Wolfram. Developers can build also their own plugins much like how iPhone developers can build iOS apps. The difference with the App Store analogy is that ChatGPT has 100x more momentum.

The aftereffects won’t be limited to our personal lives. Soon billions of working professionals will have access to copilots for their jobs. Businesses are already using Microsoft’s Office Copilot and Google is racing to release its own Workspace Copilot.

Criteria 3: Improves over time

A technology should demonstrate a pattern of continuous improvement and evolution over time. This means that the technology should not only become more efficient and effective but also more accessible and affordable to a larger user base. Computing power is a good example of a technology that has shown continuous improvement over time. With the advent of Moore's Law, which predicts that the number of transistors on a microchip doubles approximately every two years, computers have become increasingly more powerful, efficient, and affordable. This has enabled the development of more complex software and applications.

There are two components to improvement. One is performance. Another is the cost per performance. Both are progressing at rapid pace for foundation models. GPT-4 significantly outperforms GPT-3.5, which was released just over a year ago, and also surpasses human performance on academic exams that people typically spend the first 20 years of their lives preparing for.

The cost of training and hosting your own model has gone down substantially because of algorithmic and hardware improvements. GPT-3 costed OpenAI $5 million to train in 2020. Today, a similar model can be trained for just 10% of the original cost.

While observing AI communities on Discord, I've noticed that beyond the advancements in algorithms and hardware, there is an even more significant factor: the passion of open source communities dedicated to making foundation models widely available for everyone. To reinforce that point, Databricks just open sourced a ChatGPT clone, charmingly called Dolly, that can be trained by using a $30 machine in just three hours.

GPTs foster generational companies

Now that we’ve established that GPTs are GPTs, so what? General purpose technologies create an environment that allows generational companies to emerge:

1. New massive markets: GPTs give rise to entirely new markets and applications, offering generational companies the opportunity to establish a strong competitive advantage and benefit from the first-mover effect. Google (Market cap: $1.3 trillion) significantly impacted the way people access information by introducing a search engine that facilitated easier access to online content. By indexing and organizing web content, Google created new opportunities for different forms of online advertising from replacing top search results with paid ads to infuriating commercials in between videos.

2. Complementary innovations: GPTs spur innovation across various industries, allowing generational companies to leverage their commitment to R&D and develop novel products and services that disrupt existing markets. Apple (Market cap: $2.5 trillion) has been at the forefront of creating iconic consumer products like the iPhone, iPad, and Apple Watch. These devices created an ecosystem of mobile applications and services that have shaped the way we communicate and live - ride haling with Uber, food delivery with DoorDash, and digital nicotine with Tiktok.

3. Standardization: The widespread adoption of GPTs often leads to the formation of industry standards, enabling generational companies to shape these standards and ensure their offerings remain relevant and compatible with the broader ecosystem. Microsoft (Market cap: $2.2 trillion) shaped the world of operating systems and productivity tools with Windows and the ubiquitous Microsoft Office suite. The standardization of Windows OS enabled generations of developers to build higher-level applications and turned countless users into masters of Word's arcane formatting mysteries, Excel's labyrinthine formulas, and PowerPoint's hypnotic slide transitions, ultimately revolutionizing how we work, procrastinate, and communicate in colorful pie charts.

Is this just all hype? There is certainly a lot of it and skepticism is healthy. But the progress of foundation models is real. GPT-powered products created in the past 3 months are more practically valuable than what crypto has produced over the past 3 years.

The flipside

While I am an optimist and had never worried about technological progress, foundation models, specifically GPT-4, has. GPT-4 is accessible via an API, making it an infinitely replicable superhuman as long as the computer servers are up. GPT-4-powered software can automate 50% of our jobs. It is also smarter than 90% of the population based on academic exams that we have labored over for the first two decades of our lives. So for the past few weeks I’ve been gnawing on the question - are we fucked? I’ll untangle this in the next essay.

All this is to say that we are at the pivotal moment of GPTs becoming GPTs, with its pluses and minuses. This is why I'm writing Generational.

Curated reads:

Business: Keep your AI claims in check

Society: Pause Giant AI Experiments: An Open Letter

Academic: GPTs are GPTs: An early look at the labor market impact potential of large language models

Subscribe now

Special thanks to Ashish Kakran for giving feedback on a draft version!