When Diversification Fails: Why the Allocation Value of Gold Is Systematically Underestimated in the AI Era

Have you ever thought about this: you buy an S&P 500 index fund and, on paper, you hold 500 companies across technology, financials, healthcare, consumer, and energy, so it feels very well diversified. But if you open the latest weighting table, you’ll find that the top ten companies already account for about 41% of the entire index’s market cap. And most of these ten are effectively betting on the same thing: AI.

Source: RBC

What exactly does this 41% number mean?

We can compare it horizontally through history. From roughly 1980–2010, the top ten weights mostly hovered around 20%; even at the peak of the 2000 dot‑com bubble, they were only about 29%. By 2025, this has approached 41%, clearly a record high.

This is where this article begins. The goal is not to predict a crash, but to clarify one thing: after AI emerged, the underlying structure of the market quietly changed, and this change directly affects where your money actually goes, what you’re really betting on, and whether gold is now an asset worth re‑examining.

If you’re already a long‑term gold holder, this article can help you articulate the “why” more clearly. If your impression of gold is still stuck at “you only buy it in turbulent times,” you may find by the end that this intuition needs an update.

How Passive Investing Puts Everyone on the Same Boat

To truly understand the concentration issue, we first need to note: this concentration didn’t form naturally. It’s been pushed forward step by step by the mechanism of passive investing itself.

The S&P 500 is a market‑cap‑weighted index. The rule is simple: the larger a company’s market cap, the higher its weight in the index. Every new dollar flowing into an index fund is allocated according to these weights. The more inflows, the more the big companies receive; the more they receive, the higher their stock prices; the higher the prices, the larger the market caps; the larger the market caps, the higher their index weights become—then the next dollar that enters is allocated again based on this new set of weights.

This is a fully automatic positive feedback loop. In an era without a dominant narrative, that loop was relatively evenly spread across different sectors, and didn’t become too concentrated in any one place. But once the market formed a broad consensus that these few mega‑caps are the big AI winners, the loop started pushing an increasingly disproportionate amount of passive capital into the same cluster of companies.

The numbers tell the story: by 2025, ETF assets under management have surpassed $19 trillion, growing more than 30% year‑on‑year. Roughly half of U.S. mutual fund assets are now in passive vehicles, and since about 2010, around 80% of every dollar entering U.S. equities has come through the three major passive giants (Vanguard / BlackRock / State Street). These flows don’t make judgments and don’t ask whether the price is reasonable; they just allocate according to weight. Whichever company is already the most expensive gets the largest share.

The core advantages of passive investing—low fees, low turnover, and avoidance of emotional decision‑making—are all real and have helped countless investors over the past two decades. But the mechanism has a side effect: it doesn’t ask whether a company is worth its price; it only asks how big it is right now. When the majority of marginal capital in the market is passive, this lack of concern for price makes valuation self‑correction progressively harder.

This is also why, in recent years, many people feel that “I thought I bought the broad market, but my returns seem to be almost entirely driven by a handful of AI stocks.” The outperformance of the market‑cap‑weighted S&P 500 versus the equal‑weight S&P 500 has already exceeded the highs seen during the 2000 dot‑com bubble. The more market‑cap weighting outperforms equal‑weight, the more it means that the index is dominated by a small number of giants.

Source: RBC

So, each time you dollar‑cost average into an index fund every month, you think you’re doing no market timing, no stock picking, long‑term steady investing. But in reality, the moment that money enters the market, it is automatically pushed by this mechanism into a concentrated bet that AI will succeed. Not because you actively chose that bet, but because that’s simply how the system is designed to operate. Understanding this is the first step toward re‑understanding your own portfolio.

Why This Time Deserves More Caution than 2000

At this point you might think: during the tech bubble, everyone also piled into internet stocks, yet in the end people survived—so why worry?

It’s a good question, but there is one critical difference that makes today’s situation more complex than in 2000.

The 2000 tech bubble was mainly driven by active investors. Retail investors and fund managers actively chased internet stocks, and when the bubble burst, they also actively sold, allowing the market to self‑correct through discretionary buy and sell decisions. In that era, most of the money in the market had someone making a judgment behind it.

Today is different. There is now a massive pool of passive capital whose design principle is to keep buying and holding regardless of price. This means that when some companies’ valuations begin to diverge from fundamentals, the capital that should have been flowing out is offset by mechanical inflows from passive vehicles. The power of self‑correction is weakened. When a genuine correction finally comes, both the magnitude and speed of the decline may exceed historical experience, because the mechanism supporting today’s concentration is deeper, broader, and more systemic than in 2000.

Another difference: most dot‑com stocks back then were bought purely on the story that “the internet will change everything.” Many companies weren’t even profitable; the valuations were supported by narratives alone. Today’s situation is not the same: the leading AI companies are genuinely profitable, genuinely growing, with solid financial numbers. This makes many people feel that “this time is different,” that current valuations are fundamentally justified.

But even profitable companies can be overvalued. For valuations to be too high, you don’t need the company to perform badly—you only need the market’s expectations for the future to run too far ahead of reality, and that alone is enough to trigger a correction. History is full of such cases: companies that were ultimately very successful over the long run, yet investors who bought at the wrong time waited a decade just to break even. This has happened with the internet, railways, and even wireless radio.

Since 1970, whenever index concentration has sat near historic highs, it has often been followed by a period of mean reversion. For example, after the dot‑com bubble, the S&P 500 fell nearly 50% from peak to trough; in 2021–2022, after a highly concentrated tech bull market, the Nasdaq declined more than 33%. This is not a hard law, but it offers useful context: for decades, the top‑10 weights of the S&P 500 mostly hovered around 20%, and today we’re above 40%. That implies that if the index eventually reverts back toward its typical historical range, the potential “give‑back” space this time could be larger than in prior concentration peaks.

Another Layer of Risk You May Not Have Considered

So far, we’ve discussed the problem with your investment portfolio. But there’s another layer of risk related to your income, and the combination of these two is what reveals the true scope of the issue.

Most people build wealth in a similar way: they earn wages from work, save a portion of those wages, then invest those savings and let capital work for them. This path rests on a fundamental assumption: labor income is stable or even growing, and you have a steady stream of cash flow to keep investing.

AI is destabilizing this assumption, and in a direction many didn’t expect.

In this AI wave, the jobs most affected are not low‑skill blue‑collar roles but white‑collar knowledge work. That’s because AI is strongest at cognitive tasks: text processing, data analysis, content production, and customer communication—precisely the domains of mental labor. A 2026 study by BCG estimates that over the next 2–3 years, 50–55% of jobs in the U.S. will be reshaped by AI, and these jobs are concentrated mainly among mid‑income white‑collar positions. A study by the University of Pennsylvania and OpenAI similarly finds that high‑income, highly educated knowledge workers face higher AI exposure risk than low‑skill workers.

Source: WhatJobs

A 2026 economic index report by Anthropic gives a more concrete sense of speed: 49% of roles now have more than one‑quarter of tasks handled by AI, versus just 36% a year earlier—faster than most people had anticipated.

Putting these two layers together reveals a troubling symmetry: on one side, your existing assets, via passive mechanisms, are increasingly concentrated in the bet that AI will be delivered on time; on the other, your future labor income, your main source of fresh investable cash, is simultaneously facing rising uncertainty due to AI substitution. Both lines—existing capital and future income—are tightening toward the same risk factor, rather than offsetting each other.

More importantly, this is not a short‑term phenomenon. Historically, industrial restructuring has always lasted more than a decade. That means the standard path of accumulating capital through wages may be under pressure for quite some time. This reality directly affects the answer to a key question: Do I have a portion of assets that truly do not rely on this entire system functioning normally?

Real Diversification Is Not Just Owning More Companies

Now that we’ve laid out the problem, we can talk about the answer.

A common response is: “Then I’ll switch to an equal‑weight S&P 500 ETF, or add more international index exposure. Won’t that give me more diversification?”

This is directionally right, but still not enough. Equal‑weight indices do reduce top‑end concentration, and international diversification helps too—but they mainly address the degree of risk within the same risk category, rather than introducing a fundamentally different risk source. In extreme scenarios, global equity market correlations tend to spike. In past major drawdowns, markets have often fallen in near lockstep. International diversification reduces correlation somewhat, but precisely when you most need protection, markets often move together.

True diversification is not owning more companies; it’s owning assets with fundamentally different sources of value.

It’s worth pausing on this concept, because “diversification” is used too loosely. Many people think buying more ETFs equals diversification. But the real meaning of diversification is that your different assets won’t all move in the same direction when the same piece of bad news hits.

• Stocks derive value from corporate earnings and market expectations of future earnings.
• Bonds derive value from the issuer’s ability and willingness to make timely interest and principal payments.
• Real estate derives value from rents and supply–demand dynamics.

These value sources are indeed distinct, but under system‑level stress, they share a fragile common premise: they all depend on the system functioning normally. Companies need revenue, credit needs backing, and economic activity must continue. If these conditions fail, all three get hit—only to different degrees.

Gold’s value driver is different from all of the above. It doesn’t depend on any company’s earnings, any government’s credit, or whether AI rolls out as expected. It doesn’t produce cash flow, and it requires no one to honor any contract. Its value rests on thousands of years of human history, across repeated cycles of currency crises, economic collapses, and regime changes, after which it is still treated by global markets as a medium for storing and transferring value.

Statistically, gold’s long‑term correlation with U.S. equities is close to zero, and in short windows during crises (e.g., severe drawdowns), it often shows slightly negative correlation. This is no accident; it stems from entirely different value‑driving logics.

Why This Argument Is More Structural in the AI Era

One question remains: if the case for holding gold has always been valid, why say the argument is upgraded in the AI era?

The answer lies in the logical chain we just built.

An asset’s diversification value is not absolute but relative—it depends on the risk structure of the rest of your portfolio. The allocation value of gold is directly proportional to how concentrated your other assets are. When your stock portfolio is spread across many sectors without any dominant thematic bet, gold’s incremental value is modest because you’re already fairly diversified. But when your equity exposure, via passive mechanisms, becomes highly concentrated in a single theme, an asset whose value is almost independent of that theme sees its marginal usefulness rise sharply.

A late‑2025 BlackRock report noted that gold’s 2025 annual return was about 61%, the strongest year since 1979, yet among the tens of thousands of adviser‑model portfolios they track, only about 20% had any allocation to alternative assets at all, and those allocations were generally small. Gold has just delivered its strongest performance in decades, yet most investors’ portfolios remain severely under‑allocated to it.

来源：BlackRock

Why? Very likely because most people’s mental framework for gold is still stuck on “only buy it in troubled times.” When there is no perceived crisis, they feel gold isn’t necessary. When markets actually get into trouble, they panic‑buy it at high prices, often at poor entry points. This is a emotion‑driven allocation logic, not a structure‑driven one.

The structure‑driven logic is as follows: today, market concentration sits at historical extremes, driven by a systemic passive‑investment mechanism whose inertia is much harder to reverse than active buying. At the same time, labor income—your incremental cash‑flow stream—is under long‑term structural pressure from AI. With both conditions in place, the allocation value of owning an asset whose correlation with this system is near zero is as compelling as it has been at any time in recent decades.

This is what it means to say that the rationale for holding gold has been “upgraded.” Not because inflation suddenly surged, nor because a particular geopolitical crisis erupted, but because the market structure itself has made gold’s role as a true diversification tool more irreplaceable than before. Crucially, this logic does not depend on whether AI ultimately succeeds or fails. However the AI story ends, the structural diversification case still holds.

A Simple Way to Stress‑Test Your Portfolio

After all this theory, let’s end with a practical self‑check.

First, make a rough estimate of your AI bet density. Categorize your assets based on how they would react if AI delivered far less than expected:

• Holding U.S. broad‑market ETFs? Through the mechanisms described earlier, they are highly correlated with the AI theme.
• Holding tech or semiconductor funds? Even more directly tied.
• Your job: if it’s a cognition‑heavy white‑collar role—copywriting, analysis, finance, legal, customer service—then your future labor income is also part of the same bet.

Add up all these exposures and then look at the other side: how many assets can survive regardless of how the AI narrative plays out? Cash and bonds count a bit, but returns are low. Gold is a more significant piece because its value drivers have almost no overlap with the AI theme.

If, after this classification, you find that the second category is almost empty, then you’ve discovered your true situation: you’re not a diversified investor owning 500 companies; you’re a high‑conviction thematic investor deeply concentrated in a single bet, just wrapped in something that looks diversified.

Next, ask yourself: is my reason for holding (or planning to hold) gold emotional or structural?

• An emotional reason sounds like: “The news looks scary lately; I feel something bad might happen, I’ll buy some gold for safety.” Under this logic, you tend to slowly sell gold when markets are calm, and chase it at the top when fear spikes. After a few cycles, your experience with gold may be poor—not because gold is bad, but because your buy‑and‑sell timing is emotion‑driven.
• A structural reason is: “I understand that today’s market concentration is historically extreme, driven by systemic mechanisms. My other assets are already heavily tied to this structure, so I need something outside this bet to achieve real diversification.” Under this logic, when markets are calm, you won’t dismiss gold for “not going up,” because you know exactly what role it’s playing.

This difference in reasoning looks subtle but profoundly affects how you behave when markets are volatile. An investor with a structural rationale won’t panic just because gold doesn’t rise in a risk‑on rally, nor will they be shaken by short‑term pullbacks in gold, because the underlying logic hasn’t changed.

Conclusion: Real Diversification Doesn’t Wait for Crisis Headlines

To wrap up:

• In the AI era, passive investing mechanisms have channeled enormous capital into the same bet, pushing market concentration to record highs. Your seemingly diversified portfolio is likely far more concentrated than you realize.
• At the same time, AI’s substitution impact on knowledge work is eroding the main income source of many middle‑class investors from another angle; both existing capital and future income are converging on the same risk.
• In this structure, the allocation value of a truly low‑correlation asset is higher than in most historical periods, and gold is one of the best candidates that fits this description.

These points aren’t predictions about the timing of the next crisis, nor a claim that AI is doomed to fail. They assert a more fundamental investment principle: when your portfolio, because of market structure, has unknowingly taken on a historically rare concentration bet, the value of genuine diversification tools rises accordingly.

In many people’s mental models, gold still belongs to the category of “something you buy only in chaos.” But chaos never shows up on the calendar in advance so you can prepare. Once it arrives and everyone sees it, the price you can buy at is rarely cheap.

The best kind of diversification is always done before you feel you need it.

The content above is for investor education and reference only and does not constitute any investment advice. All data cited are from public sources, and past performance does not guarantee future results.