The (Mis)uses of the S&P 500

I. Introduction

The S&P 500 Index1 is among the most widely recognized and widely used stock market indices. Many of these uses are intertwined with the securities regulatory regime. Trillions of dollars are invested so as to track the Index, much of it invested by highly regulated mutual funds and ETFs.2 Mutual funds are required to report their past performance relative to a benchmark index, and the overwhelming favorite is the S&P 500. Firms commonly describe their financial performance in terms of how they did relative to the Index, and the SEC requires S&P 500 constituent firms to disclose their performance relative to the Index in their 10-Ks.

The S&P 500 is generally viewed as being a passive representation of “the market.” In fact, it simply mimics the performance of one particular portfolio of large cap domestic equities. There is nothing particularly neutral or universal about the portfolio that the Index represents: it is the result of discretionary decision-making on the part of its creators, and its constituents change over time. Having seen this, it becomes apparent that the way that the securities regulatory regime engages with the use of the Index in each of “passive” investing, mutual fund performance benchmarking, and firm performance evaluation, is fundamentally flawed.

In the empirical portion of this Article, I make the point that the S&P 500 simply captures the performance of one particular large cap portfolio by establishing two facts. First, I quantify the extent of the discretion involved in the construction of the Index and estimate the implications of this discretion for its behavior. Using two different approaches—both of which are intentionally conservative—I show that the discretionary portion of the S&P 500 represents roughly 5% of the total value of the Index. Given the aggregate size of firms on the index at the end of my sample period, this conservative estimate of 5% represents nearly $1 trillion in market capitalization. Second, I quantify the impact of constituent turnover and show that it has a substantial effect on the performance of the Index. Failure to recognize the impact of changes in the composition of the Index can render comparisons against the Index misleading.

While the empirical findings are simple, they are either being ignored, misunderstood, or overlooked. I next turn to three important uses of the Index and draw upon my empirical analysis to illuminate the fact that the securities regulatory regime is misusing the Index in all three contexts. First, despite the fact that the construction of the Index involves a substantial amount of discretionary decision-making, funds that track the performance of the Index are routinely described as “passive.” This use of the term “passive” is confusing and potentially misleading to investors. Moreover, notwithstanding the fact that the Index effectively directs how the money in these funds will be invested, the SEC does not regulate it as an investment adviser under either the Investment Company Act3 or the Investment Advisers Act.4 At the very least, the SEC should formally acknowledge this differential treatment, and take steps to ensure that investors understand the risks associated with it.5

Second, because the Index just represents a portfolio chosen by one particular group of financial market professionals, there is no reason to think that it is the “right,” or even a sensible, comparator for any particular mutual fund. And because the portfolio changes over time, it is not even the case that the group of stocks being used for that comparison is constant. While comparisons to the Index provide a patina of objectivity, a much better, and more honest, approach would be to compare mutual funds to their competitors.

The third use—the fact that the constituent firms are required to compare their performance to that of the Index—is even more problematic. Not only is there no particular reason to expect a firm’s performance to mirror that of the Index, the turnover in the Index’s constituents means that firms are forced to make nonsensical retrospective comparisons. This requirement should be abolished.

It is hard to overstate the importance of the S&P 500 to modern equity markets. Its outsized importance has been recognized in the academic literature since at least the mid-1980s, when Professor Andrei Shleifer demonstrated that stocks tend to jump after being added to the S&P 500.6 While several other articles have explored this issue,7 more than thirty years later the so-called “index inclusion” effect remains an active area of research in the finance literature.8 Another recent paper has shown that the way the S&P 500 index is typically displayed has systematic effects on financial markets,9 which further underscores the influence of the Index.

There is also a voluminous recent literature focused on the rise of index funds, much of which is, implicitly or explicitly, focused on funds that track major indices like S&P 500.10 Up to now, this literature has largely assumed that these funds simply track “the market,” and has focused on how the law should respond to this new reality. This paper contributes to this literature by demonstrating that S&P 500 funds involve a non-trivial amount of discretionary management. To the extent that scholars are concerned about the implications of the rise of such funds, my findings indicate that the analysis should begin at the index stage, rather than at the fund stage.

The remainder of this Article proceeds as follows. In Part II, I discuss three contexts in which the securities regulatory regime engages with the Index. I present my empirical analyses in Part III. In Part IV, I return to the three uses of the Index, and use these empirical results to argue that each one is in fact a misuse. Part V concludes.

II. Three Uses of the Index

A. “Passive” Investing

A first common use of the S&P 500 is for “passive,” or “index,” investing. There is a large and complex regulatory architecture that surrounds mutual funds, which is grounded in the 40 Act and the Advisers Act. Together, these statutes and the regulations promulgated by the SEC thereunder regulate both the structure and operation of mutual funds. Moreover, because the shares of mutual funds and ETFs are “securities” for the purposes of the securities laws, their issuance and trading are governed by the Securities Act of 1933 and the Securities Exchange Act of 1934.11

In contrast to an actively managed mutual fund, an index fund is a mutual fund in which the fund manager’s goal is to track some underlying index as closely as possible.12 Because of this goal, the asset allocation decisions of an index fund are made, for all practical purposes, by the index provider rather than the fund’s formal investment adviser or subadviser (colloquially known as the “fund manager”). The S&P 500 is the dominant player in the U.S. index fund market, directing over $7 trillion of investors’ money in 2022.13 To put this number in context, if it were a single mutual fund, it would represent almost the entire value of long term assets managed by BlackRock14 or Vanguard.15 And yet unlike BlackRock, Vanguard, or the investment adviser to any mutual fund, index providers like S&P Dow Jones LLC—the legal entity which owns and creates the S&P 500—are not subject to any regulation under the securities laws. In particular, unlike a traditional mutual fund manager, S&P Dow Jones LLC is not regulated as an investment adviser under either the Advisers Act or the 40 Act.

One possible rationale for this is that the S&P 500 is not engaged in any meaningful security selection. This idea is consistent with the notion of “passivity,” and the idea that an index fund does not involve decision-making is reinforced by the label “passive” that generally accompanies these funds. Whereas “active” implies intentional decision-making on the part of a fund manager, “passive” implies the absence thereof. While it is obviously the case that someone is picking the securities, for the term passive to have any content at all it must at least imply the absence of intentional individualized stock selection. S&P 500 mutual funds routinely use this language in their marketing material. Based on data from a recent exhaustive study of index mutual funds,16 about 60% of the S&P 500 tracking funds use the term “passive” in the prospectus. This figure reflects the fact that 41 of the 69 index mutual funds that track the S&P 500 in the universe of mutual funds collected for that study use the term “passive” either in describing their investment strategy or as a risk factor of the fund.17 This phenomenon is not restricted to smaller funds: a review of the underlying prospectuses reveals that 7 of the 10 largest funds by assets under management (and 3 of the 5 largest) use this language. A further three funds (out of the original 69) refer to the S&P 500 as an “unmanaged index” or an “unmanaged group.” Like all securities offered for sale in the United States, mutual funds are prohibited from using false or misleading information in their prospectuses or other marketing materials. The SEC routinely reviews mutual fund prospectuses and asks issuers to revise language that it believes may fall afoul of this rule and other regulatory requirements.18

B. Mutual Fund Benchmarking

A second common use of the S&P 500 is as a benchmark. Globally, a mind boggling $15.6 trillion was indexed or benchmarked to S&P 500 as of December 2021.19 Benchmarking is required in the mutual fund context: the SEC requires funds that have annual returns for at least one calendar year to report their returns alongside the returns of “an appropriate broad-based securities market index.”20 Around the end of the sample period for the analysis in Part III, over 40% of all dollars invested in US-focused equity mutual funds used the S&P 500 as their primary prospectus benchmark, making the Index by far the most popular benchmark and dwarfing its closest competitor.21

The SEC suggested two related rationales for the index benchmarking requirement in its proposing and adopting releases: to “provide investors with an objective standard against which they can compare the performance of the fund,” and, more specifically, to allow them to compare the performance of the fund to that of “the market.”22 The latter rationale was tied directly to the performance of fund managers: the requirement was “designed to show how much value the management of the fund added by showing whether the fund ‘outperformed’ or ‘under-performed’ the market.”23 While the adopting release explicitly rejected the suggestion that funds should instead compare themselves to other funds,24 the proposing release nevertheless hinted at the idea that the “broad market index” essentially represents a sort of outside option for investors.25 The final rule, as codified in the Federal Register, contains a sample figure illustrating how the comparison is to be made. In that figure, the comparison index is the S&P 500.26

C. Evaluating Corporate Performance

A third and final common use of the S&P 500 is as a means of evaluating individual firms, or the managers of such firms. Regulation S-K requires public companies to include a performance graph in their 10-K Annual Reports comparing the company’s total return to that of “a broad equity market index . . . that includes companies whose equity securities are traded on the same exchange or are of comparable market capitalization.”27 Moreover, if the company is a constituent of “the Standard & Poor’s 500 Stock Index, [it] must use that index,”28 as well as the returns of a published index or group of similar firms.29

This requirement was initially added as part of a package relating to executive compensation disclosures and was “[d]esigned to furnish shareholders with a more understandable presentation of the nature and extent of executive compensation.”30 This change was in part motivated by the fact that shareholder return was a commonly used metric in assessing corporate, and presumably by extension executive, performance.31 The SEC’s rationale for mandating the use of the S&P 500 as a benchmark for constituents of the Index was to “enhance inter-company comparability.”32

This executive compensation disclosure requirement was rolled into the section on general performance in 2006. At that time, the SEC proposed eliminating the requirement entirely, but retained it because of comment letters indicating that it “provides an easily accessible visual comparison of a company’s performance relative to its peers and the market, and provides a standardized source for this type of information.”33 Other than this change in location, the substantive aspects of this requirement have not changed since 1992.

III. Empirical Analysis

In this Part, I establish a very simple fact: that the S&P 500 simply captures the performance of one particular portfolio of large cap domestic equities. I do so by focusing on two features of the Index. First, its construction involves the exercise of a substantial amount of discretion by a particular group of financial market professionals. In Section III.B, I quantify the extent of the discretion involved in the construction of the Index and show that it is substantial. Second, its constituents change substantially over time. In Section III.C, I turn to the effect of these changes in composition and show that they substantially affect the performance of the Index. I begin in Section III.A by discussing the details of the methodology used in constructing the S&P 500.34 While the details of the methodology that the S&P employs are freely available on its webpage,35 they are not well understood.36 Later, in Part IV, I return to the three common uses of the Index, and show that once we recognize these fundamental features of the Index, these uses look more like misuses.

A. Details of the S&P 500

S&P Dow Jones Indices (“S&P”) aims to include about 500 securities on the S&P 500 at any given time.37 However, rather than mechanically selecting, for example, the 500 largest securities in some universe, the methodology defines certain thresholds such as size, liquidity, and financial viability. For example, the rules regarding size in effect as of April 2018 require “[u]nadjusted company market capitalization of US$ 6.1 billion or more,”38 and the liquidity threshold involved a two-part test: “the ratio of annual dollar value traded (defined as average closing price over the period multiplied by historical volume) to float-adjusted market capitalization should be at least 1.00, and the stock should trade a minimum of 250,000 shares in each of the six months leading up to the evaluation date.”39 Securities that satisfied these thresholds were then eligible to be considered for addition to the Index. Other features, such as whether the security satisfies the methodology’s domicile or listing requirements, also operated as eligibility criteria.40 Together, these rules defined the consideration set from which securities can be chosen.

While these criteria seem fairly mechanical, the reality may be more ambiguous. For example, some of the thresholds are phrased as “should” rather than “shall” or “must,” making it unclear to what extent the cutoff is binding.41 Moreover, S&P emphasizes that these are rules for addition, not necessarily for continued inclusion.42 The rationale for this is to reduce the amount of turnover on the Index, something that is viewed as being undesirable. Instead, according to the methodology, “an index constituent that appears to violate criteria for addition to that index is not deleted unless ongoing conditions warrant an index change.”43 This provision gives the Index Committee discretion to decide if and when to remove a constituent, and, by extension, if and when to change the composition of the Index.

An additional element of discretion is introduced in the constituent selection step. The methodology explicitly states that “[c]onstituent selection is at the discretion of the Index Committee and is based on the eligibility criteria.”44 For example, one factor the Index Committee  considers is “[s]ector balance, as measured by . . . GICS sector.”45 Frequent changes to the Index methodology compound the effect of discretion further: between January 1, 2015 (the first date for which the change log is available)46 and April 2018, the S&P 500 methodology changed at least eight times.47 The power to change the methodology is itself vested in the Index Committee.48

B. Discretion

There are several different ways that one could conceptualize the discretion available to the Index Committee. One approach is to take the methodology as given and focus on the amount of discretionary decision-making that it gives the Index Committee. A second approach is to treat the rules themselves as a matter of discretion, and to compare the Index to one that is constructed using a very mechanical and transparent set of rules.49 Because they capture different dimensions of discretion, I consider each of these approaches in turn. Under both approaches, I find that the construction of the Index involves a substantial amount of discretion, representing roughly 5% of the aggregate value of the securities on the Index.

1. Discretion Based on Eligibility and Inclusion Rules

I begin by estimating the amount of discretion that the Index Committee has pursuant to the Index methodology.50 To do so, I determine, as closely as possible using publicly available data, the number of securities that satisfy the inclusion criteria on each of the 735 trading days between January 1, 2015 and December 31, 2017. The larger the set of eligible securities, the more choice the Index Committee has in selecting the constituents of the Index. I have used the most conservative means available in constructing my proxies for these criteria, the details of which are provided in Appendix A-II.A.51 In order to properly capture the amount of discretion available to the Index Committee, I add to this set the securities that do not satisfy these constructed criteria but are actually on the Index on the day in question. The reason for this choice is simple: either these securities really are eligible and my constructed criteria are underinclusive, or the committee has chosen to keep them on the Index despite the fact that they do not satisfy the criteria.52 Either way, the fact that they are on the Index means that, by definition, they are within the universe of securities that could be on the Index.

Panel A of Table 1 summarizes the results of this exercise. On the median day, 615 securities (with a mean of 601) are eligible for the Index. Given that the Index normally contains about 500 securities, this indicates that, on the median day, there is a surplus of over 100 securities that could be on the Index but are not, representing over 20% of the total number of securities on the Index.

This table presents summary statistics related to the securities eligible for inclusion on the S&P 500 on each of the trading days between January 1, 2015 and December 31, 2017. Panel A presents summary statistics of the distribution of the number of securities eligible for inclusion in the S&P 500 on each of the trading days. Eligible securities include securities that satisfy the criteria constructed from the Index methodology as well as the securities that were actually on the Index that day, regardless of whether or not they satisfy the constructed criteria. Panel B presents summary statistics of the distribution of the average, median, and total size of securities on the S&P 500 on each of these trading days, as well as securities eligible for inclusion but not included.

Panel B of Table 1 provides some additional summary statistics relating to both the securities on the Index and those securities that, while eligible for inclusion, are not on the Index.53 While these securities are generally smaller than those that are included on the Index, they are still sizeable. On the median day, the average size of securities eligible but not included on the Index is about 21% of the size of the securities on the Index.54 Combined, on the median day these securities represent about 5% of the total market value of the securities on the Index.55

This analysis illustrates two features of the S&P 500. First, the analysis helps to quantify the extent to which the Index’s current rules empower the Index Committee to pick and choose the securities that ultimately make up the Index. The results in Panel A of Table 1 demonstrate that this discretion is substantial. One way to interpret the values in Panel A is that, should the Index Committee decide that it wants to remove a particular security from the Index, it has, on average, over 100 securities to choose from to replace it, which together represent about 5% of the total market value of the Index. A second, related, implication of this analysis is that the Index simply represents one particular large cap portfolio that is constructed in accordance with the decisions made by the Index Committee. It could just as easily have made different decisions and constructed a substantially different portfolio. This portfolio would contain different securities, and would, as a result, generate different returns.

2. Comparison to Alternative Construction Rules

A second approach to estimating the amount of discretion implied by the Index is to compare the constituents of the Index to a group of securities selected using an alternative methodology.56 While one could use almost any group of securities for this comparison, the most natural choice is the 500 largest securities in the Center for Research in Security Prices (CRSP) database, selected on an annual basis.57 This choice represents a very simple, mechanical, and transparent methodology. In effect, this exercise is the equivalent of asking: “suppose that instead of constructing the S&P 500 the way it is actually constructed, the committee just picked the 500 largest stocks in CRSP. How much difference would this make?”

To implement this approach, I begin by identifying, on each trading day between January 1, 1989 and December 31, 2017, two groups of securities: (1) the 500 largest securities in CRSP, as measured on the last trading day of the prior year,58 and (2) the constituents of the S&P 500. I then investigate the overlap between these two groups. The larger the overlap, the greater the similarity, ex post, between the S&P 500 selection rule and the mechanical selection rule (the “mechanical index”). The results are presented in Table 2. For ease of interpretation, I present both the average number of securities per day in each of the three populated quadrants, as well as the average and median market capitalization, and the average total market capitalization of these securities.59 I also report the average and median size of the securities on the Index in the caption.

Table 2: Characteristics of S&P 500 Constituents v. Constituents of Mechanical Index

This table presents summary statistics related to the constituents of the S&P 500, as well as the securities selected using a mechanical rule that chooses the largest 500 securities in CRSP for the period from January 1, 1989 through December 31, 2017. The mechanical index is constructed using the 500 largest securities in CRSP on the last trading day of the prior year. The first entry in each quadrant presents the average number of securities in that quadrant. This is computed by counting the number of relevant securities on each day, and then averaging these values across days. The second entry presents the average size of the securities in that quadrant. This is computed by first calculating the average size of the relevant securities on each day, and then averaging these values across days. The third entry presents the median size of the securities in that quadrant. This is computed by first calculating the median size of the relevant securities on each day, and then averaging these values across days. The fourth entry presents the average total value of the securities in that quadrant. This number is computed by first calculating the total size of the relevant securities on each day, and then averaging these values across days.

To be sure, the securities that would have been selected for the mechanical index but are not on the S&P 500 are smaller, on average, than those that are on the Index. This difference, however, is not particularly large: on the average day, the average excluded security is over 60% of size of the average S&P 500 constituent. Perhaps more surprisingly, on the average day, the median excluded security is actually slightly larger than the median S&P 500 constituent ($9.1 billion versus $9.04 billion), a difference that is statistically significant at the 1% level.60 Combined, these securities represent, on average, about 17% of the total market value of the securities on the S&P 500. At the same time, the average security that is included on the S&P 500, despite not being among the 500 largest securities, is substantially smaller—about 18% of the size of the average S&P 500 constituent. Combined, these securities represent about 5% of the total value of the S&P 500.

So far, the analysis in this section has demonstrated that the Index methodology, as implemented by the Index Committee, generates constituents that differ substantially from an alternative methodology that simply selects the 500 largest securities in CRSP. I now investigate the extent to which these differences result in different return behavior. In other words, the analysis so far in this subsection has asked “to what extent do the securities chosen by the committee differ from the purely mechanical alternative?” Next, I ask “what is the effect of these differences on the behavior of returns?”

To answer this question, I construct a value weighted portfolio of the securities selected for my mechanical index.61 When a security disappears from CRSP after the beginning of the year, the proceeds are reinvested according to the weights of the remaining securities.62 I then compute three measures of the difference in performance between my mechanical index and the S&P 500 Total Return Index, obtained from Bloomberg. The first, and primary, measure is tracking error, which is a standard measure in the mutual fund literature.63 The tracking error between two return series is defined as the standard deviation of the difference between the performance of the two series.64 In this case, the two return series are the return on the S&P 500 Total Return Index and the return on my mechanical index. For completeness, I also compute the mean absolute error and the mean squared error. Because it is difficult to interpret these measures in isolation, I also compute the same measures with respect to the difference in performance between the S&P 500 and the Vanguard 500 Index Fund Admiral Share (VFIAX). The tracking error of the VFIAX relative to the S&P 500 is a measure of the realized gap between the returns of the S&P 500 and the returns of a fund that is seeking to track the Index. The same is true of the mean absolute error and the mean squared error.

One way to interpret this analysis is that it allows me to answer the following question: suppose that the S&P 500 were trying to track my mechanical index. How large is its tracking error? I then compare this tracking error to the tracking error of the VFIAX relative to the index that it tracks (i.e., the S&P 500). I select this mutual fund because it is both one of the largest mutual funds in the world and because, anecdotally, Vanguard has a reputation for offering index funds with both low fees and low tracking errors. As a result, I restrict attention to the period beginning November 14, 2000, the first date for which the returns of the VFIAX are available in the CRSP Survivor Bias-Free U.S. Mutual Fund Database. The results are presented in Table 3.

While the gap between the mechanical index and the S&P 500 is small, Table 3 makes clear that it is substantially larger than the gap between the VFIAX and the S&P 500. For example, the VFIAX tracks the S&P 500 Index about 20 times more closely than the S&P 500 tracks the mechanical index. Using mean absolute error reveals a similar pattern, and the difference is even larger if we use mean squared error. Comparing Panels B and C to Panel A reveals that this pattern is quite stable across time periods.

Nothing in this analysis suggests that there is anything wrong with the S&P 500 construction methodology. Unlike the VFIAX, which is trying to match the performance of the S&P 500, the Index has never purported to track the performance of the 500 largest securities in CRSP. This analysis is also not concerned with the whether the Index “outperforms” or “underperforms” the mechanical index. Rather, it is simply concerned with quantifying discretion. Section III.B.1 showed that the Index methodology provide scope for a substantial amount of discretion in the construction of the Index. This Section showed that the constituents of the Index also deviate substantially from an alternative, purely mechanical index, and that these deviations lead to differences in the behavior of the Index.

Together, these empirical results provide evidence of two related features of the Index. First, they demonstrate that the decision-making involved in the construction of the Index has a substantial effect on the composition of the Index, which in turn affects the behavior of the Index. Second, they demonstrate that the Index is not “neutral” or “universal” in any meaningful sense. Rather, its construction and performance reflect a particular set of decisions made by a particular group of individuals.

Table 3: Measures of Similarity between the S&P 500 Index and the Mechanical Index

This table presents three measures of the deviation between the performance of the S&P 500 Index and the alternative mechanical index, as well as the Vanguard 500 Index Fund Admiral Shares (VFIAX). The mechanical index is a value weighted portfolio constructed annually on the first trading day of each year consisting of the 500 largest securities in CRSP on the last trading day of the prior year. If a security disappears from CRSP during the holding period, the proceeds are reinvested according to the weights of the remaining securities. Tracking error is defined as the standard deviation of the difference between the return of the S&P 500 Index and the mechanical index (the VFIAX). Mean absolute error is defined as the average of the absolute value of the difference between the return of the S&P 500 Index and the mechanical index (the VFIAX). Mean squared error is defined as the average of the square of the difference between the return of the S&P 500 Index and the mechanical index (the VFIAX). The Ratio row presents the ratio between the relevant value for the mechanical index and the VFIAX. Panel A presents the results for the full sample period (Nov. 14, 2000 – Dec. 31, 2017). Panel B presents the results for the first part of the sample period (Nov. 14, 2000 – Dec. 31, 2009). Panel C presents the results for the second part of the sample period (Jan. 1, 2010 – Dec. 31, 2017). The difference between the mechanical index and the VFIAX for each of these measures in each of the three time periods is statistically distinguishable from zero at the 0.001% level.

C. Composition Changes

The constituents of the S&P 500 Index change over time. In this section, I consider two different approaches to quantifying the extent to which changes in its composition affect its performance: an ex ante approach, which focuses on the securities on the Index at the initial date, and an ex post approach, which focuses on the securities on the Index at the terminal date. These two approaches provide useful benchmarks for evaluating different uses of the Index.

1. Ex Ante Constituents

The purpose of this analysis is to compare the performance of the S&P 500 to a counterfactual “frozen” version of the Index. This counterfactual index is the conceptual equivalent of investing $1 in the S&P 500 on the initial date and then completely ignoring the portfolio (with the exception of dividend reinvestment). To the extent that the performance of this counterfactual “ex ante” index differs from that of the S&P 500, this difference is attributable to the changes in the composition of the Index after the formation date. These changes, of course, flow from a combination of the quantitative rules of the Index, as well as the discretion exercised by the Index Committee.

To implement this analysis, I construct a value weighted portfolio consisting of the constituents of the S&P 500 on the first trading day of each of 1990, 1995, 2000, 2005, and 2010. I then hold each of these portfolios passively from the formation date through to December 31, 2017.65 If a security disappears from the CRSP dataset, it is not replaced and the portfolio is never rebalanced. Thereafter, that portion of the portfolio earns a return of zero for the remainder of the period. I then compare the returns on these buy-and-hold portfolios to the performance of the actual S&P 500.

Table 4 summarizes the results of this analysis.66 Unsurprisingly, the portfolio of ex ante constituents drastically underperforms the S&P 500. One simple factor in this underperformance is the fact that the securities that disappear from CRSP are never replaced and are assumed to earn a return of zero after their disappearance.

Table 4: Returns of Portfolio of Ex Ante S&P 500 Constituents and S&P 500 Index

This table compares the return of five different counterfactual buy and hold versions of the S&P 500 Index to the Index itself. “Total Return” under “Portfolio of Ex Ante Constituents” represents the total return of a value-weighted portfolio consisting of the constituents of the S&P 500 as of the first trading day of the Initial Year, assuming that the portfolio is held through to the end of 2017. “Annualized Return” represents the annualized version of the same return. The columns under “S&P 500” represent analogous returns (total and annualized) for the S&P 500 over the same period. 

2. Ex Post Constituents

A second way to investigate the effects of changes in the composition of the S&P 500 is to use an ex post perspective. Here, rather than “freezing” the composition of the Index, I look at the firms that ended up on the Index and compare them to the performance of the Index over a period of time.67 Of course, because it relies on information that was not known at the time of formation, it should be clear that this does not represent a feasible investment strategy. Only the companies that survived and did reasonably well ex post (and are therefore on the S&P 500 at the end of the period) are included in the analysis. Quantifying the extent to which these companies will have, on average, outperformed the Index is part of the point of this analysis, as it demonstrates the extent to which the composition of the Index is itself shaped by the past returns of the present-day constituents.68 To the extent that this seems like a problematic comparison, it is worth noting that this is a comparison that many firms do make, often because they are required to do so. I discuss an example of such a comparison in Section IV.C.

I begin by identifying the constituents of the S&P 500 on December 31, 2017 and follow each of those securities back to the first trading day of 1990, 1995, 2000, 2005, and 2010. I then compare the performance of these securities over the period from that initial date through December 31, 2017 to that of the S&P 500 over the same period.69 If the security is not in the CRSP dataset on the initial date, it is omitted from the analysis. As a result, there are many fewer securities in the early comparisons than there are in the later ones.

For each initial year, I compute the annualized returns of the relevant securities, as well as those of the S&P 500, over the relevant period. I then perform a t-test to test whether the difference between the realized annualized returns of the ex post constituents and those of the Index are distinguishable from zero. The results are presented in Table 5. These results confirm that the difference in performance between the ex post constituents of the Index and the Index itself is large, both statistically and economically.

Table 5: Returns of Ex Post S&P 500 Constituents and S&P 500 Index

*** p<0.001, ** p<0.01, * p<.05, + p<0.1

This table compares the average returns of securities that were S&P 500 constituents as of December 31, 2017 over different horizons to the returns of the Index. Column (1) contains the mean annualized return of the ex post constituents of the S&P 500 (i.e., the constituents of the Index as of December 31, 2017) over the period from the first trading day of the Initial Year through December 31, 2017. Column (2) contains the mean annualized return of the S&P 500 over the period from the first trading day of the Initial Year through December 31, 2017. Column (3) contains the difference between the two. t-statistics from a t-test under the null hypothesis that the mean difference is zero are presented in parentheses.

The results in this section are designed to make a simple point. Given that the constituents of the Index change for both discretionary and non-discretionary reasons, these results serve to quantify the impact of this turnover on the behavior of the Index. While the results are not terribly surprising once one takes the time to reflect on the construction of the Index, they seem to be either overlooked or ignored in many of its uses. I turn to these uses now.

IV. The (Mis)uses of the Index

The empirical analysis in this paper lays bare the fact that the S&P 500 simply captures the performance of a portfolio of large-cap US equities. It also highlights two specific features of this portfolio: its construction involves the exercise of a substantial amount of discretion, and its constituents change over time. I now return to the three uses of the Index outlined in Part II: “passive” index funds, mutual fund benchmarking, and firm performance evaluation. I argue that, in light of my empirical analyses, there are fundamental problems with the way that the securities regulatory regime engages with the Index in all three contexts. Whether this is because the regime has overlooked the facts underlying my analysis or because it chooses to ignore them, recognizing these facts makes it clear that each use is, in fact, a misuse of the Index.

A. “Passive” Investing

I begin with “passive” investing. The use of the term “passive” to describe a mutual fund implies that the portfolio is, in some meaningful sense, not actively managed. If not necessarily constant or mechanical, management of the fund should at least be non-discretionary. And yet my empirical analysis in Section III.B makes clear that the S&P 500 is the product of discretionary decision-making by a group of financial market professionals. The quantification exercises show how broad the scope of this discretion is: even using very conservative measures, discretion determines roughly 5% of the total value of the Index, representing nearly $1 trillion in total market capitalization. What this discretion means for the idea of passive investing is clear: a fund that tracks the S&P 500 is effectively being managed by the S&P 500 Index Committee, and the composition of the fund portfolio is effectively being determined by discretionary decisions made by this Committee.

The case of Nektar Therapeutics represents a concrete example of the Index Committee’s exercise of discretion. Effective March 19, 2018, Nektar Therapeutics was added to the S&P 500, having previously been a constituent of the S&P SmallCap 600 Index.70 This occurred despite the fact that at the time of the addition, based on data in Compustat, Nektar Therapeutics unambiguously failed to satisfy either prong of the financial viability criterion.71 This decision, moreover, was not necessarily inconsistent with the Index methodology that was in effect at that time, since the methodology allowed for the addition of a company that did not satisfy the criterion “if the Index Committee decides that such a move will enhance the representativeness of the index as a market benchmark.”72 Presumably, the Index Committee changed its mind in relatively short order: about a year and a half later, Nektar Therapeutics was removed from the S&P 500 and placed on the S&P MidCap 400 Index.73

The idea that the Index Committee is interested in enhancing the “representativeness” of the Index is consistent with other work. In studying the impact of discretionary changes to the S&P 500—one of the many dimensions of discretion available to the Index Committee—Geppert et al. conclude that these discretionary changes had the effect of making the Index more representative of macroeconomic measures of the business cycle, including inflation, industrial production, and oil prices.74 Two obvious, yet critically important, observations follow from this conclusion. First, notably absent from these macroeconomic measures is the performance of equities, let alone the performance of large-cap domestic equities. As with any joint optimization, increasing the extent to which the Index is representative of these other measures is, other things being equal, likely to make it less representative of the equity market. Second, and perhaps even more obviously, enhancing “representativeness” with respect to any external measure is a goal. It may be an entirely reasonable goal for an index provider to pursue, but it is a goal all the same. And to the extent that the Index Committee is in fact making decisions designed to pursue this goal, that decision-making represents conscious, deliberate, and active management of the Index. Because an S&P 500 fund tracks the Index, it follows that the index fund is, for all practical purposes, also being actively managed.

Notwithstanding this, such funds are routinely described as “passive,” including in their prospectuses. This use of the term is misleading. While the term “passive” is not precisely defined, if the term “passive” is to have any meaning at all it must surely mean that the fund’s portfolio is not selected using an active discretionary process. Some investors may fully appreciate how their money is being managed, but others, who rely on the description provided in the prospectus, will not. While it may well be the case that investors in S&P 500 index funds would be happy to have the benefit of this discretionary portfolio management at a relatively low price, that does not change the fact that investors are getting something different from what is being described to them. The argument that there is no problem here because investors are obtaining very cheap management is analogous to telling a customer who was handed a television that it’s not a problem that she was not given the toaster she was promised because the deal represents a bargain on televisions.

Fortunately, there is a simple solution to this problem: funds should stop doing it. If funds are not willing to do so on their own, the SEC should take the position that, in the context of an index fund where the underlying index is managed with some amount of discretion—like the S&P 500—the term “passive” is potentially materially misleading to investors.

My empirical analysis also points to a substantial regulatory mismatch. Just like the traditional investment adviser of a mutual fund, the S&P 500 Index Committee is selecting mutual fund portfolios. Perhaps the biggest difference is one of scale: while most investment advisers manage a few funds, the Index Committee effectively manages dozens of funds, representing trillions of dollars. It is, for all practical purposes, playing the role of a mutual fund subadviser: while it does not engage in all of the traditional roles of an investment adviser, like a subadviser it engages solely in the security selection element of the advisory role.75

And yet unlike investment advisers and subadvisers, which are subject to regulation under the Advisers Act and the ‘40 Act—including fiduciary duties and antifraud rules—index providers are completely unregulated. While a legal basis for this differential treatment in the context of the S&P 500 could likely be found in the publisher’s exclusion,76 it is hard to come up with any economic or theoretical basis for justifying this differential treatment. Either security selection for mutual funds is a matter of concern, and ought to be regulated, or there are enough other structural safeguards in place that regulation is unnecessary.

The SEC’s recent Request for Comment on Certain Information Providers Acting as Investment Advisers represents a welcome recognition of the importance of index providers, and their role as de facto, and in some cases de jure, investment advisers.77 It is too early to say what rules, if any, might come out of the request for comment, or what effect they will have on large indices like the S&P 500 that are excluded from the statutory definition of investment adviser by the publisher’s exclusion. While regulations targeting S&P 500 funds (rather than the index itself) might provide an indirect means of achieving some of the needed protections, such regulations do not appear to be on the immediate horizon. As a much more modest alternative to direct regulation, the SEC could use its platform and its regulation of mutual fund registration statements and marketing materials to ensure that it is clear to investors what, exactly, they are getting when they invest in an S&P 500 index fund: a portfolio of large-cap securities selected at the discretion of an index committee. This alternative would represent a compromise between direct regulation and ignoring the issue.

Low-cost diversification is highly desirable, and there is nothing in my empirical analysis suggesting that there is anything per se wrong with the selection process employed by the S&P 500 Index Committee. The problem is failing to recognize that when an investor invests in an S&P 500 index fund, what she is doing is delegating the management of her portfolio to that Index Committee. In the same way, there is nothing wrong with letting the Index Committee select one’s portfolio; the problem is then marketing that portfolio as being “passive.”

B. Mutual Fund Benchmarking

I now turn to mutual fund benchmarking. The misuse of the Index in this context appears to be rooted in an underlying assumption that the Index is somehow universal or neutral. It is neither of these things. Having established that the S&P 500 simply captures the performance of one particular large-cap portfolio, chosen with a substantial amount of discretion and with constituents that change substantially over time, it is clear why it makes little sense to use it as a mutual fund performance benchmark either to serve as an objective standard or to determine how well the manager performed relative to “the market.” This is not to say that there is anything particularly wrong with this portfolio; using a perfectly fine portfolio in an inappropriate way is still a misuse of that portfolio.

The first rationale—that the Index represents an “objective” standard—is clearly problematic. While it is certainly true that in most cases the mutual fund manager cannot directly influence the performance of the Index, this fact alone does not make such a comparison useful. Rather, for such a comparison to be useful to an investor, it must be the case that the Index provides an objective way to measure the risk-adjusted performance of this particular mutual fund. It is immediately obvious from the fact that the Index is just one particular large-cap portfolio that there is no reason to believe that it is the right, or even an appropriate, benchmark for any particular fund. Some funds may happen to have a similar risk profile, but most won’t. On top of this, the fact that the constituents of the Index change substantially over time means that the investor can’t even use it as a reliable measure of the average performance of a particular fixed set of securities.

Given these facts, it is unclear how even a fully rational, fully informed investor should be using the Index to evaluate any particular fund. For example, how is an investor to use the Index to evaluate a fund that is risker than the Index portfolio, but that also has a higher return than the Index? What about a fund that is less risky than the Index, but also has a lower return? And of course, how is the investor to know the relative risk profile of the fund versus the Index? While there are standard methods in financial economics to compare the risk-adjusted performance of two portfolios,78 none can be implemented with simply the annual return of the two portfolios. As a result, providing an investor with the realized return of the fund alongside the realized return of the Index does nothing to help her evaluate the risk-adjusted performance of the fund. Perhaps more importantly, it seems implausible that any investor who would go to the trouble of performing the analysis necessary to make these comparisons appropriately would be relying on the comparison information provided in the prospectus; such an investor would probably just use her preferred data from her preferred data provider. In short, it is hard to see how any useful information can be gleaned from the required comparison.

All of these problems would exist even if the Index were the best possible proxy for “the market.” But the fact that it is just one particular portfolio makes it doubly problematic, since, in addition to suffering from these theoretical problems, any comparison to the index also inherently reflects the discretionary choices made by the Index Committee.79 Moreover, the mere fact that the S&P 500 is the de facto default benchmark is not, itself, a reason to use it. Intuitively, one might think that if everyone is comparing the returns of every portfolio to the same arbitrary benchmark, the arbitrariness of the benchmark becomes less important. Rather, what matters is that everyone is coordinating on the same benchmark. To see the flaw in this intuition, consider the following arbitrary benchmark: one that returns nothing each period. Of course, this is the equivalent of simply reporting raw returns. Ubiquity alone, in other words, is not enough for a benchmark to provide any information beyond the raw returns.

Providing this comparison is not simply unhelpful; it is actively harmful. Investors pay for the information indirectly through the licensing fees that mutual funds pay to index providers, such as S&P. Perhaps more concerningly, investors appear to respond irrationally to these disclosures: there is evidence that investors care about a fund’s performance relative to its index in making investment decisions, above and beyond the overall performance of the fund itself.80 Given this finding, it is perhaps unsurprising that that there is also evidence that funds choose their benchmark indices strategically so as to improve their performance relative to that benchmark and attract additional investors.81

A slightly different rationale for benchmarking is that it provides the investor with a sense of “what else” she could have done with her money. In other words, it has the flavor of the investor’s “outside option”: had she not paid for active management, what would she have earned? Conceptually, thinking in terms of outside options is far more appropriate than using a portfolio benchmark. But the problem with using the Index for this task is that it implies that the investor can actually choose to invest in the Index. This idea is, of course, false: one cannot invest in an index. One can invest in a mutual fund, including a mutual fund that tracks an index like the S&P 500. What is missing is a reason why a fund that tracks that particular discretionary portfolio should be the default outside option for any particular mutual fund.

If the SEC’s goal in retaining the benchmarking requirement is to force mutual funds to compare their performance to that of another mutual fund, a much more honest, and appropriate, approach would be to simply do that. This approach would bring mutual funds into line with virtually every other product market, where comparisons are made to competitors rather than to some abstract benchmark.

Given the current state of the financial market, and the ease with which financial information can be obtained online, it is unclear how useful such a requirement would be. Private information providers such as Morningstar already make it easy for investors to compare mutual funds. To the extent that the SEC is concerned that private parties cannot, or will not, reliably make the relevant information available to investors, it could consider creating a searchable database of all registered mutual funds. All of the information required to do so is already filed with the SEC on a regular basis, alleviating the informational challenges associated with this option.

The Index is used as a benchmark by thousands of market actors; why single out the mutual fund benchmark requirement? The difference is that other investors may be using the index as a comparator because it either represents the risk profile that they have in mind, or because it does approximate their private “outside option.” But the appropriate outside option will almost certainly differ by investor. While one investor might sensibly determine that the S&P 500 is a reasonable proxy for what else she might have done with her money given her goals and the information available to her at the time of her investment, there is no reason to expect this to hold for all investors who have purchased, or might purchase, shares in a particular mutual fund. In other words, while it might be reasonable for an investor to use the Index as a comparator (given her own individual circumstances), it does not follow that it is a reasonable comparator for the investment itself.

It is also worth noting that the analysis in this section applies to many indices, not simply the S&P 500. It applies to the S&P 500 by virtue of the fact that the Index is simply one particular portfolio, and there is no reason why any particular portfolio should be viewed as a universal or default benchmark. The misuse derives not from the construction of the Index itself, but from the fact that its fundamental nature is either misunderstood or overlooked. But whereas it is unlikely that anyone, including any regulator, would mistake the “Alpha Architect Quantitative Value Index” for something that is either neutral or universal, it happens as a matter of course in the context of the S&P 500.

C. Evaluating Corporate Performance

Finally, I turn to evaluating corporate performance. It is straightforward to see why it makes little sense to use the S&P 500, which simply captures the performance of a constantly changing group of securities chosen subject to a substantial amount of discretion, for this purpose. To begin, there is no a priori reason to expect any particular company to perform as well as (or better than) the Index. This is true even if the company in question is a constituent of the Index, and even though the performance of the Index represents the weighted average performance of its constituents.

This issue is just a variation of the problem with using the Index for mutual fund benchmarking: classical finance theory holds that risk and return move together. Unless the risk of a particular company (or, for that matter, mutual fund) is the same as that of the Index, there is no reason to expect their performance to be the same either. Some will be riskier, in which case a comparison to the average will overstate, on average, the firm’s true risk-adjusted performance. Others will be less risky, causing the comparison to understate the firm’s performance, on average. Without adjusting for risk—something that is impossible to do from just the annual return information—such a comparison is, at best, completely uninformative.

Moreover, because of changes in the composition of the Index, even if one did want to compare the performance of a constituent to that of the Index, it becomes critical to determine the right reference point for this comparison. Suppose, for example, that one wanted to compare the performance of a current constituent of the Index to that of the Index itself over the last five years. As my empirical analysis has made clear, this would be a comparison to a constantly changing reference group. As a result, while the Index does a good job of providing a snapshot of the performance of an important segment of the equity market on any given day, what it does not do (and what it is not intended to do) is provide a consistent comparison group. It is even more problematic if the Index is selected based on the fact that the firm is, ex post, an Index constituent, since this implies that the comparison group is chosen based in part on the performance of the firm itself over the relevant period.

While most academic papers in the area of law and finance are sophisticated enough to avoid this problem, investors are not so lucky. As required by Regulation S–K, the annual reports of S&P 500 constituents generally include performance graphs that compare the performance of the firm to that of the Index over the past 5 years. In some cases, these comparisons include a period of time before the firm was a constituent of the Index, which further complicates the reference point issue. One example of this is IDEXX Laboratories, Inc. (“IDEXX Labs”), which was moved from the S&P MidCap 400 to the S&P 500 index on January 5, 2017.82 Prior to this announcement, the firm had been comparing its performance to that of the S&P MidCap 400 in its Annual Reports.83 From that point on, the firm switched indices and began comparing itself to the S&P 500. The first instance of this comparison was its Annual Report for the fiscal year ending on December 31, 2016, filed on February 17, 2017.84 Following standard practice, these performance comparisons have a 5-year lookback period. As a result, in its 2016 Annual Report, IDEXX Labs was comparing its performance to that of the S&P 500 for the period beginning on December 31, 2011 and ending on December 31, 2016.85 While this comparison is entirely consistent with, and is even required under, current regulatory requirements, the result is that the comparison changed midstream. In effect, this change rewrote the firm’s history with respect to the relative performance presented in the Annual Report.

A far better approach would be to credibly select a comparison group of peer firms ex ante, and barring some very specific reason to deviate, to stick with that group. In the event that the comparison group does change, the change should only apply prospectively, not retroactively. And, if nothing else, the SEC should do away with the requirement that constituent firms compare themselves to the Index. To the extent that it is providing investors a convenient visual comparison, that comparison is flawed.

V. Conclusion

In this Article, I argue that the regulatory regimes surrounding three major uses of the S&P 500 are deeply flawed in light of the fundamental features of the Index. Rather than being neutral or universal, the Index is simply one particular portfolio. The construction of that portfolio involves a substantial amount of discretionary decision-making and its composition changes substantially over time. Because of this, the treatment of the Index in the context of (i) “index” investing, (ii) mutual fund benchmarking, and (iii) firm performance evaluation represent misuses of the S&P 500. This is not to say that the Index has no valid uses. It has many. It represents a simple and convenient way to obtain a snapshot of the performance of an important segment of the equity market on any given day. It represents a portfolio chosen by a third party that sophisticated actors, who understand its construction, can use for their own purposes. Investing in a low-cost index fund that tracks the S&P 500 is an extremely cheap form of portfolio management. The problems discussed in this Article stem not from a particular problem with the Index, but rather from the fact that these uses do not properly take into account the underlying realities of the Index.

• 1Hereinafter, I refer to the S&P 500 Index as the “S&P 500” or simply the “Index.”
• 2Hereinafter, I use the term “mutual fund” to refer to both traditional open-ended investment companies and ETFs regulated as investment companies under the Investment Company Act of 1940.
• 3Investment Company Act of 1940, 15 U.S.C. §§ 80a-1 to 80a-64 (2018). Hereinafter, I refer to the Investment Company Act as the “40 Act.”
• 4Investment Advisers Act of 1940, 15 U.S.C. §§ 80b-1 to 80b-18c (2018). Hereinafter, I refer to the Investment Advisers Act as the “Advisers Act.”
• 5See discussion infra Section IV.A.
• 6Andrei Shleifer, Do Demand Curves for Stocks Slope Down?, 41 J. Fin. 579 (1986).
• 7For a sampling of other articles in this literature, see Jie Cai, What’s in the News? Information Content of S&P 500 Additions, 36 Fin. Mgmt. 113 (2008); Diane K. Denis, John J. McConnell, Alexei V. Ovtchinnikov & Yun Yu, S&P 500 Index Additions and Earnings Expectations, 58 J. Fin. 1821 (2003); Upinder Dhillon & Herb Johnson, Changes in the Standard and Poor’s 500 List, 64 J. Bus. 75 (1991); Lawrence Harris & Eitan Gurel, Price and Volume Effects Associated with Changes in the S&P 500 List: New Evidence for the Existence of Price Pressures, 41 J. Fin. 815 (1986); Prem C. Jain, The Effect on Stock Price of Inclusion in or Exclusion from the S&P 500, 43 Fin. Analysts J. 58 (1987); Anthony Lynch & Richard Mendenhall, New Evidence on Stock Price Effects Associated with Changes in the S&P 500 Index, 70 J. Bus. 351 (1997).
• 8See, e.g., Nimesh Patel & Ivo Welch, Extended Stock Returns in Response to S&P 500 Index Changes, 7 Rev. Asset Pricing Stud. 172 (2017).
• 9Samuel M. Hartzmark & David H. Solomon, Reconsidering Returns, 35 Rev. Fin Stud. 343 (2022).
• 10For example, three of the ten articles selected as The Top 10 Corporate and Securities Articles of 2019 by The Corporate Practice Commentator were about index fund or “passive” investing. The Top 10 Corporate and Securities Articles of 2019, Corporate Practice Commentator, https://perma.cc/J2VY-W72D (last visited Jul. 14, 2020). Two of these were primarily concerned with the implications of the rise of index investing. The first analyzed the implications of the rise of passive investing for corporate governance and well-functioning capital markets. Lucian Bebchuk & Scott Hirst, Index Funds and the Future of Corporate Governance: Theory, Evidence, and Policy 119 Columbia L. Rev. 2029 (2019). The second observed that passive investors are the most important development in modern capital markets, and then provided a theoretical framework for passive investment and its implications for corporate law and financial regulation. Jill E. Fisch, Assaf Hamdani & Steven Davidoff Solomon, The New Titans of Wall Street: A Theoretical Framework for Passive Investors, 168 U. Pa. L. Rev. 17 (2019).
• 1115 U.S.C. §§ 77a –77aa (2018) and 15 U.S.C. §§ 78a –78qq (2018), respectively.
• 12See James J. Choi & Adriana Z. Robertson, What Matters to Individual Investors? Evidence from the Horse’s Mouth, 75 J. Fin. 1965, n.22 (2020) (defining a passively managed stock mutual fund (also known as a stock index fund) as a fund that “holds stocks in order to match the performance of a market benchmark (such as the S&P 500 stock market index) as closely as possible”); see also Investor Bulletin: Index Funds, U.S. Securities and Exchange Commission (Aug. 6, 2018), https://perma.cc/3DV9-3PPG (defining an index fund as “a type of mutual fund or exchange-traded fund that seeks to track the returns of a market index”).
• 13S&P 500 (USD) Factsheet, S&P Dow Jones Indices 1, 1 (September 30, 2022). Shortly after the end of the sample period used in the empirical analysis in Part III—June 2018—this number was $3.4 trillion. S&P 500 (USD) Factsheet, S&P Dow Jones Indices 1, 1 (June 29, 2018).
• 14In its most recent 10-Q, filed in August 2022, BlackRock reported total long-term assets under management of $7.7 trillion as of June 30, 2022. BlackRock Inc., Quarterly Report (Form 10-Q) 42 (Aug. 5, 2022). Long-term assets under management include ETF, retail mutual funds, and institutional funds, and represent the vast majority of assets under management reported on BlackRock’s 10-Q ($7.7 trillion out of a total of $8.5 trillion). Id.
• 15Because Vanguard is not a public company, it is not required to file quarterly and annual reports with the SEC. However, as of January 2021, reports indicated that Vanguard’s total assets under management surpassed $7 trillion for the first time. Chris Flood, Vanguard’s assets hit record $7tn, Fin. Times (Jan. 12, 2021), https://perma.cc/L7PK-MCDM.
• 16Adriana Z. Robertson, Passive in Name Only: Delegated Management and “Index” Investing, 36 Yale J. on Reg. 795 (2019).
• 17Id. at 836.
• 18See generally Revised Procedures for Processing Registration Statements, Post-Effective Amendments and Preliminary Proxy Materials Filed by Registered Investment Companies, 49 Fed. Reg. 6708 (Feb. 15, 1984) (to be codified at 17 C.F.R. pt. 270) (discussing SEC review of investment company registration statements and post-effective amendments and adopting new procedures for the selective review of certain such filings). See also The Offering Documents, K&L Gates LLP 1, 9 (2013), https://perma.cc/56WF-WHH5 (discussing the exchange of comments following SEC review of registration statements).
• 19S&P 500 (USD) Factsheet, S&P Dow Jones Indices 1, 1 (September 30, 2022). As of June 2018, this figure was a more comprehensible, but still enormous, $9.9 trillion. S&P 500 (USD) Factsheet, S&P Dow Jones Indices 1, 1 (June 29, 2018). Because the empirical analysis in Part III uses data through 2017, I use the methodology as of 2018 when describing the index construction methodology (except where expressly noted).
• 20SEC, Form N-1A at 11, https://perma.cc/7NJQ-5FZB.
• 21See Robertson, supra note 16. Figures as of July of 2017. The next most popular index was the benchmark for only 8% of dollars.
• 22Disclosure and Analysis of Mutual Fund Performance Information: Portfolio Manager Disclosure, 55 Fed. Reg. 1460, 1463 (Jan. 16, 1990) (to be codified at 17 C.F.R. pts. 230, 239, 270, 274).
• 23Disclosure of Mutual Fund Performance and Portfolio Managers, 58 Fed. Reg. 19,050, 19,054 (Apr. 12, 1993) (to be codified at 17 C.F.R. pts. 239, 270, 274). See also Disclosure and Analysis of Mutual Fund Performance Information: Portfolio Manager Disclosure, 55 Fed. Reg. at 1461.
• 24Disclosure of Mutual Fund Performance and Portfolio Managers, 58 Fed. Reg. at 19,054.
• 25Disclosure and Analysis of Mutual Fund Performance Information: Portfolio Manager Disclosure, 55 Fed. Reg. at 1464 (“In some cases, of course, there will not be an index available that encompasses the types of securities in which the fund invests. Nonetheless, a broad market index could always be used to serve as a benchmark for how an alternative, unmanaged investment in the securities market performed during the period.”).
• 26Disclosure of Mutual Fund Performance and Portfolio Managers, 58 Fed. Reg. at 19,059.
• 2717 C.F.R. § 229.201(e)(1)(i) (2016).
• 28Id.
• 2917 C.F.R. § 229.201(e)(1)(ii) (2016).
• 30Executive Compensation Disclosure, 57 Fed. Reg. 48,126, 48,126 (Oct. 21, 1992) (to be codified at 17 C.F.R. pts. 228, 229, 240, 249).
• 31Executive Compensation Disclosure, 57 Fed. Reg. at 48,139. There is evidence that firms do use performance relative to the S&P 500 in evaluating executive performance. A recent study of relative performance evaluation of CEOs found that, among established indices, the S&P 500 was the most commonly used index, representing 10.0 % of compensation packages. Frances M. Tice, The Role of Common Risk in the Effectiveness of Explicit Relative Performance Evaluation 17 (SSRN, Working Paper No. 2645956, 2022).
• 32Executive Compensation Disclosure, 57 Fed. Reg. at 48,139.
• 33Executive Compensation and Related Person Disclosure, 71 Fed. Reg. 53,158, 53,168 (Sept 8, 2006) (to be codified at 17 C.F.R. 228, 229, 232, 239, 240, 245, 249, 274).
• 34In many of my empirical analyses, I compare various alternative indices (or portfolios) with the S&P 500 Index. Unfortunately, as pointed out by Hartzmark and Solomon, the S&P 500 has traditionally excluded dividends. See Hartzmark & Solomon, supra note 9, at 2. Even the variable representing the return on the S&P 500 Index in the Center for Research in Security Prices (CRSP) database does not include dividends. The S&P 500 Total Return index (which does include dividends) is only available from March 10, 1988. As a result, in my main analysis, I restrict attention to the period beginning in 1989 (and occasionally 1990). In order to ensure that my approaches to dealing with constituents of the Index, as well as its construction, are consistent with the approaches of the S&P 500, I also construct a portfolio to mimic the S&P 500 Total Return index for the period beginning in 1988 and perform a series of tests to confirm that this constructed S&P 500 Total Return index is a good proxy for the “true” S&P 500 Total Return index. The coefficient of correlation between the two indices is 0.9992 over the full period, and 0.9999 over the period beginning in 2005. I provide details on this construction in Online Appendix (“Appendix”) A-I, available on the University of Chicago Business Law Review website. The empirical results in this paper are qualitatively similar if I use the constructed version of the Index rather than the true Index.
• 35S&P 500, S&P Dow Jones Indices, https://perma.cc/RMU5-WAG7 (last visited Oct. 1, 2022).
• 36This is not to say that the construction of the Index, or the implications of its construction, has been entirely overlooked by the literature. Two existing papers have studied the implications of changes to the S&P 500. The first, and most relevant to the analysis in Section III.B, studied the effect of discretionary changes to the constituents of the S&P 500 on the performance of the Index. John M. Geppert, Stoyu I. Ivanov & Gordon V. Karels, An Analysis of the Importance of S&P 500 Discretionary Constituent Changes, 37 Rev. Quantitative Fin. & Acct. 21 (2011). As discussed in more detail in Section II.B, the analysis in that paper is focused on one element of the discretion afforded to the Index Committee, namely the ability to remove specific companies from the Index. See discussion infra note 50 and accompanying text. In their analysis, the authors of that paper find that these discretionary changes had a statistically significant effect on the performance of the Index, and that they had the effect of making the Index more “representative of the business cycle.” Id. Of course, given that the S&P 500 is an index of large-cap domestic equities, it is not clear why being representative of the business cycle as a whole is the most natural comparison. The second paper, which is more similar in spirit to the analysis in Section III.C, studied the performance of the S&P 500 relative to the performance of several counterfactual indices constructed from the original (i.e., 1957) constituents of the Index. Jeremy J. Siegel & Jeremy D. Schwartz, Long-Term Returns on the Original S&P 500 Companies, 62 Fin. Analysts J. 18 (2006). While there are substantial differences between the methodological choices in that paper and the present analysis, both find that turnover in Index constituents has a substantial effect on the performance of the Index.
• 37While the S&P 500 contains 500 companies, occasionally it contains a few more than 500 securities because multiple share class lines can be included in the index. S&P U.S. Indices Methodology, S&P Dow Jones Indices 6 (April 2018). As of 2022, this continues to be the case. S&P U.S. Indices Methodology, S&P Dow Jones Indices 7 (October 2022).
• 38S&P U.S. Indices Methodology (2018), supra note 37, at 6.
• 39Id. at 7.
• 40For example, the rules define three criteria for determining whether a company is a “U.S. company” for the purpose of the Index and include a discussion of the treatment of ambiguous cases. Id. at 5. The rules also provide a list of exchanges on which securities must have their primary listing in order to be eligible for inclusion. Id.
• 41Examples of this ambiguity include the treatment of IPOs (“IPOs should be traded on an eligible exchange for at least 12 months before being considered for addition to an index” (emphasis added)) and the financial viability criterion (“[t]he sum of the most recent four consecutive quarters’ Generally Accepted Accounting Principles (GAAP) earnings (net income excluding discontinued operations) should be positive as should the most recent quarter” (emphasis added)). Id. at 7. The methodology also explicitly provides for special treatment for the securities of a particular firm—Berkshire Hathaway Inc.—further contributing to the sense that the construction it might involve substantial deviations from nondiscretionary quantitative rules that apply equally to all firms. Id. at 8.
• 42Id. at 8.
• 43Id.
• 44Id. at 9.
• 45Id. GICS is a proprietary sector classification system that is owned by S&P and MSCI, another index provider. See MSCI Inc., Global Industry Classification Standard (GICS) (2016), https://perma.cc/A4KU-8F4B.
• 46According to an email correspondence with the Managing Director & Chairman of the Index Committee for the S&P 500, information on revisions to the Index methodology prior to 2015 does not exist. Email from David M. Blitzer, Managing Dir. & Chairman of the Index Comm., S&P Dow Jones Indices, to Sufei Xu, Access Services Coordinator, Bora Laskin L. Libr., University of Toronto Faculty of L. (Aug. 28, 2017, 16:01 EST) (on file with author). As a result, this portion of the analysis is limited to the period from 2015 through 2017.
• 47S&P U.S. Indices Methodology (2018), supra note 37, at 35–37.
• 48Id. at 27.
• 49A third approach, which is in some ways in between the first two, would be to study rule changes directly and see whether changes in eligibility are related to the behavior of the Index. I explore this approach in Appendix A-IV.
• 50The conception of discretion that I seek to capture in this section is broader than the one adopted by Geppert et al., who seek to capture the effect of discretionary changes to the Index’s constituents on the behavior of the Index. Geppert et al., supra note 36. One interpretation of their analysis is that it is focused on the implications of one particular facet of the discretion available to the Index Committee as exercised by the Committee. In contrast, my analysis in this section focuses on the amount of discretion available to the Committee pursuant to the Index Methodology. This latter conception is substantially broader, and is a better measure of the decision-making power afforded to the Index Committee. In contrast, the measure adopted by Geppert et al. is more useful in determining how the Committee chooses to exercise that discretion. This measure can be used to make inferences about the Committee’s objectives. I return to this observation in Section IV.A.
• 51I note that constructing this set of eligible securities involves some judgement. While I err on the side of conservatism in my construction of eligibility criteria wherever possible, I cannot guarantee that the set does not include some securities that the Index Committee would not consider eligible. For a more complete discussion of this issue, see Appendix A-II.A.
• 52Note that not satisfying these criteria is not per se evidence that the Index Committee is violating the rules laid out in the Index methodology. First, as discussed above, the Index methodology specifically allows for a security to remain on the Index, even if it no longer satisfies the criteria, until the Index Committee determines that “ongoing conditions warrant” its removal. S&P U.S. Indices Methodology (2018), supra note 37, at 8. Second, because some of these criteria are phrased using terms like “should,” rather than as firm requirements, it is hard to determine precisely how they are to be interpreted. Finally, because I have been as conservative as possible in constructing these criteria, they are likely to be more restrictive than those criteria employed by the Index Committee, leading me to underestimate the size of the set of eligible securities.
• 53For details on the construction of Panel B, see Appendix A-II.B.
• 54If instead of the average we look at medians, the gap between the two groups shrinks substantially: the median security eligible for but not included on the Index is about 39% of the size of the median security on the Index. This result is not surprising given the well-known fact that the distribution of firm size is highly positively skewed.
• 55Using the average day rather than the median day yields figures that are virtually the same (21% and 4%, respectively).
• 56One virtue of this approach, relative to the approach used in Section III.B.1, is that it can be implemented over a much longer period of time. Rather than relying on the eligibility rules on any particular day, this analysis requires only that I know the constituents of the Index and that I can implement the alternative methodology.
• 57For robustness, I repeat the analysis using a daily selection method and find that the results are virtually unchanged. The results for the daily selection method are presented in Appendix A-III.E.1.
• 58The details of the methodology used to do so are presented in Appendix A-III.A.1.
• 59For further details on the construction of Table 2, see Appendix A-III.C.
• 60The p-value of a t-test under the null hypothesis that the mean value of the difference between the median size of the excluded securities and the median size of the securities on the S&P 500 is zero is 0.0064.
• 61The details of the methodology used to do so are presented in Appendix A-III.A.2.a.
• 62For robustness, I repeat the analysis using the securities selected daily, using an equally weighted portfolio, and without reinvesting proceeds (as well as permutations of the above). The differences in behavior between the S&P 500 and the alternative index are greater using any of these methodologies. The results are presented in Appendix A-III.E.2. Recognizing the potential importance of delisting returns in this context, I add the delisting returns, if any, reported in CRSP to the returns reported in CRSP in all of my analyses. For a discussion of the importance of including delisting returns, see Tyler Shumway, The Delisting Bias in CRSP Data, 52 J. Fin. 327 (1997).
• 63K. J. Martijn Cremers & Antti Petajisto, How Active Is Your Fund Manager? A New Measure That Predicts Performance, 22 Rev. Fin. Stud. 3329, 3330 (2009) (noting that “[t]racking error volatility (hereafter just “tracking error”) is the traditional way to measure active management”).
• 64Id. at 3334.
• 65Consistent with the analysis in Section III.B.2, dividends are reinvested in the dividend-paying stock immediately, and delisting returns reported in CRSP, if any, are added to the returns reported in CRSP.
• 66I also perform this analysis at the security level. The results are consistent with the portfolio level approach. These results, as well as the methodology employed in that analysis, are presented in Appendix A-V.
• 67I also perform this analysis at the portfolio level. The results are consistent with the security level approach. These results, as well as the methodology employed in that analysis, are presented in Appendix A-VI.A.
• 68One potential critique of this analysis is that it assumes a relatively long time-horizon, while in some contexts the horizon may be substantially shorter. I therefore repeat this analysis at the annual level for each year from 1989 through 2017 and find results that are consistent with this analysis. The results are presented in Appendix A-VI.B.
• 69As in the prior counterfactuals, dividends are reinvested in the dividend-paying stock immediately. For consistency, I treat CRSP data in the same way as in Section III.C.1. In this counterfactual, I do not need to contend with the issue of a security disappearing from the CRSP dataset, since by construction the security must be on the S&P 500 on the terminal date to be included in the portfolio.
• 70Press Release, S&P Dow Jones Indices, Take-Two Interactive Software, SVB Financial and Nektar Therapeutics Set to Join S&P 500; Others to Join S&P MidCap 400 and S&P SmallCap 600 (Mar. 9, 2018), https://perma.cc/ZQ28-89RA.
• 71See Appendix A-II.A.4.
• 72S&P U.S. Indices Methodology (2018), supra note 37, at 7.
• 73Press Release, S&P Dow Jones Indices, Las Vegas Sands Set to Join S&P 500; Nektar Therapeutics to Join S&P MidCap 400; The Pennant Group to Join S&P SmallCap 600 (Sept. 26, 2019), https://perma.cc/8UB9-WHS7.
• 74See generally Geppert et al, supra note 36, at 31–33, 32 tbl. 5.
• 75See Gregory C. Davis, Rajib Chanda, Renee E. Laws & Melissa C. Smit, Mutual Fund Use of Sub-Advisers in Mutual Funds and Exchange Traded Funds Regulation 42-1, 42-3 to 42-4 (Clifford E. Kirsch ed., 3rd ed. 2013), https://perma.cc/EY3S-WB58 (describing the division of responsibilities between primary adviser and sub-advisers).
• 76See generally Paul G. Mahoney & Adriana Z. Robertson, Advisers by Another Name, 11 Harv. Bus. L. Rev. 311 (2021). While the language of the exclusions is slightly different between the two statutes, and neither has been tested in the context of a securities index being tracked by an index fund, both would very likely encompass a widely distributed and commonly used index like the S&P 500. The Advisers Act excludes the publisher of any “bona fide” publication “of general and regular circulation” from the definition of an investment adviser. Investment Advisers Act § 202(a)(11)(D), 15 U.S.C. § 80b-2(a)(11)(D). The definition of an investment adviser in the ‘40 Act excludes the provision of advice through “uniform publications distributed to subscribers thereto.” Investment Company Act § 2(a)(20)(i), 15 U.S.C. § 80a-2(a)(20)(i).
• 77Request for Comment on Certain Information Providers Acting as Investment Advisers, Securities and Exchange Commission (June 15, 2022), https://perma.cc/E7FL-G3QY.
• 78The most theoretically appropriate measure of the risk adjusted performance of a portfolio is its “alpha” (or Jensen’s alpha). See Jonathan Berk & Peter DeMarzo, Corporate Finance 452 (5th ed. 2020). A portfolio’s alpha can be calculated using a traditional one-factor model such as the Capital Asset Pricing Model (CAPM), or with a more complex factor model. See id. at 421–22.
• 79Note that the fact that Geppert et al. found that the Index Committee’s discretionary changes appear to make the Index more representative of macroeconomic measures of the business cycle, Geppert et al, supra note 36, does not necessarily imply that the Index is a better proxy for “the market.” “The market,” as used by the asset pricing literature, refers to the wealth portfolio, understood as a share of the total consumption stream, including “not only all stocks, but all bonds, real estate, privately held capital, publicly held capital (roads, parks, etc.), and human capital.” John H. Cochrane, Asset Pricing 169 (revised ed. 2005). This portfolio is notoriously difficult to estimate empirically.
• 80Berk A. Sensoy, Performance Evaluation and Self-Designated Benchmark Indexes in the Mutual Fund Industry, 92 J. Fin. Econ. 25 (2009).
• 81Id.
• 82Press Release, S&P Dow Jones Indices, IDEXX Laboratories Set to Join the S&P 500; Chemours to Join S&P MidCap 400; Shake Shack to Join S&P SmallCap 600 (Jan. 3, 2017), https://perma.cc/MS8J-2H64.
• 83See, e.g., IDEXX Lab’ys, Inc., Annual Report (Form 10-K) 28 (Feb. 17, 2015); IDEXX Lab’ys, Inc., Annual Report (Form 10-K) 30 (Feb. 17, 2016).
• 84IDEXX Lab’ys, Inc., Annual Report (Form 10-K) 32 (Feb. 17, 2017).
• 85Id.