Ryan Cummings and Ernie Tedeschi have a very interesting article on BriefingBook today that sheds new light on the divergence between measured sentiment and conventional macroeconomic indicators. Cummings and Tedeschi document how the move to online sampling has changed the characteristics of the University of Michigan Economic Sentiment series.
…we believe that online respondents are causing the level of overall sentiment and current index conditions to be significantly lower, making UMich’s latest data points inconsistent with data points prior to April 2024. Specifically, we use a simple statistical model to estimate the impact of the change in methodology moving from the telephone to the Internet currently results in a feeling of being 8.9 index points -or more than 11 percent- lower than it would be if the interviews were still collected by telephone. .
To show that the shift to online testing provided structural differences in the UMich series, they compared the Morning Consult series.
Source: Cummings and Tedeschi (2024).
The Morning Consult survey was online, so it serves as a control. Therefore, the main reason for the change in the measured state.
One look at the age distribution of phone and online assignments suggests that, no, it was not random in practice. For example, online respondents may have been older during those transition months, which may have affected their emotional responses. …
A formal multilevel model—which measures the probability of a person being in a certain age group after accounting for other demographic factors—confirms these differences. Respondents 65+ for example were 52 percent more likely to be in an online group in April-June, twice as likely as respondents aged 18-24, and this difference is statistically significant…
I don’t know how the greater presence of older respondents interacts with the partisan divide examined in this post (are Republican/leaning Republican voters older than the corresponding Democratic Alliance/leaning respondents?)
What I can say is that the adjusted Cummings-Tedeschi series shows less evidence of structural divergence than the official UMichigan series, where unemployment, y/y CPI inflation and the SF Fed news sentiment index are used as regressors.
Consider these two regressions, first with the official series, and second with the modified Cummings-Tedeschi series:
Now consider Chow’s 1 best iterative tests for stability.
Note that while both regressions show instability around 2020 and 2023, the adjusted Cummings-Tedeschi series does not show a structural break in 2024 regarding the shift to online voting.
That being said, the shift in survey methods does not fully explain the gap between what is seen and what is felt. I estimate a decline in the period 2016-2024M03, and predict a sample exit in 2024M04-M10.
Figure 1: University of Michigan Consumer Sentiment (bold black), adjusted Cummings-Tedeschi series (light blue), fitted (red), +/- one standard error (gray lines). The NBER has defined recession days as shaded in gray. Source: U.Michigan via FRED, BriefingBook, NBER and author’s statistics.
So less than half the gap is accounted for by changes in research methods.
Source link