{"id":596,"date":"2026-08-04T03:12:39","date_gmt":"2026-08-04T03:12:39","guid":{"rendered":"https:\/\/ubwp.buffalo.edu\/jopblog\/?p=596"},"modified":"2026-08-04T03:12:40","modified_gmt":"2026-08-04T03:12:40","slug":"how-to-prove-anything-with-the-weather","status":"publish","type":"post","link":"https:\/\/ubwp.buffalo.edu\/jopblog\/2026\/08\/04\/how-to-prove-anything-with-the-weather\/","title":{"rendered":"How to Prove Anything with the Weather"},"content":{"rendered":"\n

Summer 2026: extreme heat waves sweep across Europe, breaking longstanding records in many countries. Does personally experiencing such heat waves change people\u2019s climate attitudes or even their policy preferences? Our research indicates this is not the case. But more broadly, it emphasizes the stumbling blocks when trying to make an evidence-based claim on this question in the first place.<\/p>\n\n\n\n

Why? Because to measure the effect of extreme weather, researchers first have to define it. Our study<\/a> shows this seemingly technical step involves challenges \u2013 which have implications that go far beyond research investigating the consequences of weather.<\/p>\n\n\n\n

Nature\u2019s experiments come with fine print<\/h2>\n\n\n\n

Social scientists love so-called natural experiments. Researchers cannot randomly assign who gets exposed to heat waves, floods, refugee arrivals, or economic hardship. If this were possible, it would help tremendously in credibly tracing the consequences of such exposure, however. This is why researchers use instances of naturally occurring random chance that expose some, but spare others. Comparisons from such \u201cexperiments by nature\u201d then promise to reveal the impacts these events have on society.<\/p>\n\n\n\n

Yet, nature does not hand over a tidy dataset. Before any analysis, researchers must translate a messy real-world event into numbers. They must decide what exactly counts as \u201cexposure.\u201d That decision looks technical and innocent. It is neither.<\/p>\n\n\n\n

So what exactly is \u201cextreme\u201d weather?<\/h2>\n\n\n\n

Extreme weather illustrates this challenge well. Is a day extreme when the thermometer passes 30\u00b0C? Or when it is unusually hot for that place and that time of year? Unusual compared to the past 10 years, or the past 25? Among the hottest 5% of days, or the hottest 10%? And should we look at heat at all \u2013 or at drought, rain, or floods?<\/p>\n\n\n\n

Screening the published literature, we counted at least 280 plausible ways to measure extreme weather. Even authorities disagree: the Intergovernmental Panel on Climate Change (IPCC) and the Swiss Federal Office of Meteorology and Climatology (MeteoSwiss) define extreme weather based on different criteria. Below, we compare their two definitions for how strongly citizens in Switzerland experienced extreme heat in 2018\u20132019. The two maps show the percentage of days with extreme temperature experienced by our survey respondents across the country. The definitions give very different answers \u2014 note the diverging scales \u2014 both overall and in how extreme heat is spread across the country. So which is the correct measure for answering our question?<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Differences between operationalizations of extreme weather based on definitions by the IPCC (left) and MeteoSwiss (right). Each dot is one respondent taking part in both survey waves (2018 and 2019). Coloring indicates the percentage of extreme days experienced between the two waves, from dark blue (few) to yellow (many); note the different scales of the two maps. Dark grey areas mark uninhabited parts of Switzerland.<\/em><\/p>\n\n\n\n

The same ambiguity haunts other natural experiments. When does the arrival of immigrants count as noticeable exposure? When does a swing in crop prices lead to economic hardship? Researchers studying these questions likewise use many different operationalizations. And often, published studies do not explain why they picked their particular definition \u2014 and readers rarely see the roads not taken. Statisticians call this a garden of forking paths<\/a>: many small, invisible choices, each of which can change where you end up.<\/p>\n\n\n\n

One question, 34 answers<\/h2>\n\n\n\n

To show what is at stake, we linked a survey panel of almost 3,000 Swiss residents \u2013 interviewed at the beginning of 2018 and the end of 2019, a period which includes two record-breaking summers \u2013 to daily weather data at their home address. We then picked 34 plausible ways of operationalizing extreme weather, all used in prior applied research. To find out whether extreme weather actually affects respondents\u2019 preferences for stricter climate policies, we then ran the identical analysis 34 times, once for each of the different ways of measuring.<\/p>\n\n\n\n

The figure below shows all 34 results. Three find statistically significant \u2013 meaning unlikely to be pure chance \u2013 albeit small effects. But they disagree: two suggest extreme weather increased support for stricter climate policy, one suggests it reduced it. The remaining 31 find no significant effect. Notably, the two definitions plotted above by the IPCC and the Swiss Federal Office of Meteorology and Climatology lead to different conclusions.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Figure: The same analysis run with 34 established definitions of \u201cextreme weather.\u201d Each dot is one estimated effect on climate policy preferences; bars show statistical uncertainty. Most estimates cluster around zero.<\/em><\/p>\n\n\n\n

A researcher who picks just one definition \u2013 in many cases driven by data availability \u2013 could therefore report a positive, a negative, or a null effect from the same people, the same place, and the same period. Viewed as a whole, however, the evidence is clear: even two historic heat waves left Swiss climate policy preferences essentially unchanged. Digging deeper, we find the chain breaks at the first link\u2014those who were more exposed did not perceive the weather as more extreme than those who were less exposed.<\/p>\n\n\n\n

Why this matters beyond the seminar room<\/h2>\n\n\n\n

Our results matter more broadly. First, for trust in science: published studies overwhelmingly report that experiencing extreme weather raises climate concern, while negative findings are almost absent. Given that our 34 estimates scatter across positive, negative, and null, this imbalance could be a sign of publication bias: studies finding the expected result are more likely to be submitted to journals \u2013 and to get accepted for publication. Making measurement decisions more transparent could help journals accept seemingly \u201cuninteresting\u201d results more easily.<\/p>\n\n\n\n

Second, it matters for climate politics. A popular hope is that climate change will correct itself politically: as impacts become tangible, citizens will demand action. Our findings caution against counting on this. Personal experience alone did not move opinions, even in a country that is warming at twice the global average. Not all hope is lost, however: future research could trace whether both those personally affected and<\/em> those personally spared could simultaneously become more aware of, and willing to address, climate change.<\/p>\n\n\n\n

Show the whole menu<\/h2>\n\n\n\n

For other studies that use \u201cnatural experiments,\u201d too, the remedy is transparency, not despair. When theory cannot single out one best measurement, researchers should report results for the whole menu of plausible definitions. And when they find apparent linkages, they should also test the steps through which an effect is likely to occur \u2014 did people perceive the event, did they connect it to the underlying cause? \u2014 before claiming that the event changed minds. Our article offers a practical template and checklist for doing this.<\/p>\n\n\n\n

Hence, natural experiments remain a powerful tool for identifying cause and effect in the social sciences, but their credibility depends on decisions that too often remain hidden. Only if the measurement stage is set right will we be able to bring strong evidence to competing theoretical claims.<\/p>\n\n\n\n

https:\/\/www.journals.uchicago.edu\/doi\/10.1086\/735706<\/a><\/p>\n\n\n\n

By Franziska Quo\u00df (GESIS \u2013 Leibniz Institute for the Social Sciences \/ ETH Zurich) and Lukas Rudolph (University of Konstanz)<\/em><\/del><\/p>\n\n\n\n

Transparency notice: Claude AI and DeepL were used in preparing this blog post. The authors have reviewed and revised all AI-generated content and assume full responsibility for its accuracy.<\/p>\n","protected":false},"excerpt":{"rendered":"

Summer 2026: extreme heat waves sweep across Europe, breaking longstanding records in many countries. Does personally experiencing such heat waves change people\u2019s climate attitudes or even their policy preferences? <\/p>\n

Continue readingHow to Prove Anything with the Weather<\/span><\/a><\/div>\n","protected":false},"author":631,"featured_media":598,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-596","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","entry"],"_links":{"self":[{"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/posts\/596","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/users\/631"}],"replies":[{"embeddable":true,"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/comments?post=596"}],"version-history":[{"count":1,"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/posts\/596\/revisions"}],"predecessor-version":[{"id":599,"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/posts\/596\/revisions\/599"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/media\/598"}],"wp:attachment":[{"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/media?parent=596"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/categories?post=596"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ubwp.buffalo.edu\/jopblog\/wp-json\/wp\/v2\/tags?post=596"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}