Song the blog: Nina Simone - Feeling Good
I have a thing for meta-analyses, if you haven't quite gathered that yet. A meta-analysis takes all of the research on a topic and provides a quantitative overview of all of the findings. So, for example, there's a meta-analysis examining the effects of soda consumption on health. You may think it's pretty obvious soda consumption has negative health effects, but a few studies haven't found negative effects (with the caveat that research not finding negative health links tend to be funded by the soda industry). Even when studies all agree, though, there still may be debate about just exactly how strong effects are, such as exactly how bad soda is for your health.
Meta-analyses have a number of appealing strengths (and some weaknesses, of course). They make it easier to understand a literature that may consist of countless studies. They give us increased confidence in the strength of various relationships (such as the link between soda consumption and negative health outcomes). However, I think meta-analyses have one underappreciated and under-explored strength--the ability to inform policymakers.
For example, how do we talk to people about climate change? In particular, how can we talk to climate change skeptics in ways that make them more open to the consensus climate science? We could just tell people that the overwhelming majority of the scientific community believes in human-caused climate change. We could use analogies to help people understand climate change. We could emphasize how connecting with political conservatives' values, such as stressing how taking care of the environment is patriotic, can make people more concerned about climate change. These strategies, and countless others, have received empirical support. And there are matching media headlines for each study, like this headline, which seem to suggest the magical way to make everyone believe in climate change is simply using the one approach to climate messaging that they covered in their article.
But there's a problem with that kind of narrative--we have a ton of studies all claiming to reveal an effective climate change messaging approach! If you were the communication specialist for an environmental nonprofit, which messaging strategy should you use? You may only get to use one or two strategies. How do pick the most effective approach?
I have a thing for meta-analyses, if you haven't quite gathered that yet. A meta-analysis takes all of the research on a topic and provides a quantitative overview of all of the findings. So, for example, there's a meta-analysis examining the effects of soda consumption on health. You may think it's pretty obvious soda consumption has negative health effects, but a few studies haven't found negative effects (with the caveat that research not finding negative health links tend to be funded by the soda industry). Even when studies all agree, though, there still may be debate about just exactly how strong effects are, such as exactly how bad soda is for your health.
Meta-analyses have a number of appealing strengths (and some weaknesses, of course). They make it easier to understand a literature that may consist of countless studies. They give us increased confidence in the strength of various relationships (such as the link between soda consumption and negative health outcomes). However, I think meta-analyses have one underappreciated and under-explored strength--the ability to inform policymakers.
For example, how do we talk to people about climate change? In particular, how can we talk to climate change skeptics in ways that make them more open to the consensus climate science? We could just tell people that the overwhelming majority of the scientific community believes in human-caused climate change. We could use analogies to help people understand climate change. We could emphasize how connecting with political conservatives' values, such as stressing how taking care of the environment is patriotic, can make people more concerned about climate change. These strategies, and countless others, have received empirical support. And there are matching media headlines for each study, like this headline, which seem to suggest the magical way to make everyone believe in climate change is simply using the one approach to climate messaging that they covered in their article.
But there's a problem with that kind of narrative--we have a ton of studies all claiming to reveal an effective climate change messaging approach! If you were the communication specialist for an environmental nonprofit, which messaging strategy should you use? You may only get to use one or two strategies. How do pick the most effective approach?
This kind of example is one reason why I believe meta-analysis has a unique ability to inform environmental policymakers (I use the term policymaker to refer to anyone interested in informing people's environmental beliefs and behavior, such as someone working at an environmental nonprofit, the chief sustainability officer at a company, a legislator, or even a concerned neighbor). Like the dendrochronologist counting and interpreting tree rings, meta-analyzers need to more effectively assess and convey the state of research to policymakers. To further this conversation, I recently gave a talk at the Sustainability and Social Science Research Symposium at the University of Michigan (you can find our slides here; Mark Cohen, Michael Vandenbergh, and I also have an upcoming chapter in the Handbook of Sustainability and Social Science Research, which I'll post once it's available).
Here's some of the ideas I discussed relating to how we could use meta-analysis to better inform environmental policymakers. Let's create high quality meta-analytic databases from the beginning. So, what do policymakers need to know to develop more effective environmental policies? Well, they need to know how confident we are that the communication and behavior change strategies we use actually change environmental behaviors. And, relatedly, they want to know how strong of an effect our behavior change interventions tend to have on behavior. One meta-analysis exemplar that assess confidence in, and strength of, environmental behavior change interventions suggests that information-based home energy interventions, such as providing people with home energy audits, can lead to approximately 7% reductions in home energy use. This is helpful to policymakers. Fortunately, most meta-analyses in the area do a decent job of conveying this information (though we can also certainly improve).
But, is there more we can do as meta-analyzers? Certainly. First, we could improve how we conduct our meta-analyses. One way to do that would be to create online, publicly available summaries of all of the studies on a given topic (e.g., how can we use social influence behavior change interventions to help people become more sustainable?). This would allow policymakers themselves (or scientists working with policymakers) to explore studies that fit their policy context. For example, policymakers could ask which interventions are best for changing household environmental behavior versus best for changing employee environmental behaviors. Policymakers may want to explore different behavior change approaches depending on the context in which they work.
We could also update meta-analytic databases in real time as new studies are released (something we're calling "dynamic meta-analyses"; both similar to and distinct from what the health sciences have called "cumulative meta-analyses"). This would help ensure policymakers are always looking at the cutting edge best practices for influencing behavior. So if 20 studies suggest some overlapping and distinct ways to convey climate change information, and varying in how effective they each are, an updated dynamic meta-analysis would help the communication specialist choose the message approach most likely to work for their policy context. If 20 more studies come out, a dynamic meta-analysis would include the original 20 studies and the new 20 studies, all analyzed together, providing policymakers with updated best practices. And this could happen every time a new study is released. In a perfect world, the policymaker (or the scientist consulting with the policymaker) would use an online interface to request the most current behavior change information for their policy context, and the cutting edge, evidence-based findings would be conveyed in an easy to understand way (somewhat similar to data to text efforts for statistical programs).
These are some of the ideas I explored in my talk. In my opinion, we need to improve how we, as scientists, inform policymakers. Improving our meta-analyses would be a hue step toward that goal, including the development of dynamic meta-analyses that inform environmental policymakers of the latest trends and approaches to behavior change. There would be some difficulties, of course, such as deciding who would curate various dynamic meta-analyses. But, I believe it's an idea we need to more seriously explore. What do you think?
Here's some of the ideas I discussed relating to how we could use meta-analysis to better inform environmental policymakers. Let's create high quality meta-analytic databases from the beginning. So, what do policymakers need to know to develop more effective environmental policies? Well, they need to know how confident we are that the communication and behavior change strategies we use actually change environmental behaviors. And, relatedly, they want to know how strong of an effect our behavior change interventions tend to have on behavior. One meta-analysis exemplar that assess confidence in, and strength of, environmental behavior change interventions suggests that information-based home energy interventions, such as providing people with home energy audits, can lead to approximately 7% reductions in home energy use. This is helpful to policymakers. Fortunately, most meta-analyses in the area do a decent job of conveying this information (though we can also certainly improve).
But, is there more we can do as meta-analyzers? Certainly. First, we could improve how we conduct our meta-analyses. One way to do that would be to create online, publicly available summaries of all of the studies on a given topic (e.g., how can we use social influence behavior change interventions to help people become more sustainable?). This would allow policymakers themselves (or scientists working with policymakers) to explore studies that fit their policy context. For example, policymakers could ask which interventions are best for changing household environmental behavior versus best for changing employee environmental behaviors. Policymakers may want to explore different behavior change approaches depending on the context in which they work.
We could also update meta-analytic databases in real time as new studies are released (something we're calling "dynamic meta-analyses"; both similar to and distinct from what the health sciences have called "cumulative meta-analyses"). This would help ensure policymakers are always looking at the cutting edge best practices for influencing behavior. So if 20 studies suggest some overlapping and distinct ways to convey climate change information, and varying in how effective they each are, an updated dynamic meta-analysis would help the communication specialist choose the message approach most likely to work for their policy context. If 20 more studies come out, a dynamic meta-analysis would include the original 20 studies and the new 20 studies, all analyzed together, providing policymakers with updated best practices. And this could happen every time a new study is released. In a perfect world, the policymaker (or the scientist consulting with the policymaker) would use an online interface to request the most current behavior change information for their policy context, and the cutting edge, evidence-based findings would be conveyed in an easy to understand way (somewhat similar to data to text efforts for statistical programs).
These are some of the ideas I explored in my talk. In my opinion, we need to improve how we, as scientists, inform policymakers. Improving our meta-analyses would be a hue step toward that goal, including the development of dynamic meta-analyses that inform environmental policymakers of the latest trends and approaches to behavior change. There would be some difficulties, of course, such as deciding who would curate various dynamic meta-analyses. But, I believe it's an idea we need to more seriously explore. What do you think?
RSS Feed
