When journalists or market researchers interview the public, they describe what they’ve done in plain language. They say they’ve conducted interviews, that they’ve done a survey. When UN employees interview people they tell you, over and over, about their “empirical research.”

Anyone who doubts this is invited to watch a 70-minute Media Training Workshop that took place at the Bonn Climate Change talks in June 2009. (If you make it through the whole thing you earn a shiny gold star.)

A glossy 36-page report about climate refugees titled In Search of Shelter was launched at this workshop. The report was written by people with agendas a mile wide (see my previous post). It has not been peer-reviewed. Nevertheless, it provides an instructive example of how quickly questionable material gets incorporated into the self-perpetuating climate change consciousness.

In February of this year Canada’s Library of Parliament prepared a document that discusses this country’s legal obligations toward climate refugees. Footnote number five of that parliamentary document cites the glossy report. (Incidentally, footnotes six and eight cite the equally non-peer-reviewed Stern Review, while footnotes seven and ten cite an advocacy organization called the Environmental Justice Foundation. How reassuring to realize that Library of Parliament staffers don’t distinguish between activist literature and respectable scholarship.)

Despite the assertions that this glossy report presents “empirical research,” hard data is shockingly absent from its pages. One line of the report (located on page 33) tells us this project involved “fieldwork in twenty-three sites around the world.” If you are the careful sort of reader who jumps to the end of a document to read all the endnotes, you will also discover that this fieldwork was comprised of:

…expert interviews, a survey of migrants, and a related survey of non-migrants living in areas with documented environmental degradation…

The report, however, doesn’t bother to tell us:

• how many countries (as opposed to sites) these surveys were conducted in
• which countries were selected and why
• how many surveys in total were completed
• how many surveys were completed in each country
• how many surveys of migrants versus non-migrants were completed (in total, but also by country)
• when the surveys were completed
• what the surveys asked and did not ask
• what precautions were taken to ensure unbiased responses
• what cultural and language barriers were encountered – and how they were resolved
• how responses from different countries compare and contrast to one another

There was room in this report for five large maps illustrating simulating the effects of sea level rise, yet not a single chart is devoted to the actual data collected by this research project. We are provided with no overview of the results. We know almost nothing about the methodology. We can, therefore, make no judgment whatsoever regarding the significance of this research.

We’re told on page 11 that “the majority of migrants interviewed” said their ability to earn a living was affected by climate. On page 15 we’re advised that “the majority of interviewees complained of shifting rainfall periods.” On page 18 we read that an indeterminate number of “interviewees said they plan to move away if agricultural livelihoods do not improve.”

But these sorts of statements are journalistic – not scholarly. Before they can begin to be meaningful we need to see all the numbers. Instead, this report’s message is: Trust us. We interviewed a bunch of people in a bunch of places and now we’ve written a report incorporating their responses. Don’t worry, the details aren’t important.

Some additional info was supplied during the workshop. At the outset, one of the glossy report’s co-authors made mention of the “extensive fieldwork done by [the] UN University…on five continents with more than 2,000 interviewees.” Lead author Koko Warner, an economist employed by the United Nations University, then began her remarks (at 2 minutes 27 seconds into the video) in this manner:

As Charles indicated, there are two things coming out of our report that have never been done before. One, we present empirical research based on a survey of 23 countries worldwide asking migrants and non-migrants about their perception of the role of environmental change – including climate change on their decisions to stay or to go. What we found worldwide in our 23 case studies is that there is a clear signal of environmental factors including climate change contributing to people deciding to move or being forced to move, or displaced. It’s a clear signal. [bold added]

It’s nice that Warner thinks there’s a clear signal. But before I believe in that signal I need to see hard numerical evidence. The report offers none. Here’s another Warner quote (3 minutes, 40 seconds):

In our fieldwork we asked people questions such as “Did the environment have anything to do with your decision to move?” And usually the first thing they would say was “Well, the environment?” Kind of a question mark came up. Then we would ask them, “What do you do? What’s your livelihood?” And they’d say, “Well, I’m a farmer.” We’d say “OK, why did you move?” And they would usually say things like “Well, the rains aren’t coming in the right times anymore. The soil isn’t so good. We can’t count on the weather anymore.” [bold added]

Oh, please. Has there ever been a time in which humans could count on the weather? I mean, the thing about the weather is that it’s unpredictable. It rains before you can get the hay in the barn. It freezes before the corn has matured. It hails on the tender strawberry plants. Were these interviewers and interviewees all partaking in a shared delusion? Does anyone truly believe there was once a golden era in which humans could count on the weather?

This “empirical research” was a survey of people’s perceptions. Perceptions aren’t the same as reality. Twenty-two percent of Americans say they’ve seen or felt the presence of a ghost. This does not prove that ghosts exist. (Have I mentioned that this workshop was held in the Einstein Room?)

It appears unlikely that this massive study generated any new, reliable insight into climate change. Rather, Warner’s commentary above suggests that if a UN researcher asks enough leading questions even migrant, subsistence farmers try to provide answers they think the researcher wants to hear.

.