by Terri Ann Lowenthal
What if we took a survey and no one answered? Or, to be more realistic, only two-thirds of us did?
That’s what happened north of the border recently. The Canadian Parliament decided to do away with the nation’s mandatory long-form survey and replace it with the voluntary National Household Survey (NHS). Statistics Canada (StatCan) reported the results of the first NHS, conducted in 2011, this week. Instead of the 94 percent response rate achieved with the 2006 mandatory long form, only 68 percent of households returned the voluntary survey. Instead of having reliable data for 97 percent of the country, only three-quarters of Canada’s localities will have a picture of their socio-economic conditions.
In abolishing the mandatory survey, conservatives decried the burden on Canadians of revealing “personal” information to the government. How ironic, then, that in order to make up for projected falling response rates, StatCan increased the number of households that received the survey, from one in five to one in three. That’s a 65 percent jump!
Now that we’ve recovered from the initial shock of a proposal (H.R. 1638) to axe just about everything the Census Bureau does, legislation to make American Community Survey (ACS) response optional might seem relatively tame, if not harmless. Think again, census stakeholders.
Rep. Ted Poe (R-TX), citing “big government at its worst,” reintroduced a bill (H.R. 1078) to let people ‘just say no’ to all or part of the survey. (See my March 20, 2013, post.) A 2003 field test of a voluntary ACS, which Congress demanded, gave a glimpse of the stiff consequences of such a significant change in methodology. Response rates would plummet, especially for traditionally hard-to-measure population groups, and costs would skyrocket (by at least 30 percent), as the Census Bureau scrambles to ensure enough response to produce accurate data for towns, small counties, rural communities, neighborhoods and smaller population groups such as veterans, people with disabilities and ethnic subgroups. The Canadian experience, the first of its kind to our knowledge, bears this out.
Congress doesn’t seem in the mood to allocate more money for good data; the Census Bureau already is reeling from an 11 percent budget cut this year (13 percent if you count the $18 million dip into the Working Capital Fund). The bureau might have to follow StatCan’s lead and put a warning on all small-area data estimates: Use at your own risk due to high non-response error. Translation: The data are flawed because some population groups are less likely to respond than others and therefore skew the representation of the sample.
More likely, we might not see any data for small areas because the bureau won’t have the money to compensate for plummeting response rates by increasing the sample size (that’s sampling error, folks) like StatCan did. Forty-one percent of U.S. counties are home to less than 20,000 people; even with a mandatory ACS, the Census Bureau must aggregate data over five years to accumulate enough responses to yield statistically valid estimates for these areas.
New York? Most counties are larger, although we’d lose information about communities and neighborhoods within counties, making it difficult for local governments and businesses to target services and investment dollars. But bye-bye to most of Montana, the Dakotas, Nebraska and Kansas, Idaho and Iowa. You can wipe half of Texas, Nevada, Wyoming and Utah, much of Colorado, Missouri, Georgia, West Virginia, Kentucky, Arkansas and Minnesota, and not insignificant portions of other states off the map. No data for 95 percent of American Indian reservations and Alaska Native areas, most elementary school districts, and more than half of secondary school districts. How is anyone supposed to make rational decisions without all of this local information?
Meanwhile, joining the list of conservative voices that appreciates the value of objective, reliable data to support decision-making is The Weekly Standard. A May 20 article calls the ACS “one of the most robust and important tools we have for measuring and understanding American trends.” Ironically, The Weekly Standard admonished the Census Bureau for deciding, because ACS content is now a zero sum game, to drop the question on how many times a person has been married, to make room for questions on use of health care subsidies and premiums that will help policymakers assess the effectiveness of the Affordable Care Act (okay, Obamacare).
Raise your hand if you remember what happened the last time the Census Bureau tried to mess with a census question on marriage? Well, before the 2000 count — when the census long form still ruled the data world — the bureau thought it might streamline the short form that everyone received, by shifting a question on marital status to the sample (or long) form. You would have thought someone proposed abolishing Mother’s Day! Very conservative Sen. Jesse Helms (R-NC), incensed at the inference that marriage was no longer a “sacred institution” — and who had been complaining for years that the census form was too long — proposed an amendment (to the Transportation appropriations bill, 106th Congress) in support of keeping the question on the short form.
So, we have some conservatives railing against the public burden of so many nosy questions, and others urging the government to keep asking how many times you’ve been married. While Sen. Helms and conservative colleagues (e.g. John Ashcroft, Sam Brownback) were fighting to save the marriage question, the same Senate went on record urging Americans to answer only the long form questions they liked in the 2000 census. Yes, I feel a census headache coming on…!!
The Census Project 
“But bye-bye to most of Montana, the Dakotas, Nebraska and Kansas, Idaho and Iowa. You can wipe half of Texas, Nevada, Wyoming and Utah, much of Colorado, Missouri, Georgia, West Virginia, Kentucky, Arkansas and Minnesota, and not insignificant portions of other states off the map.”
In other words, the states populated with large numbers of Tea-Party Know-Nothings, the very people who are demanding an end to the Census, would become black holes on the map? Great! Let them have what they want. I always suspected that the Tea-Party was a self correcting problem…..kind of like an especially noxious chimp playing with a hand grenade.
Well, it is a bit ironic, Mark, that we would be left with little substate data for many areas represented by lawmakers who support making the American Community Survey voluntary. The same thing happened in Canada with the shift to the voluntary National Household Survey; many areas for which Statistics Canada could not produce reliable data are represented by the faction of Parliament that pushed to do away with the mandatory long form. Thanks for reading the blog!
How did Iowa, Nevada, Colorado, and Minnesota vote last year? Those states are not exactly dominated by “Tea-Party Know-Nothings”. Not to mention that there are significant constituencies within ALL states, regardless of political leanings, that recognize the value of thorough and accurate census data. It is essential that these groups make their voices heard and emphasize how this crucial information will help promote more efficient allocation of resources in both the private and public sectors. This blog is definitely one of the great instruments to help circulate the message.
JAS, thanks for your kind words about the blog! I honestly don’t know, without checking, how those states voted in 2012; I view the census as a bipartisan (or nonpartisan) issue. But you are absolutely right: There are important stakeholders in all states and communities, including (especially) state and local leaders who must prepare for disasters, emergency response, future schools and senior centers, assistance for people with disabilities, updated housing . . . the list of vital services many take for granted goes on and on.
“New York? Most counties are larger, although we’d lose information about communities and neighborhoods within counties, making it difficult for local governments and businesses to target services and investment dollars.”
What hat would be effect on business in New York. and what about of 95 percent and rational decision which is happening in the most counties
Eryn, if I understand your comment correctly, businesses rely heavily on data about smaller areas, such as communities, as well as data for counties, towns, etc., in deciding where to build new plants or offices and offer services, and what kinds of services and products (generally) to offer, all of which creates jobs and grows local economies. If the Census Bureau can’t produce data for smaller areas, which includes 40 percent of the counties in the country, businesses might ignore those areas or make decisions based on less reliable information; it’s never good for a community when businesses fail because they set up shop in the wrong place! The need for objective, comparable data across all communities is simply a matter of helping lawmakers at all levels (down to school boards and local law enforcement and first responders) make prudent decisions on where to allocate resources and invest both public and private dollars in ways that grow the economy and improve living conditions for all.
I’ll leave others to debate ACS, I personally don’t have a big problem with its mandatory nature. As a data user ACS data is a helpful annual update on important data points but for small area data like census tracts and truly small area geography it is practically useless because of the error margins. At least I had marginally useful data on small areas with the long-form census every ten years but now I have really useless data every year. I would recommend having a somewhat larger question set for the decennial census to provide at least a passable baseline for small area data and perhaps expanding the use of administrative data to make valid estimates of other important data points. I have found over the years that many people (not all) just want the data and don’t always care much about the quality. By promoting and defending ACS, Census and its defenders are in someways perpetrating a myth that ACS Census data is quality data. Let the buyer beware.