How to conduct a "comics survey"

(This is a departure from our usual format, but it's an important topic.)

There are a couple of "comics surveys" currently being conducted in major markets, which I know because of the complaints from cartoonists about how unfair, unscientific and invalid these things are, and how they defeat the purpose of gaining readers by catering to a small, vocal group.

I agree, and would add that, from my own professional experience, I know that these negatives are completely unnecessary.

Comics are an important part of what a newspaper offers, and the selection should be a combination of strips that will please current readers while attracting and retaining new readership.

I share the frustration of cartoonists over poorly-conducted "surveys" not only because they work against the best interests of the newspapers themselves but because I know it is not only possible but easy to gain reader feedback in a more constructive way:

Background

In 2003, I re-did the comics at a small (25,000 daily circulation) newspaper which, at the time, had 21 strips running Monday-Saturday and a pre-packaged Sunday section. At the conclusion of the process, we had dropped five strips, added six and I had renegotiated the Sunday section to make it a custom piece incorporating all our daily strips.

There were perhaps half a dozen complaints about the changes, few of them "furious" in tone, and we enjoyed a five-figure savings in annual costs, chiefly due to the renegotiated Sunday package.

Here's how:

The first step in conducting a comics survey is to
not call it a "comics survey"

Unless you are actually hiring a polling company to conduct a true survey, you will not end up with a "survey" or a "poll" but simply answers from a self-selected group that will skew older than your actual audience. Young, active people may intend to answer, but it is retired people who have the time.

Relying on the results of this type of "survey" will help you retain your oldest and most devoted readers, but it will not help you retain or attract younger ones. Older readers have value, but they are not the only audience you should be appealing to.

Assuming you do not have several thousand dollars to spend on a statistically, methodologically valid survey of readers and would-be readers, your goal should be to use feedback from current readers to help you make the best decisions.

Ask your readers for help, but retain
ownership of your newspaper

As I write this, there is a funny story going around about a British environmental agency that asked the public to help choose a name for a research vessel, and the leading entry so far is "Boaty McBoatface." However, the critical factor is that they simply asked for help and did not bind themselves to live with the results of an Internet prank.

Similarly, you should make it plain to readers that you are asking for input in a way that does not suggest a "vote." In addition to not saying "survey" or "poll," do not say "vote."

They are not voting. They are simply expressing preferences.

Here's how we did it:

We told readers we were planning changes to our comics and said that we wanted their feedback to avoid two outcomes:

1. Dropping a strip they truly enjoyed.
2. Retaining a strip few people liked.

Then we divided our strips into three categories of seven strips each:

Social commentary, which included not only Mallard Fillmore, but Non Sequitur and some edgier comics.

Family Strips such as Rose is Rose, Sally Forth and Stone Soup

Old Favorites like Beetle Bailey, Dagwood and Hagar the Horrible.

(You might set other categories, but keep it simple and keep it balanced. I chose those in order to set only three divisions and to have an equal number of strips in each.)

Readers were then asked to pick the one strip in each category that they most wanted us to keep, and the one that they would be least sorry to see depart.

If you ask about every strip, you will get positive or negative responses to strips the reader doesn't actually care about, even if you offer a "no opinion" option. Our method forced readers to focus only on the strips about which they had strong feelings.

Asking about each strip also provides an overall opinion on types rather than on individual strips: The reader will up-vote all the strips in a favored category and down-vote all the strips in a category they do not like. Assuming your goal is a balanced selection for a diverse audience, this works against it.

In our case, some readers wrote "Keep all of these!" across a particular group. We threw those responses into the trash and relied on the ones that followed instructions.

Define your demographics

We asked respondents for their age, and invited them to submit separate answers for each person in their family. Then we considered their input not by total numbers but by our desired audience mix.

As expected, we had far more respondents in the 55+ age group than in the 18-34 group. Not only was this not proportional to our subscriber base, but it was not proportional to our desired readership demographic.

Once it was clear how that older demographic felt --and they were surprisingly consistent in which strips they really liked and which ones they could live without --I honestly stopped keeping a tally.

With the smaller number of under-18s and 18-34's, however, I had to pay more attention because they were fewer and their responses therefore tended to be more individualistic. That is, they were not so statistically significant that you could simply go by numbers, and parsing their preferences required a little more interpretation.

Do not announce "results"

Since we never called it a "survey" and never invited them to "vote," there was no reason to report the exact numbers. Ranking strips simply invites discontent and sparks demands for reconsideration.

When we announced our decisions, we did note that some dropped strips had very strong negatives and that some retained strips had very strong positives, but we didn't provide specifics.

We focused, rather, on describing the new strips we were adding, and this was the occasion of a weekly series of cartoonist interviews that you can find here by searching for "Sunday Profile."

As said, we had very few real complaints, perhaps two angry responses.

The others were more along the lines of "I'm sorry you canceled _____, but I'll give these a chance," and we had one fellow who had long lobbied for a particular strip that we didn't add and he was, understandably, disappointed, but, again, not angry.

What can comics fans do about it?

If your paper is already conducting a survey, it's too late. But if it's been a while since they're freshened their comics page, why not email the editor and suggest that they consider doing it, and that they come to Comic Strip of the Day and click the link in the righthand rail to find out how simple, valid and painless it can be?

(Editors: Feel free to email me for more information.)

Comments

You can follow this conversation by subscribing to the comment feed for this post.

Geez Mike ... way too intelligently designed and carefully implemented. Thanks again for the view from the other side.

Posted by: Dave from Philadelphia | 03/22/2016 at 09:24 AM

I agree with Dave. The mathematician in me is always interested in knowing how people get their "answers". Thanks for "showing your work". It looks accurate.

Posted by: Bookworm | 03/22/2016 at 06:54 PM