How to Manage Measurement Costs

Hundreds of communicators listened to a “rerun” of a Ragan Communications webinar on communication measurement on Friday, which was followed by a live Q&A in which I participated as a panelist. This came on the heels of a great discussion on Steve Crescenzo’s Corporate Hallucinations blog about whether or not communicators should conduct their own communication audits. My position is that a full-blown audit should be performed by an objective third party. Cost can be a deterrent when it comes to third-party audits or any kind of communication measurement, but you can do things to minimize the cost. Following is an excerpt from an article I wrote with my business partner Katrina Gill of Gill Research LLC in Chicago. It first appeared in IABC’s Communication World magazine.

Hiring a consultant is not necessary for every communication project, but measurement— especially in the form of focus groups, interviews and surveys—is a task in which consultants can provide great value. The main benefit is integrity of the process. Your organization wants the most accurate, reliable information with which to make communication decisions. Having a third party conduct the research, analyze results and make recommendations helps ensure objectivity.

Time is usually money when working with consultants, so taking care of these five preliminary tasks can help conserve both.

Begin with the end in mind. Stephen Covey, author of The 7Habits of Highly Effective People, gives advice about strategic planning that also applies to measurement. Know what you want to measure and why. Is it more important to measure the effectiveness of communication vehicles or the degree to which people understand the messages? It could be that you need information about a lot of aspects of your organization’s communication activities. However, we encourage you to focus your efforts on the issues that hold the greatest potential for impact on the business. If you try to measure too many things, you will end up with a long survey that limits participation or focus group discussions that are too scattered.

Use the right tool for the job. There are many measurement tools at your disposal. One might not yield the kind of information you need to link communication activities to business goals. It could be that qualitative methods like focus groups or interviews are adequate. Perhaps a survey is necessary. Often, a combination of tools is required. For example, focus groups can provide insight into the kinds of questions to ask on a survey or help you understand why survey respondents answered the way they did. Be careful not to use more tools than are necessary or to use so few that you come up short of information. Thinking ahead about what you want to measure and why should lead you to the right tool for the job.

Line up resources ahead of time. You can save time and money by doing a few things in advance of measurement. Create a measurement team that will help with survey administration or focus group and interview arrangements. If you choose a survey, decide if it will be a census survey (in which all audience members are polled) or a random sample survey (in which a representative sample of the audience is polled). Moderating focus groups and conducting interviews should be the work of a third-party consultant who is trained in these roles. Otherwise, the integrity of the information might be compromised.

Maximize participation. Communicators don’t need consultants to tell us how important it is to communicate! This is just as true when communicating about measurement. You can save time and money by already having developed a communication plan for whatever measurement activity you decide to engage in. If a survey is coming, let potential participants know what the survey is about and how the information will be used. Be clear about the process—when it will take place, how it will be administered and by whom. Send a reminder as the closing date draws near. After the survey, thank people for their participation. Most important, communicate the results and the actions you will take based on the results. Beware of the black-hole syndrome! Use the reporting of the research findings as an opportunity to model effective communication.

The same rules apply to focus groups, interviews, focus panels and any other method of gathering information. Help people know what to expect, remind them what you want them to do, thank them for doing it and close the loop by communicating results. Not only will this help ensure strong participation in the current measurement activity, it will also help drive participation next time.

By maximizing participation, you will get your money’s worth out of the measurement process. Higher participation yields greater reliability in the data. The margin of error decreases as participation increases. Your efforts to measure communication effectiveness are in vain if the reliability of data is questionable.

Ask the right questions in the right way. Writing questions for surveys, focus groups and interviews is an important step. Flaws in the phraseology of questions can lead to unreliable data. While a consultant provides value in helping to write questions, you can shorten the process by being clear on what you want to know. This tip goes back to beginning with the end in mind. You will be tempted to ask questions that either do not produce relevant information or that produce information you do not intend to act upon. Extraneous questions cost money because they make the process longer—from asking the question to processing the responses—which decreases participation.

There are three golden rules for deciding which questions to ask:

  1. There’s no “nice to know.”
  2. Only ask about things that can be changed.
  3. Research findings must lead to solutions.

You should have a good understanding of what’s important to your specific organization. Do an analysis of the current situation. What issues and facts do you already have? What are the implications? What do you still need to know? This will lead you to the information you need and the types of questions you should ask.

Minimizing the cost of a consultant

In our work with clients on communication audits, we often find that they have given little thought to these five tasks. And who can blame them? Communicators today are stretched more than ever, yet we have more demands placed upon us, such as demonstrating a return on our organization’s investment.

This, in addition to the third-party objectivity we noted earlier, is a primary reason why hiring a measurement consultant is a good idea. Measurement experts have the experience and expertise to help you get the most out of the activity. As with any service provider, however, you should know what to look for. And there are some ways to save when working with consultants:

  1.  Start off on the right foot. Provide consultants with as much information up front as you can, including background about your organization, the specific goals and objectives of the project, previous research findings and anything else that might smooth the way.
  2. Do your homework. Educate yourself about research as best you can before the project begins. You’ll know what to expect and won’t waste time with things you may not want or need. You’ll also have a better understanding of measurement techniques.
  3. Be clear about expectations. With the consultant’s help, decide what can be done in-house and what needs to be outsourced. Let them know the areas in which you want their help and the things you can do yourself. For example, your  IT department can pull random names for the focus groups or send out the survey link via e-mail. The benefits of a third-party consultant are the objectivity, expertise and credibility they bring to the table.
  4. Tell the whole story. Don’t hide any information, including prior, less-than-stellar research findings or potential obstacles to the research process such as people who oppose research. The more the consultants know, the more they’ll be able to help you.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: