The validity of a questionnaire relies first and foremost on reliability. If the questionnaire cannot be shown to be reliable, there is no discussion of validity.
But there is good news. Demonstrating validity is easy, compared to reliability. If you have reached this point and have a reliable instrument for measuring the issues or phenomena you are after, demonstrating its validity will not be difficult.
Validity refers to whether the questionnaire or survey measures what it intends to measure. While there are very detailed and technical ways of proving validity that are beyond the level of this discussion, there are some concepts that are useful to keep in mind. The overriding principle of validity is that it focuses on how a questionnaire or assessment process is used. Reliability is a characteristic of the instrument itself, but validity comes from the way the instrument is employed.
The following ideas support this principle:
Evensen Web Design has developed more than a hundred reliable and valid instruments for organizational research. We would welcome the opportunity to discuss your situation and hope you will contact us.
- As nearly as possible, the data gathering should match the decisions you need to make. This means if you need to make a priority-focused decision, such as allocating resources or eliminating programs, your assessment process should be a comparative one that ranks the programs or alternatives you will be considering.
- Gather data from all the people who can contribute information, even if they are hard to contact. For example, if you are conducting a survey of customer service, try to get a sample of all the customers, not just those who are easy to reach, such as those who have complained or have made suggestions.
- A perfect example of a questionnaire that may have high
reliability, but poor validity is a standardized questionnaire that has been used in hundreds of companies. These instruments are marketed aggressively using promises of "industry norms" to compare your results with. Weigh carefully the value of such comparisons against the almost certain lack of fit with your culture, philosophy and way of managing. A good diagnosis of your organization is not likely to come from a generic instrument with lots of normative comparisons.
|A good diagnosis of your organization is not likely to come from a generic instrument...
- If you're going after sensitive information, protect your sources. It has been said that in the Prussian army at the turn of the century, decisions were made twice, once when officers were sober, again when they were drunk. This concept acknowledges the power of the "socially acceptable response" to questions or requests. Don't assume that a simple statement printed on the questionnaire that "all individual responses will be kept confidential" will make everybody relax and provide candid answers. Give respondents the freedom to decide which information about themselves they wish to withhold, and employ other administrative procedures, such as handing out Login IDs and Passwords separately from the e-mail inviting people to participate in the survey.
Questionnaire Links . . .
Sample of Clients
Reliability and Validity
Other Questionnaire Topics
Frequently Asked Questions
Online Questionnaire Examples
Why Don't Our Surveys Look
Like Traditional Surveys?
What's Wrong With
How do I get more information?