Validity is one of the most important concepts in survey research, as without validity, you have meaningless results.
Many people's concern with validity is whether or not their their survey or scale has it.
According to the American Psychological Association, validity "...refers to the appropriateness, meaningfulness, and usefulness of the specific inferences made from test scores." (Standards for Psychological and Educational Testing, 1985, p. 9).
In other words: your findings need to be appropriate, meaningful and useful... they need to be valid.
An item has face validity if it appears to measure the concept that it claims to measure. If items have face validity, respondents are more likely to respond to them seriously. The assessment of face validity is very informal (and therefore, my favorite...)
In some situations, we may deliberately construct items that do not have face validity... if we want to develop a scale that estimates a someone's tendency to lie or to give responses s/he thinks the researcher is looking for, we use items that appear to measure something else.
Content validity determines if the survey items are representative of the topic being measured. As a first step, you must clearly state what you are interested in measuring, then you must judge whether the items are representative of the topic.
We can assess content validity by listing the learning objectives and making sure that a large number are represented on the test and that some are not over represented. A panel of experts can also be formed to assess content validity.
Example: arithmetic ability measure that only includes multiplication, or Maths ability that only includes arithmetic, no algebra. Note that we must have a theoretical or practical idea of what it is we want to measure (the construct).
Example: Being a good psychologist - knowledge of theories, interpersonal performance, ethical, and so on.
Criterion-related validation relies on statistical analyses rather than judgments as in content validation. Criterion-related validation involves calculating a 'validity coefficient' by correlating the survey items with another measure (criteria) already known to be related to other aspects of the attribute.
Criterion-Referenced validity concerns the extent to which the current test or scale is associated with some other measure of the same concept.
Estimate the Pearson product moment correlation between two continuous measures. In order to be considered valid a new scale should have correlation of at least .6 with an existing scale.
After we develop a new scale, we may want to estimate the correlation between the new scale and an accepted existing scale.
Predictive validity assesses the extent to which the scale or test predicts future behavior or performance.
Example: The Scholastic Aptitude Test (SAT) and Graduate Record Examination (GRE) are examples of tests that are used to predict future performance. The performance is degree of success in college or graduate school.
Construct validation attempts to understand what is being measured by examining the relationship between constructs (an abstract idea used as an explanatory concept--such as mood or happiness).
Is success in college accurately measured by GPA in college? This claim is contestable as there are arguably other ways you can be a success in college -- excelling in extracurricular activities, in sports, or you may have taken very hard classes and as the result have a lower GPA than someone who took easy classes. In order for there to be construct validity, the researcher must carefully examine assumptions about the concept being measured, and the fitness of the variables to measure the concept.
Theats to Validity