<p>The basic idea is that you get a sample population. Let's say you have 100 people. You give each of those a creativity rating through some process.</p>

<p>Afterwards you give every person in the sample population a stack of questions. You run statistical tests to determine which of those questions correlate with someone being creative.  You keep all the questions that strongly correlate with your original creativity rating and throw questions that don't correlate away. </p>

<p>When making a test for a job application you also throw questions that seem to have "correct" answer and who therefore allow for cheating away. Alternatively you can also use those questions to determine the amount to which the test subject tries to game the test.</p>

CC BY-SA 3.0
2013-12-17T22:11:44.590Z
2013-12-17T22:11:44.590Z
Christian
<p>There is a large general literature evaluating the degree to which personality tests predict job performance. In particular see for example the review by Barrick et al (2001). In general such reviews find that personality measures provide a small but meaningful prediction of job performance.</p>

<h3>Ipsative testing</h3>

<p>So in general, you are asking about how personality tests work. But more specifically, you are asking about <a href="http://en.wikipedia.org/wiki/Ipsative">ipsative personality tests</a>:</p>

<blockquote>
  <p>Ipsative is a descriptor used in psychology to indicate a specific
  type of measure in which respondents compare two or more desirable
  options and pick the one that is most preferred (sometimes called a
  "forced choice" scale).</p>
</blockquote>

<p>Regarding your statement that you could not answer one of the questions, presumably the instructions asked you to indicate which statement applied most. Thus, it would be hoped that even if there were two statements that both applied to you a lot, or both applied a little, you would be able to indicate which applied more.</p>

<p>For general scientific use where you can rely on people to answer honestly, researchers typically do not use ipsative testing. Ipsative testing generally distorts the correlations between subscales.However, ipsative tests is popular in applied settings such as in selection and recruitment. The argument is that ipsative testing reduces faking. </p>

<h2>Relevant references include:</h2>

<p>Heggestad et al (2006) interpreted their results to indicate that ipsative testing was not a viable method for controlling for faking:</p>

<blockquote>
  <p>Recent research suggests multidimensional forced-choice (MFC) response
  formats may provide resistance to purposeful response distortion on
  personality assessments. It remains unclear, however, whether these
  formats provide normative trait information required for selection
  contexts. The current research evaluated score correspondences between
  an MFC format measure and 2 Likert-type measures in honest and
  instructed-faking conditions. In honest response conditions, scores
  from the MFC measure appeared valid indicators of normative trait
  standing. Under faking conditions, the MFC measure showed less score
  inflation than the Likert measure at the group level of analysis. In
  the individual-level analyses, however, the MFC measure was as
  affected by faking as was the Likert measure. Results suggest the MFC
  format is not a viable method to control faking.</p>
</blockquote>

<p>However, Bowen et al reached a more positive conclusion about ipsative testing and the degree to which it is faking-resistant:</p>

<blockquote>
  <p>The current study extends previous research by comparing empirical
  data on ipsative and normative versions of the 30-scale Occupational
  Personality Questionnaire (OPQ). Participants were randomly assigned
  to “honest” or “faking” groups. Results support the proposition that
  when the number of scales is large, the measurement dependency problem
  associated with the ipsative format is negligible. There was
  satisfactory convergent validity between ipsative and normative forms
  in both groups. On both forms, the distance between an ideal profile
  and participants' ratings were narrower in the faking group than in
  the honest group. The results suggest that though ipsative measures
  were not completely free from faking, they were relatively more
  effective in guarding against faking than more commonly used normative
  measures.</p>
</blockquote>

<p>Meade (2004) discuss ipsative measurement more generally. </p>

<blockquote>
  <p>Data are described as ipsative if a given set of responses always sum
  to the same total. However, there are many properties of data
  collection that can give rise to different types of ipsative data. In
  this study, the most common type of ipsative data used in employee
  selection (forced-choice ipsative data; FCID) is discussed as a
  special case of other types of ipsative data. Although all ipsative
  data contains constraints on covariance matrices (covariance-level
  interdependence), FCID contains additional item-level
  interdependencies as well. The psychological processes that give rise
  to FCID and the resultant psychometric properties are discussed. In
  addition, data from which both normative and ipsative responses were
  provided by job applicants illustrate very different patterns of
  correlations as well as very different selection decisions between
  normative, FCID and ipsatized measures.</p>
</blockquote>

<h3>References</h3>

<ul>
<li>Barrick, M. R., Mount, M. K., &amp; Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: what do we know and where do we go next?. International Journal of Selection and Assessment, 9(1‐2), 9-30. <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.175.8429&amp;rep=rep1&amp;type=pdf">PDF</a></li>
<li>Bowen, C. C., Martin, B. A., &amp; Hunt, S. T. (2002). A comparison of ipsative and normative approaches for ability to control faking in
personality questionnaires. International Journal of Organizational
Analysis, 10(3), 240-259.</li>
<li>Heggestad, E. D., Morrison, M., Reeve, C. L., &amp; McCloy, R. A. (2006). Forced-choice assessments of personality for selection:
Evaluating issues of normative assessment and faking resistance.
Journal of Applied Psychology, 91(1), 9.</li>
<li>Meade, A. W. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of
Occupational and Organizational Psychology, 77(4), 531-551.</li>
</ul>

CC BY-SA 3.0
2013-12-18T03:21:43.333Z
2013-12-18T03:21:43.333Z
Jeromy Anglim
<p>You have seen that one answer is to the question what do they measure (personality traits). The second answer to the question why (because empirical research repeatedly demonstrates that various personality traits can predict various types of job behaviors: mainly performance, deviance, helping, and innovativeness).</p>

<p>But the burning question...how can I fake out these questions to get the job?
A: it is unlikely that you can. Some people may tell you to try and guess the correct answer and then to maximize that response (in a forced-choice just guess the right answer). In some old school selection models, this actually makes sense. However, this assumes you know what personality-type the employer is looking for.</p>

<p>The challenge is that modern assessment vendors have a variety of response pattern detectors that are triggered when it looks like someone is trying to guess correctly and max out.  These may be actual items that only fakers will pick, like "Everyone likes me" (these are what Christian was talking about), or other more sophisticated detection methods; I work for companies like this, so can't disclose the cooler detection methods.</p>

<p>The hardest thing for the faker is that ultimately you need to know which personality traits are most important and at which level (high,med,low). There is a movement called "evidence-based management" that relies heavily on data and statistical inference in order to justify certain business practices. One such practice is how to weight the personality traits in your assessment battery for employee selection. These weightings are based on statistical models that are proprietary (both the data sources and modeling techniques), may or may not be sample-dependent, and are guaranteed to need revision over the course of a couple of years. This is because employment tests have at best a couple of years of predictive shelf-life. Trying to guess this is like trying to guess the elements of your credit score; well, probably harder. </p>

CC BY-SA 3.0
2015-04-02T22:27:44.553Z
2015-04-02T22:27:44.553Z
GoldenPandas
<p>There are different kinds of psychometric tests, I think the numerical and verbal ones are fair enough, I can see their usefulness. However, I cannot understand how the 'soft' psychometric tests work and how they apply to a particular career.</p>

<p>I just finished one which consisted of 90 'questions', except they were not questions, they were all "pick one of the two following sentences" 90 times. Some made sense, such as </p>

<blockquote>
  <ul>
  <li>Do you like to lead a group.</li>
  <li>Do you like to fit in with a group.</li>
  </ul>
</blockquote>

<p>But others I could not answer because I could not choose either option for example:</p>

<blockquote>
  <ul>
  <li>You are slow at planning.</li>
  <li>You are slow at making decisions.</li>
  </ul>
</blockquote>

<p>I have to answer one. In this case, I'd say that I am not slow in either.</p>

<p>How exactly (detail) do these types of psychometric tests work? A link to an open one would be what I am looking for. How do they create the answers? How do they pair them? Do they change based on your previous answers?</p>

<measurement><personality><io-psychology><test>
How do personality tests work that require you to choose one of two statement that applies most to you?
Sam Leach
# How do personality tests work that require you to choose one of two statement that applies most to you?

(Asked by: username_0 on 2013-12-17)

There are different kinds of psychometric tests, I think the numerical and verbal ones are fair enough, I can see their usefulness. However, I cannot understand how the 'soft' psychometric tests work and how they apply to a particular career.

I just finished one which consisted of 90 'questions', except they were not questions, they were all "pick one of the two following sentences" 90 times. Some made sense, such as

> * Do you like to lead a group.
> * Do you like to fit in with a group.

But others I could not answer because I could not choose either option for example:

> * You are slow at planning.
> * You are slow at making decisions.

I have to answer one. In this case, I'd say that I am not slow in either.

How exactly (detail) do these types of psychometric tests work? A link to an open one would be what I am looking for. How do they create the answers? How do they pair them? Do they change based on your previous answers?

**Tags:** measurement, personality, io-psychology, test


---

## 3 answers

### Answer by username_1 on 2013-12-17. Score: 2 | Accepted answer

The basic idea is that you get a sample population. Let's say you have 100 people. You give each of those a creativity rating through some process.

Afterwards you give every person in the sample population a stack of questions. You run statistical tests to determine which of those questions correlate with someone being creative. You keep all the questions that strongly correlate with your original creativity rating and throw questions that don't correlate away.

When making a test for a job application you also throw questions that seem to have "correct" answer and who therefore allow for cheating away. Alternatively you can also use those questions to determine the amount to which the test subject tries to game the test.


---

### Answer by username_2 on 2013-12-18. Score: 6

There is a large general literature evaluating the degree to which personality tests predict job performance. In particular see for example the review by Barrick et al (2001). In general such reviews find that personality measures provide a small but meaningful prediction of job performance.

### Ipsative testing

So in general, you are asking about how personality tests work. But more specifically, you are asking about [ipsative personality tests](http://en.wikipedia.org/wiki/Ipsative):

> Ipsative is a descriptor used in psychology to indicate a specific
> type of measure in which respondents compare two or more desirable
> options and pick the one that is most preferred (sometimes called a
> "forced choice" scale).

Regarding your statement that you could not answer one of the questions, presumably the instructions asked you to indicate which statement applied most. Thus, it would be hoped that even if there were two statements that both applied to you a lot, or both applied a little, you would be able to indicate which applied more.

For general scientific use where you can rely on people to answer honestly, researchers typically do not use ipsative testing. Ipsative testing generally distorts the correlations between subscales.However, ipsative tests is popular in applied settings such as in selection and recruitment. The argument is that ipsative testing reduces faking.

## Relevant references include:

Heggestad et al (2006) interpreted their results to indicate that ipsative testing was not a viable method for controlling for faking:

> Recent research suggests multidimensional forced-choice (MFC) response
> formats may provide resistance to purposeful response distortion on
> personality assessments. It remains unclear, however, whether these
> formats provide normative trait information required for selection
> contexts. The current research evaluated score correspondences between
> an MFC format measure and 2 Likert-type measures in honest and
> instructed-faking conditions. In honest response conditions, scores
> from the MFC measure appeared valid indicators of normative trait
> standing. Under faking conditions, the MFC measure showed less score
> inflation than the Likert measure at the group level of analysis. In
> the individual-level analyses, however, the MFC measure was as
> affected by faking as was the Likert measure. Results suggest the MFC
> format is not a viable method to control faking.

However, Bowen et al reached a more positive conclusion about ipsative testing and the degree to which it is faking-resistant:

> The current study extends previous research by comparing empirical
> data on ipsative and normative versions of the 30-scale Occupational
> Personality Questionnaire (OPQ). Participants were randomly assigned
> to “honest” or “faking” groups. Results support the proposition that
> when the number of scales is large, the measurement dependency problem
> associated with the ipsative format is negligible. There was
> satisfactory convergent validity between ipsative and normative forms
> in both groups. On both forms, the distance between an ideal profile
> and participants' ratings were narrower in the faking group than in
> the honest group. The results suggest that though ipsative measures
> were not completely free from faking, they were relatively more
> effective in guarding against faking than more commonly used normative
> measures.

Meade (2004) discuss ipsative measurement more generally.

> Data are described as ipsative if a given set of responses always sum
> to the same total. However, there are many properties of data
> collection that can give rise to different types of ipsative data. In
> this study, the most common type of ipsative data used in employee
> selection (forced-choice ipsative data; FCID) is discussed as a
> special case of other types of ipsative data. Although all ipsative
> data contains constraints on covariance matrices (covariance-level
> interdependence), FCID contains additional item-level
> interdependencies as well. The psychological processes that give rise
> to FCID and the resultant psychometric properties are discussed. In
> addition, data from which both normative and ipsative responses were
> provided by job applicants illustrate very different patterns of
> correlations as well as very different selection decisions between
> normative, FCID and ipsatized measures.

### References

* Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: what do we know and where do we go next?. International Journal of Selection and Assessment, 9(1‐2), 9-30. [PDF](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.175.8429&rep=rep1&type=pdf)
* Bowen, C. C., Martin, B. A., & Hunt, S. T. (2002). A comparison of ipsative and normative approaches for ability to control faking in
  personality questionnaires. International Journal of Organizational
  Analysis, 10(3), 240-259.
* Heggestad, E. D., Morrison, M., Reeve, C. L., & McCloy, R. A. (2006). Forced-choice assessments of personality for selection:
  Evaluating issues of normative assessment and faking resistance.
  Journal of Applied Psychology, 91(1), 9.
* Meade, A. W. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of
  Occupational and Organizational Psychology, 77(4), 531-551.


---

### Answer by username_3 on 2015-04-02. Score: -1

You have seen that one answer is to the question what do they measure (personality traits). The second answer to the question why (because empirical research repeatedly demonstrates that various personality traits can predict various types of job behaviors: mainly performance, deviance, helping, and innovativeness).

But the burning question...how can I fake out these questions to get the job?
A: it is unlikely that you can. Some people may tell you to try and guess the correct answer and then to maximize that response (in a forced-choice just guess the right answer). In some old school selection models, this actually makes sense. However, this assumes you know what personality-type the employer is looking for.

The challenge is that modern assessment vendors have a variety of response pattern detectors that are triggered when it looks like someone is trying to guess correctly and max out. These may be actual items that only fakers will pick, like "Everyone likes me" (these are what username_1 was talking about), or other more sophisticated detection methods; I work for companies like this, so can't disclose the cooler detection methods.

The hardest thing for the faker is that ultimately you need to know which personality traits are most important and at which level (high,med,low). There is a movement called "evidence-based management" that relies heavily on data and statistical inference in order to justify certain business practices. One such practice is how to weight the personality traits in your assessment battery for employee selection. These weightings are based on statistical models that are proprietary (both the data sources and modeling techniques), may or may not be sample-dependent, and are guaranteed to need revision over the course of a couple of years. This is because employment tests have at best a couple of years of predictive shelf-life. Trying to guess this is like trying to guess the elements of your credit score; well, probably harder.


---

