<p>Questionnaires and measurement scales typically take a lot of revision and validation. Otherwise, it is very common that you measure things you don't intend to measure, are missing relevant options, or omit critical factors entirely.</p>

<p>I'd recommend some piloting and to use existing and validated measures whenever possible.</p>

<p>Some of your items sound incredibly broad to me. Are you going to rate "influence of emotions" on a 7-point scale by asking people "On a scale of 1-7, how emotional do you feel about this situation?" That might be fine, but there's also a ton of nuance to emotion that will be missed.</p>

<p>You also have a lot of factors, so it will take a <em>lot</em> of data to resolve their relative influence (potentially hundreds of cases; thousands if you care about interactions between factors) in order to determine proper weights, and you <em>still</em> need to come up with a ground truth "conflict difficulty" outcome measure. Otherwise you are just guessing at weights which isn't very useful. </p>

<p>Make sure you have IRB approval before continuing with human subjects research (I've been surprised how unaware some tech people are about regulatory requirements, especially at Academia.SE).</p>

<p>Given this is a bachelors' thesis, it may be sufficient to <em>start</em> some of these processes rather than <em>complete</em> them. It might be simpler to just do an experiment, test two versions of your software, and compare some satisfaction outcomes. You won't know which conflicts were more difficult, but you won't necessarily have to: you can rely on random assignment to assume that the comparison groups are equal.</p>

CC BY-SA 4.0
2019-09-10T16:25:15.060Z
Thank you for your answer. I was aware that standardized questionnaires are difficult to create. I tried to make clear what I need by doing a proposal. I was looking for a questionnaire like the System Usability Scale(SUS) or Unified Theory of Acceptance and Use Technology (UTAUT) that a commonly used. Those also heavily rely on user feedback (although they are validated by other studies). I also don't strive for high accuracy but more for an indicator. About the influence of Emotions: I have 2 approaches for measuring that: One would be the questionnaire which obviously doesn't cover --
telion
CC BY-SA 4.0
2019-09-10T16:32:01.317Z
everthing, but it gives a hint. The other approach is the new textual emotion detection from nlp AIs (nlp = Natural Language Proccesing). I then can measure the amount of detected emotions and should be able to correct the hint I got from the questionnaire. If I can't measure/control for conflict difficulty I planned to do as you say and only rely on 1 or 2 conflicts. Also thanks for the IRB note. I probably need to inform myself on how that is regulated at my university (Stuttgart Germany). Thank you for your advice.
telion
CC BY-SA 4.0
2019-09-10T16:36:29.467Z
@telion Yeah, I think you might be benefited by finding a local collaborator with some expertise in this area (maybe in psychology, but also areas like human factors). I've given a general answer here that might be helpful to others, but SE isn't a great venue for figuring out all the details of a specific project. For comparison, I'm currently working on a project where 6-10 experts in the field have met weekly for the past 5 months or so designing a combination questionnaire and professional evaluation, and that's even basing it off existing work. Good luck!
Bryan Krause
CC BY-SA 4.0
2019-09-10T15:24:47.167Z
2019-09-10T15:24:47.167Z
Bryan Krause
<p><strong>Question</strong></p>

<p>I want to conduct an experiment and I found that the difficulty of a conflict to resolve could play a factor in this experiment. 
I now need an easy way of measuring the potential difficulty of a conflict, so that I can control this factor.  </p>

<p><strong>Own Proposal</strong></p>

<p>I'm coming from StackOverflow and am used to present own proposals (since psychology is not primarily my field of expertise I wanted to have your opinion for my approach: 
I thought of a questionnaire that is supposed to measure various factors that could influence the difficulty of a conflict. 
The factors I thought of are: </p>

<ul>
<li><strong>Personal Involvement</strong> (PI): The potential change and stress that a given conflict can have on the life of a conflict party. e.g. a colleague is telling you to bring out the trash while you don't feel like doing it. </li>
<li><strong>Influence of Emotions</strong> (IoE): How strong Emotions influence conflicts. </li>
<li><strong>Objective vs. Subjective</strong> (OvS): Are Arguments in the conflict based on evidence or are more testimonial kind. </li>
<li><strong>Risk</strong> (R): What is the probability and damage of an undesired outcome for a conflict party? </li>
<li><strong>Level of Society</strong> (LoS): 1-to-1 vs. 1-to-many vs. many-to-many, also is it a conflict between countries of just a few people. personal-family-inner friend circle-outer friend circle-foreign groups-countries-cultures ... just to give some example to what that could include.  </li>
<li><strong>potential violence</strong> (PV): Probability and potential dangerousness of on party using violent measures. </li>
<li><strong>explicitness</strong> of conflict result(s) (ECR): I think some conflicts have explicit results like "Do Aliens exist?" and some don't necessarily have a definite answer like "What is an appropiate outfit for a woman? (which is answered differently in different cultures)" </li>
<li><strong>conflict</strong> resolution skills of the conflict partys (CRS): This also might include negotiating and communication skills as well. </li>
<li><strong>medium</strong> (M): speech, written, social network, face-to-face, etc. </li>
</ul>

<p>I then thought of measuring those metrics on a scale of 1-7, weight them and add them add for some kind of conflict difficulty score:</p>

<p><a href="https://i.sstatic.net/RXRZ4.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/RXRZ4.gif" alt="enter image description here"></a></p>

<p>Say Risk has a 10% influence on the conflict difficulty the weight, therefore, would be 0,1*(Scale of Risk). 
I also think this way the result might be 2 different scores depending on the partys point of view. </p>

<p>I also think that some of the metrics I proposed are not needed or cannot be measured using a questionnaire like the medium for example. 
But before I really go all the way to create a new questionnaire I wanted to know if I'm on the right track, if there is literature on this topic I missed, if I missed other important factors and so on. </p>

<p><strong>Background</strong></p>

<p>I have implemented a chat tool for my bachelors thesis, that displays emotions and handles profanity in messages differently. I know want to evaluate this software. The goal of this software is to improve communication in conflict situations and a new solution for hate speech in social networks. 
I also have a software engineer background and this is my first question in the psychology sector. </p>

<social-psychology><measurement><experiment-design><social-networks><conflict>
How to measure/predict the difficulty of a conflict?
telion
# How to measure/predict the difficulty of a conflict?

(Asked by: username_0 on 2019-09-10)

**Question**

I want to conduct an experiment and I found that the difficulty of a conflict to resolve could play a factor in this experiment.
I now need an easy way of measuring the potential difficulty of a conflict, so that I can control this factor.

**Own Proposal**

I'm coming from StackOverflow and am used to present own proposals (since psychology is not primarily my field of expertise I wanted to have your opinion for my approach:
I thought of a questionnaire that is supposed to measure various factors that could influence the difficulty of a conflict.
The factors I thought of are:

* **Personal Involvement** (PI): The potential change and stress that a given conflict can have on the life of a conflict party. e.g. a colleague is telling you to bring out the trash while you don't feel like doing it.
* **Influence of Emotions** (IoE): How strong Emotions influence conflicts.
* **Objective vs. Subjective** (OvS): Are Arguments in the conflict based on evidence or are more testimonial kind.
* **Risk** (R): What is the probability and damage of an undesired outcome for a conflict party?
* **Level of Society** (LoS): 1-to-1 vs. 1-to-many vs. many-to-many, also is it a conflict between countries of just a few people. personal-family-inner friend circle-outer friend circle-foreign groups-countries-cultures ... just to give some example to what that could include.
* **potential violence** (PV): Probability and potential dangerousness of on party using violent measures.
* **explicitness** of conflict result(s) (ECR): I think some conflicts have explicit results like "Do Aliens exist?" and some don't necessarily have a definite answer like "What is an appropiate outfit for a woman? (which is answered differently in different cultures)"
* **conflict** resolution skills of the conflict partys (CRS): This also might include negotiating and communication skills as well.
* **medium** (M): speech, written, social network, face-to-face, etc.

I then thought of measuring those metrics on a scale of 1-7, weight them and add them add for some kind of conflict difficulty score:

[![enter image description here](https://i.sstatic.net/RXRZ4.gif)](https://i.sstatic.net/RXRZ4.gif)

Say Risk has a 10% influence on the conflict difficulty the weight, therefore, would be 0,1\*(Scale of Risk).
I also think this way the result might be 2 different scores depending on the partys point of view.

I also think that some of the metrics I proposed are not needed or cannot be measured using a questionnaire like the medium for example.
But before I really go all the way to create a new questionnaire I wanted to know if I'm on the right track, if there is literature on this topic I missed, if I missed other important factors and so on.

**Background**

I have implemented a chat tool for my bachelors thesis, that displays emotions and handles profanity in messages differently. I know want to evaluate this software. The goal of this software is to improve communication in conflict situations and a new solution for hate speech in social networks.
I also have a software engineer background and this is my first question in the psychology sector.

**Tags:** social-psychology, measurement, experiment-design, social-networks, conflict


---

## 1 answers

### Answer by username_1 on 2019-09-10. Score: 1 | Accepted answer

Questionnaires and measurement scales typically take a lot of revision and validation. Otherwise, it is very common that you measure things you don't intend to measure, are missing relevant options, or omit critical factors entirely.

I'd recommend some piloting and to use existing and validated measures whenever possible.

Some of your items sound incredibly broad to me. Are you going to rate "influence of emotions" on a 7-point scale by asking people "On a scale of 1-7, how emotional do you feel about this situation?" That might be fine, but there's also a ton of nuance to emotion that will be missed.

You also have a lot of factors, so it will take a *lot* of data to resolve their relative influence (potentially hundreds of cases; thousands if you care about interactions between factors) in order to determine proper weights, and you *still* need to come up with a ground truth "conflict difficulty" outcome measure. Otherwise you are just guessing at weights which isn't very useful.

Make sure you have IRB approval before continuing with human subjects research (I've been surprised how unaware some tech people are about regulatory requirements, especially at Academia.SE).

Given this is a bachelors' thesis, it may be sufficient to *start* some of these processes rather than *complete* them. It might be simpler to just do an experiment, test two versions of your software, and compare some satisfaction outcomes. You won't know which conflicts were more difficult, but you won't necessarily have to: you can rely on random assignment to assume that the comparison groups are equal.

> **Comments:**
> **username_0**: Thank you for your answer. I was aware that standardized questionnaires are difficult to create. I tried to make clear what I need by doing a proposal. I was looking for a questionnaire like the System Usability Scale(SUS) or Unified Theory of Acceptance and Use Technology (UTAUT) that a commonly used. Those also heavily rely on user feedback (although they are validated by other studies). I also don't strive for high accuracy but more for an indicator. About the influence of Emotions: I have 2 approaches for measuring that: One would be the questionnaire which obviously doesn't cover --  
> **username_0**: everthing, but it gives a hint. The other approach is the new textual emotion detection from nlp AIs (nlp = Natural Language Proccesing). I then can measure the amount of detected emotions and should be able to correct the hint I got from the questionnaire. If I can't measure/control for conflict difficulty I planned to do as you say and only rely on 1 or 2 conflicts. Also thanks for the IRB note. I probably need to inform myself on how that is regulated at my university (Stuttgart Germany). Thank you for your advice.  
> **username_1**: @username_0 Yeah, I think you might be benefited by finding a local collaborator with some expertise in this area (maybe in psychology, but also areas like human factors). I've given a general answer here that might be helpful to others, but SE isn't a great venue for figuring out all the details of a specific project. For comparison, I'm currently working on a project where 6-10 experts in the field have met weekly for the past 5 months or so designing a combination questionnaire and professional evaluation, and that's even basing it off existing work. Good luck!  

---

