<p>The paper "<a href="https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/can-robots-make-good-models-of-biological-behaviour/2EDDC170407C49D33A59D514918C2FDA" rel="nofollow noreferrer">Can robots make good models of biological behaviour</a>" by Barbara Webb comes from a slightly different area, i.e. modeling biological organisms with robotics, making artificial cockroaches for example.</p>

<p>It has a brilliant theoretical examination of modeling in terms of epistemology and philosophy of science. For your question it is possible that your skepticism is with regard dimensions 5 and 6 mentioned in the paper, i.e. structural accuracy and performance match.</p>

<p>In terms of structural accuracy for example, if a social network uses random graphs, like Erdős–Rényi, it makes an assumption that is not met by the actual degree distributions of human social networks.</p>

<p>In terms of performance match, the accuracy of the model's prediction can be assessed at various levels of precision. For example, the model may 'capture the trends' qualitatively, which may or may not be good enough for large scale simulations. </p>

<p>On the other extreme one would find precise numeric predictions and examine their fit to real data. I guess that most models fall somewhere in between.</p>

CC BY-SA 3.0
2017-08-30T20:20:11.627Z
2017-08-31T14:00:36.670Z
2017-08-31T14:00:36.670Z
nijk
<h1>Existing Papers</h1>
<p>I found three papers in the same vein with considerably more empirical evidence.</p>
<h2>1. <a href="http://www.jstor.org/stable/3118226?seq=1#page_scan_tab_contents" rel="nofollow noreferrer">Modeling the Size of Wars</a></h2>
<p>In the paper, provinces and conflicts are modeled to justify Richardson's observation that the proportion of conflict severity in relation to their frequency is described by a power law. In other words, the more space there is between each conflict, the more casualties will result. The models uses a lot of detail. A geographical map is created and conflicts (down to technological advancement, political structural change and resource allocation) are simulated in models with dozens of parameters. More importantly, the level of detail in the paper allows parameters to be set to test historical scenarios to disprove or give further evidence to the model.</p>
<h2>2. <a href="http://asr.sagepub.com/content/72/5/784.full.pdf" rel="nofollow noreferrer">The Dynamics of Polarisation</a></h2>
<p>The focus of the paper is modeling the change of public opinion in the United States as a way to provide explanation for two phenomenon:</p>
<ol>
<li>Polarization of opinion is rare despite being perceived otherwise</li>
<li>Opinion homogeneity is rare, despite being perceive otherwise.</li>
</ol>
<p>To model these phenomenon, a network similar to that of the Opinion Dynamics paper from the question is built, with imposed homophily. However, the paper grounds it's parameters in reality (for example, take-off issues that stimulate much discussion are considered rare). In contrast, the Opinion Dynamics paper creates parameters for skepticism and empathy, without giving much consideration for the mechanisms behind these attributes and how they might change over time.</p>
<h2>3. <a href="http://jasss.soc.surrey.ac.uk/20/1/13.html" rel="nofollow noreferrer">A Psychologically-Motivated Model of Opinion Change with Applications to American Politics</a></h2>
<p>This paper justifies modeling 2D distributed agents using the psychological parameters Influence, Susceptibility and Conformity and their interaction. The model is then validated on American political opinions as they shift over time.</p>
<h1>Conclusion</h1>
<p>There are papers with empirical basis in the domain of social network models. Typically, this realism is achieved by basing parameters on empirical evidence, rather than simplification of cognitive phenomena.</p>
<h1>Reply from author of Social Network papers</h1>
<p>The author of two of the papers I cited in the question, Alan Tsang, was kind enough to provide a rebuttal to my scepticism via personal correspondence:</p>
<blockquote>
<p>The premise behind both the papers was to examine the effects of a
particular psychological phenomenon by examining it in isolation.  We
base our work on more conventional agent models from economics, which
assume completely rational actors.  We want to see what happens when
the rational behavior is tweaked to incorporate a behavioral
component.  In particular, we are interested in the qualitative
effects that are produced, and the mechanisms by which these are
achieved.  An agent based simulation is the ideal way of studying
this, because it allows us to drill down and take measurements that
would be impossible or very difficult to do in an actual community.
So what we are doing is more mathematics and less science.  The
results of the papers provide qualitative insights on what are
plausible effects of these behaviors on a larger system.  The next
step could certainly be to validate the model against real data, but
the goal of the paper is not to perform &quot;full stack&quot; science.  Rather,
we hope it would prove useful component in a more detailed model to be
used on data collect in the wild.</p>
<p>That said, we are also interested in validating our model against real
data as well, but they are difficult to come by.  For instance,
Facebook almost certainly has access to data that can be used to
detect homophily in networks, and how they might affect opinions over
time.  But the data is very proprietary and, even if we were to get
it, ethical concerns might limit how it can be used.  As you've
pointed out in the follow-up post, there are a few papers that examine
political affiliation, and those could potentially be useful ground
truth.  But I don't have the background to properly assemble such a
data set from scratch (as it would need both a time series of
opinions, and the underlying network structure).  Moreover, there are
sure to be other effects at work as well, which may confound the
analysis.  We're giving some thought to gathering data for the last
Canadian election, since strategic voting was so widespread and
successful, but any data we collect will likely be disconnected from
social network structure.  Maybe some aggregate network properties
could be inferred (ex: based on region), but it would be a
multi-layered problem.</p>
</blockquote>

CC BY-SA 3.0
2015-10-27T22:10:22.200Z
2017-10-31T19:00:30.237Z
2020-06-17T09:55:31.633Z
Seanny123
<p>I've recently encountered the domain of social network modelling while reading the papers on models of strategic voting (unpublished), <a href="https://cs.uwaterloo.ca/~klarson/papers/TsangAAMAS2014.pdf">opinion dynamics</a> and <a href="http://www.cs.bris.ac.uk/~flach/ECMLPKDD2012papers/1125789.pdf">influence spread</a>.</p>

<p>All of these papers create complex computational models, usually modelling social networks as graphs and inferring that their behaviour match actual social behaviour.</p>

<p>However, I don't understand how these models can be empirically rigorous. It seems to me that the ways to validate a model is to match with empirical data, either via data matching or prediction. This doesn't seem to be done in any of those papers.</p>

<p>Consequently, how are these models validated?</p>

CC BY-SA 3.0
2016-02-08T01:04:58.237Z
You might be interested in [this blog post](https://egtheory.wordpress.com/2016/02/04/hadza/) where I discuss some of the discrepencies between what social network data looks like and how it is used in (evolutionary game theory) models.
Artem Kaznatcheev
<cognitive-modeling><social-networks><sociology>
Empirical proof for social network models
Seanny123
# Empirical proof for social network models

(Asked by: username_0 on 2015-10-18)

I've recently encountered the domain of social network modelling while reading the papers on models of strategic voting (unpublished), [opinion dynamics](https://cs.uwaterloo.ca/~klarson/papers/TsangAAMAS2014.pdf) and [influence spread](http://www.cs.bris.ac.uk/~flach/ECMLPKDD2012papers/1125789.pdf).

All of these papers create complex computational models, usually modelling social networks as graphs and inferring that their behaviour match actual social behaviour.

However, I don't understand how these models can be empirically rigorous. It seems to me that the ways to validate a model is to match with empirical data, either via data matching or prediction. This doesn't seem to be done in any of those papers.

Consequently, how are these models validated?

**Tags:** cognitive-modeling, social-networks, sociology

> **Comments:**
> **username_1**: You might be interested in [this blog post](https://egtheory.wordpress.com/2016/02/04/hadza/) where I discuss some of the discrepencies between what social network data looks like and how it is used in (evolutionary game theory) models.  

---

## 2 answers

### Answer by username_0 on 2015-10-27. Score: 5 | Accepted answer

# Existing Papers

I found three papers in the same vein with considerably more empirical evidence.

## 1. [Modeling the Size of Wars](http://www.jstor.org/stable/3118226?seq=1#page_scan_tab_contents)

In the paper, provinces and conflicts are modeled to justify Richardson's observation that the proportion of conflict severity in relation to their frequency is described by a power law. In other words, the more space there is between each conflict, the more casualties will result. The models uses a lot of detail. A geographical map is created and conflicts (down to technological advancement, political structural change and resource allocation) are simulated in models with dozens of parameters. More importantly, the level of detail in the paper allows parameters to be set to test historical scenarios to disprove or give further evidence to the model.

## 2. [The Dynamics of Polarisation](http://asr.sagepub.com/content/72/5/784.full.pdf)

The focus of the paper is modeling the change of public opinion in the United States as a way to provide explanation for two phenomenon:

1. Polarization of opinion is rare despite being perceived otherwise
2. Opinion homogeneity is rare, despite being perceive otherwise.

To model these phenomenon, a network similar to that of the Opinion Dynamics paper from the question is built, with imposed homophily. However, the paper grounds it's parameters in reality (for example, take-off issues that stimulate much discussion are considered rare). In contrast, the Opinion Dynamics paper creates parameters for skepticism and empathy, without giving much consideration for the mechanisms behind these attributes and how they might change over time.

## 3. [A Psychologically-Motivated Model of Opinion Change with Applications to American Politics](http://jasss.soc.surrey.ac.uk/20/1/13.html)

This paper justifies modeling 2D distributed agents using the psychological parameters Influence, Susceptibility and Conformity and their interaction. The model is then validated on American political opinions as they shift over time.

# Conclusion

There are papers with empirical basis in the domain of social network models. Typically, this realism is achieved by basing parameters on empirical evidence, rather than simplification of cognitive phenomena.

# Reply from author of Social Network papers

The author of two of the papers I cited in the question, Alan Tsang, was kind enough to provide a rebuttal to my scepticism via personal correspondence:

> The premise behind both the papers was to examine the effects of a
> particular psychological phenomenon by examining it in isolation. We
> base our work on more conventional agent models from economics, which
> assume completely rational actors. We want to see what happens when
> the rational behavior is tweaked to incorporate a behavioral
> component. In particular, we are interested in the qualitative
> effects that are produced, and the mechanisms by which these are
> achieved. An agent based simulation is the ideal way of studying
> this, because it allows us to drill down and take measurements that
> would be impossible or very difficult to do in an actual community.
> So what we are doing is more mathematics and less science. The
> results of the papers provide qualitative insights on what are
> plausible effects of these behaviors on a larger system. The next
> step could certainly be to validate the model against real data, but
> the goal of the paper is not to perform "full stack" science. Rather,
> we hope it would prove useful component in a more detailed model to be
> used on data collect in the wild.
>
> That said, we are also interested in validating our model against real
> data as well, but they are difficult to come by. For instance,
> Facebook almost certainly has access to data that can be used to
> detect homophily in networks, and how they might affect opinions over
> time. But the data is very proprietary and, even if we were to get
> it, ethical concerns might limit how it can be used. As you've
> pointed out in the follow-up post, there are a few papers that examine
> political affiliation, and those could potentially be useful ground
> truth. But I don't have the background to properly assemble such a
> data set from scratch (as it would need both a time series of
> opinions, and the underlying network structure). Moreover, there are
> sure to be other effects at work as well, which may confound the
> analysis. We're giving some thought to gathering data for the last
> Canadian election, since strategic voting was so widespread and
> successful, but any data we collect will likely be disconnected from
> social network structure. Maybe some aggregate network properties
> could be inferred (ex: based on region), but it would be a
> multi-layered problem.


---

### Answer by username_2 on 2017-08-30. Score: 2

The paper "[Can robots make good models of biological behaviour](https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/can-robots-make-good-models-of-biological-behaviour/2EDDC170407C49D33A59D514918C2FDA)" by Barbara Webb comes from a slightly different area, i.e. modeling biological organisms with robotics, making artificial cockroaches for example.

It has a brilliant theoretical examination of modeling in terms of epistemology and philosophy of science. For your question it is possible that your skepticism is with regard dimensions 5 and 6 mentioned in the paper, i.e. structural accuracy and performance match.

In terms of structural accuracy for example, if a social network uses random graphs, like Erdős–Rényi, it makes an assumption that is not met by the actual degree distributions of human social networks.

In terms of performance match, the accuracy of the model's prediction can be assessed at various levels of precision. For example, the model may 'capture the trends' qualitatively, which may or may not be good enough for large scale simulations.

On the other extreme one would find precise numeric predictions and examine their fit to real data. I guess that most models fall somewhere in between.


---

