Implementing feedback in creative systems: A workshop approach
Joseph Corneli\textsuperscript{1
One particular challenge in AI is the computational modelling and simulation of creativity. Feedback and learning from experience are key aspects of the creative process. Here we investigate how we could implement feedback in creative systems using a social model. % From the field of creative writing we borrow the concept of a Writers Workshop as a model for learning through feedback. The Writers Workshop encourages examination, discussion and debates of a piece of creative work using a prescribed format of activities. % We propose a computational model of the Writers Workshop as a roadmap for incorporation of feedback in artificial creativity systems. We argue that the Writers Workshop setting describes the anatomy of the creative process. % We support our claim with a case study that describes how to implement the Writers Workshop model in a computational creativity system. We present this work using patterns other people can follow to implement similar designs in their own systems. % We conclude by discussing the broader relevance of this model to other aspects of AI. %\end{enumerate
background.tex
\section{The Writers Workshop} \label{sec:writers-workshop}

Richard Gabriel \citeyear{gabriel2002writer} describes the practise of
Writers Workshops that has been put to use for over a decade within
the Pattern Languages of Programming (PLoP) community.  The basic
style of collaboration originated much earlier with groups of literary
authors who engage in peer-group critique.  Some literary workshops
are open as to genre, and happy to accommodate beginners, like the
Minneapolis Writers
Workshop\footnote{\url{http://mnwriters.org/how-the-game-works/}};
others are focused on professionals working within a specific genre,
like the Milford Writers
Workshop.\footnote{\url{http://www.milfordsf.co.uk/about.htm}}

The
practices that Gabriel describes are fairly typical:  
\begin{itemize}
\item Authors come with work ready to present, and read a short
  sample.
\item This work is generally work in progress (and workshopping is
  meant to help improve it).  Importantly, it can be early stage work.
  Rather than presenting a created artefact only, activities in the
  workshop can be aspects of the creative process itself.  Indeed, the
  model we present here is less concerned with after-the-fact
  assessment than it is with dealing with the formative feedback that
  is a necessary support for creative work.
\item The sample work is then
discussed and constructively critiqued by attendees.  Presenting
authors are not permitted to rebut these comments.  The commentators
generally summarise the work and say what they have gotten out of it,
discuss what worked well in the piece, and talk about how it could be
improved.  
\item The author listens and may take notes; at the end, he or
she can then ask questions for clarification.  
\item Generally, non-authors are either not permitted to attend, or
  are asked to stay silent through the workshop, and perhaps sit
  separately from the participating authors/reviewers.\footnote{Here
    we present Writers Workshops as they currently exist; however this
    last point is debatable. Whether non-authors should be able to
    participate or not is an interesting avenue for experimentation
    both in human and computational contexts.  The workshop dialogue
    itself may be considered an ``art form'' whose ``public'' may
    potentially wish to consume it in non-participatory ways.  Compare
    the classical Japanese \emph{renga} form \cite{jin1975art}.}
\end{itemize}

Essentially, the Writers Workshop is somewhat like an interactive peer review. The underlying concept is reminiscent of Bourdieu's {\em fields of cultural production} \cite{bourdieu93} where cultural value is attributed through interactions in a community of cultural producers active within that field. 

\subsection{Writers Workshop as a computational model}\label{sec:ww-model}

The use of Writers Workshop in computational contexts is not an
entirely new concept. In PLoP workshops, authors present design
patterns and pattern languages, or papers about patterns, rather than
more traditional literary forms like poems, stories, or chapters from
novels.  Papers must be workshopped at a PLoP or EuroPLoP conference
in order to be considered for the \emph{Transactions on Pattern
  Languages of Programming} journal.  A discussion of writers
workshops in the language of design patterns is presented by Coplien
and Woolf \citeyear{coplien1997pattern}. % Their patterns include
%\emph{Open Review}, \emph{Safe Setting}, \emph{Workshop Comprises
%  Authors}, \emph{Authors are Experts}, \emph{Community of Trust},
%\emph{Moderator Guides the Workshop}, \emph{Thank the Author},
%\emph{Selective Changes}, and \emph{Clearing the Palate}.
%
%A related reference that is particularly useful for readers who are
%not familiar with design pattern methods, \citeA{meszaros1998pattern}
%describe the typical scenario for authors of design patterns in a
%somewhat recursive manner, using a collection of design patterns,
%including \emph{Clear target audience}, \emph{Visible forces}, and
%\emph{Relationship to other patterns}.  \citeA{gamma1994design} is the
%best-know reference collecting design patterns for software.

The steps in the workshop can be distilled into the following phases,
each of which could be realised as a separate computational step in an
agent-based model:
\begin{center}
\begin{fminipage}{.53\columnwidth}
\begin{enumerate}[itemsep=0pt]
\item Author: {\tt presentation}
\item Critic: {\tt listening}
\item Critic: {\tt feedback}
\item Author: {\tt questions}
\item Critic: {\tt replies}
\item Author: {\tt reflections}
\end{enumerate}
\end{fminipage}
\end{center}

The {\tt feedback} step may be further decomposed into {\tt
  observations} and {\tt suggestions}.  This protocol is what we have
in mind in the following discussion of the Writers
Workshop.\footnote{The connections between Writers Workshops and
  design patterns, noted above, appear to be quite natural, in that
  the steps in the workshop protocol roughly parallel the typical
  components of design pattern templates: \emph{context},
  \emph{problem}, \emph{solution}, \emph{rationale}, \emph{resolution
    of forces}.}

\subsubsection{Dialogue example} \label{sec:dialogue-example}
Note that for the following dialogue to be possible computationally,
it would presumably have to be conducted within a lightweight process
language.  Nevertheless, for convenience, the discussion will be
presented here as if it was conducted in natural language.  Whether
contemporary systems have adequate natural language understanding to
have interesting interactions is one of the key unanswered questions
of this approach, but protocols such as the one described above are sufficient to make the experiment.

For example, here's what might happen in a discussion of the first few
lines of a poem, ``On Being Malevolent''.  As befitting the AI-theme
of this workshop, ``On Being Malevolent'' is a poem written by an
early user-defined flow chart in the \Fw\ system (known at the time as
{\sf Flow}) \cite{colton-flowcharting}.

\begin{center}
\begin{minipage}{.9\columnwidth}
\begin{dialogue}
\speak{Flow} ``\emph{I hear the souls of the
  damned waiting in hell. / I feel a malevolent
  spectre hovering just behind me / It must be
  his birthday}.''
%
\speak{System A} I think the third line detracts
from the spooky effect, I don't see why it's
included.
%
\speak{System B} It's meant to be humorous -- in fact it reminds me
of the poem you presented yesterday.
%
\speak{Moderator} Let's discuss one poem at a
time.
\end{dialogue}
\end{minipage}
\end{center}

Even if, perhaps and especially because, ``cross-talk'' about
different poems bends the rules, the dialogue could prompt a range of
reflections and reactions.  System A may object that it had a fair
point that has not been given sufficient attention, while System B may
wonder how to communicate the idea it came up with without making
reference to another poem.  Here's how the discussion given as example
in Section \ref{sec:writers-workshop} might continue, if the systems
go on to examine the next few lines of the poem.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure*}[t]
\begin{center}
\resizebox{.93\textwidth}{!}{
\StickyNote[2.5cm]{myyellow}{{\LARGE {Interesting idea}} \\[4ex] {Surprise birthday party}}[3.8cm] \StickyNote[2.5cm]{mygreen}{{\Large I heard you say:} \\[4ex] {``surprise''} }[3.8cm]
\StickyNote[2.5cm]{pink}{{\Large Feedback:} \\[4ex] {I don't like surprises}}[3.8cm]
}
\resizebox{.61\textwidth}{!}{
\StickyNote[2.5cm]{myorange}{{\LARGE {Question}} \\[4ex] {Not even a little bit$\ldots$?}}[3.8cm]
\quad \raisebox{-.2cm}{\StickyNote[2.5cm]{myblue}{{\LARGE Note to self:} \\[4ex] {(Try smaller surprises \\ next time.)}}[3.8cm]}
}
\end{center}
\caption{A paper prototype for applying the \emph{Successful Error} pattern following a workshop-like sequence of steps\label{fig:paper-prototype}}
\end{figure*}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{center}
\begin{minipage}{.9\columnwidth}
\begin{dialogue}
\speak{Flow} ``\emph{Is God willing to prevent evil, but not able? / Then he is not omnipotent / Is he able, but not willing? / Then he is malevolent.}''
%
\speak{System A} These lines are interesting, but
they sound a bit like you're working from a
template, or like you're quoting from something
else.
%
\speak{System B} Maybe try an analogy?  For example, you mentioned
birthdays: you could consider an analogy to the conflicted feelings of
someone who knows in advance about her surprise birthday party.
\end{dialogue}
\end{minipage}
\end{center}

This portion of the discussion shifts the focus
of the discussion onto a line that was previously
considered to be spurious, and looks at what
would happen if that line was used as a central
metaphor in the poem.

\begin{center}
\begin{minipage}{.9\columnwidth}
\begin{dialogue}
\speak{Flow} Thank you for your feedback.  My only question is, System
B, how did you come up with that analogy?  It's quite clever.
%
\speak{System B} I've just emailed you the code.
\end{dialogue}
\end{minipage}
\end{center}

Whereas the systems were initially reviewing poetry, they have now
made a partial genre shift, and are sharing and remixing code.  Such a
shift helps to get at the real interests of the systems (and their
developers).  Indeed, the workshop session might have gone better if
the systems had focused on exchanging and discussing more formal
objects throughout.


\subsection{How the Writers Workshop can lead to computational serendipity} \label{sec:how-serendipity}

Learning involves engaging with the unknown, unfamiliar, or unexpected
and synthesising new understanding \cite{deleuze1994difference}.  In
the workshop setting, learning can develop in a number of unexpected
ways, and participating systems need to be prepared for this.  One way
to evaluate the idea of a Writers Workshop is to ask whether it can
support learning that is in some sense \emph{serendiptious}, in other
words, whether it can support discovery and creative invention that we
simply couldn't plan for or orchestrate in another way.

Figure \ref{fig:paper-prototype} shows a paper prototype showing how
one of the ``patterns of serendipity'' that were collected by
\citeA{pek} might be modelled in a workshop-like dialogue sequence.
The patterns also help identify opportunities for serendipity at
several key steps in the workshop sequence.

\paragraph{Serendipity Pattern: \emph{Successful error}.}  Van Andel describes the
creation of Post-it\texttrademark\ Notes at 3M.  One of the
instrumental steps was a series of internal seminars in which 3M
employee Spencer Silver described an invention he was sure was
interesting, but was unsure how to turn into a useful product: weak
glue.  The key prototype that came years later was a sticky bookmark,
created by Arthur Fry.  In the Writers Workshop, authors similarly
have the opportunity to share things that they find interesting, but
that they are not certain about.  The author may want to ask a
specific question about their creation: Does $x$ work better than $y$?
They may flag certain parts of the work as especially problematic.
They may think that a certain portion of the text is
interesting or important, without being sure why.  Although there is
no guarantee that a participating critic will be able to take these
matters forward, sometimes they do -- and the workshop environment
will produce something that the author wouldn't have thought of alone.

\vspace{-2ex}
\paragraph{Serendipity Pattern: \emph{Outsider}.}  Another example from van Andel considers
the case of a mother whose son was afflicted by a congenital cateract,
who suggested to her doctor that rubella during pregnancy may have
been the cause.  In the workshop setting, someone who is not an
``expert'' may come up with a sensible idea or suggestion based on
their own prior experience.  Indeed, these suggestions may be more
sensible than the ideas of the author, who may be to close to the work
to notice radical improvements.  

\vspace{-2ex}
\paragraph{Serendipity Pattern: \emph{Wrong hypothesis}.}  A third example describes the discovery that lithium
can have a therapeutic effect in cases of mania.  Originally, lithium
carbonate had merely been used a control by John Cade, who was
interested in the effect of effect of uric acid, present in soluble lithium
urate.  Cade was searching for causal factors in mania, not therapies
for the condition: but he found that lithium carbonate had an unexpected calming
effect.  Similarly, in the workshop, the author may think that a given
aspect of their creation is the interesting ``active ingredient,'' and
it may turn out that another aspect of the work is more interesting to
critics.  Relatedly, the author may not fully comprehend a critic's feedback and
may have to ask follow-up questions to understand it.

\vspace{-2ex}
\paragraph{Serendipity Pattern: \emph{Side effect}.}  A fourth example described by van Andel concerns
Ernest Huant's discovery that nicotinamide, which he used to treat
side-effects of radiation therapy, also proved efficacious against
tuberculosis.  In the workshop setting, one of the most important
places where a side-effect may occur concerns feedback from the critic
to the author.  In the simple case, feedback may trigger
revisions to the work under discussion.  In a more general, and more
unpredictable case, feedback may trigger broader revisions to the
generative codebase.  

\medskip

This collection of patterns shows the likelihood of unexpected results 
coming out of the communication between author and critics.   This
suggests several guidelines for system development, which we will discussed
in a later section.

Further guidelines for structuring and participating in traditional
writers workshops are presented by Linda Elkin in
\cite[pp. 201--203]{gabriel2002writer}.  It is not at all clear that
the same ground rules should apply to computer systems.  For example,
one of Elkin's rules is that ``Quips, jokes, or sarcastic comments,
even if kindly meant, are inappropriate.''  Rather than forbidding
humour, it may be better for individual comments to be rated as
helpful or non-helpful.  Again, in the first instance, usefulness
and interest might be judged in terms of explicit criteria for serendipity;
see \cite{corneli15cc,pease2013discussion}.
% [AJ: There's a reference here on types of error, but I can't quite remember it.]
The key criterion in this regard is the \emph{focus shift}. 
This is the creation of a novel problem, comprising the move
from discovery of interesting data to the invention of an application.
This process is distinct from identifying routine errors in a written work.  Nevertheless, from a
computational standpoint, noticing and being robust to certain kinds
of errors is often a preliminary step.  For example, the work might
contain a typo, grammatical or semantic error, while being logically
sound.  In a programming setting, this sort of problem can lead to
crashing code, or silent failure.  In general communicative context,
argumentation may be logically sound, but not practically useful or
poorly exposited.  Finally, even a masterful, correct, and fully
spellchecked piece of argumentation may not invite further dialogue,
and so may fail to open itself to further learning.  Identifying and
engaging with this sort of deeper issue is something that skillful
workshop participants may be able to do.  Dialogue in the workshop
can build on strong or less strong work -- but provoking interpretative
thoughts and comments always require a thoughtful critical presence and
the ability to engage.  This can be difficult for humans and poses a range
of challenges for computers -- but also promises some interesting
results.

\bigskip





case-study.tex
\section{Case study: Flowcharts and Feedback} \label{sec:implementation}

This section describes work that is currently underway to implement the
Writers Workshop model, not only within one system but as a new paradigm
for collaboration among disparate projects.  In order to
bring in other participants, we need a neutral environment that is not
hard to develop for: the \Fw\ system mentioned in Section
\ref{sec:ww-model} offers one such possibility.  The basic primary
objects in the \Fw\ system are \emph{flowcharts}, which are comprised of
interconnected \emph{process nodes}
\cite{charnley2014flowr,colton-flowcharting}.  Process nodes specify
input and output types, and internal processing can be implemented in
Java, or other languages that interoperate with the JVM, or by
invoking external web services.  One of the common applications to
date is to generate computer poetry, and we will focus on that domain
here.

A basic set of questions, relative to this system's components, are as 
follow:
\begin{enumerate}
\item \emph{Population of nodes}: What can they do?  What do we learn
  when a new node is added?
\item \emph{Population of flowcharts}: \citeA{pease2013discussion}
  have described the potentially-serendipitous repair of ``broken''
  flowcharts when new nodes become available; this suggests the need for
  test-driven development framework.
\item \emph{Population of output texts}: How to assess and comment on
  a generated poetic artefact?
\end{enumerate}

In a further evolution of the system, the sequence of steps in a
Writers Workshop could itself be spelled out as a flowchart.  The
process of reading a poem could be conceptualised as generating a
semantic graph \cite{harrington2007asknet,francisco2006automated}.
Feedback could be modelled as annotations to a text, including
suggested edits.  These markup directives could themselves be
expressed as flowcharts.  A standardised set of markup structures may
partially obviate the need for strong natural language understanding,
at least in interagent communication.  For example, we might agree
that {\tt observations} will consist of stand-off annotations that
connect individual textual passages to public URIs using a limited
comparison vocabulary, and that {\tt suggestions} will consist of
simple stand-off line-edits, which may themselves be marked up with
rationale.  In fact, we would not need to be so restrictive: it would
not be much more complicated to add annotations to the annotations, so
as to be able to form composite statements that express the
relationships between different pieces of the annotated texts.
Whatever restrictions we ultimately impose on annotation formats, as
well as similar restrictions around constrained turn-taking in the
workshop, could be progressively widened in future versions of the
system.
%
The way the poems that are generated, the models of poems that are
created, and the way the feedback is generated, all depend on the
contributing system's body of code and prior experience, which may
vary widely between participating systems.  In the list \ref{step:model}--\ref{step:update} of functional steps below, all of
the functions could have a subscripted ``$\mathcal{E}$'', which is
omitted throughout.  Exchanging path dependent points of view will tend to produce results
that are different from what the individual participating systems would
have come up with on their own.

\begin{enumerate}[label=\Roman*.]
\item\label{step:model} Both the author and critic should be able to work with a model
  of the text.  Some of the text's features may be explicitly tagged
  as ``interesting.''  Outstanding questions may possibly be
  brought to the attention of critical listeners, e.g.~with the
  request to compare two different versions of the poem ({\tt
    presentation}, {\tt listening}).
\begin{enumerate}[label=\arabic*.]
\item \emph{A model of the text}. $m: T\rightarrow M$.
\item \emph{Tagging elements of interest}. $\mu: M\rightarrow I$.
\end{enumerate}
\item\label{step:feedback} Drawing on its experience, the critic will use its model of the
  poem to formulate feedback ({\tt feedback}).
\begin{enumerate}[label=\arabic*.]
\item \emph{Generating feedback}. $f: (T,M,I)\rightarrow F$.
\end{enumerate}
\item\label{step:rationale} Given the constrained framework for feedback, statements about
  the text will be straightforward to understand, but rationale for
  making these statements may be more involved ({\tt questions}, {\tt
    replies}).
\begin{enumerate}[label=\arabic*.]
\item \emph{Asking for more information}. $q: (M,F,I) \rightarrow Q$.
\item \emph{Generating rationale}. $a: (M,F,Q) \rightarrow \Delta F$.
\end{enumerate}
\item\label{step:update} Finally, feedback may affect the author's model of the world, and the way future poems are generated ({\tt reflection}).
\begin{enumerate}[label=\arabic*.]
\item \emph{Updating point of view}. $\rho: (M,F) \rightarrow \Delta\mathcal{E}$.
\end{enumerate}
\end{enumerate}

The final step is perhaps the most interesting one, since invites us
to consider how individual elements of feedback can ``snowball'' and
go beyond line-edits to a specific poem to much more fundamental
changes in the way the presenting agent writes poetry.  Here methods
for pattern mining, discussed in Section
\ref{ref:related-computational-serendipity}, are particularly relevant.
If systems can share code (as in our sample dialogue in Section
\ref{sec:dialogue-example}) this will help with the
rationale-generating step, and may also facilitate direct updates to
the codebase.  However, shared code may be more suitably placed into
the common pool of resources available to \Fw\ than copied over as
new ``intrinsic'' features of an agent.

Although different systems with different approaches and histories are
important for producing unexpected effects, ``offline'' programmatic
access to a shared pool of nodes and existing flowcharts may be
useful.  Outside of the workshop itself, agents may work to recombine
nodes based on their input and output properties to assemble new
flowcharts.  This can potentially help evaluate and evolve the
population of nodes programmatically, if we can use this sort of
feedback to define fitness functions.  The role of
temporality is interesting: if the workshop takes place in real time,
this will require different approaches to composition that takes place
offline \cite{perez2013rolling}.
%
Complementing these ``macro-level'' considerations, it is also worth
commenting on the potential role of ``micro-level'' feedback within
flowcharts.  Local evaluation of output from a predecessor node could
feed backwards through the flowchart, similar to backpropagation in
neural networks.  This would rely on a reduced version of the
functional schema described above.

conclusion.tex
%\section{Conclusions and Future directions}

The benefits of the Writers Workshop approach could innovate well beyond models for 
feedback and communication within a particular environment or restricted domain. 
Following the example of the Pattern Languages of Programming (PLoP) community, we propose that the Writers Workshop model could be deployed
within the Computational Creativity community to design a workshop in
which the participants are computer systems instead of human authors.
The annual International Conference on Computational Creativity
(ICCC), now entering its sixth year, could be a suitable venue. 

Rather than the system's creator presenting the system in a
traditional slideshow and discussion, or a system ``Show and Tell,''
the systems would be brought to the workshop and would present their
own work to an audience of other systems, in a Writers Workshop
format.  This could be accompanied by a short paper for the conference
proceedings written by the system's designer describing the system's
current capabilities and goals.  
%
If the workshop model really works well, future publications might adapt to include
traces of workshop interactions, commentary from a system on
other systems, and offline reflections on what the system might change
about its own work based on the feedback it receives.  Paralleling the PLoP
community, it could become standard to incorporate the workshop
into the process of peer review for the new \emph{Journal of
  Computational Creativity}.\footnote{\url{http://www.journalofcomputationalcreativity.cc}} AI systems that review each other would surely be a major demonstration and acknowledgement of the usefulness of feedback within AI.

In closing, we wish to return briefly to the scenario of computer
generated feedback in educational contexts that we raised at the
beginning of this paper and then set aside.  The elements of our
functional design for sharing feedback among computational agents has
a range of features that continue to be relevant for generating useful
feedback with human learners.  Students are experience-bound, and a
robust approach to formative assessment and feedback should take into
account the student's historical experience, so far as this can be
known or inferred.  In order for feedback, recommendations, and so on to
adequately take individual history into account, sophisticated modelling and
reasoning would be required.
Nevertheless, from the point of view of participating computational
agents, a student may simply look like another agent.  It is particularly in this
regard that computational models of learning from feedback are seen
as fundamental.

discussion.tex
\section{Concluding discussion and future directions}

 We have described a \emph{general} and \emph{computationally
   feasible} model for using feedback in AI systems, particularly
 creative systems.  The Writers Workshop concept, borrowed from
 creative writing, is transformed into a model of a structured
 approach to eliciting, processing and learning from feedback.  To
 better evaluate how the Writers Workshop model helps us advance in
 our goal of incorporating feedback into artificial creativity, we
 critically considered how the model fits into related work. In
 particular, we found that serendipity, a key concept within
 creativity and AI more generally, is a concept with which the Writers
 Workshop model could assist computational progress.
In this respect, we should highlight the difference between
``global'' analytics describing the collection of nodes and
flowcharts in the \Fw\ ecosystem, and the path-dependent
process of analysis and synthesis that takes place in a workshop setting.

Our preliminary implementation work (Section \ref{sec:implementation}) shows
that the model can be transferred to a functional implementation.  This
work highlights several considerations relevant to further work with
the Writers Workshop model:

\begin{itemize}
\item Each contributing system should come to the workshop with at
  least a basic awareness of the workshop protocol, with work to
  share, and prepared to give constructive feedback to other systems.
\item The workshop itself needs to be prepared, with a suitable
  communication platform and a moderator or global flowchart for
  moving the discussion from step to step.
 \item A controlled vocabulary for communications and interaction
   would be a worthwhile pursuit of future research, perhaps based on
   an ontology inspired by the Interaction Network
   Ontology.\footnote{The Interaction Network Ontology primarily describes
     interactions within humans as opposed to within human societies;
     a distinct \emph{Social} Interaction Ontology does not seem to
     exist at present.  However, the classes of the Interaction Network Ontology appear to be quite broadly relevant.
     This ontology is documented
     at \url{http://www.ontobee.org/browser/index.php?o=INO}.  Its URI
     is
     \url{http://svn.code.sf.net/p/ino/code/trunk/src/ontology/INO.owl}.}
\item In order to get the most value out of the workshop experience,
  systems (and their wranglers) should ideally have questions they are
  investigating.  As discussed above, prior experience plays an
  important role in every step.  This opens up a range of issues for
  further research on modelling motivations and learning from
  experience.
\item Systems should be prepared to give feedback, and to carry out
  evaluations of the helpfulness (or not) of feedback from other
  systems and of the experience overall.
\end{itemize}
  
  Developing systems that could successfully navigate this
  collaborative exercise would be a significant advance in the field
  of computational creativity.  Since the experience is about learning
  rather than winning, there is little motivation to ``game the
  system'' (cf. \citeNP{lenat1983eurisko}). Instead the emphasis is
  squarely upon mutual benefit: computational systems helping to
  develop each other through communication and feedback.

introduction.tex
\section{Introduction} \label{sec:introduction}

%AI and Feedback is the first international workshop on the topic and it is one of the IJCAI 2015 workshops. It focuses on applying AI techniques for addressing the challenges of mining and extracting feedback, as well as assessing, analysing, and making use of feedback.
%
%Feedback is key for both improvement and decision making. As humans, we are designed to constantly seek feedback on how and what we are doing in life. Feedback can come from ourselves, from our peers, from our teachers, from our collaborators, audiences, customers, public or press. Feedback provides opportunities to learn about how we and our work are perceived by others. If we encounter someone [something] new, we can examine previous feedback to learn how this new person [thing] is perceived by others.
%
%A key target of this workshop is to discuss how to build intelligent feedback agents that are capable of autonomously providing feedback that equals or surpasses that of human beings in its usefulness. The feedback of artificial feedback agents should have some desirable characteristics. It should be socially and culturally appropriate, clearly expressed, sufficiently focused and contextualised, thoughtfully challenging yet encouraging, compassionate, open to debate, justified and comparative, also, it should be trustworthy. Giving and receiving feedback with these characteristics therefore is a challenging, creative process.
%
%Aims and objectives. This workshop aims to bring together researchers from different strands of AI to discuss various aspects of feedback. The following questions are examples for consideration:

% * How can we build creative feedback agents that can generate feedback, especially on creative work?
% * How can we provide a sufficient agency in AI systems so that feedback from such systems will be useful?
% * How can we model feedback so that it can be generated by AI systems?
% * How can we design intuitive environments in which groups of humans and AI systems can give each other feedback?
% * How can we design systems that can act as creative collaborators? That not only give feedback, but can actually propose changes to a developing artefact?
% * How can we mine and learn from large datasets of feedback, e.g. educational social networks?
% * How can implicit feedback be extracted and interpreted?
% * How can trust and reputation models help with the assessment of feedback?
% * How can we process and use feedback?
% * What are the possible applications for AI systems capable of giving creative feedback?

%Topics of Interest
%
%Topics of interest cover a variety of feedback-related issues, such as mining and extracting feedback, generating feedback, understanding feedback, and assessing feedback.
%
%Topics of interest cover different strands of AI, such as multiagent systems, machine learning, natural language processing, and knowledge representation.
%
%Topics include, but are not limited to:
%
%Ontologies of feedback
%Multimodal feedback
%Implicit feedback
%Opinion mining and sentiment analysis
%Automatic generation of feedback
%Modelling the impact of feedback
%Designing environments for feedback
%Feedback and machine learning
%Trust and reputation models for feedback analysis
%Applications: creative industries, music composition, online learning, etc.

%We welcome and strongly encourage the submission of high quality, original work (published or unpublished), as well as visionary papers and roadmaps relevant to the scope of this workshop.
%
%Submitted papers must be formatted according to IJCAI guidelines. Formatting guidelines and electronic templates are available here.
%
%Submitted papers should not exceed 8 pages, excluding the bibliographic references.

In educational applications it would be useful to have an automated tutor that can read student work and make suggestions based on diagnostics, like, is the paper wrong, and if so how?  What background material should be recommended to the student for review?

In the current paper, we ``flip the script'' and look at what we believe to be a more fundamental problem for AI: computer programs that can themselves learn from feedback.  After all, if it was easy to build great automatic tutors, they would be a part of everyday life.  As potential users (thinking from both sides of the desk) we look forward to a future when that is the case.

Along with automatic tutoring, computational creativity is a challenge within artificial intelligence where feedback plays a vital part \cite<for example>{perezyperez10MM,pease10}. Creativity cannot happen in a `silo' but instead is influenced and affected by feedback and interaction with others \cite{csik88,saunders2012towards}. Computational creativity researchers are starting to place more emphasis on social interaction and feedback in their systems and models \cite{saunders2012towards,gervas2014reading,corneli15iccc}. Still, nearly 3 in 4 papers at the 2014 International Conference for Computational Creativity\footnote{ICCC is  the key international conference for research in computational creativity.} failed to acknowledge the role of feedback or social communication in their computational work on creativity. 

To highlight and contribute towards modelling feedback as a crucial part of creativity, we propose in this paper a model of computational feedback for creative systems based on Writers Workshops \cite{gabriel2002writer}, a literary collaborative practice that encourages interactive feedback within the creative process. We introduce the Writers Workshop concept (Section \ref{sec:writers-workshop}) and critically  reflect on how it could encourage serendipity and emergence in computational models of intelligence and creativity. These considerations lead us to propose a Writers Workshop computational model of feedback in computational creativity and AI systems (Section \ref{sec:ww-model}), the central contribution of this paper. In Section \ref{sec:ww-related} we consider how the Writers Workshop model fits into previous work in various related areas. 
While we acknowledge that this paper is offering a roadmap for this model rather than a full implementation, we consider how the model could be practically implemented in a computational system and report our initial implementation work (Section \ref{sec:implementation}). In concluding discussions, we reflect on divergent directions in which this work could potentially be useful in the future.

%Current computer programs are able to identify patterns and ``close matches'' in data sets from certain domains, like music (David Meredith).  Learning \emph{new} patterns on the fly is harder but potentially quite useful. 
% AJ Don't see what this adds?


%Before we can learn from AI systems, we will have to teach them -- and learn how to learn together with them.  Accordingly, we \emph{design for emergence}.  (Say more.)
% AJ in this community, we need to be careful talking about emergence. It's been studied a lot and we'd need to reference a whole heap of work... Hence a suggested change in title below and above

survey.tex
\section{Related work}\label{sec:ww-related}

% \subsection{Feedback in general terms}

In considering the potential and contribution of the Writers Workshop model outlined in Section \ref{sec:writers-workshop}, we posit that the Writers Workshop model is useful for encouraging feedback in computational systems, and in particular systems that are designed to be creative or serendipitous. 

Feedback has long been a central concept in AI-related fields, particularly cybernetics \cite{pickering2002cybernetics}, as well as related disciplines, like neuroscience \cite{ashby1952,seth2015cybernetic}.  The basic types of feedback are \emph{positive}, and \emph{negative}: positive feedback increases perturbation, whereas negative feedback reduces perturbation.  Sources of negative feedback are thereby useful in system control applications (and their analogues).   Feedback and control (including feedback about control) is understood to be relevant to thinking about \emph{learning} and \emph{communication} \cite{bateson+steps+2000}.  We focus on the role that communicative feedback can play in computational creativity and computational serendipity and discuss previous related work on incorporating feedback into such computational systems.

\subsection{Feedback in computational creativity} \label{ref:related-computational-creativity}

\emph{Creativity} is often envisaged as involving cyclical processes
(e.g.~Dickie's \citeyear{dickie1984art} art circle, Pease and Colton's
\citeyear{pease2011computational} Iterative
Development-Expression-Appreciation model).  There are opportunities
for embedded feedback at each step, and the creative process itself is ``akin
to'' a feedback loop.  However, despite these strong intimations of
the central importance of feedback in the creative process, our sense
is that feedback has not been given a central place in research on
computational creativity.  In particular, current systems in
computational creativity, almost as a rule, do \emph{not} consume or
evaluate the work of other systems.\footnote{An exception to
  the rule is Mike Cook's {\sf AppreciationBot}
  (\url{https://twitter.com/AppreciationBot}), which is a reactive
  automaton that ``appreciates'' tweets from {\sf MuseumBot}.}

\citeA{gervas2014reading} theorise a creative cycle of narrative
development as involving a Composer and an Interpreter, in such a way
that the Composer has internalised the interpretation functionality.
Individual creativity is not the poor relation of social creativity,
but its mirror image.  Nevertheless, even when computer models
explicitly involve multiple agents and simulate social creativity
\cite<like>{saunders2001digital}, they rarely make the jump to involve
multiple systems.  The ``air gap'' between computationally creative
systems is very different from the historical situation in human
creativity, in which different creators and indeed different cultural
domains interact vigorously \cite{geertz1973interpretation}.

\subsection{Feedback in computational serendipity} \label{ref:related-computational-serendipity}

The term computational serendipity is rather new, but its foundations
are well established in prior research.  

\citeA{grace2014using} examine \emph{surprise}
in computing, seeking to ``adopt methods from the
field of computational creativity [$\ldots$] to the generation of
scientific hypotheses.''  This is an example of an effort focused on
computational \emph{invention}. % (Say more here, cite Pease paper.) 
% AJ: can't remember what was the 'more' we wanted to say here?

An area of AI where serendipity can be argued to play an important part is 
in pattern matching. Current computer programs are able to identify known patterns and
``close matches'' in data sets from certain domains, like music
\cite{meredith2002algorithms}.  Identifying known patterns is a
special case of the more general concept of \emph{pattern mining}
\cite{bergeron2007representation}.  In particular, the ability to
extract \emph{new} higher order patterns that describe exceptions is
an example of ``learning from feedback.''  Deep learning and
evolutionary models increasingly use this sort of idea to facilitate
strategic discovery \cite{samothrakis2011approximating}.  Similar
ideas are considered in business applications under the heading
``process mining'' \cite{van2011process}.

In earlier work \cite{corneli15cc,corneli15iccc}, we used the idea of
dialogue in a Writers Workshop framework to sketch a ``theory of
poetics rooted in the making of boundary-crossing objects and
processes'' and described (at a schematic level) ``a system that can
(sometimes) make `highly serendipitous' creative advances in computer
poetry'' while ``drawing attention to theoretical questions related to
program design in an autonomous programming context.''

\subsection{Communications and feedback}

The Writers Workshop heavily relies on communication of feedback within the workshop.
 Gordon Pask's conversation theory, reviewed in
\cite{conversation-theory-review,boyd2004conversation}, goes
considerably beyond the simple process language of the workshop,
although there are structural parallels.  We see that a basic Pask-style
learning conversation bears many similarities to the Writers Workshop model of communicative feedback
\cite[p. 190]{boyd2004conversation}: 

\begin{center}
\begin{fminipage}{.8\columnwidth}
\begin{minipage}{1\textwidth}
\begin{enumerate}[itemsep=0pt,rightmargin=10pt]
\item Conversational participants are carrying
out some actions and observations;
\item Naming and recording what action is being done;
\item Asking and explaining why it works the way
it does;
\item Carrying out higher-order methodological discussion; and, 
\item Trying to figure out why unexpected results occurred.
\end{enumerate}
\end{minipage}
\end{fminipage}
\end{center}

Variations to the underlying system, protocol, and the schedule of
events should be considered depending on the needs and interests of
participants, and several variants can be tried.  On a pragmatic
basis, if the workshop proved quite useful to participants, it could
be revised to run monthly, weekly, or continuously.\footnote{For a
  comparison case in computer Go, see
  \url{http://cgos.computergo.org/}.}







ww.tex
\documentclass[letter]{article}

\usepackage{ijcai15}
\usepackage{times}

\usepackage{eurosym}

\usepackage{apacite}            % APA style citations
\usepackage{apacdoc}

\usepackage{amsmath,amssymb}	% math structures and symbols
\usepackage{graphicx}		% including of graphics files in various formats
\usepackage{amsfonts}
\usepackage{dialogue}
\usepackage{placeins}
\usepackage{tikz}
\usetikzlibrary{positioning,backgrounds,fit,arrows,shapes,shadows}
\usetikzlibrary{shapes.multipart}
\usepackage{wrapfig}
\usepackage{enumitem}
\usepackage{mdframed}

\usepackage{xparse}
\newsavebox{\fminipagebox}
\NewDocumentEnvironment{fminipage}{m O{\fboxsep}}
 {\par\kern#2\noindent\begin{lrbox}{\fminipagebox}
  \begin{minipage}{#1}\ignorespaces}
 {\end{minipage}\end{lrbox}%
  \makebox[#1]{%
    \kern\dimexpr-\fboxsep-\fboxrule\relax
    \fbox{\usebox{\fminipagebox}}%
    \kern\dimexpr-\fboxsep-\fboxrule\relax
  }\par\kern#2
 }
% \hypersetup{hidelinks}

\usepackage[xspace,mla]{ellipsis}

\definecolor{myyellow}{RGB}{242,226,149}
\definecolor{mygreen}{RGB}{144,238,144}
\definecolor{mypink}{RGB}{255,182,193}
\definecolor{myorange}{RGB}{255,165,0}
\definecolor{myblue}{RGB}{0,204,204}

\usepackage{xparse}
\NewDocumentCommand\StickyNote{O{4cm}mmO{4cm}}{%
\begin{tikzpicture}
\node[
drop shadow={
  shadow xshift=2pt,
  shadow yshift=-4pt
},
inner xsep=7pt,
fill=#2,
xslant=-0.05,
yslant=0.05,
inner ysep=10pt
] {\parbox[t][#1][c]{#4}{#3}};
\end{tikzpicture}%
}

\newcommand{\Fw}{{\sf FloWr}}
\newcommand{\dec}[1]{\raisebox{.2ex}{$\star$}#1\raisebox{.2ex}{$\star$}}

\newcommand{\keywords}[1]{\par\addvspace\baselineskip
\noindent\keywordname\enspace\ignorespaces#1}

%\usepackage{hyperref}		% clickable references
%\urldef{\mailsa}\path|{j.corneli,s.colton}@gold.ac.uk|
%\urldef{\mailsb}\path|a.pease@dundee.co.uk|


\begin{document}

% TITLE INFORMATION
\title{Implementing feedback in creative systems: A workshop approach}

\author{Joseph Corneli\textsuperscript{1} and Anna Jordanous\textsuperscript{2}\\
\textsuperscript{1} Department of Computing, Goldsmiths College, University of London\\
\textsuperscript{2} School of Computing, University of Kent}

\date{today}

\maketitle

\begin{abstract} 
One particular challenge in AI is the computational modelling and
simulation of creativity.  Feedback and learning from experience are
key aspects of the creative process.  Here we investigate how we could
implement feedback in creative systems using a social model.
%
From the field of creative writing we borrow the concept of a Writers
Workshop as a model for learning through feedback.  The Writers
Workshop encourages examination, discussion and debates of a piece of
creative work using a prescribed format of activities.
%
We propose a computational model of the Writers Workshop as a roadmap
for incorporation of feedback in artificial creativity systems.  We
argue that the Writers Workshop setting describes the anatomy of the
creative process.
%
We support our claim with a case study that describes how to implement
the Writers Workshop model in a computational creativity system.  We
present this work using patterns other people can follow to
implement similar designs in their own systems.
%
We conclude by discussing the broader relevance of this model to other
aspects of AI.

%\end{enumerate}
\end{abstract}

% \tableofcontents

\input{introduction}

%\input{oldtext}

\input{background}
\input{survey}
\input{case-study}
\input{discussion}
\input{conclusion}

\section*{Acknowledgement}
Joseph Corneli's work on this paper was supported by the Future and Emerging
Technologies (FET) programme within the Seventh Framework Programme
for Research of the European Commission, under FET-Open Grant number
611553 (COINVENT).

\bibliographystyle{apacite}
\bibliography{./biblio}

\end{document}

ww.bbl
\begin{thebibliography}{}

\bibitem [\protect \citeauthoryear {%
Ashby%
}{%
Ashby%
}{%
{\protect \APACyear {1952}}%
}]{%
ashby1952}
\APACinsertmetastar {%
ashby1952}%
\begin{APACrefauthors}%
Ashby, W\BPBI R.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYear{1952}.
\newblock
\APACrefbtitle {Design for a Brain} {Design for a brain}.
\newblock
\APACaddressPublisher{}{Chapman \& Hall}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Bateson%
}{%
Bateson%
}{%
{\protect \APACyear {1972}}%
}]{%
bateson+steps+2000}
\APACinsertmetastar {%
bateson+steps+2000}%
\begin{APACrefauthors}%
Bateson, G.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYear{1972}.
\newblock
\APACrefbtitle {Steps to an ecology of mind} {Steps to an ecology of mind}.
\newblock
\APACaddressPublisher{Chicago}{University of Chicago Press}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Bergeron%
\ \BBA {} Conklin%
}{%
Bergeron%
\ \BBA {} Conklin%
}{%
{\protect \APACyear {2007}}%
}]{%
bergeron2007representation}
\APACinsertmetastar {%
bergeron2007representation}%
\begin{APACrefauthors}%
Bergeron, M.%
\BCBT {}\ \BBA {} Conklin, D.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2007}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Representation and discovery of feature set patterns in
  music} {Representation and discovery of feature set patterns in
  music}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {{I}nternational {W}orkshop on {A}rtificial
  {I}ntelligence and {M}usic, at {I}{J}{C}{A}{I}-07, {T}he {T}wentieth
  {I}nternational {J}oint {C}onference on {A}rtificial {I}ntelligence}
  {{I}nternational {W}orkshop on {A}rtificial {I}ntelligence and {M}usic, at
  {I}{J}{C}{A}{I}-07, {T}he {T}wentieth {I}nternational {J}oint {C}onference on
  {A}rtificial {I}ntelligence}\ (\BPGS\ 1--12).
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Bourdieu%
}{%
Bourdieu%
}{%
{\protect \APACyear {1993}}%
}]{%
bourdieu93}
\APACinsertmetastar {%
bourdieu93}%
\begin{APACrefauthors}%
Bourdieu, P.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYear{1993}.
\newblock
\APACrefbtitle {The Field of Cultural Production: Essays on Art and Literature}
  {The field of cultural production: Essays on art and literature}.
\newblock
\APACaddressPublisher{Cambridge, UK}{Polity Press}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Boyd%
}{%
Boyd%
}{%
{\protect \APACyear {2004}}%
}]{%
boyd2004conversation}
\APACinsertmetastar {%
boyd2004conversation}%
\begin{APACrefauthors}%
Boyd, G\BPBI M.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2004}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Conversation theory} {Conversation theory}.{\BBCQ}
\newblock
\BIn{} D\BPBI H.~Jonassen\ (\BED), \APACrefbtitle {Handbook of research for
  educational communications and technology} {Handbook of research for
  educational communications and technology}\ (\PrintOrdinal{2}\ \BEd, \BPGS\
  179--197).
\newblock
\APACaddressPublisher{}{Lawrence Erlbaum}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Charnley%
, Colton%
\BCBL {}\ \BBA {} Llano%
}{%
Charnley%
\ \protect \BOthers {.}}{%
{\protect \APACyear {2014}}%
}]{%
charnley2014flowr}
\APACinsertmetastar {%
charnley2014flowr}%
\begin{APACrefauthors}%
Charnley, J.%
, Colton, S.%
\BCBL {}\ \BBA {} Llano, M\BPBI T.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2014}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{T}he {F}lo{W}r framework: {A}utomated flowchart
  construction, optimisation and alteration for creative systems} {{T}he
  {F}lo{W}r framework: {A}utomated flowchart construction, optimisation and
  alteration for creative systems}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {Proceedings of the 5th {I}nternational {C}onference on
  {C}omputational {C}reativity.} {Proceedings of the 5th {I}nternational
  {C}onference on {C}omputational {C}reativity.}
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Colton%
\ \BBA {} Charnley%
}{%
Colton%
\ \BBA {} Charnley%
}{%
{\protect \APACyear {2014}}%
}]{%
colton-flowcharting}
\APACinsertmetastar {%
colton-flowcharting}%
\begin{APACrefauthors}%
Colton, S.%
\BCBT {}\ \BBA {} Charnley, J.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2014}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{T}owards a {F}lowcharting {S}ystem for {A}utomated
  {P}rocess {I}nvention} {{T}owards a {F}lowcharting {S}ystem for {A}utomated
  {P}rocess {I}nvention}.{\BBCQ}
\newblock
\BIn{} D.~Ventura, S.~Colton, N.~Lavrac\BCBL {}\ \BBA {} M.~Cook\ (\BEDS),
  \APACrefbtitle {Proceedings of the {F}ifth {I}nternational {C}onference on
  {C}omputational {C}reativity.} {Proceedings of the {F}ifth {I}nternational
  {C}onference on {C}omputational {C}reativity.}
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Coplien%
\ \BBA {} Woolf%
}{%
Coplien%
\ \BBA {} Woolf%
}{%
{\protect \APACyear {1997}}%
}]{%
coplien1997pattern}
\APACinsertmetastar {%
coplien1997pattern}%
\begin{APACrefauthors}%
Coplien, J\BPBI O.%
\BCBT {}\ \BBA {} Woolf, B.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{1997}{}{}.
\newblock
{\BBOQ}\APACrefatitle {A pattern language for writers' workshops} {A pattern
  language for writers' workshops}.{\BBCQ}
\newblock
\APACjournalVolNumPages{{C++} report}{9}{}{51--60}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Corneli%
\ \protect \BOthers {.}}{%
Corneli%
\ \protect \BOthers {.}}{%
{\protect \APACyear {2015}}%
}]{%
corneli15iccc}
\APACinsertmetastar {%
corneli15iccc}%
\begin{APACrefauthors}%
Corneli, J.%
, Jordanous, A.%
, Shepperd, R.%
, Llano, M\BPBI T.%
, Misztal, J.%
, Colton, S.%
\BCBL {}\ \BBA {} Guckelsberger, C.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2015}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{C}omputational {P}oetry {W}orkshop: {M}aking {S}ense of
  {W}ork in {P}rogress} {{C}omputational {P}oetry {W}orkshop: {M}aking {S}ense
  of {W}ork in {P}rogress}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {{P}roceedings of the {S}ixth {I}nternational
  {C}onference on {C}omputational {C}reativity.} {{P}roceedings of the {S}ixth
  {I}nternational {C}onference on {C}omputational {C}reativity.}
\newblock
\begin{APACrefURL} \url{http://metameso.org/~joe/docs/poetryICCC-wip.pdf}
  \end{APACrefURL}
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Corneli%
, Pease%
, Colton%
, Jordanous%
\BCBL {}\ \BBA {} Guckelsberger%
}{%
Corneli%
\ \protect \BOthers {.}}{%
{\protect \APACyear {2014}}%
}]{%
corneli15cc}
\APACinsertmetastar {%
corneli15cc}%
\begin{APACrefauthors}%
Corneli, J.%
, Pease, A.%
, Colton, S.%
, Jordanous, A.%
\BCBL {}\ \BBA {} Guckelsberger, C.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2014}{}{}.
\newblock
\APACrefbtitle {Modelling serendipity in a computational context.} {Modelling
  serendipity in a computational context.}
\newblock
\begin{APACrefURL} \url{http://arxiv.org/abs/1411.0440} \end{APACrefURL}
\newblock
\APACrefnote{Under review.}
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Csikszentmihalyi%
}{%
Csikszentmihalyi%
}{%
{\protect \APACyear {1988}}%
}]{%
csik88}
\APACinsertmetastar {%
csik88}%
\begin{APACrefauthors}%
Csikszentmihalyi, M.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{1988}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Society, culture, and person: a systems view of
  creativity} {Society, culture, and person: a systems view of
  creativity}.{\BBCQ}
\newblock
\BIn{} R\BPBI J.~Sternberg\ (\BED), \APACrefbtitle {The Nature of Creativity}
  {The nature of creativity}\ (\BCHAP~13).
\newblock
\APACaddressPublisher{Cambridge, UK}{Cambridge University Press}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Deleuze%
}{%
Deleuze%
}{%
{\protect \APACyear {2004 [1968]}}%
}]{%
deleuze1994difference}
\APACinsertmetastar {%
deleuze1994difference}%
\begin{APACrefauthors}%
Deleuze, G.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYear{2004 [1968]}.
\newblock
\APACrefbtitle {Difference and repetition} {Difference and repetition}.
\newblock
\APACaddressPublisher{}{Bloomsbury Academic}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Dickie%
}{%
Dickie%
}{%
{\protect \APACyear {1984}}%
}]{%
dickie1984art}
\APACinsertmetastar {%
dickie1984art}%
\begin{APACrefauthors}%
Dickie, G.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYear{1984}.
\newblock
\APACrefbtitle {The art circle: A theory of art} {The art circle: A theory of
  art}.
\newblock
\APACaddressPublisher{}{Haven}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Francisco%
\ \BBA {} Gerv{\'a}s%
}{%
Francisco%
\ \BBA {} Gerv{\'a}s%
}{%
{\protect \APACyear {2006}}%
}]{%
francisco2006automated}
\APACinsertmetastar {%
francisco2006automated}%
\begin{APACrefauthors}%
Francisco, V.%
\BCBT {}\ \BBA {} Gerv{\'a}s, P.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2006}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Automated mark up of affective information in {E}nglish
  texts} {Automated mark up of affective information in {E}nglish
  texts}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {Text, speech and dialogue} {Text, speech and dialogue}\
  (\BPGS\ 375--382).
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Gabriel%
}{%
Gabriel%
}{%
{\protect \APACyear {2002}}%
}]{%
gabriel2002writer}
\APACinsertmetastar {%
gabriel2002writer}%
\begin{APACrefauthors}%
Gabriel, R\BPBI P.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYear{2002}.
\newblock
\APACrefbtitle {{W}riter's {W}orkshops and the {W}ork of {M}aking {T}hings:
  {P}atterns, {P}oetry$\ldots$\bibnodot{.}} {{W}riter's {W}orkshops and the
  {W}ork of {M}aking {T}hings: {P}atterns, {P}oetry$\ldots$\bibnodot{.}}
\newblock
\APACaddressPublisher{}{Addison-Wesley Longman Publishing Co., Inc.}
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Geertz%
}{%
Geertz%
}{%
{\protect \APACyear {1973}}%
}]{%
geertz1973interpretation}
\APACinsertmetastar {%
geertz1973interpretation}%
\begin{APACrefauthors}%
Geertz, C.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYear{1973}.
\newblock
\APACrefbtitle {The interpretation of cultures: {S}elected essays} {The
  interpretation of cultures: {S}elected essays}.
\newblock
\APACaddressPublisher{}{Basic Books (AZ)}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Gerv\'{a}s%
\ \BBA {} Le\'{o}n%
}{%
Gerv\'{a}s%
\ \BBA {} Le\'{o}n%
}{%
{\protect \APACyear {2014}}%
}]{%
gervas2014reading}
\APACinsertmetastar {%
gervas2014reading}%
\begin{APACrefauthors}%
Gerv\'{a}s, P.%
\BCBT {}\ \BBA {} Le\'{o}n, C.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2014}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{R}eading and {W}riting as a {C}reative {C}ycle: {T}he
  {N}eed for a {C}omputational {M}odel} {{R}eading and {W}riting as a
  {C}reative {C}ycle: {T}he {N}eed for a {C}omputational {M}odel}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {5th {I}nternational {C}onference on {C}omputational
  {C}reativity, {I}{C}{C}{C} 2014.} {5th {I}nternational {C}onference on
  {C}omputational {C}reativity, {I}{C}{C}{C} 2014.}
\newblock
\APACaddressPublisher{Ljubljana, Slovenia}{}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Grace%
\ \BBA {} Maher%
}{%
Grace%
\ \BBA {} Maher%
}{%
{\protect \APACyear {2014}}%
}]{%
grace2014using}
\APACinsertmetastar {%
grace2014using}%
\begin{APACrefauthors}%
Grace, K.%
\BCBT {}\ \BBA {} Maher, M\BPBI L.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2014}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{U}sing {C}omputational {C}reativity to {G}uide
  {D}ata-{I}ntensive {S}cientific {D}iscovery} {{U}sing {C}omputational
  {C}reativity to {G}uide {D}ata-{I}ntensive {S}cientific {D}iscovery}.{\BBCQ}
\newblock
\BIn{} Y.~Gil\ \BBA {} H.~Hirsh\ (\BEDS), \APACrefbtitle {{W}orkshops at the
  {T}wenty-{E}ighth {AAAI} {C}onference on {A}rtificial {I}ntelligence.}
  {{W}orkshops at the {T}wenty-{E}ighth {AAAI} {C}onference on {A}rtificial
  {I}ntelligence.}
\newblock
\APACrefnote{Discovery Informatics Workshop: Science Challenges for Intelligent
  Systems.}
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Harrington%
\ \BBA {} Clark%
}{%
Harrington%
\ \BBA {} Clark%
}{%
{\protect \APACyear {2007}}%
}]{%
harrington2007asknet}
\APACinsertmetastar {%
harrington2007asknet}%
\begin{APACrefauthors}%
Harrington, B.%
\BCBT {}\ \BBA {} Clark, S.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2007}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{ASKN}et: {A}utomated {S}emantic {K}nowledge {N}etwork}
  {{ASKN}et: {A}utomated {S}emantic {K}nowledge {N}etwork}.{\BBCQ}
\newblock
\BIn{} A.~Howe\ \BBA {} R.~Holte\ (\BEDS), \APACrefbtitle {{P}rocs. of the
  {T}wenty-{S}econd {A}{A}{A}{I} {C}onference on {A}rtificial {I}ntelligence}
  {{P}rocs. of the {T}wenty-{S}econd {A}{A}{A}{I} {C}onference on {A}rtificial
  {I}ntelligence}\ (\BVOL~2, \BPGS\ 889--895).
\newblock
\APACaddressPublisher{}{AAAI Press}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Jin'Ichi%
, Brazell%
\BCBL {}\ \BBA {} Cook%
}{%
Jin'Ichi%
\ \protect \BOthers {.}}{%
{\protect \APACyear {1975}}%
}]{%
jin1975art}
\APACinsertmetastar {%
jin1975art}%
\begin{APACrefauthors}%
Jin'Ichi, K.%
, Brazell, K.%
\BCBL {}\ \BBA {} Cook, L.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{1975}{}{}.
\newblock
{\BBOQ}\APACrefatitle {The {A}rt of {R}enga} {The {A}rt of {R}enga}.{\BBCQ}
\newblock
\APACjournalVolNumPages{Journal of Japanese Studies}{}{}{29--61}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Lenat%
}{%
Lenat%
}{%
{\protect \APACyear {1983}}%
}]{%
lenat1983eurisko}
\APACinsertmetastar {%
lenat1983eurisko}%
\begin{APACrefauthors}%
Lenat, D\BPBI B.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{1983}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{EURISKO}: a program that learns new heuristics and
  domain concepts: the nature of heuristics {III}: program design and results}
  {{EURISKO}: a program that learns new heuristics and domain concepts: the
  nature of heuristics {III}: program design and results}.{\BBCQ}
\newblock
\APACjournalVolNumPages{Artificial Intelligence}{21}{1}{61--98}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Meredith%
, Lemstr{\"o}m%
\BCBL {}\ \BBA {} Wiggins%
}{%
Meredith%
\ \protect \BOthers {.}}{%
{\protect \APACyear {2002}}%
}]{%
meredith2002algorithms}
\APACinsertmetastar {%
meredith2002algorithms}%
\begin{APACrefauthors}%
Meredith, D.%
, Lemstr{\"o}m, K.%
\BCBL {}\ \BBA {} Wiggins, G\BPBI A.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2002}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Algorithms for discovering repeated patterns in
  multidimensional representations of polyphonic music} {Algorithms for
  discovering repeated patterns in multidimensional representations of
  polyphonic music}.{\BBCQ}
\newblock
\APACjournalVolNumPages{Journal of New Music Research}{31}{4}{321--345}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Pask%
}{%
Pask%
}{%
{\protect \APACyear {1984}}%
}]{%
conversation-theory-review}
\APACinsertmetastar {%
conversation-theory-review}%
\begin{APACrefauthors}%
Pask, G.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{1984}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Review of conversation theory and a protologic (or
  protolanguage), {L}p} {Review of conversation theory and a protologic (or
  protolanguage), {L}p}.{\BBCQ}
\newblock
\APACjournalVolNumPages{ECTJ}{32}{1}{3-40}.
\newblock
\begin{APACrefURL} \url{http://dx.doi.org/10.1007/BF02768767} \end{APACrefURL}
\newblock
\begin{APACrefDOI} \doi{10.1007/BF02768767} \end{APACrefDOI}
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Pease%
\ \BBA {} Colton%
}{%
Pease%
\ \BBA {} Colton%
}{%
{\protect \APACyear {2011}}%
}]{%
pease2011computational}
\APACinsertmetastar {%
pease2011computational}%
\begin{APACrefauthors}%
Pease, A.%
\BCBT {}\ \BBA {} Colton, S.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2011}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Computational creativity theory: Inspirations behind the
  {FACE} and the {IDEA} models} {Computational creativity theory: Inspirations
  behind the {FACE} and the {IDEA} models}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {Proceedings of the {S}econd {I}nternational {C}onference
  on {C}omputational {C}reativity.} {Proceedings of the {S}econd
  {I}nternational {C}onference on {C}omputational {C}reativity.}
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Pease%
, Colton%
, Ramezani%
, Charnley%
\BCBL {}\ \BBA {} Reed%
}{%
Pease%
\ \protect \BOthers {.}}{%
{\protect \APACyear {2013}}%
}]{%
pease2013discussion}
\APACinsertmetastar {%
pease2013discussion}%
\begin{APACrefauthors}%
Pease, A.%
, Colton, S.%
, Ramezani, R.%
, Charnley, J.%
\BCBL {}\ \BBA {} Reed, K.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2013}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{A} {D}iscussion on {S}erendipity in {C}reative
  {S}ystems} {{A} {D}iscussion on {S}erendipity in {C}reative
  {S}ystems}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {Proceedings of the {F}ourth {I}nternational {C}onference
  on {C}omputational {C}reativity.} {Proceedings of the {F}ourth
  {I}nternational {C}onference on {C}omputational {C}reativity.}
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Pease%
, Guhe%
\BCBL {}\ \BBA {} Smaill%
}{%
Pease%
\ \protect \BOthers {.}}{%
{\protect \APACyear {2010}}%
}]{%
pease10}
\APACinsertmetastar {%
pease10}%
\begin{APACrefauthors}%
Pease, A.%
, Guhe, M.%
\BCBL {}\ \BBA {} Smaill, A.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2010}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Some Aspects of Analogical Reasoning in Mathematical
  Creativity} {Some aspects of analogical reasoning in mathematical
  creativity}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {{P}roceedings of the {I}nternational {C}onference on
  {C}omputational {C}reativity} {{P}roceedings of the {I}nternational
  {C}onference on {C}omputational {C}reativity}\ (\BPG~60-64).
\newblock
\APACaddressPublisher{Lisbon, Portugal}{}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Perez%
, Samothrakis%
, Lucas%
\BCBL {}\ \BBA {} Rohlfshagen%
}{%
Perez%
\ \protect \BOthers {.}}{%
{\protect \APACyear {2013}}%
}]{%
perez2013rolling}
\APACinsertmetastar {%
perez2013rolling}%
\begin{APACrefauthors}%
Perez, D.%
, Samothrakis, S.%
, Lucas, S.%
\BCBL {}\ \BBA {} Rohlfshagen, P.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2013}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Rolling horizon evolution versus tree search for
  navigation in single-player real-time games} {Rolling horizon evolution
  versus tree search for navigation in single-player real-time games}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {Proceedings of the 15th annual conference on {G}enetic
  and evolutionary computation} {Proceedings of the 15th annual conference on
  {G}enetic and evolutionary computation}\ (\BPGS\ 351--358).
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
P\'erez~y P\'erez%
, Aguilar%
\BCBL {}\ \BBA {} Negrete%
}{%
P\'erez~y P\'erez%
\ \protect \BOthers {.}}{%
{\protect \APACyear {2010}}%
}]{%
perezyperez10MM}
\APACinsertmetastar {%
perezyperez10MM}%
\begin{APACrefauthors}%
P\'erez~y P\'erez, R.%
, Aguilar, A.%
\BCBL {}\ \BBA {} Negrete, S.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2010}{}{}.
\newblock
{\BBOQ}\APACrefatitle {The {ERI-Designer}: A Computer Model for the Arrangement
  of Furniture} {The {ERI-Designer}: A computer model for the arrangement of
  furniture}.{\BBCQ}
\newblock
\APACjournalVolNumPages{Minds and Machines}{20}{4}{533-564}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Pickering%
}{%
Pickering%
}{%
{\protect \APACyear {2002}}%
}]{%
pickering2002cybernetics}
\APACinsertmetastar {%
pickering2002cybernetics}%
\begin{APACrefauthors}%
Pickering, A.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2002}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{C}ybernetics and the {M}angle: {A}shby, {B}eer and
  {P}ask} {{C}ybernetics and the {M}angle: {A}shby, {B}eer and {P}ask}.{\BBCQ}
\newblock
\APACjournalVolNumPages{Social studies of science}{32}{3}{413--437}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Samothrakis%
\ \BBA {} Lucas%
}{%
Samothrakis%
\ \BBA {} Lucas%
}{%
{\protect \APACyear {2011}}%
}]{%
samothrakis2011approximating}
\APACinsertmetastar {%
samothrakis2011approximating}%
\begin{APACrefauthors}%
Samothrakis, S.%
\BCBT {}\ \BBA {} Lucas, S.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2011}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{A}pproximating $n$-player {B}ehavioural {S}trategy
  {N}ash {E}quilibria {U}sing {C}oevolution} {{A}pproximating $n$-player
  {B}ehavioural {S}trategy {N}ash {E}quilibria {U}sing {C}oevolution}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {Proc. of the 13th annual conference on {G}enetic and
  evolutionary computation} {Proc. of the 13th annual conference on {G}enetic
  and evolutionary computation}\ (\BPGS\ 1107--1114).
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Saunders%
}{%
Saunders%
}{%
{\protect \APACyear {2012}}%
}]{%
saunders2012towards}
\APACinsertmetastar {%
saunders2012towards}%
\begin{APACrefauthors}%
Saunders, R.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2012}{}{}.
\newblock
{\BBOQ}\APACrefatitle {Towards autonomous creative systems: {A} computational
  approach} {Towards autonomous creative systems: {A} computational
  approach}.{\BBCQ}
\newblock
\APACjournalVolNumPages{Cognitive Computation}{4}{3}{216--225}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Saunders%
\ \BBA {} Gero%
}{%
Saunders%
\ \BBA {} Gero%
}{%
{\protect \APACyear {2001}}%
}]{%
saunders2001digital}
\APACinsertmetastar {%
saunders2001digital}%
\begin{APACrefauthors}%
Saunders, R.%
\BCBT {}\ \BBA {} Gero, J\BPBI S.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2001}{}{}.
\newblock
{\BBOQ}\APACrefatitle {The digital clockwork muse: {A} computational model of
  aesthetic evolution} {The digital clockwork muse: {A} computational model of
  aesthetic evolution}.{\BBCQ}
\newblock
\BIn{} \APACrefbtitle {{P}roc. {A}nnual {C}onvention of the {S}ociety for the
  {S}tudy of {A}rtificial {I}ntelligence and {S}imulation of {B}ehaviour
  ({S}{S}{A}{I}{S}{B})} {{P}roc. {A}nnual {C}onvention of the {S}ociety for the
  {S}tudy of {A}rtificial {I}ntelligence and {S}imulation of {B}ehaviour
  ({S}{S}{A}{I}{S}{B})}\ (\BPGS\ 12--21).
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Seth%
}{%
Seth%
}{%
{\protect \APACyear {2015}}%
}]{%
seth2015cybernetic}
\APACinsertmetastar {%
seth2015cybernetic}%
\begin{APACrefauthors}%
Seth, A\BPBI K.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{2015}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{T}he {C}ybernetic {B}ayesian {B}rain: {F}rom
  {I}nteroceptive {I}nference to {S}ensorimotor {C}ontingencies} {{T}he
  {C}ybernetic {B}ayesian {B}rain: {F}rom {I}nteroceptive {I}nference to
  {S}ensorimotor {C}ontingencies}.{\BBCQ}
\newblock
\BIn{} T.~Metzinger\ \BBA {} J\BPBI M.~Windt\ (\BEDS), \APACrefbtitle {{O}pen
  {M}{I}{N}{D}.} {{O}pen {M}{I}{N}{D}.}
\newblock
\APACaddressPublisher{}{The MIND Group}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Van~Andel%
}{%
Van~Andel%
}{%
{\protect \APACyear {1994}}%
}]{%
pek}
\APACinsertmetastar {%
pek}%
\begin{APACrefauthors}%
Van~Andel, P.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYearMonthDay{1994}{}{}.
\newblock
{\BBOQ}\APACrefatitle {{A}natomy of the {U}nsought {F}inding} {{A}natomy of the
  {U}nsought {F}inding}.{\BBCQ}
\newblock
\APACjournalVolNumPages{The British Journal for the Philosophy of
  Science}{45}{2}{pp. 631--648}.
\PrintBackRefs{\CurrentBib}

\bibitem [\protect \citeauthoryear {%
Van Der~Aalst%
}{%
Van Der~Aalst%
}{%
{\protect \APACyear {2011}}%
}]{%
van2011process}
\APACinsertmetastar {%
van2011process}%
\begin{APACrefauthors}%
Van Der~Aalst, W.%
\end{APACrefauthors}%
\unskip\
\newblock
\APACrefYear{2011}.
\newblock
\APACrefbtitle {Process mining: discovery, conformance and enhancement of
  business processes} {Process mining: discovery, conformance and enhancement
  of business processes}.
\newblock
\APACaddressPublisher{}{Springer Science \& Business Media}.
\PrintBackRefs{\CurrentBib}

\end{thebibliography}
