\section{Intro}\label{intro}

A variety of programming environments and tools can provide the
\emph{impression of changing a program while it is
running}\autocites{Tanimoto1990VVL}{trenouth1991survey}{Nick2003LCL}.
Nowadays, this impression is often described as \emph{liveness}. While
research on \emph{liveness} is not part of mainstream research on
programming, still a broad spectrum of contributions exists. In
particular, three ideas incorporate this capability as an integral part:
\emph{live coding}, \emph{live programming}, and \emph{exploratory
programming}. As each term has its own research community, shared
approaches and potential synergies might be overlooked. Further, the
whole range of research on liveness only becomes visible when looking at
all these communities. To delineate this potential across communities
and illustrate the range of contributions, we conducted a literature
study.

Looking at prominent publications from each community approximate
differences become apparent. Generally speaking \emph{live coding} is
often concerned with the creation of art through changing source code as
a performance in front of an audience
\autocites{Nick2003LCL}{blackwell2005programming}. \emph{Live
programming} in contrast often seems to put the very activity of
programming in its focus \autocites{Tanimoto2013PEL}{Hancock2003RTP}.
Correspondingly, the term seems to be used when describing programming
tools which provide immediate feedback on the dynamic behavior of a
program even while programming. The term \emph{exploratory programming}
often refers to a particular workflow during programming whenever
requirements are not fully defined but are yet to be discovered
\autocites{trenouth1991survey}{sheil1983power}. It is supported by
exploratory programming environments that incorporate changing a running
system to make exploration of unknown domains or of design alternatives
easier.

The communities around these ideas differ in their motives for dealing
with changing a program while it is running, the fundamental perspective
on the activity of programming, applications domains in which liveness
is used, and desired outcomes of their research (system architectures,
workflows, experiences). Besides their differences, they share the
common notion of creating an impression of changing a program while it
is running. Their differing approaches to this notion could be leveraged
for cross-polination between the communities to advance all three of
them. For example, \emph{exploratory programming} systems could benefit
from the specialized mechanisms and tools investigated under the term
\emph{live programming}. At the same time the \emph{live programming}
tools could be examined through the lense of experience reports from
programmers as it is done in the \emph{live coding} community.

All three communities are academic communities. Thus, we treat the
publications published under the corresponding terms as artifacts which
can be studied to gain insights into the values and practices of each
community. Hence, we relate the communities around these terms to each
other by surveying the similarities and differences in a sample of the
corresponding literature. In particular, our contributions in this paper
are:

\begin{itemize}
\tightlist
\item
  A systematically collected sample of 191 publications indexed in the
  ACM DL, IEEE Xplore, and DBLP on the terms \emph{live programming},
  \emph{live coding}, and \emph{exploratory programming}\footnote{We
    will publish the bibliography files on Zenodo with the final version
    of this paper}.
\item
  A thematic analysis \autocite{braun2006using} of the sample regarding
  the following aspects:

  \begin{itemize}
  \tightlist
  \item
    motivations for liveness
  \item
    applications domains in which the liveness is used
  \item
    types of contributions
  \item
    historic distribution of terms
  \item
    modes of running a program
  \item
    prominent publications for each term
  \item
    fields related to the three terms
  \end{itemize}
\item
  A comparison of the results of the thematic analysis between the
  corpus for each of the three terms
\end{itemize}

The remainder of the paper is structured as follows. We begin with the
specific research questions guiding the literature study in
\autoref{researchQuestions}. We will then illustrate our methodology in
\autoref{methodology}. We describe the characteristics of the three
corpora in \autoref{characteristics} and the results of the study and
the relation to the research questions in \autoref{results}. We discuss
correlations in the study results, threats to validity, and potential
consequences in \autoref{discussion} and summarize our findings and
point out further potential research in \autoref{conclusion}.

\section{Research Questions}\label{researchQuestions}

The goal of this survey is to determine differences and commonalities
between the three communities around the terms ``live coding'', ``live
programming'', and ``exploratory programming''. In this context, we
posed eight research questions.

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  The notion of an impression of changing a program while it is running
  has been around since the 1950s and the term \emph{liveness} has been
  used since the 1990s
  \autocites{McCarthy:1978:HL:960118.808387}{Tanimoto1990VVL}. However,
  communities around these terms seem to have formed later, thus we want
  to determine \emph{What is the distribution of publications on the
  different terms over time?}
\item
  While live coding seems to be mainly focused on programming for
  performance art, live programming and exploratory programming do not
  seem to have such a strong focus on one application domain. Therefore,
  we want to find out \emph{In the context of which domains has liveness
  been used?}
\item
  The motivation for applying the technical concept for liveness can be
  telling about the values of the communities, thus we want to determine
  \emph{What are the motivations for having liveness?}
\item
  In order to determine what kind of knowledge each community
  contributes we want to investigate \emph{Which kinds of contributions
  are described?}
\item
  Besides the general motivation for liveness, the general reason for
  having a running program varies greatly between different approaches.
  For example, the Smalltalk community regards their systems as ``live''
  as they are constantly running and any change to a Smalltalk program
  is actually a change in the object graph of the running system. In
  order to determine whether the three communities have a particular
  stance on this aspect we want to determine *What is the general reason
  for having a running version of the program while changing it?"
\item
  Each community has works which define their field. Assuming that these
  are the most cited works in their field we want to find out \emph{What
  are the most prominent publications?}
\item
  The three terms used in this study are only one way of categorizing
  the field or research on programming. Simliar and overlapping
  perspectives might indicate further related work and future research
  potential. Thus, we want to determine \emph{Which keywords have
  authors applied to their work?}
\end{enumerate}

Finally, besides these research questions related to the publication we
want to determine a list of artifacts providing liveness. These
artifacts (such as tools, user interfaces, programming environments)
created by the communities can be the foundation for determining common
patterns in the design of a live programming experience.

\section{Methodology}\label{methodology}

In order to determine the range of topics and compare the literature
related to the three terms we conduct a thematic analysis on three
corpora, one for each term. We did not aim to create a comprehensive
systematic literature survey of the three terms but to determine the
general distribution of themes of each corpus regarding to our research
questions.

In general, we followed the SALSA (Search, AppraisaL, Synthesis,
Analysis) process \autocite{grant2009typology}. First, we created an
overall corpus with a systematic search and an appraisal of works. We
then synthesized data for the individual research questions by
extracting data from the publications and by doing a thematic analysis
\autocite{braun2006using}. We analyzed the resulting data by determining
correlations, comparing overlaps, and differences between the corpora
defined by the three terms.

\subsection{Search: Systematically Determined
Corpora}\label{search-systematically-determined-corpora}

For each term we created an initial corpus and then created the hull of
references. We determined the initial corpora by using the full-text
search on three major indexing services :

\begin{itemize}
\tightlist
\item
  IEEE Xplore
  (\url{http://ieeexplore.ieee.org/search/searchresult.jsp?queryText=.QT.KEYWORD.QT.\&newsearch=true})
\item
  ACM Digital Library (ACM DL): Guide to Computing
  (\url{http://dl.acm.org/exportformats_search.cfm?query=KEYWORD\&filtered=\&within=owners\%2Eowner\%3DGUIDE\&dte=\&bfr=\&srt=\%5Fscore\&expformat=bibtex})
\item
  Digital Bibliography \& Library Project (DBLP)
  (\url{http://dblp.uni-trier.de/search/publ/api?q=KEYWORD\%24\&h=1000\&format=bib1\&rd=1a})
\end{itemize}

We retrieved these search results through an automated process using the
URLs denoted in brackets (insertion of the keyword denoted as KEYWORD).
Depending on the capabilities of the search API the keywords were either
entered as exact matches (for example \texttt{"live\ programming"} at
IEEE) or AND queries (for example \texttt{live+programming} for DBLP).

Further, we manually added publications from manually selected
scientific venues (For a detailed list of keywords and venues per term,
see table 1). Namely, we added the International Conference on Live
Coding (ICLC) and the Live Programming Workshop (LIVE). Other major
venues, such as CHI, UIST, or PLATEAU are fully indexed by the three
services we used and thus appeared automatically through our initial
search. We then selected appropriate publications from this initial
corpus using our appraisal criteria.

\begin{longtable}[]{@{}lll@{}}
\toprule
Term & Keywords & Venues\tabularnewline
\midrule
\endhead
live coding & livecoding, live coding & ICLC 2015, ICLC 2016, ICLC
2017\footnote{Only contains the publications which were actually made
  available from the conference website.}\tabularnewline
live programming & live programming & LIVE 2013, LIVE 2016, LIVE
2017\tabularnewline
exploratory programming & exploratory programming & -\tabularnewline
\bottomrule
\end{longtable}

\subsection{Appraisal}\label{appraisal}

The appraisal criteria are concerned with the nature of the publication
as well as the content of publications. The complete appraisal process
was done manually.

For a publication to be accepted it has to be peer-reviewed. This covers
publications for conferences, workshops, and journals, as well as
doctoral theses. To ensure that the papers contained enough information
for us to evaluate the content, we removed all publications with only
one page.

In general, a publication is added to a term-specific corpus if it
contains the corresponding term in the title, keywords, abstract or the
title of the publication it is part of, for example the title of the
proceedings. Further, whenever the term occurred we manually determined
whether it actually referred to the particular experience of programming
we want to investigate. This is necessary as the three terms live
programming, live coding, and exploratory programming are also used in
other domains. Consequently, we only selected papers adhering to the
following notion of programming:

\begin{quote}
``the human activity of describing a process run by a computer''
\end{quote}

This excludes the following domains from the corpora:

\begin{itemize}
\tightlist
\item
  planning and modification of television schedules
\item
  encoding video streams during recording and broadcasting
\item
  algorithms for artifical intelligence which explore a solution space
\item
  planning health programs
\item
  analysis of concurrent programs for the liveness property
\end{itemize}

Further, the term ``live coding'' is used in \emph{teaching research} to
refer to a class room constellation in which teachers program ``live''
in the sense that students can follow the teachers' activities in a
programming environment on a projector. We also excluded publications on
this topic as they do not include an explict notion of changing a
program while it is running. The fact that the mere writing of source
code is done ``live'' in front of an audience (without continuously
executing the program) those not entail any particular connection
between the code and its dynamic behavior.

Also, while exploratory programming environments refers to environments
which often include some form of liveness, the single term ``exploratory
programming'' can also refer to a particular workflow
\autocite{trenouth1991survey}. This ``exploratory workflow'' is creating
new alternatives and trying them out. This does not entail liveness but
can also be done in an edit-compile-run cycle with a long roundtrip
time. We excluded papers exclusively referring to exploratory workflows
and did not mention exploratory programming environments. {[}This list
comprised 10 papers{]}

\subsection{Synthesis: Thematic
Analysis}\label{synthesis-thematic-analysis}

We derived the data for research questions 1, 6, and 7 directly and
manually from the individual publications. The specific procedures used
are reported in {[}results{]}.

As the goal of the study is to illustrate the spectrum of research on
the topic of liveness, we applied thematic analysis instead of a
pre-defined code book to gather data for research questions 2 to 4
\autocite{braun2006using}. That means we determined the codes for the
themes application domain, motivation, and types of contribution during
coding. Specifically, we used theoretical thematic analysis with the
themes defined by our research questions \autocite{braun2006using}. The
codes for the theme modes of running were determined beforehand.

All publications were coded as one corpus in order to prevent bias of
the coder towards codes who they might deem fitting to the corpus a
publication belongs to. The codes are described in detail in the related
sections in {[}results{]}. All coding was done by one coder. During the
first pass the set of codes changed, as codes were added or the focus of
a code changed. Thus, we did a second pass through the corpus for each
theme revising the codes.

Afterwards, the coder formalized the codes in the descriptions. We
determined the inter-coder reliability for a 20\% sample using Cohen's
kappa. We report the inter-coder reliablity with each result.

\subsection{Analysis}\label{analysis}

The main goal of the study is to illustrate the spectrum of research on
liveness. Thus, we directly report the determined codings for the
overall corpus. We do not report on the distribution of codes for the
overall corpus, as the three terms are disproportionally represented and
consequently the overall distribution would be skewed to the
distribution of the term with the most publications.

Another goal of the study is to investigate the overlap and differences
of the terms \emph{live programming}, \emph{live coding}, and
\emph{exploratory programming}. Hence, as a further analysis we compare
the distribution of the codes between the three term-specific corpora.

\section{Corpus Characteristics}\label{characteristics}

\subsection{Selection}\label{selection}

From how many papers to how many papers?

\subsection{Publication Data}\label{publication-data}

\subsubsection{Overlap}\label{overlap}

\subsubsection{Venues and Journals}\label{venues-and-journals}

\begin{longtable}[]{@{}ll@{}}
\toprule
\begin{minipage}[b]{0.33\columnwidth}\raggedright\strut
Term\strut
\end{minipage} & \begin{minipage}[b]{0.61\columnwidth}\raggedright\strut
Venues / Journals (sorted alphabetically)\strut
\end{minipage}\tabularnewline
\midrule
\endhead
\begin{minipage}[t]{0.33\columnwidth}\raggedright\strut
live programming\strut
\end{minipage} & \begin{minipage}[t]{0.61\columnwidth}\raggedright\strut
CHI, CHI, COP, ComposableWeb, ECOOP, FARM@ICFP, HCC, ICSE, ISMM, LIVE,
MOVILESoft, OOPSLA, Onward!, PLDI, PROMOTO, PX, Programming Journal,
SIGMOD, SPLASH, UIST, VL/HCC, VRST\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.33\columnwidth}\raggedright\strut
exploratory programming\strut
\end{minipage} & \begin{minipage}[t]{0.61\columnwidth}\raggedright\strut
CHI, COMPCON, ECOOP, HOPL, ICSENG, IMCSIT, ISCA, OOPSLA, Onward!, PX,
VL, VL/HCC\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.33\columnwidth}\raggedright\strut
live coding\strut
\end{minipage} & \begin{minipage}[t]{0.61\columnwidth}\raggedright\strut
AM, Blocks and Beyond, C\&C, CLEI, DUXU, EVA, EvoMUSART, FARM, ICFP,
ICLC, ICSE, LIVE, MM, NIME, OZCHI, SIGCSE, TEI, VL/HCC\strut
\end{minipage}\tabularnewline
\bottomrule
\end{longtable}

\subsubsection{Distribution over Time}\label{distribution-over-time}

\subsubsection{Keywords}\label{keywords}

In order to determine other perspectives on the field of research on
programming related to liveness, we extracted the keywords of each
publication. We only recorded keywords mentioned in the actual
publication. We also only recorded the keywords chosen by the authors
and ignored general tags and categories pre-determined by the publisher
(for example the ACM subject classification).

We cleaned the data marginally by merging the keywords ``programming
environment'' and ``programming environments'' as well as ``end-user
development'' and ``end-user programming''. Finally, we selected all
keywords mentioned more then once.

\begin{longtable}[]{@{}ll@{}}
\toprule
\begin{minipage}[b]{0.33\columnwidth}\raggedright\strut
Term (works with terms / corpora size)\strut
\end{minipage} & \begin{minipage}[b]{0.61\columnwidth}\raggedright\strut
Keywords (count)\strut
\end{minipage}\tabularnewline
\midrule
\endhead
\begin{minipage}[t]{0.33\columnwidth}\raggedright\strut
live programming (50/72)\strut
\end{minipage} & \begin{minipage}[t]{0.61\columnwidth}\raggedright\strut
live programming (32), live coding (6), end-user programming (5),
debugging (4), direct manipulation (4), Smalltalk (4), live programming
environment (3), music (3), mashups (3), spreadsheets (3), programming
environment (3), Datalog (3), web services (2), prototyping (2),
integrated development environment (2), data analysis (2), incremental
maintenance (2), streaming data (2), LogiQL (2), testing (2), LogicBlox
(2), Javascript (2), liveness (2), mashup tools (2), data mining
(2)\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.33\columnwidth}\raggedright\strut
exploratory programming (12/27)\strut
\end{minipage} & \begin{minipage}[t]{0.61\columnwidth}\raggedright\strut
exploratory programming (6), programming environment (4), Self (2),
direct manipulation (2), gestures (2), liveness (2), programming by
demonstration (2), virtual machine (2), live programming (2), Forms/3
(2)\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.33\columnwidth}\raggedright\strut
live coding (29/101)\strut
\end{minipage} & \begin{minipage}[t]{0.61\columnwidth}\raggedright\strut
live coding (22), music (6), generative art (4), software engineering
(2), visual programming (2), liveness (2), art (2), domain specific
languages (2), creative coding (2), generative music (2), programming
environment (2), functional programming (2), music performance (2), web
audio (2), improvisation (2), visualization (2)\strut
\end{minipage}\tabularnewline
\bottomrule
\end{longtable}

\section{Study Results}\label{results}

For each research question state: - coding scheme - inter coder
reliability - results

Three results regarding the liveness described in the publications, One
result regarding research approach

\subsection{Which domains was the liveness used in (specifically, if it
was only given as an example it should not be counted)? (not mutually
exclusive)}\label{which-domains-was-the-liveness-used-in-specifically-if-it-was-only-given-as-an-example-it-should-not-be-counted-not-mutually-exclusive}

\subsubsection{Coding Scheme}\label{coding-scheme}

To answer the question for which application domains liveness plays a
role in each corpus, we coded the intended application domain described
in a publication. Specifically, we coded the domain of application for
which the \emph{liveness} was relevant. Further, the domain had to be a
major theme. We did not code domains which were only mentioned as
examples. The codes are not mutually exclusive.

\begin{description}
\item[Education and training] The program is created to either learn programming or to learn something from the program (for example a model of ecological processes).
\item[Simulation] Systems and programs for simulation
\item[Data analysis] The analysis of existing data to gain insights
\item[Visual performance art] The in-time creation of visual art in front of an audience or for recording. 
\item[Audio performance art] The in-time creation of audible art in front of an audience or for recording.
\item[Performance art] Any other form of performance than pure visuals and audio, for example writing poems, dance, general performance art reflections
\item[Web development] The construction of applications running in or on the web. This does not involve the construction of programs through an environment implemented as a web application.
\item[User interfaces] The design and implementation of user interfaces.
\item[Databases] The programming and manipulation of databases.
\item[Programming languages] The design and implementation of programming languages (syntax and semantics).
\item[Systems programming] Development of execution environments or fundamental libraries
\item[Computer graphics] Creating programs concerned with generating visual output (though not as a performance in contrast to visual performance art)
\item[General software development] General software development with no focus on a particular application domain.
\item[Artifical intelligence] The design and implementation of artifical intelligence systems.
\item[Physical computing] All systems which have some interface with the physical world (actuators or sensors).
\item[Virtual/Augmented reality] 
\item[Enterprise-Resource-Planning (ERP) systems]
\item[Computer games]
\item[Software verification]
\item[Text editing]
\end{description}

\subsection{Intended Outcome}\label{intended-outcome}

TODO: outcome is not the right word TODO: explain program vs system in
more detail: system is about modifying the runtime state which in turn
might represent a program TODO: Jens put it like: What is the goal of
the user of the meta system? The notion of ``an impression of changing a
program while it is running'' subsumes different modes of ``running'' a
program.

\subsubsection{Coding Scheme}\label{coding-scheme-1}

We distinguish three modes of ``running a program'' according to the
intended outcome:

\begin{description}
\item[Computing a result] The programmer wants some output and therefore runs a program. The program is only secondary and might be discarded after the computation, this covers in-time output such as generated sound as well as discrete outputs created by a data analysis algorithm.
\item[Creating a program] The programmer wants to create a program. The program is currently running to get feedback on the dynamic behavior of it but the resulting artifact of the activity are some static representation of the program behavior.  
\item[Evolving a system] The programmer wants to evolve a system which is constantly running. Thus, programmers change behavior descriptions as well as the state of the system. Examples are image-based systems or database systems.
\end{description}

Notably, all three modes of running a program are possible for most
activities and programs (for example a programmer might keep a Python
REPL for data analysis open for a long time). Therefore, we assigned the
code only to the emphasis in the described work. Although, the
categories are mutually exclusive, we allowed assigning multiple codes
to a publication when it described multiple aspects of the activity of
the programmer.

\subsection{Motivation for the term
stated}\label{motivation-for-the-term-stated}

While liveness is generally an intriguing idea, the motivations for it
vary greatly between authors and communities. However, the initial
motivation influences designs and research methodology and is thus an
integral element of the definition of a field.

\subsubsection{Coding Scheme}\label{coding-scheme-2}

The following coding scheme reflects the variety of actual motivations
for liveness. When determining the motivation in a publication we only
assigned a code when it was a) explicitly stated or b) a fundamental
theme of the work. As we extracted the list directly from the
publications, the items do not always have the same level of
abstraction.

\begin{description}
\item[Productivity] Liveness improves productivity, most often through making some activity during programming faster
\item[Creativity] This applies to all works specifically mentioning creativity as their primary motication.
\item[Exploration of alternatives] This is the very specific notation of livness enabling programmers to explore distinct alternative solutions (It does not suffice to say it supports "exploratory programming" as this would often be self-referential; mentioning "exploratory work" falls in this category though)
\item[Live tuning] Liveness as a means to explore a continuous parameter space of a design
\item[Comprehension] Ease the process of comprehension and learning of program behavior or the program domain
\item[In-time output] Livness is an integral part of the activity itself. The output has to be created and modified with a short delay as the changing of the output is the relevant action; this does often correlate with live coding but does not have to. <!-- There are argumentations which argue that industrial live programming is similar due to pair programming -->
\item[Collaboration] The liveness enables collaboration between users because others can see the effects of owns changes 
\item[Accessibility] Liveness makes programming more accessible. This involves comprehension but is more specific towards enabling less experience programmers
\item[Engagement] Getting users more enganged through liveness
\item[Enstrangement] Live programming as an extreme form of programming which enables reflections on the nature of programming
\item[Can not be determined] We added this code explictly in order to determine how often liveness is taken as granted.
\end{description}

Again, these codes are not mutually exclusive.

\subsection{What type of contribution is
described?}\label{what-type-of-contribution-is-described}

As illustrated before, the three communities differ greatly in their
motivatios for liveness and desired outcomes. These different world
views also reflect in the inquiries and the contributions communicated
within the community.

\subsubsection{Coding Scheme}\label{coding-scheme-3}

We code the contribution of each publication. Some of these codes are
hierarchical so a code can have several sub-codes which also count in
the super-code set (see for example the empirical codes).

\begin{description}
\item[Empirical studies] Works which investigated an existing artifact or phenomena. As this covers a wide range of contributions we made codes more specific with the following additional information: a) fixed / flexible setup\footnote{Was the methodology completely determined before the observation or was it adapted during the observation [TODO: cite real world research]} b) quantitative / qualitative outcome c) humans as subject / systems as subjects 
\item[Survey] Surveys of existing artifacts, for example publications or systems
\item[Technical design] The description of a technical implementation
\item[User interface design] Describes the design of a particular user interaction. This can be a novel form of interaction, a new input device, or a new way of output provided by the system.
\item[Programming interface design] This involves programming language design (syntax and semantics) as well as the design of the interface of libraries.
\item[Tool design] The description of the design of a programming tool or environment
\item[Individual account of experience] An individual subjective account
\item[Formal methods] The description of a formal model or the application of an existing model
\item[Algorithm design] The description of an algorithm
\item[Practices] The description of practices either empirically observed or from subjective experience
\item[Design principles] A collection of rules or values for designs
\item[Essay] An application of theoretical models or principles to an existing phenomena [TODO: wat?]
\item[Performance] The description of an artistic performance 
\item[Curriculum design] The description of a new curriculum or teaching model
\end{description}

\subsection{Prominent Publications}\label{prominent-publications}

In order to determine the which publications define and form each field,
we determined the most prominent publications referenced by a corpus.
Therefore, we collected all references from publications of one corpus.
The cited publication did not have to be in the corpus itself in order
to be collected. We list the five most cited references. If there are
multiple publications with the same citation count at position five we
included all publications with that citation count.

\subsubsection{Exploratory Programming}\label{exploratory-programming}

\begin{itemize}
\tightlist
\item
  (9) Smalltalk-80: The Language and Its Implementation
  \autocite{Goldberg:1983:SLI:273}
\item
  (7) Smalltalk and exploratory programming \autocite{Sandberg1988SEP}
\item
  (5) Using Prototypical Objects to Implement Shared Behavior in
  Object-Oriented Systems \autocite{Lieberman:1986:UPO:28697.28718}
\item
  (4) Self: The Power of Simplicity \autocite{Smith:1994:SPS:974939}
\item
  (4) The Design and Implementation of the Self Compiler, an Optimizing
  Compiler for Object-Oriented Programming Languages
  \autocite{xxx1992TDI}
\item
  (4) An Efficient Implementation of SELF - a Dynamically-Typed
  Object-Oriented Language Based on Prototypes \autocite{xxx1989AEI}
\item
  (4) Back to the Future: The Story of Squeak, a Practical Smalltalk
  Written in Itself \autocite{ingalls_1997_squeak}
\end{itemize}

\subsubsection{Live Programming}\label{live-programming}

\begin{itemize}
\tightlist
\item
  (18) Real-time Programming and the Big Ideas of Computational Literacy
  \autocite{xxx2003RPB}
\item
  (14) Smalltalk-80: The Language and Its Implementation
  \autocite{Goldberg:1983:SLI:273}
\item
  (13) VIVA: A Visual Language for Image Processing
  \autocite{Tanimoto1990VVL}
\item
  (12) Living It Up with a Live Programming Language
  \autocite{McDirmid2007LUL}
\item
  (11) Usable Live Programming
  \autocite{McDirmid:2013:ULP:2509578.2509585}
\end{itemize}

\subsubsection{Live Coding}\label{live-coding}

\begin{itemize}
\tightlist
\item
  (26) Live Coding in Laptop Performance \autocite{Nick2003LCL}
\item
  (12) The programming language as a musical instrument
  \autocite{blackwell2005programming}
\item
  (12) Gibber: Live Coding Audio in the Browser
  \autocite{Roberts2012GLC}
\item
  (11) Live Coding of Consequence \autocite{xxx2011LCC}
\item
  (11) Aa-Cell in Practice: An Approach to Musical Live Coding
  \autocite{Sorensen2007ACP}
\item
  (11) Rethinking the Computer Music Language: SuperCollider
  \autocite{mccartney2002rethinking}
\item
  (11) Visualisation of Live Code \autocite{Alex2010VLC}
\item
  (11) The IXI Lang: A SuperCollider Parasite for Live Coding
  \autocite{Magnusson2011TIL}
\item
  (11) Tidal - Pattern Language for the Live Coding of Music
  \autocite{xxx2010TPL}
\end{itemize}

\section{Discussion}\label{discussion}

\subsection{Is there an Exploratory Programming Environments
Community?}\label{is-there-an-exploratory-programming-environments-community}

\begin{itemize}
\tightlist
\item
  Exploratory programming: yes!
\item
  exp prog envs: Not clear
\item
  Smalltalk and Self in keywords suggests its particular language
  communities
\end{itemize}

\subsection{Threats to Validity}\label{threats-to-validity}

\begin{itemize}
\item
  Missing hull: For example, the original Viva paper is missing
\item
  Selection based on indexing services
\item
  Threats to validity: Indexing services + OCR and thereby maybe wrong
  positive
\end{itemize}

\section{Conclusion}\label{conclusion}

\appendix

\section{Detailed Data Sets}\label{detailed-data-sets}

\subsection{lpsSynthesisDomains}\label{lpssynthesisdomains}

\begin{longtable}[]{@{}lllll@{}}
\toprule
codes & all & live prog. & expl. prog. & live coding\tabularnewline
\midrule
\endhead
Artifical intelligence & 3 & 0 & 2 & 1\tabularnewline
Audio performance art & 78 & 4 & 0 & 78\tabularnewline
Computer graphics & 11 & 7 & 0 & 6\tabularnewline
Data analysis & 15 & 11 & 3 & 3\tabularnewline
Database development & 5 & 5 & 0 & 0\tabularnewline
Education and training & 24 & 10 & 2 & 16\tabularnewline
Enterprise-Resource-Planning (ERP) systems & 3 & 3 & 0 &
0\tabularnewline
Game development & 1 & 1 & 0 & 0\tabularnewline
General software development & 70 & 45 & 17 & 13\tabularnewline
Performance art & 15 & 0 & 0 & 15\tabularnewline
Physical computing & 7 & 5 & 0 & 2\tabularnewline
Programming languages & 5 & 4 & 0 & 1\tabularnewline
Simulation & 6 & 2 & 4 & 0\tabularnewline
Software verification & 2 & 2 & 0 & 0\tabularnewline
Systems programming & 7 & 2 & 3 & 2\tabularnewline
Text editing & 1 & 0 & 0 & 1\tabularnewline
User interfaces & 13 & 7 & 4 & 2\tabularnewline
Virtual/Augmented reality & 4 & 2 & 0 & 2\tabularnewline
Visual performance art & 21 & 3 & 0 & 20\tabularnewline
Web development & 6 & 6 & 0 & 0\tabularnewline
Total & 297 & 119 & 35 & 162\tabularnewline
\bottomrule
\end{longtable}

\subsection{lpsSynthesisIntendedEffect}\label{lpssynthesisintendedeffect}

\begin{longtable}[]{@{}lllll@{}}
\toprule
codes & all & live prog. & expl. prog. & live coding\tabularnewline
\midrule
\endhead
Computing a result & 101 & 18 & 3 & 86\tabularnewline
Creating a program & 51 & 35 & 6 & 13\tabularnewline
Evolving a system & 43 & 23 & 16 & 5\tabularnewline
Not applicable & 6 & 0 & 2 & 4\tabularnewline
Total & 201 & 76 & 27 & 108\tabularnewline
\bottomrule
\end{longtable}

\subsection{lpsSynthesisMethodOfInquiry}\label{lpssynthesismethodofinquiry}

\begin{longtable}[]{@{}lllll@{}}
\toprule
codes & all & live prog. & expl. prog. & live coding\tabularnewline
\midrule
\endhead
Algorithm design & 6 & 2 & 0 & 4\tabularnewline
Can not be determined & 3 & 0 & 2 & 1\tabularnewline
Curriculum design & 4 & 1 & 1 & 3\tabularnewline
Design principles & 11 & 5 & 3 & 4\tabularnewline
Empirical study (system) & 27 & 13 & 8 & 6\tabularnewline
Empirical study (users) & 36 & 16 & 10 & 17\tabularnewline
Essay & 20 & 2 & 0 & 18\tabularnewline
Formal methods & 9 & 8 & 0 & 1\tabularnewline
Individual account of experience & 14 & 0 & 2 & 12\tabularnewline
Performance & 19 & 2 & 0 & 19\tabularnewline
Practices & 11 & 0 & 0 & 11\tabularnewline
Programming interface design & 37 & 15 & 7 & 18\tabularnewline
Survey & 3 & 1 & 0 & 3\tabularnewline
Technical design & 75 & 28 & 14 & 36\tabularnewline
Tool design & 83 & 50 & 11 & 26\tabularnewline
User interface design & 34 & 15 & 3 & 18\tabularnewline
Total & 392 & 158 & 61 & 197\tabularnewline
\bottomrule
\end{longtable}

\subsection{lpsSynthesisMotivation}\label{lpssynthesismotivation}

\begin{longtable}[]{@{}lllll@{}}
\toprule
codes & all & live prog. & expl. prog. & live coding\tabularnewline
\midrule
\endhead
Accessibility & 23 & 19 & 2 & 3\tabularnewline
Can not be determined & 33 & 14 & 8 & 11\tabularnewline
Collaboration & 17 & 1 & 1 & 15\tabularnewline
Comprehension & 25 & 22 & 1 & 5\tabularnewline
Creativity & 3 & 0 & 1 & 2\tabularnewline
Engagement & 4 & 1 & 0 & 3\tabularnewline
Enstrangement & 2 & 0 & 0 & 2\tabularnewline
Exploration & 32 & 15 & 11 & 7\tabularnewline
In-time output & 81 & 8 & 0 & 78\tabularnewline
Live tuning & 5 & 2 & 0 & 3\tabularnewline
Productivity & 17 & 8 & 6 & 6\tabularnewline
Total & 242 & 90 & 30 & 135\tabularnewline
\bottomrule
\end{longtable}
