# Intro A variety of programming environments and tools can provide the *impression of changing a program while it is running*[@Tanimoto1990VVL; @trenouth1991survey; @Nick2003LCL]. Nowadays, this impression is often described as *liveness*. While research on *liveness* is not part of mainstream research on programming, still a broad spectrum of contributions exists. In particular, three ideas incorporate this capability as an integral part: *live coding*, *live programming*, and *exploratory programming*. As each term has its own research community, shared approaches and potential synergies might be overlooked. Further, the whole range of research on liveness only becomes visible when looking at all these communities. To delineate this potential across communities and illustrate the range of contributions, we conducted a literature study. Looking at prominent publications from each community approximate differences become apparent. Generally speaking *live coding* is often concerned with the creation of art through changing source code as a performance in front of an audience [@Nick2003LCL; @blackwell2005programming]. *Live programming* in contrast often seems to put the very activity of programming in its focus [@Tanimoto2013PEL; @Hancock2003RTP]. Correspondingly, the term seems to be used when describing programming tools which provide immediate feedback on the dynamic behavior of a program even while programming. The term *exploratory programming* often refers to a particular workflow during programming whenever requirements are not fully defined but are yet to be discovered [@trenouth1991survey; @sheil1983power]. It is supported by exploratory programming environments that incorporate changing a running system to make exploration of unknown domains or of design alternatives easier. The communities around these ideas differ in their motives for dealing with changing a program while it is running, the fundamental perspective on the activity of programming, applications domains in which liveness is used, and desired outcomes of their research (system architectures, workflows, experiences). Besides their differences, they share the common notion of creating an impression of changing a program while it is running. Their differing approaches to this notion could be leveraged for cross-polination between the communities to advance all three of them. For example, *exploratory programming* systems could benefit from the specialized mechanisms and tools investigated under the term *live programming*. At the same time the *live programming* tools could be examined through the lense of experience reports from programmers as it is done in the *live coding* community. All three communities are academic communities. Thus, we treat the publications published under the corresponding terms as artifacts which can be studied to gain insights into the values and practices of each community. Hence, we relate the communities around these terms to each other by surveying the similarities and differences in a sample of the corresponding literature. In particular, our contributions in this paper are: - A systematically collected sample of 191 publications indexed in the ACM DL, IEEE Xplore, and DBLP on the terms *live programming*, *live coding*, and *exploratory programming*^[We will publish the bibliography files on Zenodo with the final version of this paper]. - A thematic analysis [@braun2006using] of the sample regarding the following aspects: - motivations for liveness - applications domains in which the liveness is used - types of contributions - historic distribution of terms - modes of running a program - prominent publications for each term - fields related to the three terms - A comparison of the results of the thematic analysis between the corpus for each of the three terms The remainder of the paper is structured as follows. We begin with the specific research questions guiding the literature study in \autoref{researchQuestions}. We will then illustrate our methodology in \autoref{methodology}. We describe the characteristics of the three corpora in \autoref{characteristics} and the results of the study and the relation to the research questions in \autoref{results}. We discuss correlations in the study results, threats to validity, and potential consequences in \autoref{discussion} and summarize our findings and point out further potential research in \autoref{conclusion}. # Research Questions {#researchQuestions} The goal of this survey is to determine differences and commonalities between the three communities around the terms "live coding", "live programming", and "exploratory programming". In this context, we posed eight research questions. 1. The notion of an impression of changing a program while it is running has been around since the 1950s and the term *liveness* has been used since the 1990s [@McCarthy:1978:HL:960118.808387; @Tanimoto1990VVL]. However, communities around these terms seem to have formed later, thus we want to determine *What is the distribution of publications on the different terms over time?* 2. While live coding seems to be mainly focused on programming for performance art, live programming and exploratory programming do not seem to have such a strong focus on one application domain. Therefore, we want to find out *In the context of which domains has liveness been used?* 3. The motivation for applying the technical concept for liveness can be telling about the values of the communities, thus we want to determine *What are the motivations for having liveness?* 4. In order to determine what kind of knowledge each community contributes we want to investigate *Which kinds of contributions are described?* 5. Besides the general motivation for liveness, the general reason for having a running program varies greatly between different approaches. For example, the Smalltalk community regards their systems as "live" as they are constantly running and any change to a Smalltalk program is actually a change in the object graph of the running system. In order to determine whether the three communities have a particular stance on this aspect we want to determine *What is the general reason for having a running version of the program while changing it?" 6. Each community has works which define their field. Assuming that these are the most cited works in their field we want to find out *What are the most prominent publications?* 7. The three terms used in this study are only one way of categorizing the field or research on programming. Simliar and overlapping perspectives might indicate further related work and future research potential. Thus, we want to determine *Which keywords have authors applied to their work?* Finally, besides these research questions related to the publication we want to determine a list of artifacts providing liveness. These artifacts (such as tools, user interfaces, programming environments) created by the communities can be the foundation for determining common patterns in the design of a live programming experience. # Methodology {#methodology} In order to determine the range of topics and compare the literature related to the three terms we conduct a thematic analysis on three corpora, one for each term. We did not aim to create a comprehensive systematic literature survey of the three terms but to determine the general distribution of themes of each corpus regarding to our research questions. In general, we followed the SALSA (Search, AppraisaL, Synthesis, Analysis) process [@grant2009typology]. First, we created an overall corpus with a systematic search and an appraisal of works. We then synthesized data for the individual research questions by extracting data from the publications and by doing a thematic analysis [@braun2006using]. We analyzed the resulting data by determining correlations, comparing overlaps, and differences between the corpora defined by the three terms. ## Search: Systematically Determined Corpora For each term we created an initial corpus and then created the hull of references. We determined the initial corpora by using the full-text search on three major indexing services : - IEEE Xplore () - ACM Digital Library (ACM DL): Guide to Computing () - Digital Bibliography & Library Project (DBLP) () We retrieved these search results through an automated process using the URLs denoted in brackets (insertion of the keyword denoted as KEYWORD). Depending on the capabilities of the search API the keywords were either entered as exact matches (for example `"live programming"` at IEEE) or AND queries (for example `live+programming` for DBLP). Further, we manually added publications from manually selected scientific venues (For a detailed list of keywords and venues per term, see table 1). Namely, we added the International Conference on Live Coding (ICLC) and the Live Programming Workshop (LIVE). Other major venues, such as CHI, UIST, or PLATEAU are fully indexed by the three services we used and thus appeared automatically through our initial search. We then selected appropriate publications from this initial corpus using our appraisal criteria. | Term | Keywords | Venues | |--------------------------+---------------------------------------+----------------------------------------------------------------------| | live coding | livecoding, live coding | ICLC 2015, ICLC 2016, ICLC 2017^[Only contains the publications which were actually made available from the conference website.] | | live programming | live programming | LIVE 2013, LIVE 2016, LIVE 2017 | | exploratory programming | exploratory programming | - | ## Appraisal The appraisal criteria are concerned with the nature of the publication as well as the content of publications. The complete appraisal process was done manually. For a publication to be accepted it has to be peer-reviewed. This covers publications for conferences, workshops, and journals, as well as doctoral theses. To ensure that the papers contained enough information for us to evaluate the content, we removed all publications with only one page. In general, a publication is added to a term-specific corpus if it contains the corresponding term in the title, keywords, abstract or the title of the publication it is part of, for example the title of the proceedings. Further, whenever the term occurred we manually determined whether it actually referred to the particular experience of programming we want to investigate. This is necessary as the three terms live programming, live coding, and exploratory programming are also used in other domains. Consequently, we only selected papers adhering to the following notion of programming: > "the human activity of describing a process run by a computer" This excludes the following domains from the corpora: - planning and modification of television schedules - encoding video streams during recording and broadcasting - algorithms for artifical intelligence which explore a solution space - planning health programs - analysis of concurrent programs for the liveness property Further, the term "live coding" is used in *teaching research* to refer to a class room constellation in which teachers program "live" in the sense that students can follow the teachers' activities in a programming environment on a projector. We also excluded publications on this topic as they do not include an explict notion of changing a program while it is running. The fact that the mere writing of source code is done "live" in front of an audience (without continuously executing the program) those not entail any particular connection between the code and its dynamic behavior. Also, while exploratory programming environments refers to environments which often include some form of liveness, the single term "exploratory programming" can also refer to a particular workflow [@trenouth1991survey]. This "exploratory workflow" is creating new alternatives and trying them out. This does not entail liveness but can also be done in an edit-compile-run cycle with a long roundtrip time. We excluded papers exclusively referring to exploratory workflows and did not mention exploratory programming environments. [This list comprised 10 papers] ## Synthesis: Thematic Analysis We derived the data for research questions 1, 6, and 7 directly and manually from the individual publications. The specific procedures used are reported in [results]. As the goal of the study is to illustrate the spectrum of research on the topic of liveness, we applied thematic analysis instead of a pre-defined code book to gather data for research questions 2 to 4 [@braun2006using]. That means we determined the codes for the themes application domain, motivation, and types of contribution during coding. Specifically, we used theoretical thematic analysis with the themes defined by our research questions [@braun2006using]. The codes for the theme modes of running were determined beforehand. All publications were coded as one corpus in order to prevent bias of the coder towards codes who they might deem fitting to the corpus a publication belongs to. The codes are described in detail in the related sections in [results]. All coding was done by one coder. During the first pass the set of codes changed, as codes were added or the focus of a code changed. Thus, we did a second pass through the corpus for each theme revising the codes. Afterwards, the coder formalized the codes in the descriptions. We determined the inter-coder reliability for a 20% sample using Cohen's kappa. We report the inter-coder reliablity with each result. ## Analysis The main goal of the study is to illustrate the spectrum of research on liveness. Thus, we directly report the determined codings for the overall corpus. We do not report on the distribution of codes for the overall corpus, as the three terms are disproportionally represented and consequently the overall distribution would be skewed to the distribution of the term with the most publications. Another goal of the study is to investigate the overlap and differences of the terms *live programming*, *live coding*, and *exploratory programming*. Hence, as a further analysis we compare the distribution of the codes between the three term-specific corpora. # Corpus Characteristics {#characteristics} ## Selection From how many papers to how many papers? ## Publication Data ### Overlap ### Venues and Journals | Term | Venues / Journals (sorted alphabetically) | |-----------------------------------------+---------------------------------------------------------------------------| | live programming | CHI, CHI, COP, ComposableWeb, ECOOP, FARM@ICFP, HCC, ICSE, ISMM, LIVE, MOVILESoft, OOPSLA, Onward!, PLDI, PROMOTO, PX, Programming Journal, SIGMOD, SPLASH, UIST, VL/HCC, VRST | | exploratory programming | CHI, COMPCON, ECOOP, HOPL, ICSENG, IMCSIT, ISCA, OOPSLA, Onward!, PX, VL, VL/HCC | | live coding | AM, Blocks and Beyond, C&C, CLEI, DUXU, EVA, EvoMUSART, FARM, ICFP, ICLC, ICSE, LIVE, MM, NIME, OZCHI, SIGCSE, TEI, VL/HCC | ### Distribution over Time ### Keywords In order to determine other perspectives on the field of research on programming related to liveness, we extracted the keywords of each publication. We only recorded keywords mentioned in the actual publication. We also only recorded the keywords chosen by the authors and ignored general tags and categories pre-determined by the publisher (for example the ACM subject classification). We cleaned the data marginally by merging the keywords "programming environment" and "programming environments" as well as "end-user development" and "end-user programming". Finally, we selected all keywords mentioned more then once. | Term (works with terms / corpora size) | Keywords (count) | |-----------------------------------------+---------------------------------------------------------------------------| | live programming (50/72) | live programming (32), live coding (6), end-user programming (5), debugging (4), direct manipulation (4), Smalltalk (4), live programming environment (3), music (3), mashups (3), spreadsheets (3), programming environment (3), Datalog (3), web services (2), prototyping (2), integrated development environment (2), data analysis (2), incremental maintenance (2), streaming data (2), LogiQL (2), testing (2), LogicBlox (2), Javascript (2), liveness (2), mashup tools (2), data mining (2) | | exploratory programming (12/27) | exploratory programming (6), programming environment (4), Self (2), direct manipulation (2), gestures (2), liveness (2), programming by demonstration (2), virtual machine (2), live programming (2), Forms/3 (2) | | live coding (29/101) | live coding (22), music (6), generative art (4), software engineering (2), visual programming (2), liveness (2), art (2), domain specific languages (2), creative coding (2), generative music (2), programming environment (2), functional programming (2), music performance (2), web audio (2), improvisation (2), visualization (2) | # Study Results {#results} For each research question state: - coding scheme - inter coder reliability - results Three results regarding the liveness described in the publications, One result regarding research approach ## Which domains was the liveness used in (specifically, if it was only given as an example it should not be counted)? (not mutually exclusive) ### Coding Scheme To answer the question for which application domains liveness plays a role in each corpus, we coded the intended application domain described in a publication. Specifically, we coded the domain of application for which the *liveness* was relevant. Further, the domain had to be a major theme. We did not code domains which were only mentioned as examples. The codes are not mutually exclusive. \begin{description} \item[Education and training] The program is created to either learn programming or to learn something from the program (for example a model of ecological processes). \item[Simulation] Systems and programs for simulation \item[Data analysis] The analysis of existing data to gain insights \item[Visual performance art] The in-time creation of visual art in front of an audience or for recording. \item[Audio performance art] The in-time creation of audible art in front of an audience or for recording. \item[Performance art] Any other form of performance than pure visuals and audio, for example writing poems, dance, general performance art reflections \item[Web development] The construction of applications running in or on the web. This does not involve the construction of programs through an environment implemented as a web application. \item[User interfaces] The design and implementation of user interfaces. \item[Databases] The programming and manipulation of databases. \item[Programming languages] The design and implementation of programming languages (syntax and semantics). \item[Systems programming] Development of execution environments or fundamental libraries \item[Computer graphics] Creating programs concerned with generating visual output (though not as a performance in contrast to visual performance art) \item[General software development] General software development with no focus on a particular application domain. \item[Artifical intelligence] The design and implementation of artifical intelligence systems. \item[Physical computing] All systems which have some interface with the physical world (actuators or sensors). \item[Virtual/Augmented reality] \item[Enterprise-Resource-Planning (ERP) systems] \item[Computer games] \item[Software verification] \item[Text editing] \end{description} ## Intended Outcome TODO: outcome is not the right word TODO: explain program vs system in more detail: system is about modifying the runtime state which in turn might represent a program TODO: Jens put it like: What is the goal of the user of the meta system? The notion of "an impression of changing a program while it is running" subsumes different modes of "running" a program. ### Coding Scheme We distinguish three modes of "running a program" according to the intended outcome: \begin{description} \item[Computing a result] The programmer wants some output and therefore runs a program. The program is only secondary and might be discarded after the computation, this covers in-time output such as generated sound as well as discrete outputs created by a data analysis algorithm. \item[Creating a program] The programmer wants to create a program. The program is currently running to get feedback on the dynamic behavior of it but the resulting artifact of the activity are some static representation of the program behavior. \item[Evolving a system] The programmer wants to evolve a system which is constantly running. Thus, programmers change behavior descriptions as well as the state of the system. Examples are image-based systems or database systems. \end{description} Notably, all three modes of running a program are possible for most activities and programs (for example a programmer might keep a Python REPL for data analysis open for a long time). Therefore, we assigned the code only to the emphasis in the described work. Although, the categories are mutually exclusive, we allowed assigning multiple codes to a publication when it described multiple aspects of the activity of the programmer. ## Motivation for the term stated While liveness is generally an intriguing idea, the motivations for it vary greatly between authors and communities. However, the initial motivation influences designs and research methodology and is thus an integral element of the definition of a field. ### Coding Scheme The following coding scheme reflects the variety of actual motivations for liveness. When determining the motivation in a publication we only assigned a code when it was a) explicitly stated or b) a fundamental theme of the work. As we extracted the list directly from the publications, the items do not always have the same level of abstraction. \begin{description} \item[Productivity] Liveness improves productivity, most often through making some activity during programming faster \item[Creativity] This applies to all works specifically mentioning creativity as their primary motication. \item[Exploration of alternatives] This is the very specific notation of livness enabling programmers to explore distinct alternative solutions (It does not suffice to say it supports "exploratory programming" as this would often be self-referential; mentioning "exploratory work" falls in this category though) \item[Live tuning] Liveness as a means to explore a continuous parameter space of a design \item[Comprehension] Ease the process of comprehension and learning of program behavior or the program domain \item[In-time output] Livness is an integral part of the activity itself. The output has to be created and modified with a short delay as the changing of the output is the relevant action; this does often correlate with live coding but does not have to. \item[Collaboration] The liveness enables collaboration between users because others can see the effects of owns changes \item[Accessibility] Liveness makes programming more accessible. This involves comprehension but is more specific towards enabling less experience programmers \item[Engagement] Getting users more enganged through liveness \item[Enstrangement] Live programming as an extreme form of programming which enables reflections on the nature of programming \item[Can not be determined] We added this code explictly in order to determine how often liveness is taken as granted. \end{description} Again, these codes are not mutually exclusive. ## What type of contribution is described? As illustrated before, the three communities differ greatly in their motivatios for liveness and desired outcomes. These different world views also reflect in the inquiries and the contributions communicated within the community. ### Coding Scheme We code the contribution of each publication. Some of these codes are hierarchical so a code can have several sub-codes which also count in the super-code set (see for example the empirical codes). \begin{description} \item[Empirical studies] Works which investigated an existing artifact or phenomena. As this covers a wide range of contributions we made codes more specific with the following additional information: a) fixed / flexible setup\footnote{Was the methodology completely determined before the observation or was it adapted during the observation [TODO: cite real world research]} b) quantitative / qualitative outcome c) humans as subject / systems as subjects \item[Survey] Surveys of existing artifacts, for example publications or systems \item[Technical design] The description of a technical implementation \item[User interface design] Describes the design of a particular user interaction. This can be a novel form of interaction, a new input device, or a new way of output provided by the system. \item[Programming interface design] This involves programming language design (syntax and semantics) as well as the design of the interface of libraries. \item[Tool design] The description of the design of a programming tool or environment \item[Individual account of experience] An individual subjective account \item[Formal methods] The description of a formal model or the application of an existing model \item[Algorithm design] The description of an algorithm \item[Practices] The description of practices either empirically observed or from subjective experience \item[Design principles] A collection of rules or values for designs \item[Essay] An application of theoretical models or principles to an existing phenomena [TODO: wat?] \item[Performance] The description of an artistic performance \item[Curriculum design] The description of a new curriculum or teaching model \end{description} ## Prominent Publications In order to determine the which publications define and form each field, we determined the most prominent publications referenced by a corpus. Therefore, we collected all references from publications of one corpus. The cited publication did not have to be in the corpus itself in order to be collected. We list the five most cited references. If there are multiple publications with the same citation count at position five we included all publications with that citation count. ### Exploratory Programming - \(9\) Smalltalk-80: The Language and Its Implementation [@Goldberg:1983:SLI:273] - \(7\) Smalltalk and exploratory programming [@Sandberg1988SEP] - \(5\) Using Prototypical Objects to Implement Shared Behavior in Object-Oriented Systems [@Lieberman:1986:UPO:28697.28718] - \(4\) Self: The Power of Simplicity [@Smith:1994:SPS:974939] - \(4\) The Design and Implementation of the Self Compiler, an Optimizing Compiler for Object-Oriented Programming Languages [@xxx1992TDI] - \(4\) An Efficient Implementation of SELF - a Dynamically-Typed Object-Oriented Language Based on Prototypes [@xxx1989AEI] - \(4\) Back to the Future: The Story of Squeak, a Practical Smalltalk Written in Itself [@ingalls_1997_squeak] ### Live Programming - \(18\) Real-time Programming and the Big Ideas of Computational Literacy [@xxx2003RPB] - \(14\) Smalltalk-80: The Language and Its Implementation [@Goldberg:1983:SLI:273] - \(13\) VIVA: A Visual Language for Image Processing [@Tanimoto1990VVL] - \(12\) Living It Up with a Live Programming Language [@McDirmid2007LUL] - \(11\) Usable Live Programming [@McDirmid:2013:ULP:2509578.2509585] ### Live Coding - \(26\) Live Coding in Laptop Performance [@Nick2003LCL] - \(12\) The programming language as a musical instrument [@blackwell2005programming] - \(12\) Gibber: Live Coding Audio in the Browser [@Roberts2012GLC] - \(11\) Live Coding of Consequence [@xxx2011LCC] - \(11\) Aa-Cell in Practice: An Approach to Musical Live Coding [@Sorensen2007ACP] - \(11\) Rethinking the Computer Music Language: SuperCollider [@mccartney2002rethinking] - \(11\) Visualisation of Live Code [@Alex2010VLC] - \(11\) The IXI Lang: A SuperCollider Parasite for Live Coding [@Magnusson2011TIL] - \(11\) Tidal - Pattern Language for the Live Coding of Music [@xxx2010TPL] # Discussion {#discussion} ## Is there an Exploratory Programming Environments Community? - Exploratory programming: yes! - exp prog envs: Not clear - Smalltalk and Self in keywords suggests its particular language communities ## Threats to Validity - Missing hull: For example, the original Viva paper is missing - Selection based on indexing services - Threats to validity: Indexing services + OCR and thereby maybe wrong positive # Conclusion {#conclusion} \appendix # Detailed Data Sets ## lpsSynthesisDomains | codes | all | live prog. | expl. prog. | live coding | |---+---+---+---+---| | Artifical intelligence | 3 | 0 | 2 | 1 | | Audio performance art | 78 | 4 | 0 | 78 | | Computer graphics | 11 | 7 | 0 | 6 | | Data analysis | 15 | 11 | 3 | 3 | | Database development | 5 | 5 | 0 | 0 | | Education and training | 24 | 10 | 2 | 16 | | Enterprise-Resource-Planning (ERP) systems | 3 | 3 | 0 | 0 | | Game development | 1 | 1 | 0 | 0 | | General software development | 70 | 45 | 17 | 13 | | Performance art | 15 | 0 | 0 | 15 | | Physical computing | 7 | 5 | 0 | 2 | | Programming languages | 5 | 4 | 0 | 1 | | Simulation | 6 | 2 | 4 | 0 | | Software verification | 2 | 2 | 0 | 0 | | Systems programming | 7 | 2 | 3 | 2 | | Text editing | 1 | 0 | 0 | 1 | | User interfaces | 13 | 7 | 4 | 2 | | Virtual/Augmented reality | 4 | 2 | 0 | 2 | | Visual performance art | 21 | 3 | 0 | 20 | | Web development | 6 | 6 | 0 | 0 | | Total | 297 | 119 | 35 | 162 | ## lpsSynthesisIntendedEffect | codes | all | live prog. | expl. prog. | live coding | |---+---+---+---+---| | Computing a result | 101 | 18 | 3 | 86 | | Creating a program | 51 | 35 | 6 | 13 | | Evolving a system | 43 | 23 | 16 | 5 | | Not applicable | 6 | 0 | 2 | 4 | | Total | 201 | 76 | 27 | 108 | ## lpsSynthesisMethodOfInquiry | codes | all | live prog. | expl. prog. | live coding | |---+---+---+---+---| | Algorithm design | 6 | 2 | 0 | 4 | | Can not be determined | 3 | 0 | 2 | 1 | | Curriculum design | 4 | 1 | 1 | 3 | | Design principles | 11 | 5 | 3 | 4 | | Empirical study (system) | 27 | 13 | 8 | 6 | | Empirical study (users) | 36 | 16 | 10 | 17 | | Essay | 20 | 2 | 0 | 18 | | Formal methods | 9 | 8 | 0 | 1 | | Individual account of experience | 14 | 0 | 2 | 12 | | Performance | 19 | 2 | 0 | 19 | | Practices | 11 | 0 | 0 | 11 | | Programming interface design | 37 | 15 | 7 | 18 | | Survey | 3 | 1 | 0 | 3 | | Technical design | 75 | 28 | 14 | 36 | | Tool design | 83 | 50 | 11 | 26 | | User interface design | 34 | 15 | 3 | 18 | | Total | 392 | 158 | 61 | 197 | ## lpsSynthesisMotivation | codes | all | live prog. | expl. prog. | live coding | |---+---+---+---+---| | Accessibility | 23 | 19 | 2 | 3 | | Can not be determined | 33 | 14 | 8 | 11 | | Collaboration | 17 | 1 | 1 | 15 | | Comprehension | 25 | 22 | 1 | 5 | | Creativity | 3 | 0 | 1 | 2 | | Engagement | 4 | 1 | 0 | 3 | | Enstrangement | 2 | 0 | 0 | 2 | | Exploration | 32 | 15 | 11 | 7 | | In-time output | 81 | 8 | 0 | 78 | | Live tuning | 5 | 2 | 0 | 3 | | Productivity | 17 | 8 | 6 | 6 | | Total | 242 | 90 | 30 | 135 |