# Implementation 
<!--  
  #Meta notes
  - Scripts in <script>foo()</script> are executed sequentially in the same context (the world or container)
    - the source is not shown, but it's result (string or HTMLElement) will be displayed by default at the position of the script
  - code in markdown blocks ```javasscript {.SomeClass}... foo()``` are syntax highlighted and displayed
    - they are not excuted by default
  - script with `runExampleButton`
    - e.g. runExampleButton("Run", this, ["TimeoutExample", "JoeBase"])
    - will show a button named "run"
    - will executed everything with the dependcies "TimeoutExample", "JoeBase" (and "Run") #Refactor
-->

<style>
  pre:not(#LOG) {
    background-color:  rgba(240,240,250,1);
    padding: 6px;
    width: 800px;
  }
  
  .indexElement {
    position: relative;
    top: -10px;
    left: -20px;
    height: 0px;
    
  
  }
</style>

<script>
import {hideHiddenElements, toggleLayer, showVariable, runExampleButton} from "src/client/essay.js"
""
</script>



## Scoping Async Callbacks: setTimeout

JavaScript basic async API consists of functions `setTimeout`, `setIntervall`, and `requestAnimationFrame` that takes callbacks as arguments that will be executed later in time.

- #Idea: should we make all callbacks keep their layer activations
  - Pro: this would be very general and setTimeout would work out of the box
  - Cons: 
    - the semantics of layer scoping seem to be very domain specific, so it is not clear if this is desired behavior
    - ContextJS and other Meta-libraries uses functions / callbacks as an implementation technique. This is not a "user" level. 



###  Base Behavior

```javascript {.LogMorph .Hidden}
// Some log tool
var logComonent  = lively.query(this, "#LOG")
logComonent.mode  ="text/plain"
logComonent.parentElement.setAttribute("title", "Log")
logComonent.textContent = ""

function log(s) {
   logComonent.textContent += s +"\n" 
}
```

Change the color of a rectangle to red.

```javascript {.JoeBase .NoResult}
var joe = lively.query(this, "#Joe")
joe.foo  = function() {
    log("Base>>foo")
    this.style.backgroundColor = "red"
}
```

<div id="examplespace" style="position:sticky; top: 10px;
width: 340px; left: 850px; background:rgba(240,240,240,0.8); padding: 10px; border: 1px dashed gray">
<i>example</i><br>
  <div style="display: inline-block; width: 100px; height: 100px; border: 1px solid gray; vertical-align: top; padding: 10px" id="Joe">Joe
  </div>
  <pre id="LOG" style="display: inline-block; overflow:auto; width: 200px; height: 300px; backgroundColor: gray;border: 1px solid gray; margin: 0px; vertical-align: top; padding: 5px"></pre>
</div>

```javascript {.JoeBaseCall}
joe.foo();
```
<script>runExampleButton("Run", this, ["LogMorph", "JoeBase", "JoeBaseCall"])</script>

### Behavioral Adaptation and Dynamic Scoping

First we have to include ContextJS, to be able to define behavioral adaptations and scope them. 

```javascript {.includes .CountSetTimeoutLayer .PromiseExample .TimeoutExample}
import * as cop  from 'src/client/ContextJS/src/contextjs.js'

```

The layer `L1` changes refines the behavior of the rectangle `joe` method's `foo` 
so that the the color will be set to green green. 

```javascript {.JoeL1Def}
cop.layer(self, "L1").refineObject(joe, {
  foo() {
    log("L1>>foo")
    this.style.backgroundColor = "green"
  }
});
```

When calling foo within the scope of the dynamic layer activation the rectangle will be green. 

```javascript {.JoeL1Dynamic}
// (2) sync layered call
cop.withLayers([L1], () => {
    joe.foo()
})
```

<script>runExampleButton("Run", this, ["LogMorph", "TimeoutExample", "JoeBase", "JoeL1Def", "JoeL1Dynamic"])</script>


### Example: Keep active Layers through setTimeout

But not everything is executed immediately and therefore within the dynamic scope. The `setTimeout` method for example does not execute the function handed to it in its first argument, but rembers it and schedules it to be executed later in time, 100ms as specified by the second argument. Since `L1` will not be active at that time, the 
foo method will just execute its base behavior and the rectangle will be red again. 

```javascript {.JoeAsync .Hidden}
if(self.DelayedLayerActivationLayer) {
  DelayedLayerActivationLayer.beNotGlobal()
}
```


```javascript {.JoeAsync .JoeAsyncWithLayers }
joe.foo();
// async layered call
cop.withLayers([L1], () => {
  setTimeout(() => {
    joe.foo()
  }, 100);
})
```

<script>runExampleButton("run", this, ["LogMorph", "TimeoutExample", "JoeBase", "JoeAsync"])</script>


## Layering setTimeout

The `cop.withLayers(layers, func)` statement by default only affects the synchronous execution of `func`.

We can use COP, to change the behavior of `setTimeout`, e.g. to count and tally it, see [Count Timeout Layer](count-timeout).

## Delayed Layer Activation

But what we really want is to adept `setTimeout` (and potentially other JavaScript function) so that they preserve layer activations and allow to scope them asynchronously. 

We have to import `LayerStack` to get access to internal ContextJS layer composition state. 

```javascript {.DelayedLayerActivationLayer, .LayerStack .DelayedLayerActivationLayer .ReplayLayerActivationsLayer .PromiseExample}
import {LayerStack} from 'src/client/ContextJS/src/Layers.js'
```
And we can preserve and replay such a stack by replaying layer (de-)activations with `replayStack`:

```javascript {.ReplayStack .PromiseExample .DelayedLayerActivationLayer }
function replayStack(layerStack, index, callback) {
  if (index >= layerStack.length) { 
    return callback()
  }
  var comp = layerStack[index]
  var cont = function() {
    return replayStack(layerStack, index + 1, callback)
  }
  if (comp.withLayers)
    return cop.withLayers(comp.withLayers, cont)
  if (comp.withoutLayers)
    return cop.withoutLayers(comp.withoutLayers, cont)
  throw new Error("unknown layer composition")
};
```

In our current implementation, #ContextJS cannot refine plain functions, such as setTimeout, but only methods. #Bug 

We can work around the for by replacing `setTimeout` with a function that call the original `setTimeout` as a method. Since this is overly complicated we plan to implement support for layering (global?) functions.

```javascript {.setTimeoutMethod .TimeoutExample .CountSetTimeoutLayer .NoResult}
if (!window.setTimeoutMethod) {
  window.setTimeoutMethod = window.setTimeout
} 
window.setTimeout = function(callback, delay) {
  return window.setTimeoutMethod(callback, delay)
}
```

```javascript {.DelayedLayerActivationLayer}
cop.layer(self, "DelayedLayerActivationLayer").refineObject(self, {
  setTimeoutMethod(callback, delay) {
    var layerStack = Array.from(LayerStack)
    return cop.proceed(() => {
      return replayStack(layerStack, 1, callback);
    }, delay);
  }
});
DelayedLayerActivationLayer.beGlobal();
```

So, we can run the example again, and hopefully, the layer is active now and Joe, the rectangle, should get green. 

<script>runExampleButton("run", this, ["LogMorph", "TimeoutExample", "JoeBase", "LayerStack", "DelayedLayerActivationLayer", "JoeAsyncWithLayers"])</script>


## Extending the Dynamic Scope To Promises

The `setTimeout`  function is not the only source of async behavior. Instead of extending the replay of layer activations to possible other places of async behavior such as `fetch` browser API, we can make Promises layer aware.

So lets, see how our adaptation example behaves with Promises. Similar to `setTimeout`, JavaScript Promises are executed in an asynchronous manner. The rectangle will first be set to red (1) and even though the callback will be  defined in the scope of `L1` (2), the layer will not be active by the time it gets executed. 

```javascript {.JoePromiseAsync}
joe.foo(); // (1)

// (2) async layered call, where layer activation keep active beyon async fetch call
cop.withLayers([L1], () => {
  fetch(lively4url + "/").then(() => { 
     joe.foo()
  })
})
```


<script>
runExampleButton("run", this, ["LogMorph", "JoeBase", "JoeL1Def","JoePromiseAsync"])
</script>


We cannot refine constructors JavaScript classes directly, so we have first have to replace the class by a subclass with a hook. #FutureWork This works only for classes that have not already been subclassed or used otherwise in other places, since we are changing the identity of the class we are adapting. For a general solution, one has to adapt the class early in the code loading process.

```javascript {.PromiseConstructor .PromiseExample .NoResult}
if (!self.OriginalPromise) {
  self.OriginalPromise = Promise;
}
self.Promise = function Promise(...args) {
  if (this.constructorHook) {
    var p = (...rest) => {
      return new OriginalPromise(...rest)
    }
    return this.constructorHook(p, args)
  }
  return new OriginalPromise(...args)
};
self.Promise.prototype = OriginalPromise.prototype;
self.Promise.__proto__ = OriginalPromise // meta-class inheritance... Promise.all should work
self.Promise.prototype.constructorHook = function(p, args) {
 return p(...args)
}
```

```javascript {.ReplayLayerActivationsLayer .PromiseExample}

cop.layer(window, "ReplayLayerActivationsLayer").refineClass(Promise, {
  constructorHook(proceed, args) {
    var result = cop.proceed(proceed, args)
    // log && log("construct promise")
    return result
  },


  then(onFulfilled, onRejected) {
    var layerStack = Array.from(LayerStack)
    return cop.proceed(
      onFulfilled ? (...args) => {
        return replayStack(layerStack, 1, () => onFulfilled(...args));
      } : undefined,
      onRejected ? (...args) => {
        return replayStack(layerStack, 1, () => onRejected(...args));
      } : undefined,);
  }
})
ReplayLayerActivationsLayer.beGlobal()
```


<script>
runExampleButton("run", this, ["LogMorph","JoeBase", "JoeL1Def", "LayerStack", "ReplayStack", "PromiseConstructor", "ReplayLayerActivationsLayer" , "JoePromiseAsync"])
</script>


We can put this in a library. 

```javascript {.ContextJSAsync}
import "src/client/ContextJS/src/async.js"
```


<script>
runExampleButton("run", this, ["LogMorph","JoeBase", "JoeL1Def", "ContextJSAsync" , "JoePromiseAsync"])
</script>


## Constructing a Call graph of Promises

```javascript {.PromiseGraph .PromiseExample}

var promiseGraph = new Map();

function promiseToNode(promise, parentNode) {
  var node = {
    name: "[pending]",
    value: promise, 
    parent: parentNode,
    children: []
  };
  // ingore tooling usage of promise while we are tracing them... 
  cop.withoutLayers([TracePromiseLayer], () => {
    promise.then(
      result => node.name = "" + result,
      reject => node.name = "[rejected]" 
    )
  })
  return node
}

var lastPromise

cop.layer(window, "TracePromiseLayer").refineClass(Promise, {
  constructorHook(proceed, args) {
    var result = cop.proceed(proceed, args)
    if(lastPromise) {
      console.log('lastPromise ' + lastPromise)
      var node = promiseToNode(result, lastPromise)
      node.independent = true
      var lastNode = promiseGraph.get(lastPromise)
      if (lastNode) {
        lastNode.children.push(node)   
      } 
      promiseGraph.set(result, node)
    }
    return result
  },

  then(onFulfilled, onRejected) {
    var node = promiseGraph.get(this)
    if (!node) {
      node = promiseToNode(this)
      promiseGraph.set(this, node)
    }
    var nextPromise = cop.proceed((...args) => {
      lastPromise = nextPromise // #Querstion do we need a stack?
      var result = onFulfilled(...args)
      lastPromise = undefined
      return result
    }, onRejected) // #BUG returns promised undefined
    var nextNode = promiseToNode(nextPromise, node)
    nextNode.independent = false
    promiseGraph.set(nextPromise, nextNode)
    node.children.push(nextNode)   
    return nextPromise
  }
})
```

### Examples and Visualization of Promise Graph

```javascript {.PromiseGraphViz .PromiseExample}
async function visualizePromiseGraph(promiseGraph) {
  var treeData = {
    name: "", 
    children: 
      Array.from(promiseGraph).map(ea => ea[1]).filter(ea => !ea.parent)
  }
  if (treeData.children.length == 1) {
    treeData = treeData.children[0]
  }
  //<div style="position: relative;  height: 400px">
  return lively.create("lively-d3-tree").then(tree => {
      lively.setPosition(tree, pt(0,0))
      tree.style.width = "800px"
      tree.style.height = "200px"
      tree.setTreeData(treeData)
      return tree
    })
    //}</div>
}

async function visualizePromises(cb, timeout) {
  promiseGraph = new Map(); // #TODO store graph in layer instance
  var promise 
  cop.withLayers([TracePromiseLayer], () => {
    var promise = cb()
  })
  await lively.sleep(timeout)
  return visualizePromiseGraph(promiseGraph)
}
```

```javascript {.PromiseGraphExample1 .PromiseExample}
visualizePromises(() => {
  var root = Promise.resolve("root")
  root.then(() => {
    return "hello"
  }).then(() => {
    return "world"
  })
  return root.then(() => {
  
    new Promise(resolve => resolve("bar")).then(() => "baz");
    
    return "foo"
  })
}, 100)
```

<script>runExampleButton("PromiseExample", this)</script>

```javascript {.PromiseGraphExample2}
visualizePromises(() => {
  return Promise.resolve(3).then(r => {
    console.log("result 1: " + r)
    return r
  }).then(r => {
    console.log("result 2: " + r)
    return r
  })
}, 100)

```

<script>runExampleButton("PromiseGraphExample2", this)</script>


```javascript {.PromiseGraphExample3 }
visualizePromises(() => {
  cop.withLayers([TracePromiseLayer], () => {
    System.import("src/client/html.js")
  })
},100)
```

<script>runExampleButton("PromiseGraphExample3", this)</script>



```javascript {.PromiseGraphExample4}
visualizePromises(() => {
  var root = Promise.resolve("root")
  root.then(() => {
    return "hello"
  }).then(() => {
    return "world"
  })
  return root.then(async () => {
  
    await new Promise(resolve => resolve("bar")).then(() => "baz");
    
    return "foo"
  })
}, 100)
```

<script>runExampleButton("run", this, ["PromiseGraphExample4"])</script>


## Hidden Promises in Async / Await


When the promise is still visible, e.g. `then` is called. Our layers are still active.

```javascript {.JoePromiseAsyncAwait}
cop.withLayers([L1], async () => {
  await fetch(lively4url + "/").then(() => { 
      joe.foo()
  })
})
```

<script>
runExampleButton("run", this, ["LogMorph", "JoeBase", "JoeL1Def","JoePromiseAsyncAwait"])
</script>

When the promise are hidden inside the language through async/await... we have a problem. 

```javascript {.JoePromiseAsyncAwaitAfter}
cop.withLayers([L1], async () => {
  await fetch(lively4url + "/")
  debugger
  joe.foo() // #Bug, L1 is here not activated...
})
```

<script>
runExampleButton("run", this, ["LogMorph", "JoeBase", "JoeL1Def","JoePromiseAsyncAwaitAfter"])
</script>


### What can we do?

- find a globally accessible function / method we can adapt to control layer activation.
- create a babel plugin that takes care of layer (de-)activation after `await`.
- we can fix this specific example, by waiting in "cop.withLayers" on the promise, before we (de-)activate it. But this could create possible nasty side effects since other code gets executed and the layer should not be active then. 

### Test A

```javascript {.LogExpect .Hidden}
var logExpect = function(name, value, expect) {
  if (value != expect) {
    log(name + " " + value + " (but expected " + expect +")")
  } else {
    log(name + " " + value)
  }
}
```

```javascript {.JoePromiseAsyncTestA}
var o = {m() { return 3}}
var l = cop.layer("AsyncLayer").refineObject(o, {
  m() { return cop.proceed() + 4}
})

logExpect("(a) ", o.m(), 3)
cop.withLayers([l], async () => {
  logExpect("(b) " , o.m(), 7)
  await Promise.resolve().then(() => {
    logExpect("(c) ", o.m(), 7)
  })
  logExpect("(d) ", o.m(), 7) 
})
```

<script>
runExampleButton("run", this, ["LogMorph", "LogExpect", "ReplayLayerActivationsLayer", "JoePromiseAsyncTestA"])
</script>

We can solve this test by waiting in `withLayers`


### Test B (overlapping execution)

```javascript {.JoePromiseAsyncTestB}
var o = {m() { return 3}}
var l = cop.layer("AsyncLayer").refineObject(o, {
  m() { return cop.proceed() + 4}
});

(async () => {
  logExpect("(a)", o.m(),  3)
  cop.withLayers([l], async () => {
    logExpect("(b)", o.m(), 7)
    await lively.sleep(10)
    logExpect("(d)" , o.m(), 7) 
  })
  await lively.sleep(5)
  logExpect("(c)", o.m(), 3) 
})()
```

<script>
runExampleButton("run", this, ["LogMorph", "LogExpect", "JoePromiseAsyncTestB"])
</script>




### Test C (overlapping execution in anonymous function)

```javascript {.JoePromiseAsyncTestC}
var o = {m() { return 3}}
var l = cop.layer("AsyncLayer").refineObject(o, {
  m() { return cop.proceed() + 4}
});

(async () => {
  logExpect("(a)", o.m(),  3)
  cop.withLayers([l], () => {
    (async () => {
      logExpect("(b)", o.m(), 7)
      await lively.sleep(10)
      logExpect("(d)" , o.m(), 7) 
    })()
  })
  await lively.sleep(5)
  logExpect("(c)", o.m(), 3) 
})()
```

<script>
runExampleButton("run", this, ["LogMorph", "LogExpect", "JoePromiseAsyncTestC"])
</script>

### Questions

- give up support for ContextJS in aysnc / await JavaScript?
- if not:
  - we can solve A, but how can we deal with B and C ?
  - will `await` create a promise and we can intercept that?


- title: What does dynamic scoping mean with coroutines. 
  - `yield` has the same problems...


### Debugging into  it


```javascript {.LogMorph2 .Hidden}
var example  = lively.query(this, "#examplespace")
example.style.left = "300px"
var logComonent  = lively.query(this, "#LOG")
logComonent.textContent = ""
logComonent.style.width = "800px"
example.style.width = "800px"
example.querySelector("#Joe").style.display = "none"
```

```javascript {.JoePromiseAsyncTrace}
import * as Layers from 'src/client/ContextJS/src/Layers.js'
log = log || function(...args) { console.log(...args)};

var standardLayers = (Layers && Layers.currentLayers()) || []
var currentLayers = function() {
  if (!Layers) return ""
  return Layers.currentLayers().filter(ea => !standardLayers.includes(ea)).map(ea => ea.name).join(" ")
};

window.currentLayers = currentLayers

var promiseIdCounter = 1

cop.layer(window, "DebugPromiseLayer").refineClass(Promise, {
  constructorHook(proceed, args) {
    var id = promiseIdCounter ++
    log("construct " + id)
    var result = cop.proceed(proceed, args)
    result._debugId = id
    log("construct after " + id)
    
    cop.withoutLayers([DebugPromiseLayer], () => {
      result.then( r => {
        try {
          log("result " + r + " " + currentLayers())
        } catch(e) {
          // egal
        }
      })
    })
    return result
  },

  then(...args) {    
    // we were missing the constructor?
    if (!this._debugIgnore) {
      this._debugId = this._debugId || promiseIdCounter++
      log("when " +  (this._debugParentId ?  this._debugParentId + ","  : "") + (this._debugId +" then..."))
    }
    if (this._debugId == 1) {
      debugger
    }    
    var result = cop.proceed((...innerArgs) => {
      if (!this._debugIgnore) {
        log("BEGIN resolved " + this._debugId + " then...")
      }
      try {
        var innerResult =  args[0](...innerArgs)
      } finally {
        if (!this._debugIgnore) {
          if (result) {
            result._debugId = result._debugId || promiseIdCounter++
          }
          log("END resolved " + this._debugId + " then " + (result ? result._debugId  : "no promise"))
          if (result && result._debugId == 2) {
            window.LastPromise = result
            debugger
          }
        }
      }
      return innerResult
    }, args[1])
    result._debugParentId = this._debugId
    // console.log("then", this)
    return result
  },

})
```

### Wrap Await Layer....

Since we lack both: the dynamical scope of the stack and and explicit object connection via promises, we have to adapt the code in our `async/await` scenario.  


```javascript {.JoePromiseWrapAwaitLayerDef}
cop.layer(window, "WrapAwaitLayer").refineClass(Promise, {
  then(...args) {
    if (this.isMySpecialAwait) {
      // activate layers
      this.isMySpecialAwait.forEach(ea => ea.beGlobal())
      log("activate layers " + this.isMySpecialAwait)
    }  
    var result = cop.proceed(...args)
    if (this.isMySpecialAwait) {
      result.then(() => {
        // deactivate this.isMySpecialAwait
        this.isMySpecialAwait.forEach(ea => ea.beNotGlobal())
        log("deactivate layers " + this.isMySpecialAwait)
      })
    }
    return result
  }  
});
WrapAwaitLayer.beGlobal();

var wrapAwait =  function(p) {
  // var a  = currentLayers()
  var r = p.then(x => x)
  // r.isMySpecialAwait = Array.from(LayerStack)
  r.isMySpecialAwait = Layers.currentLayers().filter(ea => ea.isAsyncLayer)
  log("remember " + r.isMySpecialAwait)
  return r
} 
```

This prototype works, but to make it more general we need something different, so that we do not interfere with global layer activation list. 

```javascript {.JoePromiseWrapAwaitLayerRun}
var o = {m() { return 3}}
var l = cop.layer(window, "AsyncLayer").refineObject(o, {
  m() { return cop.proceed() + 4}
});
ReplayLayerActivationsLayer && ReplayLayerActivationsLayer.beNotGlobal()

var p = Promise.resolve();
AsyncLayer.isAsyncLayer = true


logExpect("(a)", o.m(),  3)
var ignorePromise = cop.withLayers([l], async () => {
  
  logExpect("(b)" , o.m(), 7) 
  await wrapAwait(p)

  logExpect("(c)" , o.m(), 7) 
  await wrapAwait(Promise.resolve())
  
  logExpect("(d)" , o.m(), 7) 
  await wrapAwait(Promise.resolve())

  logExpect("(e)" , o.m(), 7) 
  
});
logExpect("(f)", o.m(),  3)

setTimeout(() => {
  WrapAwaitLayer.beNotGlobal()
}, 5000)

```

## Open Question:

- What is the dominant scope?
  - Lexical Scope?
  - dynamic scope?
  - Promise scope?
    - What is Promise scope? 
      - #Works through layering `then`, because it is a clear hierarchy
      - pass a reference of a promise... and the promise replays it's active layers?
      - But: can conflict with new lexical scope!

- #Problem: ContextJS assumes a single stack execution semantic and therefore maintains also just one LayerStack. The problem is that async/await  and Promises introduce new execution stacks. We therefore should maintain separate execution stacks. 

<script>
var button = runExampleButton("run", this, ["includes", "LayerStack", "ReplayStack", "PromiseConstructor", "LogMorph", "LogMorph2", "LogExpect", "JoePromiseAsyncTrace", "JoePromiseWrapAwaitLayerDef", "JoePromiseWrapAwaitLayerRun"])
//  "ReplayLayerActivationsLayer"

button
</script>


## Idea


```javascript

cop.withLayers([l1] , async() => { // (a) 
    await p  // (b) 
    o.m() // (c)
  }) (d) // return p_a
```

- (a) we have a list of active layers `[l1]` and a async function, that returns a promise `p_a`
- (b) waits on `p`
- (c) is executed sometime after the async function returned the promise `p_a`




<script>hideHiddenElements(this)</script>