## 2018-03-23 #IndexDB #Benchmarks #Performance #SWX Found performance bug in IndexDB. Reading a lot of data seems to be very slow? ```javascript import {Dictionary} from "src/external/lively4-serviceworker/src/dictionary.js" var dict = new Dictionary("response-cache") (async () => { var all = [] var start = performance.now() for(var i=0; i < 20; i++) { all.push(dict.match("OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js")) } return Promise.all(all).then(p => performance.now() - start) })() // 8149.899999960326 ``` ``` dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 21.300000022165477ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 25.000000023283064ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 28.599999961443245ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 32.40000002551824ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 36.00000008009374ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 39.299999945797026ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 43.49999991245568ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 46.6999999480322ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 50.00000004656613ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 53.59999998472631ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 107.4999999254942ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 800.3999999491498ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 883.4999999962747ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 886.5999999688938ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 889.9999998975545ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 893.7999999616295ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 897.4999999627471ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 905.2999999839813ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 909.8999999696389ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 912.8999999957159ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 915.4000000562519ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 972.7000000420958ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1742.3999999882653ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1825.7000000448897ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1828.3999999985099ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1831.7000000970438ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1835.0000000791624ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1838.2999999448657ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1842.1000000089407ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1846.4999999850988ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1850.399999995716ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1858.8000000454485ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 1916.1000000312924ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2618.099999963306ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2707.799999974668ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2711.399999912828ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2714.8000000743195ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2718.2000000029802ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2721.6000000480562ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2725.399999995716ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2728.899999987334ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2733.0000000074506ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 2737.3000000370666ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3577.199999941513ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3579.799999948591ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3582.9000000376254ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3586.4999999757856ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3590.3999999864027ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3594.7000000160187ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3602.6000001002103ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3607.1000000229105ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3614.9000000441447ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3618.100000079721ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 3684.1000000713393ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4411.399999982677ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4499.500000034459ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4502.499999944121ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4505.899999989197ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4510.200000018813ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4513.600000063889ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4517.400000011548ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4520.800000056624ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4524.3000000482425ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4528.099999995902ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 4613.399999914691ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5335.1000000257045ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5420.799999963492ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5423.999999999069ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5427.299999981187ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5430.399999953806ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5434.000000008382ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5437.699999893084ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5442.89999990724ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5449.499999987893ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5453.300000051968ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 5515.200000023469ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6223.800000036135ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6306.699999957345ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6309.400000027381ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6312.5ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6315.7000000355765ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6319.400000036694ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6323.399999993853ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6330.19999996759ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6335.800000000745ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6345.900000073016ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 6398.699999903329ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7106.599999940954ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7192.299999995157ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7194.900000002235ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7197.899999911897ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7201.399999903515ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7205.19999996759ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7210.000000079162ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7219.800000078976ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7234.30000001099ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7240.099999937229ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7297.200000029989ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 7996.099999989383ms dictionary.js:44 dictionary lookup OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js took 8078.900000080466ms ``` Doing it in sync does not change the result ```javascript (async () => { var start = performance.now() for(var i=0; i < 100; i++) { await dict.match("OPTIONS https://lively-kernel.org/lively4/lively4-jens/src/external/lively4-serviceworker/src/dictionary.js") } return performance.now() - start })() ``` Can we plot this graph? @Stefan? It takes 8000 ms to read it 100 times... but it should take 300ms or even less? ## Dexi I thought, maybe we used IndexBD wrong... but this seems to be not the case. We could reproduce the bad performance with our other FileCache API that uses Dexie. ```javascript (async () => { var all = [] var start = performance.now() for(var i=0; i < 100; i++) { all.push(FileCache.current().db.files.where("name").equals("2016-03-08.md").first()) } return Promise.all(all).then(p => performance.now() - start) })() // RESOLVED: 7822.599999955855 ``` It also takes 8seconds for 1000 queries. Shit! ```javascript import Dexie from "https://unpkg.com/dexie@2.0.0-beta.11/dist/dexie.js" var db = new Dexie("file_cache"); ``` ## LocalStorage Performance ```javascript (async () => { var all = [] var start = performance.now() for(var i=0; i < 100; i++) { localStorage["test"] } return Promise.all(all).then(p => performance.now() - start) })() // RESOLVED: 2.4999999441206455 ``` Oh my God! ## Its a Chrome Bug or a bug of [my instance..](https://lively-kernel.org/lively4/lively4-jens/demos/benchmarks/dexie.html) - running it in an isolated page in my instance under - my default chrome: 1979.300000006333ms - microsoft edge browser: 146.50003049761597ms - firefox: fail (async syntax?) - new incognito window chrome: 20.299999974668026ms So the performance of IndexDB must be affected by other databases in the same domain / web page! Damn! At least I found it!