MantisBT - NetSurf
View Issue Details
0002366NetSurfJavascriptpublic2015-10-18 17:492016-02-16 14:13
ReporterHarriet Bazley 
Assigned ToMichael Drake 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformARMOSRISC OSOS Version5.19
Product Version3.4 
Target VersionFixed in Version3.4 
Fixed in CI build #3032
Reported in CI build #3000
URL of problem pagehttps://1940sexperiment.wordpress.com/
Summary0002366: JS lockup
DescriptionPage https://1940sexperiment.wordpress.com/ goes into single-tasking loop with JavaScript enabled. (Stuck on the Fetching, Fetching, Processing stage with desktop unresponsive.)
Additional InformationIyonix Aria, RISC OS 5.20
TagsNo tags attached.
has duplicate 0002364closed Michael Drake JavaScript-related lockup 
has duplicate 0002348closed  Looping 
Attached Filespng duktape-unicode-regex.png (472,919) 2015-10-21 22:07
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=336&type=bug
png duktape-fast-regex.png (671,558) 2015-10-24 14:49
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=338&type=bug

Notes
(0000974)
Vincent Sanders   
2015-10-21 21:07   
(1.340669) content/content.c:82 content__init: url https://s1.wp.com/_static/??-eJyVzTEOwjAMheELYaxCQCyIs6SNCS6uGzkuFbenCwNLUaenf/j0cC7QjeqkjsMERabMWtFndieDVmL3bNkSFCZUepGRJta87+sOt9jy37FQghx -> 0x2c0c7d0
(1.340682) content/content.c:664 content_add_user: content https://s1.wp.com/_static/??-eJyVzTEOwjAMheELYaxCQCyIs6SNCS6uGzkuFbenCwNLUaenf/j0cC7QjeqkjsMERabMWtFndieDVmL3bNkSFCZUepGRJta87+sOt9jy37FQghx (0x2c0c7d0), user 0x4db240 0x2b7f4e0
(1.340703) content/content.c:282 content_convert: content https://s1.wp.com/_static/??-eJyVzTEOwjAMheELYaxCQCyIs6SNCS6uGzkuFbenCwNLUaenf/j0cC7QjeqkjsMERabMWtFndieDVmL3bNkSFCZUepGRJta87+sOt9jy37FQghx (0x2c0c7d0)
(1.340714) render/html_script.c:273 convert_script_sync_cb: script 27 done 'https://s1.wp.com/_static/??-eJyVzTEOwjAMheELYaxCQCyIs6SNCS6uGzkuFbenCwNLUaenf/j0cC7QjeqkjsMERabMWtFndieDVmL3bNkSFCZUepGRJta87+sOt9jy37FQghxFyN6/taLKWB3uEtmwPqItF99d0G24NiGE5nQ+HC/9B5XxV+w='
(1.340720) render/html_script.c:275 convert_script_sync_cb: 0 fetches active
(7.355472) javascript/duktape/dukky.c:376 js_exec: Uncaught error in JS: ReferenceError: identifier 'jQuery' undefined
(7.355498) javascript/duktape/dukky.c:378 js_exec: was at: javascript/duktape/dukky.c line 1
(7.355504) javascript/duktape/dukky.c:379 js_exec: Stack trace: ReferenceError: identifier 'jQuery' undefined
    duk_js_var.c:1241
    eval javascript/duktape/dukky.c:1 preventsyield
(7.355551) render/html_script.c:532 html_process_script: content 0x2c06690 parser 0x2c06c30 node 0x2b50a90
(7.355668) javascript/duktape/dukky.c:383 js_exec: Returning false
(7.355692) render/html_script.c:532 html_process_script: content 0x2c06690 parser 0x2c06c30 node 0x2b83860
(0000975)
Vincent Sanders   
2015-10-21 21:08   
It does not appear to hang, just takes some time to run the javascript
(0000976)
Michael Drake   
2015-10-21 22:14   
(Last edited: 2015-10-21 22:26)
Uploaded a kcachegrind profile; duktape-unicode-regex.png

As I understand it duktape's implementation of Unicode case insensitive regular expressions is written for low memory usage, rather than for speed.

As such its taking almost 90% of the time to start NetSurf with that page just to do the regular expressions. In the remaining 10% of the time NetSurf starts, decodes all the html and css, generates the dom, does all the non-regex javascript, selects styles for all the elements, lays out the page, renders it, decodes the images and renders them, and then quits.

So there's a lot of room for improvement. We've already reported the issue upstream to the duktape maintainer.

(0000979)
Michael Drake   
2015-10-24 14:47   
(Last edited: 2015-10-24 14:48)
From the duktape maintainer:

< svaarala_> tlsa: thanks - i'll try to figure out a minimal fix for this issue as soon as i get some free time
< svaarala_> tlsa: a more comprehensive solution would be to provide a speed optimized unicode implementation in general, but this particular case is much worse than other places
< svaarala_> tlsa: actually, i wrote a quick branch which uses a 64k entry lookup for the regexp canonicalization
< svaarala_> tlsa: the cost is 128kB of footprint but it makes a 30 second pathological test run in 30ms
< svaarala_> tlsa: i don't plan on this being the long term solution but this might fix your immediate issue if that 128k is not too much?
< svaarala_> tlsa: anyway, if you're interested, i'll push the test branch and it'd then be interesting to know if that fixes the netsurf issue
< svaarala_> tlsa: at the very least it'd tell us that this is the only real problem in those cases
< svaarala_> tlsa: here's a snapshot with a prototype fix included: http://duktape.org/snapshots/duktape-1.3.99-20151022195439-v1.3.0-138-g4045393-regexp-canonicalize-lookup.tar.xz
< svaarala_> tlsa: and here's the pull: https://github.com/svaarala/duktape/pull/411

(0000980)
Michael Drake   
2015-10-24 14:55   
Using the prototype duktape fix, the performance is much better. Most of the time is still spent processing javascript, but its not dominated by regex work.

I've attached another profile image which shows the improvement: duktape-fast-regex.png

From the 2 images we can see the total instruction fetch cost is reduced to 12% of what it was (from 60 billion to 7 billion).
(0001005)
Michael Drake   
2015-10-31 18:25   
Fixed by http://git.netsurf-browser.org/netsurf.git/commit/?id=c22706b9049c4c85205a8cc9a3dc72a4e4a52be5
(0001133)
Harriet Bazley   
2016-01-02 23:19   
This problem seems to have reappeared after I upgraded from v3222 to 3244.
(0001137)
Michael Drake   
2016-01-05 13:37   
Fixed in 3250, thanks.
(0001249)
Vincent Sanders   
2016-02-16 14:13   
Confirmed fixed in 3.4 release

Issue History
2015-10-18 17:49Harriet BazleyNew Issue
2015-10-20 21:58Vincent SandersStatusnew => acknowledged
2015-10-20 21:58Vincent SandersProduct Version => 3.4
2015-10-21 21:07Vincent SandersNote Added: 0000974
2015-10-21 21:08Vincent SandersNote Added: 0000975
2015-10-21 22:07Michael DrakeFile Added: duktape-unicode-regex.png
2015-10-21 22:14Michael DrakeNote Added: 0000976
2015-10-21 22:16Michael DrakeStatusacknowledged => confirmed
2015-10-21 22:26Michael DrakeNote Edited: 0000976bug_revision_view_page.php?bugnote_id=976#r1568
2015-10-24 14:47Michael DrakeNote Added: 0000979
2015-10-24 14:48Michael DrakeNote Edited: 0000979bug_revision_view_page.php?bugnote_id=979#r1570
2015-10-24 14:49Michael DrakeFile Added: duktape-fast-regex.png
2015-10-24 14:55Michael DrakeNote Added: 0000980
2015-10-30 14:09Vincent SandersRelationship addedhas duplicate 0002364
2015-10-31 18:24Michael DrakeFixed in CI build # => 3032
2015-10-31 18:24Michael DrakeAssigned To => Michael Drake
2015-10-31 18:24Michael DrakeStatusconfirmed => resolved
2015-10-31 18:24Michael DrakeResolutionopen => fixed
2015-10-31 18:24Michael DrakeFixed in Version => 3.4
2015-10-31 18:25Michael DrakeNote Added: 0001005
2015-10-31 18:46Vincent SandersRelationship addedhas duplicate 0002348
2016-01-02 23:19Harriet BazleyNote Added: 0001133
2016-01-02 23:19Harriet BazleyStatusresolved => feedback
2016-01-02 23:19Harriet BazleyResolutionfixed => reopened
2016-01-05 13:37Michael DrakeNote Added: 0001137
2016-01-05 13:37Michael DrakeStatusfeedback => resolved
2016-01-05 13:37Michael DrakeResolutionreopened => fixed
2016-02-16 14:13Vincent SandersNote Added: 0001249
2016-02-16 14:13Vincent SandersStatusresolved => closed