MantisBT - Hubbub
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0002090Hubbub[All Projects] Generalpublic2014-03-09 14:482014-06-12 12:24
ReporterAchal 
Assigned ToJohn-Mark Bell 
PrioritynormalSeverityfeatureReproducibilityalways
StatusacknowledgedResolutionopen 
PlatformlinuxOSubuntuOS Version13.10
Product Version 
Target VersionFixed in Version 
Fixed in CI build #
Reported in CI build #
Summary0002090: updating some tokeniser test to html5lib
Descriptioncdata.test
entities.test
numericEntities.test
test1.test
test2.test
test3.test
test4.test
unicodeChars.test

are the test suits which are updated to html5lib tests and passed on tokeniser2 after this patch
Additional InformationFROM TEST4.TEST
{"description":"Maximum non-BMP numeric entity",
"input":"",
"output":["ParseError", ["Character", "\uDBFF\uDFFF"]]},

and

FROM NUMERICENTITIES.TEST
{"description": "Invalid numeric entity character U+10FFFF",
"input": "",
"output": ["ParseError", ["Character", "\uFFFD"]]},

are two conflicting tests. So I used first one because it is in accordance with html5 specs.
TagsNo tags attached.
Relationships
Attached Filesdiff libhubbub-test.diff (293,757) 2014-03-09 14:48
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=93&type=bug
diff rewrite.diff (301,032) 2014-03-11 16:29
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=94&type=bug
patch 0001-Rewriting-whole-tokenizer-and-updating-its-tests.patch (255,232) 2014-03-11 17:00
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=95&type=bug
patch 0002-Replacing-content-model-with-states-except-SCRIPT-DA.patch (57,023) 2014-03-11 17:00
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=96&type=bug
patch 0003-Fix-tokeniser-test-executer-for-content-model-replac.patch (4,014) 2014-03-11 17:01
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=97&type=bug
patch 0004-Fixing-rcdata-and-rawtext-close-tag-open-state-for-b.patch (4,433) 2014-03-11 17:01
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=98&type=bug

Notes
(0000260)
Achal   
2014-03-09 14:58   
contentModelFlags.test
escapeFlag.test
regression.test

are currently failing. I am working on them, will upload second patch soon.
(0000267)
Achal   
2014-03-11 16:00   
New branch to track : https://github.com/Achal-Aggarwal/hibbub/compare/rewrite

What to expect from this?
Tokeniser updated to HTML5 spec with some few exceptions(working on that).
http://www.whatwg.org/specs/web-apps/current-work/multipage/tokenization.html

What is remaining in tokeniser?
SCRIPT DATA states (few implemented, few remaining...)
named entities
double escaped tests (unicodeCharsProblematic.test)

Tree Builder
tests are failing currently might be because of updated tokeniser.

Issue History
Date ModifiedUsernameFieldChange
2014-03-09 14:48AchalNew Issue
2014-03-09 14:48AchalFile Added: libhubbub-test.diff
2014-03-09 14:58AchalNote Added: 0000260
2014-03-11 16:00AchalNote Added: 0000267
2014-03-11 16:29AchalFile Added: rewrite.diff
2014-03-11 17:00AchalFile Added: 0001-Rewriting-whole-tokenizer-and-updating-its-tests.patch
2014-03-11 17:00AchalFile Added: 0002-Replacing-content-model-with-states-except-SCRIPT-DA.patch
2014-03-11 17:01AchalFile Added: 0003-Fix-tokeniser-test-executer-for-content-model-replac.patch
2014-03-11 17:01AchalFile Added: 0004-Fixing-rcdata-and-rawtext-close-tag-open-state-for-b.patch
2014-06-12 12:24Vincent SandersAssigned To => John-Mark Bell
2014-06-12 12:24Vincent SandersStatusnew => acknowledged