MantisBT - NetSurf
View Issue Details
0002583NetSurfAmiga-specificpublic2017-12-10 17:142019-07-19 09:30
ReporterChris Young 
Assigned ToChris Young 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformAmigaOSAmigaOSOS Version3.5
Product Version3.8 
Target Version3.9Fixed in Version3.9 
Fixed in CI build #4584
Reported in CI build #4250
URL of problem pagehttps://www.howsmyssl.com
Summary0002583: SSL not working in OS3 build
DescriptionSSL is not working in OS3 build since OpenSSL upgrade.
TagsNo tags attached.
Attached Filesdiff openssl.diff (177,615) 2018-07-23 23:01
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=549&type=bug
jpg Annotation 2019-03-14 201208.jpg (163,753) 2019-03-14 20:26
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=577&type=bug
jpg

Notes
(0001694)
Chris Young   
2017-12-10 17:16   
curl command-line is not working:

curl: (35) error:14195006:SSL routines:tls_construct_cke_ecdhe:EVP lib
(0001699)
Chris Young   
2017-12-11 22:31   
As far as I can tell, this error is being thrown up because evp_pkey_asn1_ctrl() returns <0.
This suggests the problem lies somewhere in the ASN1 code, but I have no idea what I'm looking for.
(0001703)
Chris Young   
2018-01-12 12:41   
Suspect the answer to this might be to upgrade the version of GCC we are using in the toolchain. There is a patched GCC6 here: https://github.com/bebbo/gcc
At the moment, I'm unable to even rebuild the ppc-amigaos toolchain (GCC 5 or 6) due to conflicting autotools dependencies, and m68k-amigaos is stuck on an even older version of binutils, so I'm not going to attempt switching to GCC6 until I at least get it building for ppc-amigaos.
(0001740)
Chris Young   
2018-01-27 17:42   
GCC6 for OS3 here: http://git.netsurf-browser.org/toolchains.git/log/?h=chris/gcc6-os3

Unable to build much as it appears to have some header dependency issue - conflicting gettimeofday in sys/time.h when building libiconv, for example.
(0001743)
Chris Young   
2018-01-29 21:58   
gcc6 build fixed to the extent it now builds working binaries without include errors.

A couple of outstanding issues:
* gcc-ranlib does not delete - this needs to be deleted otherwise libtool tries to use it and it calls our old ranlib with an option that hadn't been invented back then. (rm command in makefile not working)
* locale_t needs fixing - this is a problem with the clib2 includes which Olaf is aware of. (quick fix is to addd typedef void* locale_t to wchar.h)
* openssl e_os.h tries to include netinet/tcp.h - this doesn't exist in clib2, looks like OpenSSL doesn't actually need anything from it anyway. (quick fix is to touch netinet/tcp.h)

Even with this new version of gcc, OpenSSL still doesn't work. New error message is:
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to howsmyssl.com
(0001781)
Chris Young   
2018-07-23 23:01   
Attached diff between our OpenSSL and the one used by AmiSSL (which works), in case the problem is with our patches rather than the compiler.
(0001782)
Chris Young   
2018-07-24 00:10   
Applied a patch which looked important but hasn't resolved this issue (at least from curl; didn't try netsurf). Will need to check file for other patches we might be missing.
(0001783)
Chris Young   
2018-07-24 22:34   
(Last edited: 2018-07-24 22:35)
Can't see anything else important that we're missing :(

(0001844)
Chris Young   
2018-09-09 14:43   
(Last edited: 2018-09-09 14:44)
Turns out that SSL is largely working (netsurf-browser.org domains work with latest CI build), but https://www.howsmyssl.com still gives "unable to fetch document" (as does https://en.wikipedia.org)

So there's something wrong, but it isn't all SSL/TLS as previously thought.

(0001845)
Vincent Sanders   
2018-09-10 22:23   
it is not something as simple as the OS3 port using the default certificate verification prompt code? (which simply says the user refused the certificate)
(0001846)
Chris Young   
2018-09-10 23:47   
I don't think so, as the certificate prompt worked last time I checked and there shouldn't be any reason for those sites to prompt.
(0001896)
Chris Young   
2019-03-10 23:47   
(Last edited: 2019-03-11 00:05)
I've built Curl against AmiSSL and the Curl executable gives me the same error, however it also fails downloading from sites that work in NetSurf (OS3 build with OpenSSL), so not sure if this is Curl not working properly with AmiSSL or AmiSSL having similar - but more severe - problems as our OpenSSL build.

curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.howsmyssl.com:443

refs: https://github.com/jens-maus/amissl/issues/15#issuecomment-471366123

update: doesn't work in OS4 either, which suggests the libcurl-AmiSSL interface is broken.

(0001897)
Chris Young   
2019-03-10 23:59   
and... an experimental build of NetSurf using libcurl built against AmiSSL doesn't connect to any secure sites either.
(0001898)
Chris Young   
2019-03-14 20:26   
(Last edited: 2019-03-14 20:35)
Fixed Curl+AmiSSL which now downloads pages from sites which Curl+OpenSSL didn't.
I'm confident that this will fix the problem with NetSurf downloading these pages.
At the moment a test build of NetSurf against libcurl-AmiSSL crashes before the main window opens. The same binary run on OS4 opens and allows browsing of secure pages.

(0001899)
Chris Young   
2019-03-30 15:12   
chris/amissl branches contain a working build with AmiSSL.
(0001911)
Chris Young   
2019-05-07 11:24   
Now built against AmiSSL instead of OpenSSL.
(0002009)
Vincent Sanders   
2019-07-19 09:30   
we believe this issue has been resolved in NetSurf 3.9

Issue History
2017-12-10 17:14Chris YoungNew Issue
2017-12-10 17:14Chris YoungStatusnew => assigned
2017-12-10 17:14Chris YoungAssigned To => Chris Young
2017-12-10 17:16Chris YoungStatusassigned => confirmed
2017-12-10 17:16Chris YoungNote Added: 0001694
2017-12-10 17:16Chris YoungRelationship addedrelated to 0002582
2017-12-10 23:24Chris YoungRelationship deletedrelated to 0002582
2017-12-11 22:31Chris YoungNote Added: 0001699
2018-01-12 12:41Chris YoungNote Added: 0001703
2018-01-27 17:42Chris YoungNote Added: 0001740
2018-01-29 21:58Chris YoungNote Added: 0001743
2018-07-23 23:01Chris YoungFile Added: openssl.diff
2018-07-23 23:01Chris YoungNote Added: 0001781
2018-07-24 00:10Chris YoungNote Added: 0001782
2018-07-24 22:34Chris YoungNote Added: 0001783
2018-07-24 22:35Chris YoungNote Edited: 0001783bug_revision_view_page.php?bugnote_id=1783#r2047
2018-08-28 11:40Vincent SandersTarget Version3.8 => 3.9
2018-09-09 14:43Chris YoungNote Added: 0001844
2018-09-09 14:44Chris YoungNote Edited: 0001844bug_revision_view_page.php?bugnote_id=1844#r2062
2018-09-10 22:23Vincent SandersNote Added: 0001845
2018-09-10 23:47Chris YoungNote Added: 0001846
2019-03-10 23:47Chris YoungNote Added: 0001896
2019-03-10 23:59Chris YoungNote Added: 0001897
2019-03-11 00:05Chris YoungNote Edited: 0001896bug_revision_view_page.php?bugnote_id=1896#r2072
2019-03-14 20:26Chris YoungFile Added: Annotation 2019-03-14 201208.jpg
2019-03-14 20:26Chris YoungNote Added: 0001898
2019-03-14 20:35Chris YoungNote Edited: 0001898bug_revision_view_page.php?bugnote_id=1898#r2074
2019-03-30 15:12Chris YoungNote Added: 0001899
2019-05-07 11:24Chris YoungStatusconfirmed => resolved
2019-05-07 11:24Chris YoungResolutionopen => fixed
2019-05-07 11:24Chris YoungFixed in Version => 3.9
2019-05-07 11:24Chris YoungFixed in CI build # => 4584
2019-05-07 11:24Chris YoungNote Added: 0001911
2019-07-19 09:30Vincent SandersStatusresolved => closed
2019-07-19 09:30Vincent SandersNote Added: 0002009