MantisBT - NetSurf
View Issue Details
0002301NetSurfAmiga-specificpublic2015-04-01 18:292016-02-16 15:30
ReporterChris Young 
Assigned ToVincent Sanders 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformAmigaOSAmigaOSOS Version4.1
Product Version3.4 
Target VersionFixed in Version3.4 
Fixed in CI build #2695
Reported in CI build #2690
URL of problem page
Summary0002301: Backing store not creating 'm' dir and block writes not working
DescriptionThe backing store is not creating the 'm' directory even though the 'd' directory and files are present.
The mblk and dblk directories only contain one 0-byte file 'A'.
Steps To ReproduceRun NetSurf, browse a few pages, and quit.
Additional InformationLogfile attached.
Listing of cache directory after quitting also attached - filtered to show files only, root of cache is as follows:

  Directory "Apps:Internet/NetSurf/Users/chris/Cache" on Wednesday 01-Apr-15
  blocks 4096 ----rw-d Today 18:19:43
  entries 8160 ----rw-d Today 18:19:43
  mblk Dir ----rwed Today 18:07:45
  d Dir ----rwed Today 18:16:25
  dblk Dir ----rwed Today 18:07:45
  CACHEDIR.TAG 191 ----rw-d Today 18:07:36
  control 12 ----rw-d Today 18:07:36
  4 files - 12K bytes - 3 directories - 36 blocks used

TagsNo tags attached.
Attached Filesgz ns.log.gz (111,799) 2015-04-01 18:29
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=274&type=bug
gz ns-cache-dir.txt.gz (2,233) 2015-04-01 18:29
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=275&type=bug
gz ns.log-new.gz (148,137) 2015-04-02 19:19
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=276&type=bug

Notes
(0000814)
Vincent Sanders   
2015-04-01 20:24   
ok, firstly, the lack of an m directory was fine. The cache never tried to create one! all the metadata filers were less than a kilobyte long and were therefore being stored in the block files.

Which is where we actually hit the issue.

these three lines from the log tell me what the issue is:

  content/fs_backing_store.c set_store_entry 933: url:http://www.google.co.uk/favicon.ico

first attempt to store a data object

  content/fs_backing_store.c store_open 1058: opening PROGDIR:Users/chris/Cache/dblk/A

which succeeded at opening the block file

  content/fs_backing_store.c store_write_block 1609: Write failed -1 of 5430 bytes from 0x54bf6420 at 0x2000 block 1 errno 29

from the ppc sys/errno.h
#define ESPIPE 29 /* Illegal seek */

so errno 29 is telling us that the pwrite operation was trying to seek to offset 0x2000 (8k) into the block file and write the 5430 bytes of data and got told it could not seek there.

On other operating systems seeking past the end of a file opened for writing simply extends the file. Is this the not case on amigaos 4?
(0000815)
Chris Young   
2015-04-01 23:27   
No, you can't seek past the end of a file. I just double-checked that in the autodocs (http://wiki.amigaos.net/amiga/autodocs/dos.txt):
"ChangeFilePosition() beyond the limits of the file will cause failure."
(0000816)
Vincent Sanders   
2015-04-02 00:02   
ok so, how about an amigaos wrapper that tries the seek and if it gets errno 29 calls the ChangeFileSize() to extend to the offset, retries the lseek and then does a write.

I already added a wrapper for RISC OS in utils/config.h and utils/utils.c so extending it for amigaos would seem relatively straightforward
(0000817)
Chris Young   
2015-04-02 00:11   
That sounds like it should work.
(0000818)
Vincent Sanders   
2015-04-02 17:44   
ok added a universal pread/pwrite implementation to libnsutils as it was horribly hacky and removed the compat cruft from NetSurf itself.

The new implementation of pwrite *should* work on amigaos now. Please can you start from a clean cache again and post the log to the bug so I can check?
(0000819)
Chris Young   
2015-04-02 19:20   
Attached. The mblk and dblk 'A' file now has size, so looks good.
(0000820)
Vincent Sanders   
2015-04-02 19:50   
phew. all looks good and the 1.5megabytes/sec overall write rate looks a lot healthier than previous scheme.

Thanks for your patience and testing, sorry we had a bit of a rough start. I do hope this improves things for you though please let me know on list either way.
(0001321)
Vincent Sanders   
2016-02-16 15:30   
Confirmed resolved in 3.4 release

Issue History
2015-04-01 18:29Chris YoungNew Issue
2015-04-01 18:29Chris YoungStatusnew => assigned
2015-04-01 18:29Chris YoungAssigned To => Vincent Sanders
2015-04-01 18:29Chris YoungFile Added: ns.log.gz
2015-04-01 18:29Chris YoungFile Added: ns-cache-dir.txt.gz
2015-04-01 18:35Chris YoungAdditional Information Updatedbug_revision_view_page.php?rev_id=1461#r1461
2015-04-01 20:24Vincent SandersNote Added: 0000814
2015-04-01 20:24Vincent SandersStatusassigned => feedback
2015-04-01 20:24Vincent SandersDescription Updatedbug_revision_view_page.php?rev_id=1463#r1463
2015-04-01 20:24Vincent SandersAdditional Information Updatedbug_revision_view_page.php?rev_id=1464#r1464
2015-04-01 23:27Chris YoungNote Added: 0000815
2015-04-01 23:27Chris YoungStatusfeedback => assigned
2015-04-02 00:02Vincent SandersNote Added: 0000816
2015-04-02 00:02Vincent SandersStatusassigned => feedback
2015-04-02 00:11Chris YoungNote Added: 0000817
2015-04-02 00:11Chris YoungStatusfeedback => assigned
2015-04-02 17:44Vincent SandersNote Added: 0000818
2015-04-02 17:44Vincent SandersStatusassigned => feedback
2015-04-02 19:19Chris YoungFile Added: ns.log-new.gz
2015-04-02 19:20Chris YoungNote Added: 0000819
2015-04-02 19:20Chris YoungStatusfeedback => assigned
2015-04-02 19:50Vincent SandersFixed in CI build # => 2695
2015-04-02 19:50Vincent SandersNote Added: 0000820
2015-04-02 19:50Vincent SandersStatusassigned => resolved
2015-04-02 19:50Vincent SandersResolutionopen => fixed
2015-04-02 19:50Vincent SandersFixed in Version => 3.4
2016-02-16 15:30Vincent SandersNote Added: 0001321
2016-02-16 15:30Vincent SandersStatusresolved => closed