MantisBT - Buildsystem
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0002633Buildsystem[All Projects] Generalpublic2019-01-14 03:592020-07-25 19:06
ReporterSamuel Holland 
Assigned Tokinnison 
PrioritynormalSeverityminorReproducibilityalways
StatusacknowledgedResolutionopen 
PlatformLinuxOSGentoo muslOS Version
Fixed in CI build #
Reported in CI build #
Summary0002633: Test runner can't handle certain buffering cases, causing spurious test failures
DescriptionWhen running the tests for libcss on musl libc, specifically "test_parse-auto parse/properties.dat", the output of the test program is buffered just so that the first line returned by read() is a partial line, and the first and last lines are both "PASS". Strace output from the test program:

13451 writev(1, [{iov_base="PASS\nTest 488: PASS\nTest 489: PASS\nTest 490: PASS\nTest 491: PASS\nTest 492: PASS\nTest 493: PASS\nTest 494: PASS\nTest 495: PASS\nTest 496: PASS\nTest 497: PASS\nTest 498: PASS\nTest 499: PASS\nTest 500: PASS\nTest 501: PASS\nTest 502: PASS\nTest 503: PASS\nTest 504: PASS\nTest 505: PASS\nTest 506: PASS\nTest 507: PASS\nTest 508: PASS\nTest 509: PASS\nTest 510: PASS\nTest 511: PASS\nTest 512: PASS\nTest 513: PASS\nTest 514: PASS\nTest 515: PASS\nTest 516: PASS\nTest 517: PASS\nTest 518: PASS\nTest 519: PASS\nTest 520: PASS\nTest 521: PASS\nTest 522: PASS\nTest 523: PASS\nTest 524: PASS\nTest 525: PASS\nTest 526: PASS\nTest 527: PASS\nTest 528: PASS\nTest 529: PASS\nTest 530: PASS\nTest 531: PASS\nTest 532: PASS\nTest 533: PASS\nTest 534: PASS\nTest 535: PASS\nTest 536: PASS\nTest 537: PASS\nTest 538: PASS\nTest 539: PASS\nTest 540: PASS\nTest 541: PASS\nTest 542: PASS\nTest 543: PASS\nTest 544: PASS\nTest 545: PASS\nTest 546: PASS\nTest 547: PASS\nTest 548: PASS\nTest 549: PASS\nTest 550: PASS\nTest 551: PASS\nTest 552: PASS\nTest 553: PASS\nTest 554: PASS\nPASS\n", iov_len=1015}, {iov_base=NULL, iov_len=0}], 2) = 1015

This confuses the test runner, and it ends up with $last = "Test 554: PASSPASS" at the end of the run. This does not equal "PASS", so it claims the test case has failed. I'm no perl programmer, but I was able to get it to work by checking the array index instead of the line contents:

--- buildsystem-1.7/testtools/testrunner.pl
+++ buildsystem-1.7/testtools/testrunner.pl
@@ -159,20 +159,20 @@ sub run_test
 
                 if ($fh == $out) {
                     # Child's stdout
- foreach my $l (@lines) {
+ while (my ($i, $l) = each @lines) {
                         # Last line of previous read
                         # was incomplete, and this is
                         # the first line of this read
                         # Simply contatenate.
                         if ($outcont == 1 &&
- $l eq $lines[0]) {
+ $i eq 0) {
                             print LOG "$l\n";
                             $last .= $l;
                         # Last char of this read was
                         # not '\n', so don't terminate
                         # line in log.
                         } elsif ($lastchar ne '\n' &&
- $l eq $lines[-1]) {
+ $i eq (scalar(@lines) - 1)) {
                             print LOG " $l";
                             $last = $l;
                         # Normal behaviour, just print
Steps To ReproduceBuild libcss 0.8.0 against musl libc (e.g. on Alpine Linux or with a musl cross-compiler) and attempt to run the test suite with the buildsystem 1.7. Running the test case manually will show that it passes (the last line is PASS by itself, as shown in the output above), but the test runner will claim that it fails.
TagsNo tags attached.
Relationships
Attached Filespatch netsurf-buildsystem-1.7-testrunner.patch (792) 2020-07-25 19:05
https://bugs.netsurf-browser.org/mantis/file_download.php?file_id=680&type=bug

Notes
(0002043)
kinnison   
2019-08-01 10:22   
Could you please try with a more up-to-date version? That one is from a year ago.

If it still happens, please *attach* a patch to the git version so that we can apply it.
(0002276)
Samuel Holland   
2020-07-25 19:06   
I can still reproduce this with libcss 0.9.1 and buildsystem 1.9. The patch still applies cleanly to buildsystem git master.

Issue History
Date ModifiedUsernameFieldChange
2019-01-14 03:59Samuel HollandNew Issue
2019-08-01 10:22kinnisonNote Added: 0002043
2019-08-01 10:22kinnisonAssigned To => kinnison
2019-08-01 10:22kinnisonStatusnew => assigned
2019-08-01 10:22kinnisonStatusassigned => acknowledged
2020-07-25 19:05Samuel HollandFile Added: netsurf-buildsystem-1.7-testrunner.patch
2020-07-25 19:06Samuel HollandNote Added: 0002276