2024-11-24 15:06 UTC
  

View Issue Details [ Jump to Notes ] [ Issue History ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0002633Buildsystem[All Projects] Generalpublic2019-01-14 03:592020-07-25 19:06
ReporterSamuel Holland 
Assigned Tokinnison 
PrioritynormalSeverityminorReproducibilityalways
StatusacknowledgedResolutionopen 
PlatformLinuxOSGentoo muslOS Version
Summary0002633: Test runner can't handle certain buffering cases, causing spurious test failures
DescriptionWhen running the tests for libcss on musl libc, specifically "test_parse-auto parse/properties.dat", the output of the test program is buffered just so that the first line returned by read() is a partial line, and the first and last lines are both "PASS". Strace output from the test program:

13451 writev(1, [{iov_base="PASS\nTest 488: PASS\nTest 489: PASS\nTest 490: PASS\nTest 491: PASS\nTest 492: PASS\nTest 493: PASS\nTest 494: PASS\nTest 495: PASS\nTest 496: PASS\nTest 497: PASS\nTest 498: PASS\nTest 499: PASS\nTest 500: PASS\nTest 501: PASS\nTest 502: PASS\nTest 503: PASS\nTest 504: PASS\nTest 505: PASS\nTest 506: PASS\nTest 507: PASS\nTest 508: PASS\nTest 509: PASS\nTest 510: PASS\nTest 511: PASS\nTest 512: PASS\nTest 513: PASS\nTest 514: PASS\nTest 515: PASS\nTest 516: PASS\nTest 517: PASS\nTest 518: PASS\nTest 519: PASS\nTest 520: PASS\nTest 521: PASS\nTest 522: PASS\nTest 523: PASS\nTest 524: PASS\nTest 525: PASS\nTest 526: PASS\nTest 527: PASS\nTest 528: PASS\nTest 529: PASS\nTest 530: PASS\nTest 531: PASS\nTest 532: PASS\nTest 533: PASS\nTest 534: PASS\nTest 535: PASS\nTest 536: PASS\nTest 537: PASS\nTest 538: PASS\nTest 539: PASS\nTest 540: PASS\nTest 541: PASS\nTest 542: PASS\nTest 543: PASS\nTest 544: PASS\nTest 545: PASS\nTest 546: PASS\nTest 547: PASS\nTest 548: PASS\nTest 549: PASS\nTest 550: PASS\nTest 551: PASS\nTest 552: PASS\nTest 553: PASS\nTest 554: PASS\nPASS\n", iov_len=1015}, {iov_base=NULL, iov_len=0}], 2) = 1015

This confuses the test runner, and it ends up with $last = "Test 554: PASSPASS" at the end of the run. This does not equal "PASS", so it claims the test case has failed. I'm no perl programmer, but I was able to get it to work by checking the array index instead of the line contents:

--- buildsystem-1.7/testtools/testrunner.pl
+++ buildsystem-1.7/testtools/testrunner.pl
@@ -159,20 +159,20 @@ sub run_test
 
                 if ($fh == $out) {
                     # Child's stdout
- foreach my $l (@lines) {
+ while (my ($i, $l) = each @lines) {
                         # Last line of previous read
                         # was incomplete, and this is
                         # the first line of this read
                         # Simply contatenate.
                         if ($outcont == 1 &&
- $l eq $lines[0]) {
+ $i eq 0) {
                             print LOG "$l\n";
                             $last .= $l;
                         # Last char of this read was
                         # not '\n', so don't terminate
                         # line in log.
                         } elsif ($lastchar ne '\n' &&
- $l eq $lines[-1]) {
+ $i eq (scalar(@lines) - 1)) {
                             print LOG " $l";
                             $last = $l;
                         # Normal behaviour, just print
Steps To ReproduceBuild libcss 0.8.0 against musl libc (e.g. on Alpine Linux or with a musl cross-compiler) and attempt to run the test suite with the buildsystem 1.7. Running the test case manually will show that it passes (the last line is PASS by itself, as shown in the output above), but the test runner will claim that it fails.
TagsNo tags attached.
Fixed in CI build #
Reported in CI build #
Attached Files
  • patch file icon netsurf-buildsystem-1.7-testrunner.patch (792 bytes) 2020-07-25 19:05 - 
    --- buildsystem-1.7/testtools/testrunner.pl
+++ buildsystem-1.7/testtools/testrunner.pl
@@ -159,20 +159,20 @@ sub run_test
 
 				if ($fh == $out) {
 					# Child's stdout
-					foreach my $l (@lines) {
+					while (my ($i, $l) = each @lines) {
 						# Last line of previous read
 						# was incomplete, and this is
 						# the first line of this read
 						# Simply contatenate.
 						if ($outcont == 1 && 
-							$l eq $lines[0]) {
+							$i eq 0) {
 							print LOG "$l\n";
 							$last .= $l;
 						# Last char of this read was 
 						# not '\n', so don't terminate
 						# line in log.
 						} elsif ($lastchar ne '\n' &&
-							$l eq $lines[-1]) {
+							$i eq (scalar(@lines) - 1)) {
 							print LOG "    $l";
 							$last = $l;
 						# Normal behaviour, just print
    patch file icon netsurf-buildsystem-1.7-testrunner.patch (792 bytes) 2020-07-25 19:05 +

-Relationships
+Relationships

-Notes
kinnison

~0002043

kinnison (administrator)

2019-08-01 10:22

 Could you please try with a more up-to-date version? That one is from a year ago.

If it still happens, please *attach* a patch to the git version so that we can apply it.
Samuel Holland

~0002276

Samuel Holland (reporter)

2020-07-25 19:06

 I can still reproduce this with libcss 0.9.1 and buildsystem 1.9. The patch still applies cleanly to buildsystem git master.
+Notes

-Issue History
Date Modified Username Field Change
2019-01-14 03:59 Samuel Holland New Issue
2019-08-01 10:22 kinnison Note Added: 0002043
2019-08-01 10:22 kinnison Assigned To => kinnison
2019-08-01 10:22 kinnison Status new => assigned
2019-08-01 10:22 kinnison Status assigned => acknowledged
2020-07-25 19:05 Samuel Holland File Added: netsurf-buildsystem-1.7-testrunner.patch
2020-07-25 19:06 Samuel Holland Note Added: 0002276
+Issue History