[Cuis-dev] GitHub jobs passing even when tests fail

[Juan Vuletich]

Hi Nico,

On 12/29/2024 4:59 PM, Nicolás Papagna Maldonado via Cuis-dev wrote:
> Turns out the failing tests were expected in OSX, that's why the exit 
> code is indeed 0.
>
> It looks like there is a feature that allows us to mark some tests as 
> "broken" (didn't know about this one). See:
>
> FloatTest expectedFailures
>   Smalltalk platformName = 'Mac OS' ifTrue: [
>     ^#(testE testExp testPrimExpAndLn) ].
>   ^#().
>
> What is confusing is that these tests are reported as failures in the 
> console:
>
> --------
> 2052 
> <https://github.com/npapagna/Cuis-Smalltalk-Dev-Fix-CI-Tests/actions/runs/12538106004/job/34962890559#step:5:2053>3 
> FAILURE(S)
> 2053 
> <https://github.com/npapagna/Cuis-Smalltalk-Dev-Fix-CI-Tests/actions/runs/12538106004/job/34962890559#step:5:2054>-------- 
>
> 2054 
> <https://github.com/npapagna/Cuis-Smalltalk-Dev-Fix-CI-Tests/actions/runs/12538106004/job/34962890559#step:5:2055>&#16r274C; 
> FloatTest>>#testE has failed because: Assertion (3) failed.
> 2055 
> <https://github.com/npapagna/Cuis-Smalltalk-Dev-Fix-CI-Tests/actions/runs/12538106004/job/34962890559#step:5:2056>&#16r274C; 
> FloatTest>>#testExp has failed because: Assertion (2) failed.
> 2056 
> <https://github.com/npapagna/Cuis-Smalltalk-Dev-Fix-CI-Tests/actions/runs/12538106004/job/34962890559#step:5:2057>&#16r274C; 
> FloatTest>>#testPrimExpAndLn has failed because: Assertion (1) failed.
>
> Maybe we should use another color/category to let users know these 
> were expected failures?

Yes, that would be nice. Maybe yellow.

>
> Best,
> Nico PM
>
> -- 
>
> Nicolás Papagna

Cheers,

-- 
Juan Vuletich
cuis.st
github.com/jvuletich
researchgate.net/profile/Juan-Vuletich
independent.academia.edu/JuanVuletich
patents.justia.com/inventor/juan-manuel-vuletich
linkedin.com/in/juan-vuletich-75611b3
twitter.com/JuanVuletich

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20241230/1a6a5a40/attachment.htm>