[Cuis-dev] Jitter source during benchmarks

Wed May 27 14:22:24 PDT 2026

The best thing in order to help you would be to share a minimal example
benchmark, and try to reproduce your findings first.

Questions:

1. Did you try already with old images and VMs, 32 vs 64 bits, different
OSes, different hardware?
2. Did you try disabling JIT?
3. Are you documenting your findings so people can help you by trying to
replicate your results?
4. Is the benchmark really coupled with Cuis' Aconcagua, or can you think
of something similar, but more generic that would allow you/us to also test
with Squeak and Pharo?

Ideas:
1. One benchmark variant that does almost no allocation, and another that
allocates much more, then compare whether the spike frequency changes. If
the spike cadence tracks allocation volume, you are looking at a runtime
maintenance cycle; if it stays fixed, it is more likely a periodic
heartbeat/interrupt source.
2. Test whether the spikes survive when you run a completely empty loop
under the same measurement wrapper.

-- 
Eze

El vie, 15 may 2026 a la(s) 5:12 p.m., Nicolás Papagna Maldonado via
Cuis-dev (cuis-dev at lists.cuis.st) escribió:

> Hi folks,
>
> I've tried the following without success. The same jitter is still present
> in these scenarios:
>
>    - Running the experiment using the -headless VM flag
>    - Running the experiment using the -noheartbeat VM flag
>    - Running the experiment inside a block with valueUnpreemtively
>    - Configuring Smalltalk vmParameterAt: 26 put: 10 before running the
>    experiment
>    - Replace idle process: 50ms relinquish instead of 1ms -> this
>    required making the change and re-starting the image to run the experiment.
>
>
> I'll try to provide exact steps to reproduce, but for now, I have no clue
> about where this is coming from.
>
> Is there anything else I could be missing?
>
> Thanks in advance,
> Nico PM
>
>
> On Wed, May 13, 2026 at 11:00 AM Phil B via Cuis-dev <
> cuis-dev at lists.cuis.st> wrote:
>
>> I have not looked recently, but Morphic used to have an idle throttle
>> where (IIRC) if there weren't keyboard/mouse inputs for a period of time,
>> Morphic updates would occur at a decreased rate.  It occurred after some
>> number of seconds without input.  Assuming that functionality is still in
>> place, you might be seeing the other side of it (i.e. when the Morphic
>> update rate is higher your code is less performant, when it is lower your
>> code is more performant.  While the *drawing* may be on another thread, the
>> Morphic Smalltalk overhead would still be sharing/competing for a single
>> core with the rest of the image.)  I don't recall if it was hard-coded or a
>> preference but I'm sure Juan would be able to speak to the current state of
>> that functionality if it still exists.  If this information is out of date,
>> please disregard...
>>
>> On Tue, May 12, 2026 at 3:18 PM Nicolás Papagna Maldonado via Cuis-dev <
>> cuis-dev at lists.cuis.st> wrote:
>>
>>> Hi folks,
>>>
>>> I'm in the process of doing performance benchmarks of my Code Coverage
>>> tool for my thesis, and I've encountered something interesting while
>>> measuring the test execution times of the Aconcagua package.
>>>
>>> *Observation:*
>>> When running the Aconcagua test suite 150 times in a clean image, the
>>> execution times are not normally distributed. Instead, they fall into two
>>> distinct "modes":
>>>
>>> [image: image.png]
>>> A Base Mode (~6.28 ms): Representing roughly 72% of the iterations.
>>> A Spike Mode (~7.0 ms): Representing roughly 28% of the iterations.
>>>
>>> As the next plot shows, the spikes are not random; they seem to be
>>> kind-of deterministic (roughly every 4th iteration).
>>> I think these perturbations are visible because I'm taking very small
>>> measurements (~6.28 ms).
>>> [image: image.png]
>>>
>>> *Question:*
>>> I am trying to isolate whether this "heartbeat" is originating from
>>> within the Cuis image or the VM/OS layer (I'm thinking of VM timer/events,
>>> maybe UI timers?)
>>> Does anyone know what could be causing this?
>>> Are there any known events that run at this particular frequency?
>>>
>>> I repeated the experiment a couple of times and the perturbations are
>>> consistent (in terms of ms and frequency).
>>>
>>> *Methodology:*
>>>
>>>    - Measurements were taken in batches of 50 test runs.
>>>    - Each batch was run in a fresh image, waiting 2 minutes for the VM
>>>    to stabilize Cuis code.
>>>    - 10 warmup runs were done and discarded before measuring test run
>>>    times.
>>>    - Measurements were taken using Time primHighResClock and converted
>>>    to ms using Time highResTimerTicksPerMillisecond.
>>>    - VM stats were collected before and after measurements. A full GC
>>>    was run before taking measurements.
>>>    - No GC activity was observed while measuring the tests run (VM
>>>    stats report no full nor incremental GCs).
>>>    - The test suite was created and run from a workspace
>>>
>>> *Cuis environment:*
>>>
>>>    - Cuis Smalltalk commit:
>>>    https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev/commit/b221722348cc46ebb9f5d9f3a057a3f361aa3d55
>>>    - Code Coverage commit:
>>>    https://github.com/npapagna/cuis-code-coverage/commit/db19de02250a3fd50dfc4e0b4d0893b656e31e9b
>>>    - Aconcagua commit:
>>>    https://github.com/Cuis-Smalltalk/Measures/commit/05aa02aada64f7b64898a5e1452ae8fb76f79427
>>>
>>> *Machine environment:*
>>>
>>>    - Model Name: MacBook Pro
>>>    - Model Identifier: MacBookPro18,3
>>>    - Model Number: MKGR3LL/A
>>>    - Chip: Apple M1 Pro
>>>    - Total Number of Cores: 8 (6 Performance and 2 Efficiency)
>>>    - Memory: 16 GB
>>>    - System Firmware Version: 13822.81.10
>>>    - OS Loader Version: 13822.81.102
>>>    - APPLE SSD AP0512R (NVMe, avg read 5671 MB/s, avg write 5670 MB/s)
>>>    - macOs 26.3 (25D125)
>>>    - A dedicated fresh profile was created for the benchmarks
>>>    (indexing, time machine, screen lock/saver, display shutdown, bluetooth and
>>>    wifi disabled).
>>>
>>> Thanks in advance,
>>> Nico PM
>>>
>>> --
>>> Cuis-dev mailing list
>>> Cuis-dev at lists.cuis.st
>>> https://lists.cuis.st/mailman/listinfo/cuis-dev
>>>
>> --
>> Cuis-dev mailing list
>> Cuis-dev at lists.cuis.st
>> https://lists.cuis.st/mailman/listinfo/cuis-dev
>>
>
>
> --
>
> Nicolás Papagna
> --
> Cuis-dev mailing list
> Cuis-dev at lists.cuis.st
> https://lists.cuis.st/mailman/listinfo/cuis-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20260527/92e6e172/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 175232 bytes
Desc: not available
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20260527/92e6e172/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 48058 bytes
Desc: not available
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20260527/92e6e172/attachment-0003.png>