<div dir="ltr"><div>Hi Christian,</div><div>concerning the thresholds, the Opensmalltalk VM stops allocating in eden for 2^16 slots and above. </div><div>It allocates in oldSpace instead numSlots > self maxSlotsForNewSpaceAlloc ifTrue: [numSlots > self maxSlotsForAlloc ifTrue: [coInterpreter primitiveFailFor: PrimErrUnsupported. ^nil]. newObj := self allocateSlotsInOldSpace: numSlots format: instSpec classIndex: classIndex] ifFalse: [newObj := self allocateSlots: numSlots format: instSpec classIndex: classIndex]. </div> <div>Hence the 66,000 threshold you observe with new: (it's 65536).</div><div> </div><div>For 41000, we start we size 10, and double size at each growth, then, at 11th growth we get a size 2^11 * 10, 4096*10 = 40,960 </div><div>At next growth (adding the 41961), the array size will get over the 65535 threshold. </div><div>Hence it's about the same threshold that we observe. </div><div>For 82000, this gets more interesting. This time, the size allocated is 163,840 slots. </div><div>With 8 bytes per slot (assuming 64 bits VM), we're getting just over a MiByte. </div><div>No time to dig more in VM source code, but it might be related to object memory growth... </div><div> </div><div>Now let's observe some interesting figures in Squeak: </div><div><div>[Array new: 65535] bench. '1,650 per second. 605 microseconds per run. 60.35841 % GC time.' </div><div> [Array new: 65536] bench. '226 per second. 4.42 milliseconds per run. 91.64405 % GC time.' </div><div> </div><div>Notice the high percentage spent at GC once we're in the old space : the cost is likely to be dominated by GC.</div><div> </div><div>And in Cuis:</div><div> [Array new: 65535] bench. '2.02 k runs per second' . [Array new: 65536] bench. '1.31 k runs per second' . </div><div>Ah ah ! Cuis image is much smaller, hence full GC much cheaper ! <div>I guess that Pharo images are much larger, hence the cost...</div><div> </div><div>So, </div><div>- the cost is dominated by GC in OpenSmalltalk images </div><div>- starting at 2^16 slot and above, oldSpace is allocated, and that ends up with full GC </div><div>- the larger the image, the less efficient the allocation </div><div>VW memory policy is more optimized than Opensmalltalk, at least for this benchmark. </div><div>Probably a virtue of large space (segments reserved for large objects). </div><div>Nicolas </div></div></div></div> <div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le dim. 3 mars 2024 à 19:36, Christian Haider via Cuis-dev <<a href="mailto:cuis-dev@lists.cuis.st">cuis-dev@lists.cuis.st</a>> a écrit : </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-8657701997952365470"><div lang="DE" style="overflow-wrap: break-word;"><div class="m_-8657701997952365470WordSection1">Hi, I was going to complain that #new: on OrderedCollection was removed.I thought that it is common wisdom that #new: is a very important optimization when allocating OrderedCollections, especially big ones. Therefore, I measured the differences with the following script:| time time1 time2 |(0 to: 100000 by: 1000) collect: [:size | time := time1 := time2 := 0. 1000 timesRepeat: [ time := time + (Time microsecondsToRun: [ | list | list := OrderedCollection new. 1 to: size do: [:i | list add: i]])]. time1 := time / 1000. time := 0. 1000 timesRepeat: [ time := time + (Time microsecondsToRun: [ | list | list := OrderedCollection new: size. 1 to: size do: [:i | list add: i]])]. time2 := time / 1000. Array with: size with: time1 with: time2] I ran the script with the current Cuis 6.3, Pharo 10, Squeak 6.0 and VW 9.3.1.The results are in the attached file from which I created the charts below with Excel.The size of the created collections are on the x-axis, while the y-axis shows the microseconds. The results are interesting!<ol style="margin-top:0cm" start="1" type="1"><li class="m_-8657701997952365470MsoListParagraph" style="margin-left:0cm">As expected, VW new: is the fastest and grows linear with the collection size.</li><li class="m_-8657701997952365470MsoListParagraph" style="margin-left:0cm">As expected, #new is slower than #new: (except for Cuis – more below)</li><li class="m_-8657701997952365470MsoListParagraph" style="margin-left:0cm">There are certain threshold sizes from which on the creation is consistently slower (look at 40,000 and 82,000 for #new:)</li><li class="m_-8657701997952365470MsoListParagraph" style="margin-left:0cm">Surprising is the slowness of Pharo(?)</li></ol> <img width="1264" height="737" style="width: 13.1666in; height: 7.677in;" id="m_-8657701997952365470Diagramm_x0020_1" src="cid:ii_18e065fd7c55b16b21"> Extremely surprising is Cuis. The next chart shows just Cuis and Squeak to spread the scale a bit.In Cuis, both methods have about the same performance and are growing linear with the size!!! How is this possible? What is the magic? Or is the measurement wrong? I am quite impressed.And I am not going to complain about the missing #new: J. Happy hacking,Christian <img width="1264" height="737" style="width: 13.1666in; height: 7.677in;" id="m_-8657701997952365470Diagramm_x0020_2" src="cid:ii_18e065fd7c99374b62"> </div></div>-- Cuis-dev mailing list <a href="mailto:Cuis-dev@lists.cuis.st" target="_blank">Cuis-dev@lists.cuis.st</a> <a href="https://lists.cuis.st/mailman/listinfo/cuis-dev" rel="noreferrer" target="_blank">https://lists.cuis.st/mailman/listinfo/cuis-dev</a> </div></blockquote></div>