[Cuis-dev] Rebuilding on events: Experiments in simplifying UI development in Cuis inspired by Mithril.js

[Paul D. Fernhout]

Thanks for the additional background, Hilaire. Not surprising the 
reference you mentioned relates to the web given a lot of experiments 
there over the past decade with new UI libraries (including Mithril and 
Elm).

On legacy complexity and Cuis compared to web UI development, that may 
be debatable in some specific cases -- even as, in general, Cuis is 
simpler and more approachable than the overwhelming complexity of the 
JavaScript ecosystem.

For an example related to Cuis and unexpected complex behavior, I was 
debugging why the background color of the experiment I worked on 
suddenly turned light brown as soon as the window was rebuilt. It 
ultimately had to do with the original LayoutMorphs having a transparent 
background color somehow, but for some reasons the new ones the code 
made had a light brown color. I could not easily figure out why that was 
the case (like where the original LayoutMorph got its color set). I 
spent a lot of time drilling down into the submorphs of the window, past 
layers and layers of LayoutMorphs.

And my conclusion from all that debugging was that the idea of an 
invisible LayoutMorph that Cuis uses is an anti-pattern. :-)

Those extra morphs made it harder to understand what was going on when 
inspecting windows. I'd rather see morphs hold directly onto their 
(visible) submorphs and hold onto a layout object of some sort that 
moves submorphs around as needed. That would make it easier to inspect a 
tree of (visible) morphs. To me, every morph having an optional layout 
object to arrange submorphs also would be simpler conceptually than 
having mysterious invisible-sometimes morphs in between all other 
morphs, because it would be a separation of concerns of layout from 
display. With a replaceable layout object, people could also more easily 
experiment with all sorts of layout options like Swing and HTML have 
(like MiGLayout or FlexBox which I  have found useful).

Here is the latest version of the experimental code which resolves the 
updating issues I mentioned in a previous email to the list (at a 
potential cost of more redrawing) and also has a fix for the background 
color issue mentioned above:
https://github.com/pdfernhout/Cuis-Smalltalk-RedrawWhenClicked/blob/main/KfExperiments-v5.st#L250-L317

That version also adds initial layout support to the builder 
(KfMorphBuilder) -- inspired by MiGLayout from Swing, but very primitive 
comparatively (and also the layout code should move to its own class 
eventually). The highlighted code at the link above is now shorter than 
the previous version because no extra code is needed to add the created 
submorphs to LayoutMorphs as the builder does that for you.

Granted though, you have to understand something about how the layout 
code works, which is an extra complexity. Right now, only "wrap" and 
"newline" are supported as layout options of what is outlined here (and 
not even that well):
https://www.miglayout.com/whitepaper.html

There is still an issue with how the Panel morph gets resized when the 
Kelvin field is added and removed. The show/hide button and input field 
also extend beyond the boundaries of the Panel sometimes, and the half 
of the button morph hanging over the panels' bottom edge sometimes is 
surprising not clickable. There is also another issue with how the other 
fields may have their height adjusted as the Kelvin field is added and 
removed. I can't help but think that layout issue might be easier to 
debug without all the layers of LayoutMorphs in the way -- but that may 
also just be me not understanding the layout approach well enough yet.

As far as efficiency and wasted computing, to get a definitive answer as 
to how big an issue it was, we would need to profile things. I have not 
yet looked into how to do that in Cuis and the CogSpur VM at a low level.

What I saw surprisingly though is that the window I made was responding 
to mouse moved events (not rejecting them), even though no code I added 
had explicitly listened for mouse move events. (This was also leading to 
a flickering issue as the Transcript flashed to the top every time it 
was written to on such events and then submerged again, which may be a 
bug or a feature arguably.) I could not figure out why such events we 
being processed by the window and not rejected -- and still don't know 
for sure. I though maybe at first there were hidden scrollbars listening 
for events? Now I think maybe it may have to do with the way the button 
color highlighting changes after you click and then move the mouse?

If most windows are already processing frequent mouse move events for 
whatever reasons, then there is already some CPU load there. In general, 
it seems to me that Morphic does a lot of querying of most morphs on the 
screen whenever an event happens, because Morphic is trying to determine 
who might want to process and event and does a lot of back-and-forth 
message sending between an event and nested morphs. If Morphic is 
already doing all that, then maybe some extra redrawing on top of all 
that won't make much of a difference?

For Dr. Geo for example, presumably there is a lot of redrawing going on 
when various geometric elements are moved? So, I can wonder for such a 
dynamic application how much more CPU would be needed if some extra UI 
elements were also redrawn? Again, we would need to profile to be sure.

And in a worst case, there are other techniques to reduce excessive 
drawing of complex displays, like by having multiple layers of bitmaps 
composed together. In one application I wrote about twenty-five years 
ago involving a constructivist application in a gardening microworld, I 
had about six layers of bitmaps to get the best UI responsiveness.

I started programming on a KIM-1 with 1K of memory (about a decade after 
the time the Dynabook concept was invented). Cheap computers like a 
Raspberry Pi now have about a million times more speed and memory than 
back then. That legacy still causes me to want to be frugal with CPU and 
memory -- and that is one other reason I like Cuis, that it could in 
theory run frugally on bare metal. But, at this point in my life, in my 
late 50s and with realistically only an all-too-limited number of 
reasonably effective programmer years ahead of me, the thing more 
valuable to me than CPU and memory -- given those changes -- is my time 
spent programming and how effective that time can be. And to me, I feel 
that the rebuild-on-event approach to UI development maximizes my 
productivity in that UI area compared to setting up and debugging 
observer dependencies.

Also, from another direction, generic Smalltalk code generally runs much 
slower than BitBlit code or other compiled plugin code (like presumably 
for the newer vector graphics rendering approach?). So, if (speculating) 
more drawing needed to be done at a low level with the rebuild approach 
but less Smalltalk code needs to run at a high level (because of less 
complexity with dependencies), then in theory there could be a CPU use 
reduction with a system that redraws more but manages dependencies less. 
I don't actually think that is likely here, just that it might be a 
plausible possibility. Again, without profiling it is hard to be sure 
given some uncertainty involving multiple code layers with different 
performance characteristics.

In general though, I agree that overlapping window pose a big challenge 
to the efficiency of the rebuilding approach compared to setting up 
dependencies explicitly. I outlined some possible ways for a 
rebuild-on-events approach to deal more efficiently with multiple 
windows in a previous email. But right now I am thinking it may be good 
enough tradeoff at the start if I just ignore that? Then maybe 
eventually someday we can find some way to transcend the tradeoffs and 
get the best results for both developer ergonomics and CPU efficiency? 
As an experiment, however, such results can't be guaranteed.

And surprisingly, when I looked around, it does seem like there are 
places Cuis already does a more general redraw like when there is a code 
change to a method and all browsers refresh? So that also suggests ad 
hoc solutions to refreshing other windows may be good enough?

Anyway, thanks again for your interest in this topic.

--Paul Fernhout (pdfernhout.net)
"The biggest challenge of the 21st century is the irony of technologies 
of abundance in the hands of those still thinking in terms of scarcity."

On 5/14/23 07:13, Hilaire Fernandes via Cuis-dev wrote:
> Sorry I don't have the reference at hand.  It is an accidental reading. 
> But I remember it is a way of doing in the web and from my understanding 
> to circumvent its inherent complexity in GUI. We don't have this legacy 
> complexity.
> 
> In most WEB GUI you don't have overlapping windows. With overlapping 
> windows, I fear rebuilding at each time may result in wasting a lot of 
> computing. Imagine a model represented visually with several different 
> visible forms (Morphs in fact), these morph embedded in more complex 
> Morphs. Will it required the whole redraw ?