[Cuis-dev] Rebuilding on events: Experiments in simplifying UI development in Cuis inspired by Mithril.js

[Paul D. Fernhout]

Thanks for the reply, Hilaire. Some comments below inline.

On 5/6/23 09:42, Hilaire Fernandes via Cuis-dev wrote:> Coincidentally, 
a few weeks ago, I was reading about model-less GUI, not
> any particular implementation like the ones you referred in your email, 
> but about the general idea to re-construct the whole or part of the GUI 
> when an event occurs.

Would be curious to read the "model-less GUI" article you mention if you 
could supply a link.

I updated the subject of this email to reflect that more general idea of 
rebuilding the UI in response to events (which indeed is what Mithril does).

> It is true that it is not obvious to track the source of events with the 
> observer pattern: triggered in one place, hooked in another ones. The 
> tools to help the developer comprehension of events is not really there. 
> The fact that it is based on symbol and not object does not help. I 
> guess that what we all do is to search for symbol in the source code, 
> i.e. #selected.

Agreed that the use of symbols can make things harder depending on the 
convenient browser menu options.

In the 1990s I made a change to VisualWorks base code (locally to a 
company with ENVY) so that asking for senders of a method also found 
methods that included the same symbol. That was to support a 
specification-driven UI system I wrote where rather large methods 
defined how to hookup UI widget dependencies for forms -- but where it 
could be hard to find those methods when browsing senders of the methods 
they used via symbols.

It is great that in Cuis you can select a symbol in the browser (without 
the leading #) and then search on senders of it or implementors of it 
with the popup Smalltalk editor menu. It did take me a while to notice 
that menu option though. Whereas I had at first thought that feature was 
unavailable and that I would need to do the VisualWorks kludge again 
because the obvious buttons in the Browser for "senders" and 
"implementors" did not do that.

> Regarding the model, Morph itself is very neutral regarding the 
> necessity to have one. Pluggable morph requires one but many others 
> embed it (TextParagraphMorph, TextEnty Morph). And when required, there 
> are generic models (ValueHolder, ListModel, ...).

Thanks for the pointer to the non-plugable morphs.

It's true there are generic models like you mentioned. But because they 
inherit from Object which defines the dependency infrastructure for the 
observer/dependency pattern, it is hard to know at a glance how 
decoupled those models are from observer/dependency use. For example, 
the PluggableListMorph (which uses a ListModel) defines an update: 
method, and also sets a model: (which establishes a dependency). So it 
is not clear to me how much in practice those more generic models are 
being used outside of the observer/dependency pattern in Cuis. But that 
may just be showing my current ignorance of Cuis details.

I made a third version of the Proof of Concept that included a 
subcomponent to edit the Fahrenheight temperature that is a 
TextModelMorph --- but that is in the PluggableMorph hierarchy.

I ended up having a method triggered by a button press event pull the 
value out of the edit rather than having the edit put changed text into 
an instance variable as the editor makes changes. That method also need 
to call "self redrawNeeded" because the mouse click that triggers that 
button press by itself does not trigger the redraw.

No doubt I could add some more wrapper code or such hopefully to make 
the TextModelMorph push changes somewhere though. But even then I will 
probably need to make additional changes to trigger the redraw.

I'm starting to think that ultimately, were this idea to move forward to 
its ultimate extent, it will end up involving grabbing bits and pieces 
from existing Cuis classes to build essentially a parallel class 
hierarchy that does not use dependencies (and which runs while the 
existing Cuis development tools also run). The goal would be to reach a 
point where stuff like the actionMap instance variable and the change 
method can be removed from Object and all the (parallel) development 
tools in Cuis would still work. I would suggest Cuis would be "simpler" 
then while still delivering the same functionality. (Others might 
disagree based on the CPU use efficiency point or depending what other 
kludges are needed to keep multiple morphs in sync.)

Of course, such a change to Object would break essentially all existing 
Cuis applications and UI tools. But there could always be some sort of 
changeset people could file in if they still wanted legacy dependency 
support.

> *A few points of thinking.*

Thanks! :-)

> It is possible to hook programmatically mouse/keyboard events to a 
> Morph. In your example the handlesMouseDown: and mouseButtons1xxx can be 
> set programmatically, so it avoid subclassing. It may help to compose 
> the GUI with sub morphs, using sub morphs improves the efficiency. In 
> your example the whole rectangle  of the GUI is rebuild even if only a 
> small part needs to be updates.

Thanks for the suggestion to programmatically hook up mouse/keyboard 
events to a Morph. Perhaps that may help with setting up a redraw after 
events.

Indeed, more will likely be redrawn that strictly needed. That is a 
tradeoff of CPU vs. human thought. But I am suggesting in practice it is 
a good tradeoff as it reduces code complexity the programmer needs to 
think about.

And I might also suggest that so much redrawing goes on so often that it 
might not be that significant of an increase, especially with some minor 
optimizations mentioned below.

But, this tradeoff ideally should be quantified. Personally I am less 
interested in doing that quantification myself than in just being able 
to write UIs that are simpler to implement and maintain. But 
academically, quantification of before and after for CPU use and memory 
use and lines of code expended and so on might make an interesting project.

> I am wondering if the simplicity you get from just rebuilding GUI comes 
> with a hidden cost.

I agree that, as Frederick Brooks wrote about in "No Silver Bullet -- 
Essence and Accident in Software Engineering" that systems have a 
certain level of "essential complexity" -- complexity that you can at 
best push around somewhere else. And that relates to "hidden costs" you 
bring up if this different approach just makes something else 
significantly worse (which it might). The deep question here is whether 
the observer pattern in Smalltalk may instead be "accidental complexity" 
(as Brooks also mentions) for UI building that is unneeded these days?

And that is why it eventually might make sense to quantify CPU cycles 
used or other measures to see how many resources if any the observer 
pattern really saves over a somewhat broader redraw approach these days. 
And I say "these days" because needs and implementations have changed 
since the 1980s (or earlier) when this Observer pattern got started. It 
might truly have been the best or only feasible choice back then. Now 
that computers are 1000s of time faster, it may not matter much in most 
cases. And when it does matter, those special cases can perhaps be 
optimized individually.

It is also fair to debate whether code that does more computation but is 
easier for a programmer to understand or change is "simpler". Rich 
Hickey (mentioned in some Cuis docs) argues that "easier" things aren't 
necessarily "simpler". He says easier things are just usually more 
familiar and "at hand" mentally (even if they may be more complex). But 
I think Rich Hickey would also say easier things can also be simpler as 
long as the underlying code or processing is also simpler.

Related by Rich Hickey on "complect" versus "compose":
https://www.infoq.com/presentations/Simple-Made-Easy/
https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/SimpleMadeEasy-mostly-text.md
"So "complect" actually means to braid together. And "compose" means to 
place together. And we know that. Everybody keeps telling us. What we 
want to do is make composable systems. We just want to place things 
together, which is great, and I think there is no disagreement. 
Composing simple components, simple in that same respect, is the way we 
write robust software."

As a key question, is the redraw-on-events approach that Mithril uses 
more composable for UI development than the (complected?) 
observer/dependency pattern that Cuis uses?

In this case, I think redrawing entirely when a click (or any event 
listened for) happens would meet that "simpler" definition by being not 
"complected" between model and UI, which are essentially "composed" 
side-by-side. Whereas an observer pattern with dependencies and changed 
messages is "complected" because it tightly links ("braids together") 
references to multiple objects pointing to each other and involves a lot 
more message traffic (usually bidirectionally) between them (compared to 
than a simple request by drawing code for a current value of a model).

But perhaps people might argue that, since the UI ultimately still 
depends on the model in the drawing code, just without the push approach 
of change messages. So there is a dependency between the drawing code 
and the model. It is just an implicit dependency defined by how the 
drawing code works -- instead of, say, explicit dependency defined by 
setting up responses to changed messages in a separate initialization 
method somewhere.

But because the implicit dependencies in this event-driven-rebuilding 
approach are also defined in the same method that defines the UI, I 
would argue, from a cognitive burden perspective, it is less cognitive 
overhead to reason about the behavior of one draw method than to reason 
simultaneously about both a draw (or update) method an an initialization 
method which are "complected". And if you refactor larger methods that 
draw/build the UI into smaller methods, I would still suggest those 
smaller methods each defining part of the UI are composed side-by-side 
more than "complected".

But maybe  at this point  I am just so used to the newer way of defining 
UIs offered by Mithril so it seems simpler but is not? Especially when 
considering the drawbacks -- in a Cuis environment -- of the challenge 
of knowing what windows need to be redrawn when instance variables 
change. As outlined above with a list below of options for dealing with 
that, they all introduce a bit more complexity to what a user does with 
in UI actions or in coding.

And yes, redrawing more than is strictly needed reduces one sort of 
efficiency as a "hidden cost".

As I mentioned in my original post, there is a challenge for using this 
idea compared to Mithril/JavaScript because there is an assumption in 
Mithril that you only need to refresh one browser page at most when an 
event happens, not every browser page. But in Cuis, there is no obvious 
similar distinction between Cuis windows (unless one is imposed).

This is probably not a big deal for rerendering every Cuis window when a 
mouse button is clicked (assuming typical windows that redraw nearly 
instantaneously). A mouse click presumably happens at most once every 
few seconds. The same is true for a redraw initiated by a typical timer 
or typical network event that also might fire every few seconds.

But it is a much bigger CPU load if a morph (or window) responds to key 
presses which might happen a few times a second. Even worse is if a 
morph responds to mouse move events or mouse scroll events as there 
might be dozens of those a second.

As many games demonstrate, modern CPUs (with GPU support) can indeed 
often keep up with 60 to 120 frames per second of rendering. But that 
takes a lot of power and computers tend to run hot then. So it would be 
better to avoid having to redraw everything so frequently.

Here is a typical example of the need to redraw multiple windows 
frequently. Let's say you make a morph that draws something inside its 
bounds under the x and y coordinates of the mouse as the user moves the 
mouse over the morph. The morph stores those mouseX and mouseY values in 
instance variables when a mouse move event happens and used that 
information in its drawing method (triggered from the event). So, this 
morph is redrawing every time the mouse moves. This morph would be a 
window or be in a window, with the window defining the boundary of how 
much is redrawn when a morph thinks it is dirty (everything in the 
window, which presumably may all be using data in closely related 
models). Then you open some sort of companion inspector morph on that 
other morph which displays the mouseX and mouseY values as text. The 
inspector morph really should update itself whenever those mouseX and 
mouseY values in the first morph change, which can happen every time the 
mouse moves inside that first morph's boundaries. But how will the 
second morph know to redraw itself because the first morph redrew itself 
because of a mouse event only the first morph was listening for?

This same issue will arise as with the Cuis DependencyExamples with 
shared data across multiple windows about a list of items and which item 
is selected. But in practice that example only needs to redraw when an 
item is clicked, so even if the entire display is redraw every few 
seconds in response to a click event, that is probably not a big deal in 
extra CPU use.

I've been thinking on this a bit, and here are a few possibilities to 
keep secondary morphs in sync when they are inspecting objects/models 
that other morphs are modifying:

1. Entirely ignore the problem and require the user to do "restore 
display" as needed.

2. Have an update button on the secondary inspector morph.

3. Redraw the inspector morph anytime it is clicked anywhere.

4. Have the inspector morph redraw frequently using a timer.

5. As an optimization building on #4, the inspector morph could cache a 
copy of the values it was inspecting and only redraw when they have 
changed when a timer fires. This caching could be generalized somehow so 
any morph might use this approach, like perhaps at the beginning of the 
drawOn: method there could be code that caches all the values needed for 
the drawing. This could get ugly if drawing required a lot of data and 
adds to the complexity. But for an inspector on only a couple of values, 
this approach might not be that ugly.

6. Establish a dependency between the second morph and the first morph 
for redraw events. This starts to recreate the complexity of the 
observer pattern and a push model of updates sent from a model to an 
observer, but to a lesser extent because it only happens on a redraw. 
So, there is not much for the user to think about in setting this up and 
not as much to go wrong as with the regular observer pattern. This would 
require redraw code not to change data that might trigger other redraws 
in the first morph (to avoid "ringing"), but it is a best practice 
anyway not to modify data in a drawing method (other than maybe to cache 
a display-specific calculation result).

7. Have the redraw event in the first morph trigger a hierarchical 
cascade of messages to all morphs visible on the screen. Morphs could 
decide whether to redraw based on some additional information in those 
messages like the morph initiating the redrawing. The second inspector 
morph would implement a method that redraws itself if it sees such a 
message with a reference to the morph it is inspecting. This may seem 
like an observer pattern -- except that, unlike ActiveModel, there is no 
knowledge in the first morph about what other morphs specifically are 
observing it. I think it is less complex in that sense than the usual 
observer/dependency pattern.

8. Essentially the same as #7, but with the extra information being an 
id or other abstract reference to some multi-window concept representing 
a set of morphs that all want to redraw together. Essentially, this 
shared concept becomes similar to the idea of a browser page in 
Mithril/JavaScript. Redraws then are restricted to only groups of 
cooperating morphs that may all redraw when any morph in the group redraws.

There might be other options as well. On a practical basis, I think 
option #4 (using a timer to redraw in any inspector) might be 
satisfactory in most cases for inspectors (at least at first). Using 
caching of inspected values as in #5 could then be used in the rare 
cases there was any performance issues. So, this approach pushes a bit 
of the complexity into inspectors. But since morphs already have support 
for handling timer events, I don't see the #4 option as being that bad.

But ultimately something like #8 might make the most sense, where 
windows of morphs somehow agree they are in the same group and they will 
all redraw together. And where other windows not in that group don't 
redraw as an optimization.

With #8 implemented this way, an inspector on a morph would somehow have 
to add itself to the window group though. And arguably this introduces a 
new form of observer pattern, even if it is less specific than what is 
now used. But establishing this dependency between windows is an example 
of the "hidden cost" you suggested -- even if arguably it is a lesser 
cost in cognitive burden than creating a Cuis UI now. It has been said 
that that"efficiency is where elegance goes to die" -- and the question 
is how much that applies to the rebuilding on events idea in Cuis? And I 
still don't fully understand the tradeoffs there, this being an experiment.

> Concerning model, don't you need one at some point to represent the 
> state of your application ? So most the time the model is not a bargain 
> as it already exists. Ok may be not for your converter example.
Regarding a model, yes all these UIs require a model in the sense of 
needing data stored somewhere related to state.

The issue is whether that model is just plain Smalltalk objects or 
whether the data needs to be held in objects that send #changed messages 
or similar every time methods are called with new data. And whether you 
then need to explicitly hook up that dependency with #when:send:to: or 
whether you instead can just have an implicit dependency in the draw 
code that reaches out to other objects to query their current state.

In that JavaScript example above, this is the model (just a plain 
JavaScript object):

     const model = {
	amount: "100.00",
	currency: "dollars",
	precision: "2",
	currencyError: false,
	precisionError: false
     }

In the Cuis Proof of Concept Fahrenheit to Celsius converter example, 
the "model" is essentially just the two instance variables temperatureF 
and temperatureC. Plus, the "model" includes arguably the code for 
directly changing or transforming those values, which conceptually could 
be in a dedicated model object with its own class that the convertor 
morph could hold on to, and which other morphs might also reference 
(like an inspector morph or a charting morph or a sound-generating morph 
or whatever).

Although as above, that POC is complicated by the morph subcomponent 
added in the third version for text editing also having its own state 
(both the text and an insertion point).

In general, any subcomponent may arbitrarily have its own local state -- 
or through some sort of plugability may retrieve state from elsewhere 
(typically an enclosing component, but not necessarily).

> In the Cuis-Smalltalk-UI, there is a UI-Mold package to describe field 
> input form, with data validation. It should be possible to add a kind of 
> computed ouput object.

Thanks for mentioning Cuis-Smalltalk-UI and UI-Mold. I just imported 
that and plan to poke around in it. It definitely shows the initial 
value of having a wrapping system in any new approach to leverage all 
those existing Cuis morphs.
https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-UI

Relating to form layout, one thing missing from the Proof of Concept is 
having some sort of layout manager in the "drawOn:" method.

While the quickest way to make a POC was to override drawOn:, a better 
approach might be to have a new "view" method. In practice with Mithril 
the "view" methods define a nested set of HTML DOM elements and also 
custom components.

I'd ideally like to be able to define Cuis UIs in a way similar to this 
example where I create a nested structure using arrays that defines UI 
elements that are then laid out automatically:
https://github.com/pdfernhout/mithril-demos/blob/master/currency-converter/currencyConverterFew.js#L72-L91

So ultimately, I would like something more elegant than that Proof of 
Concept drawOn: approach. For a form, you would essentially just create 
an array or tree of all the widgets (text, editor areas, buttons, and 
all the specialized variants for form input like in Cuis-Smalltalk-UI) 
and where a layout manager you specified handles the details of 
placement (similar to what happens in an HTML window) considering 
various constraints.

There are lots of interesting layout managers in various languages that 
Cuis could include -- and might already, as I have not looked for any 
beyond the row and column layouts. I was fond of migLayout in Java for a 
time, although with JavaScript I increasingly use the CSS Flexbox layout 
approach. But improving layout managers is a different issue mostly 
independent of event-driven rebuilding.

To be clear, most JavaScript developers don't know this pattern that 
Mithril uses, and so they don't know how much accidental complexity they 
are using when they build on, say, React with Redux. Here is a related 
essay I wrote on that, explaining why Angular and React have unnecessary 
complexity, which has a section on "Redraw approach" and "Data model: 
Component & Services vs. Component or Redux vs. Flexible" which are 
similar to the points I have made here about the observer/dependency 
accidental complexity in Smalltalk: 
https://github.com/pdfernhout/choose-mithril

I know it is a bit rude to come into a community and suggest a 
fundamental part of a community's development paradigm could change. 
Sorry about that. I do admire Cuis' quest for simplicity though -- and 
all the related previous work on that -- which is why I am suggesting 
these ideas here and not over on the Pharo or Squeak lists. And of 
course I love the Squeak (and now Cuis) ideal of a self-reflective 
self-generating system with great tools for development which now can 
potentially run in any browser like via SqueakJS and also potentially 
even run on bare metal without an OS at all. Having been away from 
Smalltalk development for many years while journeying through 
JavaScript/HTML/CSS land with all its diversity (discovering Mithril's 
rebuild-on-events design pattern along the way), I feel that there is an 
even more awesomely simpler Cuis possible if it discards the 
observer/dependency complexity that make UI development in modern 
Smalltalks more complected -- and more difficult -- than it has to be.

--Paul Fernhout (pdfernhout.net)
"The biggest challenge of the 21st century is the irony of technologies 
of abundance in the hands of those still thinking in terms of scarcity."