[Cuis-dev] Rerendering when touched: Experiments in simplifying UI development in Cuis inspired by Mithril.js

[Mariano Montone]

Hi Paul,

      I'm late to this, hope you read this message.

I've tried a couple of GUI experiments, one that does Immediate Mode, 
and another that does Reactive updates.

Here are my IMGUI experiments: 
https://bitbucket.org/mmontone/mold/src/master/IMGUI.pck.st.
Instead of rebuilding morphs like in yours, there's no Morph tree 
maitained (that's why it is not a persistent GUI, and it is immediate). 
Everything happens in the drawOn: method, logic, rendering, rendering of 
subcomponents, etc.
I would qualify them as half-failed and a bit excessive, because I was 
trying to do some extremely difficult thing (for me, at least) with GUI, 
some structuring editing, and wanted to have a very dynamic funcional 
oriented gui (view = f (model)).
Unlike normal Immediate GUI frameworks, it doesn't redraw on every 
frame, but only when an event occurs.

My other experiment is a reactive GUI: 
https://bitbucket.org/mmontone/mold/src/master/ReactiveMorphs.pck.st.
Like your approach it rebuilds the morphs automatically, but there's 
some plugging involved, as it doesn't every submorph blindly, it 
rebuilds the affected parts of the gui based on what reactive variables 
changed.

The comparison in Javascript world would be SolidJs library, I believe.

This is an example of a TODO list, with add, remove and conditional 
updates to the UI:

todoListExample

     | todos todoList todosMorph textInput |

     todos :=​ RxCollection new.

     todosMorph :=​ LayoutMorph newColumn.

     todoList :=​ todos
         ifEmpty: [LabelMorph contents: 'Nothing TODO. Add below.']
         ifNotEmpty: [todos doCol: [:todo |            |todoItem|
             todoItem :=​ LayoutMorph newRow.
             todoItem addMorph: (    LabelMorph contents: todo).
             todoItem addMorph: (PluggableButtonMorph model:[todos 
remove: todo]
                                                 action: #value label: 'x').
             todoItem]].
     todoList color: Color lightBlue.
     todosMorph addMorph: todoList.

     textInput :=​ TextModelMorph withText: ''.
     textInput morphHeight: 30.
     todosMorph addMorph: textInput.

     todosMorph addMorph: (PluggableButtonMorph
                             model: [todos add: textInput text asString]
                             action: #value
                             label: 'Add todo').
     todosMorph openInWorld.

I attach a gif with a demo of the resulting Morph.

Note that the interface is completely specified in that method (there 
are no hidden parts) and is completely declarative, there are no manual 
morph manipulations after its construction. When the reactive variables 
change, the GUI is updates automatically.
I would call it ingenious, but less excessive than the IMGUI approach, 
and I think viable as an alternative Morph layer with enough effort.

Cheers,

Mariano


El 5/5/23 a las 00:53, Paul D. Fernhout via Cuis-dev escribió:
> Hi Cuis developers. First time Cuis list poster here -- someone who is 
> new to Cuis but old to Smalltalk and programming in general. Really 
> liking Cuis for the focus on simplicity. But one thing seems still too 
> complex, which is that you need to build Morphic UIs with an observer 
> pattern using special models instead of just rerendering the UI 
> anytime the UI is touched or otherwise "dirty". Attached is some demo 
> code with a Proof of Concept experiment of that (arguably) simpler 
> approach in Cuis. Below is an explanation of why that design pattern 
> may be of interest.
>
> --Paul Fernhout
>
> ==== More details
>
> Let's say you are building a UI in Cuis like, say, a window where you 
> can enter a temperature in Fahrenheit, click a button, and have the 
> value converted to Celsius. (I know, not great UX...)
>
> To do that now in Cuis, you would make a model (like one derived from 
> ActiveModel) and hook up dependencies (like using when:send:to:) so 
> that whenever the model containing those temperature values changes, 
> then any morph who is interested in that change is notified and can 
> update their display or sometimes their internal state.
>
> I started writing an example like that to learn more about Cuis using 
> two TextModelMorphs and a PluggableButtonMorph. And after getting to 
> the point where I needed to hook up all the dependencies, I thought, I 
> don't want to go back thirty years to the worse part of my ObjectWorks 
> and VisualWorks days (adding endless #changed messages or similar 
> everywhere for example, and some other observer/dependency challenges) 
> when I have been happier in recent years developing UIs using a 
> different design pattern.
>
> I know there may be some performance efficiency advantages to that 
> observer/model pattern. There may also be some UI extension benefits 
> like being able to hook up events to other Morphs like Dan Ingalls 
> demonstrates with Pronto or is used with VisualAge Smalltalk.
>
> But there is a lot of complexity there too involving defining special 
> model classes, establishing dependencies, and avoiding ringing where 
> one dependency triggers another that re-triggers the original 
> dependency. People may end up with complicated buildMorphicWindow 
> methods in Cuis -- and then, when you want to add a new part to your 
> window, you presumably usually need to close the window and open it 
> again to rerun that definition method to hook up all the dependencies 
> just right.
>
> One alternative to using a model and observer dependencies is, like in 
> many 3D graphics system, just re-rendering the entire UI frequently. 
> How frequently? For games the rendering may be done every display 
> refresh cycle, so like 60 times a second if possible (or more for 
> higher refresh screens).
>
> The advantage of rerendering frequently is that you can just draw the 
> UI based on the latest data available. That makes the UI much easier 
> to reason about because you only need to understand how a drawing 
> method takes available data and draws something. You don't have to try 
> to reason about dependencies (usually). Also, you can just modify the 
> drawing code (which can include defining self-contained components) 
> which on a redraw will update your entire UI (including new UI 
> elements if needed, depending). And best of all, there are no special 
> models needed. You just put your data anywhere in the image in no 
> special way, and your drawing algorithm just fetches it as needed.
>
> Of course, rerendering that many times sounds inefficient, especially 
> on constrained mobile devices. So, to optimize rerendering, instead of 
> brute-force high refresh rates, libraries like Mithril.js for 
> JavaScript instead more elegantly use an approach of considering some 
> widget on the screen as dirty and needing to be redrawn if it was 
> listening for an event that it received. (This is sort of like having 
> handlesMouseDown: return true for a morph in Cuis.)
>
> So, if a widget is listening for a mouse down, and it gets one, the 
> widget will rerender entirely after handling the mouse down. Same for 
> keystrokes or mouse move events. Network events and timers can also 
> potentially be hooked into this automatic redraw system, although 
> sometimes you just need to add a redraw call yourself for special 
> cases or to avoid modifying the base system too much. Also, you may 
> want to override the default behavior of automatic redrawing in 
> special cases like if you listen for keystrokes but only care about 
> one specific one and want to avoid redrawing on others.
>
> While that rerendering approach works OK usually in a web browser page 
> where your data usually only affects that window, admittedly it is 
> more problematical in a Smalltalk environment where in theory any 
> window can draw data from anywhere. Of course, that is what the 
> "Restore display" menu item is for in a worst case. But that issue is 
> an admitted weakness in this idea. How serious a weakness is still in 
> question.
>
> Mithril is where I first learned this redrawing-when-touched design 
> pattern, but a similar approach is also in Elm (which predates Mithril).
>
> For those familiar with some common JavaScript libraries, what Mithril 
> does is different than what React does. React essentially only 
> rerenders when you call a method to modify state data in a widget. In 
> the end that requires all sorts of complex plumbing like Redux to 
> share state across an application and only update widgets when the 
> special shared state changes. Essentially React in practice using 
> Redux or similar for a complex application end up being somewhat like 
> using an observer pattern with a special model (a Redux store).
>
> Angular is much closer to this rerendering approach, but rather than 
> using the elegant approach Mithril uses of hooking functions that 
> connect up UI events so they have an automatic redraw, Angular uses 
> more complex interventions in the heart of JavaScript with "Zones" 
> that trigger rerenders that makes such applications harder to debug 
> with endless extra low-level confusing layers to wade through for any 
> UI event handler.
>
> Attached is a simple demonstration of a Mithril-inspired rerendering 
> idea I made today in Cuis of a "KfMithrilInspiredExperimentMorph". You 
> can click on some areas of a BoxedMorph subclass to enter a 
> temperature and to convert it. As the class comment shows, use 
> "KfMithrilInspiredExperimentMorph new openInWorld" to start it.
>
> Right now, rather than a button to do the conversion, an area of the 
> screen occupied by some text "CONVERT" is used to trigger the 
> conversion. Ideally there would be some way to bridge between existing 
> Cuis morphs for buttons and text editors and this re-rendering 
> approach (at least at first).
>
> To get a sense of how this development process feels, try modifying 
> the drawOn: method in some way and clicking on the morph to force a 
> redraw. Of course, once you could include other re-rendering Morphs 
> like a button or text area in the drawOn: method things would get more 
> interesting, but this proof of concept is not there yet.
>
> You could also make changes to the mouseButton1Down:localPosition: 
> method -- where unfortunately approximate bounding boxes are currently 
> hardcoded for areas with text. Ideally those would be defined as 
> nested components somehow. Mithril works by using existing HMTL DOM 
> elements, so in theory wrapping existing Cuis morphs at first should 
> be feasible (although Mithril uses a VDOM approach to do that, which 
> it would be best to avoid in Cuis).
>
> Hoping to spark some ideas and more experiments including by others 
> towards further simplifications of Cuis along these lines -- at least 
> as an alternative at first for UI building. Someone who already knows 
> Cuis well might be able to take this rerendering-on-touched design 
> pattern much further much faster than I can. Then maybe if those 
> experiments prove the concept then the core Cuis browsers and so on 
> maybe could be rewritten this way to increase the simplicity of the 
> core system (at least, simplicity from one perspective, even if it 
> might use more CPU cycles sometimes perhaps, to be evaluated).
>
> Of course, this idea might not work out in practice. It's an 
> experiment after all. Maybe such a transformation of Cuis to this 
> design pattern and its quantitative evaluation might even make an 
> interesting academic thesis project for someone?
>
> Don't know of any modern Smalltalks that use this design pattern, 
> although would like to hear of one if such a thing existed already. 
> Perhaps the earliest Smalltalks might have been a bit closer to this 
> approach by using polling for events which could lead to a redraw? But 
> that is less efficient and has other issues than being purely 
> event-driven.
>
> My thanks go to Juan and everyone else here for remaking Squeak in a 
> simpler way to become Cuis -- including because it makes experiments 
> like this one more feasible.
>
> --Paul Fernhout (pdfernhout.net)
> "The biggest challenge of the 21st century is the irony of 
> technologies of abundance in the hands of those still thinking in 
> terms of scarcity."
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ReactiveMorphs.gif
Type: image/gif
Size: 47678 bytes
Desc: not available
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20230529/bce54191/attachment-0001.gif>