[Cuis-dev] StyledTextEditor PR & InnerTextMorph Was: Non-ASCII characters in source code files

[Bernhard Pieber]

Hi Juan,

I updated the PR for StyledTextEditor to fix SUnit tests. I fixed 18 errors and am now down to just 1 failure and 7 errors out of 56 tests. :-)

For the tests to work I needed an additional accessor in InnerTextMorph to explicitly set its editor. I could have done it as an extension in StyledTextEditor but thought it fits better in the base image. See attachment.

The method is currently only needed in setup code in StyledTextEditor SUnit tests. I need to set the editor explicitly because InnerTextMorph>>#hasUnacceptedEdits: now calls convertAndStyleIfNeeded which assumes InnerTextMorph has an editor.

Some design questions arose:
Should convertAndStyle be used for a StyledTextEditor? As it should be able to contain Smalltalk code I would say it should. What do you think?

Should InnerTextMorph always have an editor, or are there use cases where it does not need & should not have one?

Right now InnerTextMorph is responsible for setting up its own editor in InnerTextMorph<<#installEditorAndTextComposition. Is it ok to set it from the outside, e.g. for test setup, or should we try to avoid that?

Cheers,
Bernhard


------- Original Message -------
Bernhard Pieber via Cuis-dev <cuis-dev at lists.cuis.st> schrieb am Donnerstag, 14. September 2023 um 22:19:


> 
> 
> Hi Juan,
> 
> Thanks for the clarification. I can now understand the changes in String>>#iso8859s15ToRTFEncoding in 0b58898 and in StyleSet>>#createFeaturesParagraphStyleSet and StyleSet>>#createSampleParagraphStyleSet in 57ce260. It is really great how well this works!
> 
> 
> The changes in RTFConversionTest>>#textSample6 in c1d67b5 and in StyledTextTest>>#testOldInstanceDeserialization in 57ce260 look dubious to me, though.
> 
> 
> I decided to create the PR for StyledTextEditor anyway:
> https://github.com/Cuis-Smalltalk/StyledTextEditor/pull/7
> 
> With these changes STE can be loaded again. There are no more test failures and errors than before the font improvement. Feel free to improve my code. ;-)
> 
> Cheers,
> Bernhard
> 
> 
> ------- Original Message -------
> Juan Vuletich via Cuis-dev cuis-dev at lists.cuis.st schrieb am Donnerstag, 14. September 2023 um 19:38:
> 
> 
> 
> > Hi Bernhard,
> > 
> > On 9/14/2023 10:16 AM, Bernhard Pieber via Cuis-dev wrote:
> > 
> > > Hi Juan,
> > > 
> > > Thanks for the explanation. Did I understand correctly that old ISO-8859-15-encoded source files should be converted to UTF-8 before they are loaded in Unicode-enabled Cuis? Or does Cuis somehow do that automatically if possible?
> > 
> > Almost all the code is pure ASCII. Nothing to be done here.
> > 
> > For non-ASCII parts, usually they are "invalid UTF-8", they can't be
> > mistaken for UTF-8 content. Cuis converts these on the fly to the new
> > UnicodeString objects. Then, when you save, the new file is UTF-8. So,
> > the conversion is done automatically. There is a very small risk of some
> > old non-ASCII stuff to be mistaken for UTF-8, leading to wrong code. But
> > it is really small. A bit of checking should be enough.
> > 
> > > IIUC the files on https://github.com/Cuis-Smalltalk/StyledTextEditor should have been converted to UTF-8 already. If yes, I still don't understand why the string literal in RTFExporting.pck.st was changed by resaving the package file? (I did not use my repo for the test.)
> > 
> > That's what I described now above. Auto convert the instances on load,
> > then save in UTF-8 format.
> > 
> > > How did you do the conversion? Did you use some external tool? (I could not find any code for this in Cuis except from UnicodeString>>#fromBytesStream: but there are no senders.)
> > 
> > No. Just loaded, saved, and checked that everything looked ok. It did.
> > For instance, #nextUtf8BytesAndCodePointInto:into: ends calling
> > #utf8BytesAndCodePointFor:byte2:byte3:byte4:into:into: . Check the
> > comments in these methods. I had hoped these would be informative enough.
> > 
> > > Regarding the method iso8859s15ToRTFEncoding, I am pretty sure this is the correct string from the comment (Test for Cent and Euro characters):
> > > self assert: 'A¢€' iso8859s15ToRTFEncoding = 'A\u162?\u8364?'
> > 
> > Cool.
> > 
> > > Instead of #iso8859s15ToRTFEncoding a new method #toRTFEncoding or #asRTF polymorphic to String and UnicodeString is probably needed, right?
> > 
> > Yes. That sounds like a good idea. Still, I'd check recent RTF
> > documentation. I'd be really surprised if they don't handle UTF-8
> > encoding as part of the standard. If the do, maybe all that can simply
> > be removed, and just replaced with UTF-8 stuff.
> > 
> > > Cheers,
> > > Bernhard
> > 
> > Cheers,
> > 
> > --
> > Juan Vuletich
> > cuis.st
> > github.com/jvuletich
> > researchgate.net/profile/Juan-Vuletich
> > independent.academia.edu/JuanVuletich
> > patents.justia.com/inventor/juan-manuel-vuletich
> > linkedin.com/in/juan-vuletich-75611b3
> > twitter.com/JuanVuletich
> > 
> > --
> > Cuis-dev mailing list
> > Cuis-dev at lists.cuis.st
> > https://lists.cuis.st/mailman/listinfo/cuis-dev
> 
> --
> Cuis-dev mailing list
> Cuis-dev at lists.cuis.st
> https://lists.cuis.st/mailman/listinfo/cuis-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 6032-CuisCore-BernhardPieber-2023Sep17-15h15m-bp.001.cs.st
Type: application/vnd.sailingtracker.track
Size: 187 bytes
Desc: not available
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20230917/945717b1/attachment-0001.bin>