[Cuis-dev] TrueType font import problems

Thu Sep 14 06:16:56 PDT 2023

Hi Guys,

this removal of TextFontFamilyAndSize breaks loading persistent Text objects
that use its instances as a text attribute. I encountered this kind of 
problem
with my ToDo-application that stores texts in my PlanF database,
but I presume it will also happen when one
uses a ReferenceStream for persistence.

As a stopgap measure I exported TextFontFamilyAndSize from
an old image and added it directly t by database package.
I seems to work (but it is ugly).

In a nutshell:
Some kind of objects, like strings, byte arrays, texts and their
associated classes, are likely to be persistent in some way.
These classes should not be removed
without providing a backward-compatibility package.

In other words:
There is not much sense in providing
means for persistence, like ReferenceStream, in the base
image and than break this very persistence by removing classes.
(Sorry for my harsh words).

Just my 0.01€,

Gerald

On 9/14/23 1:33 PM, Juan Vuletich via Cuis-dev wrote:
> Hi Bernhard,
>
> We have two unrelated issues.
>
> 1)- Yes, I removed TextFontFamilyAndSize. When I fixed loading of 
> Optima.tcc I reviewed font handling in general, and I thought it would 
> be more comfortable to users to control Font ans Size as two separate 
> attributes. So now we have TextFontFamily and TextFontSize. I fixed 
> the editors and any tests for the base image. I forgot about 
> StyledTextEditor. Apologies. Any code that was aware of 
> TextFontFamilyAndSize now needs to be aware of TextFontFamily and 
> TextFontSize. Changes are relatively straightforward.
>
> 2)- Non-ASCII characters in source code files. Until about one year 
> ago, Cuis didn't care much about Unicode. Files were saved in 
> ISO-8859-15 encoding. This means ASCII [0..127] and a slight 
> modification of Latin-1 [160..255], and some hand picked Math symbols 
> in [128..159]. Around one year ago I added full support for Unicode in 
> Cuis. This also meant converting all files to UTF-8 encoding, where 
> non-ASCII characters use more than one byte. There are very few places 
> where this makes a difference, and you found a few in those tests. I'm 
> pretty sure I was careful enough to do those conversions correctly, 
> including StyledTextEditor. So, I think that whatever is at 
> https://github.com/Cuis-Smalltalk/StyledTextEditor is correct. Maybe 
> you'd need to update your fork. In any case, a review for correctness 
> is more than welcome.
>
> Hope this helps.
>
> Thanks,
>
>
> On 9/14/2023 6:38 AM, Bernhard Pieber via Cuis-dev wrote:
>> Hi Juan,
>>
>> Thanks for the fix. It works as expected.
>>
>> I found the fonts improvement broke loading StyledTextEditor, though, 
>> because it still references TextFontFamilyAndSize which was removed. 
>> I guess this might also affect other packages?
>>
>> I prepared a branch which fixes it without breaking additional tests:
>> https://github.com/bpieber/Cuis-Smalltalk-StyledTextEditor/tree/fix-STE
>>
>> I wanted to create a PR but then saw that there are some changes I 
>> did not do on purpose. You can see an example here in line 10:
>> https://github.com/bpieber/Cuis-Smalltalk-StyledTextEditor/commit/0b58898b75167454db54298db0803cd2c75c4156 
>>
>>
>> This is how it is shown in SourceTree:
>>   !String methodsFor: '*rtfExporting' stamp: 'jmv 9/5/2016 20:27:00'!
>>   iso8859s15ToRTFEncoding
>>       "Convert the given string to RTF escaped Unicode from the 
>> internal encoding: ISO Latin 9 (ISO 8859-15)"
>>       "
>> -    self assert: ('A¢§' iso8859s15ToRTFEncoding) hex = ' 
>> 'A\u162?\u8364?''
>> +    self assert: ('A¬¢‚Ç¨' iso8859s15ToRTFEncoding) hex = ' 
>> 'A\u162?\u8364?''
>>       "
>>
>> In the terminal it looks still a bit different:
>> git show 0b58898 -- RTFExporting.pck.st
>>
>>   !String methodsFor: '*rtfExporting' stamp: 'jmv 9/5/2016 20:27:00'!
>>   iso8859s15ToRTFEncoding
>>          "Convert the given string to RTF escaped Unicode from the 
>> internal encoding: ISO Latin 9 (ISO 8859-15)"
>>          "
>> -       self assert: ('A<A2><A4>' iso8859s15ToRTFEncoding) hex = ' 
>> 'A\u162?\u8364?''
>> +       self assert: ('A¢€' iso8859s15ToRTFEncoding) hex = ' 
>> 'A\u162?\u8364?''
>>
>> I could reproduce the behaviour like this:
>> 1. Start with the latest image and install updates.
>> 2. Checkout the master branch of StyledTextEditor.
>> 3. Temporaryly recreate class TextFontFamilyAndSize.
>> 4. Feature require: 'StyledText'.
>> 5. Open Installed Packages.
>> 6. Click on RTFExporting.
>> 7. Click save.
>>
>> I think the problem already happened in step 4 when Cuis was reading 
>> the package file. Saving in step 7 makes it apparent.
>>
>> This type of behaviour happened in other places as well. See line 710 
>> here:
>> https://github.com/bpieber/Cuis-Smalltalk-StyledTextEditor/commit/c1d67b56ffb707e189d144b696c31ee8f3d2bf21#diff-fc4d98b72cc240e23f1fbf7486239a158bcb860548588a80825fe6abf3f0b6a0 
>>
>>
>> And I am afraid it also happened in past changes which are already in 
>> master. See e.g. line 853 here:
>> https://github.com/bpieber/Cuis-Smalltalk-StyledTextEditor/commit/57ce26012da883def313cbf005a6ac60081545e8#diff-ef2065cf0d5ae47c54dfc101485dcdd7a4f02c4a91eb164f6cfa4e1cdc2e442c 
>>
>>
>>
>> Why does this happen? Are the changes correct or erroneous?
>>
>> Cheers,
>> Bernhard
>>
>>
>> ------- Original Message -------
>> Juan Vuletich via Cuis-dev<cuis-dev at lists.cuis.st> schrieb am 
>> Dienstag, 12. September 2023 um 16:20:
>>
>>
>>>
>>> Fix now at GitHub.
>>>
>>> Thanks!
>>>
>>> On 9/11/2023 5:07 PM, Bernhard Pieber via Cuis-dev wrote:
>>>
>>>> Hi Juan,
>>>>
>>>> Thanks for improving this. I had problems getting it to work, though.
>>>>
>>>> At first I tried this:
>>>> TrueTypeFontFamily readTrueTypeFontEntry: '/System/Library/Fonts' 
>>>> asDirectoryEntry // 'Optima.ttc'.
>>>>
>>>> However, this method still has the old behavior: Optima ExtraBlack 
>>>> replaces Optima Regular as the baseTTFontDescription.
>>>>
>>>> After some digging I found I needed to copy Optima.ttc to its own 
>>>> directory and use this method instead:
>>>> TrueTypeFontFamily readAllTrueTypeFontsIn: DirectoryEntry 
>>>> smalltalkImageDirectory / 'TrueTypeFonts' / 'Optima'.
>>>>
>>>> This works as you described. Now I two families Optima and 
>>>> Optima-ExtraBlack.
>>>>
>>>> The method TrueTypeFontFamily<<#readTrueTypeFontEntry: is used when 
>>>> installing a font file from a file list, so it should probably be 
>>>> fixed as well.
>>>>
>>>> Cheers,
>>>> Bernhard
>>>>
>>>> ------- Original Message -------
>>>> Juan Vuletichjuan at cuis.st schrieb am Mittwoch, 30. August 2023 um 
>>>> 01:00:
>>>>
>>>>> Hi Bernhard,
>>>>>
>>>>> I pushed your fix to support V2 headers.
>>>>>
>>>>> But I also did a few more things. When importing fonts like Optima,
>>>>> ExtraBlack and other styles beyond regular / italic / bold / bold 
>>>>> italic
>>>>> will be turned into different FontFamilies.
>>>>>
>>>>> I also changed the handling of fonts and sizes in TextEditors quite a
>>>>> bit. I think it is nicer now.
>>>>>
>>>>> Please pull, update, and test.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> On 8/27/2023 5:23 AM, Bernhard Pieber via Cuis-dev wrote:
>>>>>
>>>>>> Here is a slightly improved version.
>>>>>>
>>>>>>> Am 26.08.2023 um 20:59 schrieb Bernhard Pieber via 
>>>>>>> Cuis-devcuis-dev at lists.cuis.st:
>>>>>>>
>>>>>>> Hi everyone,
>>>>>>>
>>>>>>> I tried importing a TrueType font I had on my system 
>>>>>>> (Optima.ttc). I copied the file to a new folder named Fonts and 
>>>>>>> used the following:
>>>>>>> TrueTypeFontFamily readAllTrueTypeFontsIn: DirectoryEntry 
>>>>>>> currentDirectory / 'Fonts'
>>>>>>>
>>>>>>> This resulted in the error "This TTF Collection has an 
>>>>>>> unsupported header version“ in TTFontReader>>#readTTCFontFrom:.
>>>>>>>
>>>>>>> The reason is that the font uses a TTC Header Version 2.0, see:
>>>>>>> https://docs.fileformat.com/font/ttc/
>>>>>>>
>>>>>>> I changed the code to allow both header versions. If I 
>>>>>>> understand the difference between version 1 and 2 of the header 
>>>>>>> formats correctly, this should be a safe change. Installing the 
>>>>>>> font worked with this change.
>>>>>>>
>>>>>>> However, instead of Optima Regular it used Optima ExtraBlack 
>>>>>>> when I selected Optima as the font. I found no way to select 
>>>>>>> Optimal Regular as the font style.
>>>>>>>
>>>>>>> Optima.ttc contains five styles:
>>>>>>> Optima Regular
>>>>>>> Optima Italic
>>>>>>> Optima Bold
>>>>>>> Optima Bold Italic
>>>>>>> Optima ExtraBlack
>>>>>>>
>>>>>>> The code for importing the TTC file imports all 5 styles:
>>>>>>> TTFontReader readTTCFrom: (DirectoryEntry currentDirectory / 
>>>>>>> 'Fonts' // 'Optima.ttc') binaryContents.
>>>>>>>
>>>>>>> However, only 4 of those 5 are saved in the font family:
>>>>>>> TrueTypeFontFamily availableFamilies at: 'Optima'
>>>>>>>
>>>>>>> There seems to be a restriction TrueTypeFontFamily that there 
>>>>>>> can only be one font variant per TextEmphasis combination.
>>>>>>>
>>>>>>> As a workaround I changed TTFontDescription>>#emphasis to 
>>>>>>> identify Black as Bold. Now Optima Regular is used as default. 
>>>>>>> However, I cannot use Optima Bold now.
>>>>>>>
>>>>>>> I attach the ChangeSet with the fix and the workaround.
>>>>>>>
>>>>>>> Given that we use only TrueType fonts now, I think the best 
>>>>>>> solution would be to remove bold an italic from TextEmphasis and 
>>>>>>> use the all variants of the TrueType font families instead.
>>>>>>>
>>>>>>> By the way, is there a package with a nice font viewer?
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Bernhard
>>>>>>> -- 
>>>>>>> Cuis-dev mailing list
>>>>>>> Cuis-dev at lists.cuis.st
>>>>>>> https://lists.cuis.st/mailman/listinfo/cuis-dev
>>>>>>> <5984-CuisCore-BernhardPieber-2023Aug26-15h22m-bp.001.cs.st>
>>>>> -- 
>>>>> Juan Vuletich
>>>>> cuis.st
>>>>> github.com/jvuletich
>>>>> researchgate.net/profile/Juan-Vuletich
>>>>> independent.academia.edu/JuanVuletich
>>>>> patents.justia.com/inventor/juan-manuel-vuletich
>>>>> linkedin.com/in/juan-vuletich-75611b3
>>>>> twitter.com/JuanVuletich
>>>
>>>
>>> -- 
>>> Juan Vuletich
>>> cuis.st
>>> github.com/jvuletich
>>> researchgate.net/profile/Juan-Vuletich
>>> independent.academia.edu/JuanVuletich
>>> patents.justia.com/inventor/juan-manuel-vuletich
>>> linkedin.com/in/juan-vuletich-75611b3
>>> twitter.com/JuanVuletich
>>>
>>> -- 
>>> Cuis-dev mailing list
>>> Cuis-dev at lists.cuis.st
>>> https://lists.cuis.st/mailman/listinfo/cuis-dev
>
>