[Cuis-dev] Any quick way to convert a UTF16 text to Latin1 ?

[Juan Vuletich]

Hi Nicola,

Something like this should work:


utf16 _ 'expo-test-IT-UTF16.xml' asFileEntry binaryContents.
possibleBOM _ utf16 copyFrom: 1 to: 2.
isLittleEndian _ true. "use your best guess"
possibleBOM = #[255 254] ifTrue: [
     isLittleEndian _ true.
     utf16 _ utf16 copyFrom: 3 to: utf16 size ].
possibleBOM = #[254 255] ifTrue: [
     isLittleEndian _ false.
     utf16 _ utf16 copyFrom: 3 to: utf16 size ].
String streamContents: [ :out |
     index _ 1.
     [index < utf16 size] whileTrue: [
         codePoint _ utf16 unsignedShortAt: index bigEndian: 
isLittleEndian not.
         out nextPut: (Character codePoint: codePoint).
         index _ index + 2 ]].

As you see, after messing a bit with the silly ByteOrderMark (BOM), all 
we do is grab 2 bytes and store them as a Character.

Good luck with your project!

Cheers,

On 6/11/2022 5:02 AM, Nicola Mingotti via Cuis-dev wrote:
> Hi Jaun,
>
> I can share this data file, it is attached.
> It is the output reading of a machine who does a NIR analysis on a 
> corn sample.
> Once the machine GUI has been set to work in Italian.
>
> bye
> Nicola
>
>
>
>
>
> On Sat, Jun 11, 2022 at 2:02 AM Juan Vuletich <JuanVuletich at zoho.com 
> <mailto:JuanVuletich at zoho.com>> wrote:
>
>     On 6/10/2022 8:25 PM, Nicola Mingotti via Cuis-dev wrote:
>>     Hi guys,
>>
>>     as title says, I am converting a script from Python to Cuis.
>>     It reads an XML file and does stuff consequently. Unfortunately the
>>     XML is in UTF-16 (we are running in Windows).
>>
>>     I am sure the file content is convertible to Latin-1 since it is
>>     Italian
>>     text stuff.
>>
>>     Do you have any quick way to solve it from Cuis ?
>>
>>     The XML part is working, already tested converting UTF16 to Latin-1
>>     with an external editor.
>>
>>     bye
>>     Nicola
>
>     Hi Nicola,
>
>     Can you send me (to the mail list or to juan at jvuletich.org
>     <mailto:juan at jvuletich.org> ) one of those files in a zip file?
>     I'll show you how to do it, but I need an example. The zip file is
>     to ensure that I get it exactly as you have it.
>
>     Cheers,
>
>     -- 
>     Juan Vuletich
>     www.cuis-smalltalk.org  <http://www.cuis-smalltalk.org>
>     https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev
>     https://github.com/jvuletich
>     https://www.linkedin.com/in/juan-vuletich-75611b3
>     https://independent.academia.edu/JuanVuletich
>     https://www.researchgate.net/profile/Juan-Vuletich
>     https://patents.justia.com/inventor/juan-manuel-vuletich
>     https://twitter.com/JuanVuletich
>


-- 
Juan Vuletich
www.cuis-smalltalk.org
https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev
https://github.com/jvuletich
https://www.linkedin.com/in/juan-vuletich-75611b3
https://independent.academia.edu/JuanVuletich
https://www.researchgate.net/profile/Juan-Vuletich
https://patents.justia.com/inventor/juan-manuel-vuletich
https://twitter.com/JuanVuletich

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20220611/14d9c2ad/attachment.htm>