[Cuis-dev] [ODBC] Using accented characters in resultset

[Juan Vuletich]

Hi Olivier,

It looks like you're getting UTF-8 bytes for Unicode, and the ODBC 
package is turning them into a (Byte)String. Take a look at the attach. 
This is my take on how a UnicodeString should be created instead.

I'm not aware of anyone using or maintaining that package right now. I 
can not actually test it, as I don't have any ODBC data source on my 
machine. But the protocols in Cuis for creating UnicodeString from UTF-8 
bytes, and extracting the UTF-8 bytes from a String/UnicodeString (to 
save to the database) are pretty straightforward, as shown in the attach.

If you can fix the problem, and test that the fix works, we can 
integrate your contribution to the package, so it will help others in 
the future.

Thanks!

On 2025-12-30 7:17 PM, H. Hirzel via Cuis-dev wrote:
> Hi Olivier
>
> A contribution towards an answer: yes it is an encoding issue.
>
> If I use Notepad++
>
> type
>
> invité
>
> (encoding is UTF8 there by default) and then choose ANSI in the 
> 'encoding' menu I get what you mention:
>
> 'invité'.
>
> However how to do this in Cuis I have not yet managed to do. I guess 
> there is a wayto tell the database driver that you want UTF8. That 
> should actually be the default if things are fine.
>
> --Hannes
>
> On 29/12/2025 10:40 am, olivier auverlot via Cuis-dev wrote:
>> Hi,
>>
>> I am trying to use the ODBC package for Cuis Smalltalk. I used the 
>> documentation by Mark Volkmann 
>> (https://mvolkmann.github.io/blog/topics/#/blog/smalltalk/50-databases/?v=1.1.1). 
>> Thank you, Mark, for your tutorial!
>>
>> I'm connecting to a PostgreSQL database but I'm having an issue with 
>> French accented characters. For example, instead of 'Invité', I get 
>> the string 'invité'.
>>
>> I suspect an encoding issue, although I'm using the psqlodbcw.so 
>> library which supports Unicode.
>>
>> Does anyone have experience with this ODBC package and can help me 
>> resolve this issue?
>>
>> Best regards
>> Olivier
>>
-- 
Juan Vuletich
www.cuis.st
github.com/jvuletich
researchgate.net/profile/Juan-Vuletich
independent.academia.edu/JuanVuletich
patents.justia.com/inventor/juan-manuel-vuletich
-------------- next part --------------
'From Cuis7.7 [latest update: #7780] on 29 December 2025 at 2:34:47 pm'!

!ODBCColumn methodsFor: 'private - type convertion' stamp: 'jmv 29/Dec/2025 14:27:56'!
stringFromBuffer
	"convert the buffer to a String"
	| len |
	len := bufferLength value min: BUFFERSIZE.
	^UnicodeString fromUtf8Bytes: (buffer copyFrom: 1 to: len)! !


!ODBCStatement methodsFor: 'executing' stamp: 'jmv 29/Dec/2025 14:34:41'!
fillArg: buf with: arg
	"Fill a bound parameter with a new value. Answer true if successful, false otherwise"
	| argHandle |
	buf ifNil:[^false].
	argHandle := buf handle.
	argHandle isNull ifTrue:[^false].
	buf sqlType caseOf: {
		[SQLVARCHAR] -> [
			arg isString ifFalse:[^false].
			arg size > buf size ifTrue:[^false].
			"I'd say here too..."
			1 to: arg size do:[:b| argHandle unsignedByteAt: b put: (arg uint8At: b)].
			^true
		].
		[SQLINTEGER] -> [
			arg isInteger ifFalse:[^false].
			arg digitLength > 4 ifTrue:[^false].
			argHandle signedLongAt: 1 put: arg.
			^true
		].
		[SQLDOUBLE] -> [
			arg isFloat ifFalse:[^false].
			argHandle doubleAt: 1 put: arg.
		].
		[SQLTIME] -> [
			(arg isMemberOf: Time) ifFalse:[^false].
			(SQLTime fromHandle: argHandle)
				hour: arg hour;
				minute: arg minute;
				second: arg second.
			^true
		].
		[SQLDATE] -> [
			(arg isMemberOf: Date) ifFalse:[^false].
			(SQLDate fromHandle: argHandle)
				year: arg year;
				month: arg monthIndex;
				day: arg dayOfMonth.
			^true
		].
		[SQLTIMESTAMP] -> [
			(arg isKindOf: DateAndTime) ifFalse:[^false].
			(SQLTimestamp fromHandle: argHandle)
				year: arg year;
				month: arg month;
				day: arg dayOfMonth;
				hour: arg hour;
				minute: arg minute;
				second: arg second.
			^true
		].
	} otherwise:[^false].
! !