@koan

Quote:

I am happy to work with you on Thai support, will PM you my email address.

The thai charset and language drivers and the thai keymaps for OS4 are in betatest,
and the first two thai catalogs go to betatest tonight.

Thank you very much for your help!

@TetiSoft

Quote:

The thai charset and language drivers and the thai keymaps for OS4 are in betatest, and the first two thai catalogs go to betatest tonight.

Very glad to help. It's been a pleasure working with you.

@keisangi

Its cool of you to be willing to spend time on this for free. Did you ever hear back from the person Hyperion referred you to?

_________________
Pegasos2 G3 running AOS 4.1 and MorphOS 2.0
Amikit user, tinkering with Icaros VM (AROS)
EFIKA owner
Amiga 1200

anything new on this?
i find it kind of interesting. :)

_________________
AOS4 Betatester, Peg2, G4@1ghz, Radeon 9250 256mb, 1gb RAM.

http://www.asciiarena.com
http://www.uprough.net

@spotUP

Me too

_________________
BACK FOR THE FUTURE

http://www.betatesting.it/backforthefuture

Sam440ep Flex 800 Mhz 1 GB Ram + AmigaOS 4.1 Update 6
AmigaOne XE G3 800 Mhz - 640 MB Ram - Radeon 9200 SE + AmigaOS 4.1 Update 6

@samo79

read this thread...

http://www.amigans.net/modules/newbb/viewtopic.php?topic_id=952&forum=3&post_id=12029#forumpost12029

They asked me help out, but Olsen does not know how to deal whit NDA stuff, right now, I tell you what; I probably enjoy working on some tiny part of the OS, just for fun.

(Anyway I need to wait for my CPU module to return, a few more weeks…)

_________________
http://lifeofliveforit.blogspot.no/
Facebook::LiveForIt Software for AmigaOS

@fairlanefastback
I don't think that Keisangi will be involved in this (or OS4) anymore, since Rogue feels unable to discuss things with him, AND he has just been "permanently suspended" from Amigans.net (the only Amiga forum that Rogue frequently visits).

FWIW, I do understand why Rogue feels like that, and why Amigans.net took their actions, but there's no point discussing it here.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@ChrisH

Quote:

I don't think that Keisangi will be involved in this (or OS4) anymore, since Rogue feels unable to discuss things with him, AND he has just been "permanently suspended" from Amigans.net (the only Amiga forum that Rogue frequently visits).

FWIW, he is suspended on AW.net too (29 days to left).

@hatschi

Seems he's really angry with everything Amiga these days. Complaining
loudly (without listening to reason) because AmigaOS doesn't support
whatever features he wants.
The funny thing is that nobody disagrees with him. Just that his behaviour
was a bit over the top.

_________________
/Harry

[SOLD] µA1-C - 750GX 800MHz - 512MB - Antec Aria case

Avatar by HNL_DK!

Quote:

I don't think that Keisangi will be involved in this (or OS4) anymore, since Rogue feels unable to discuss things with him, AND he has just been "permanently suspended" from Amigans.net (the only Amiga forum that Rogue frequently visits).

I don't know why our japanese friend was banned from Amigans.net but if he can help with OS4 ...

He can always contact Rogue in a private e-mail if he want, so I hope for the better, expecially in this way, one day I like to use the arabic charset too !

Last edited by samo79 on 22-Aug-2007 at 11:44 PM.

_________________
BACK FOR THE FUTURE

http://www.betatesting.it/backforthefuture

Sam440ep Flex 800 Mhz 1 GB Ram + AmigaOS 4.1 Update 6
AmigaOne XE G3 800 Mhz - 640 MB Ram - Radeon 9200 SE + AmigaOS 4.1 Update 6

@samo79

Quote:

one day I like to use the arabic charset too !

OS4 comes with a charset driver for ISO-8859-6, with more than a dozen country
files for countries with arabic language, and with several fonts with arabic glyphs.

It comes without an arabic language driver and without an arabic keymap, for
the reasons I'm quoting OS4Localization.doc of the SDK:
Quote:

OS4 currently only supports the left-to-right writing direction. If your language is written in another direction you have to wait until OS4 supports e.g. the Unicode bi-directional writing algorithm, sorry. This excludes e.g. Arabic and Hebrew. OS4 does currently not support character combining. There exist scripts where the visual appearance of a character (its glyph) may be modified by neighbour characters or marks, e.g. Arabic, Hangul, Syriac. Supporting those would require a font layout engine and an API for it which doesnt exist yet for OS4, sorry.

IMHO its possible to create applications which support arabic, e.g. a word
processor would "just" need to support the "wrong" writing direction and
an arabic keymap, however, I dont know if displaying arabic text with the
ISO-8859-6 presentation forms (as single characters) and not as complete
words is acceptable for arabic users.

Does AbiWord support arabic? Didnt download it yet.

@TetiSoft

Hi.

Beforehand: I use only OS3.x but I'm interested in OS4, too.

I didn't know about the existing of this tread so please apologise for jumping in (and then that late) and also that I'm going a little bit OT:


Quote:

Thats a small library which contains the locale.library ToUpper() ToLower() IsDigit() etc functions for a specific charset.

Because OS4 already tries to implement a sub-set of Unicode wouldn't it be of more benefit if you would add a Unicode core library that can do such things? I don't see the benefit of defining ToUpper and such things on a single-byte character set basis when it is possible for the whole code point range defined by the Unicode Consortium. Furthermore, I would also suggest to implement those functions which can be of benefit for render engines - such as obtaining whether a character is non-spacing-marker or not.

In addition, I think it's about time of thinking about how those mappings from Unicode code points to ISO-8859-x character codes and vice versa can be avoided, except for codepages used by the input system.
The benefit I currently see of so late jumping onto the train called Unicode is that such a support can be cleanly encapsulated.

So, may I ask whether it is planned in order to implement these features?


J.v.d.Loo


Who ever makes jokes about my surname gets to know me - more than 6 foots tall, more than 190 pounds heavy and what's very dangerous, wearer the smelling sock.

@Jolo

Quote:

Quote: Thats a small library which contains the locale.library ToUpper() ToLower() IsDigit() etc functions for a specific charset. Because OS4 already tries to implement a sub-set of Unicode wouldn't it be of more benefit if you would add a Unicode core library that can do such things? I don't see the benefit of defining ToUpper and such things on a single-byte character set basis when it is possible for the whole code point range defined by the Unicode Consortium.

You did not read the OS3 autodoc for ConvToUpper()/IsDigit() etc before asking.
They say
Quote:

NOTE This function(s) require(s) a full 32-bit character(s) be passed-in in order to support multi-byte character sets.

which implies that its no big problem to create a LOCALE:CharSets/UTF-32BE.charset
driver for OS4 which would contain those functions. In fact an experimental version
of such a charset driver already exists on my harddisk, it parses
http://www.unicode.org/Public/UNIDATA/UnicodeData.txt to get the character properties.
It is still missing the StrnCmp() function which would need something like a collation table
and I currently dont have time to work on it.

Quote:

Furthermore, I would also suggest to implement those functions which can be of benefit for render engines - such as obtaining whether a character is non-spacing-marker or not.

Until now I didnt bother looking at render engines in detail, they seem to
all prefer C++ and cryptic configure scripts which I dont like (both).

Quote:

In addition, I think it's about time of thinking about how those mappings from Unicode code points to ISO-8859-x character codes and vice versa can be avoided, except for codepages used by the input system.

I think it's time when its needed. IMHO its a big mistake to define new APIs
before you need them. This approach resulted in such "broken by design" things
like
Quote:

/*------ Font Flags -------------------------------------------------*/ #define FPB_REVPATH 2 /* designed path is reversed (e.g. left) */

in the graphics/text.h include file. This right-to-left writing direction
flag for a font could never work because there never existed a charset
which is written from right to left only, the existing arabic and hebrew
charsets all include US-ASCII...

Quote:

The benefit I currently see of so late jumping onto the train called Unicode is that such a support can be cleanly encapsulated.

There were users which asked me to add UTF-8 autodetection which would be a
mistake IMHO, others asked for adding UTF-8 support only for new programs
which would be a mistake IMHO, now you want its "cleanly encapsulated".
Currently I dont see a valid reason to handle UTF-8 in any way different
than e.g. ISO-8859-7 (greek), when the system default charset would be UTF-8
this should be the system default charset for everything. Of course the
average non-english user would switch to UTF-8 only after converting most
of his documents but for the english/US-ASCII user it would not make
much of a difference switching to UTF-8 IMHO.

Quote:

So, may I ask whether it is planned in order to implement these features?

Which features? First, you wont get any definite answer, just because its
a mistake to announce or deny something when you are a developer which signed
an NDA, and second, why should I accept suggestions which only suggest
vague terms like "encapsulation" or "sandbox" (this term was used in another
thread). Such words say as much as "multimedia": Nothing. My ToDo list is
long enough and wont be filled with incomplete or vague suggestions.

@TetiSoft

Quote:

You did not read the OS3 autodoc for ConvToUpper()/IsDigit() etc before asking. They say Quote: NOTE This function(s) require(s) a full 32-bit character(s) be passed-in in order to support multi-byte character sets.

I'm aware of this function but currently these function doesn't help one who wants to use Unicode code points, so I wrote my own.
But I now understand that you will change this function in order to support 32-bit character-codes.


Quote:

I think it's time when its needed. IMHO its a big mistake to define new APIs before you need them. This approach resulted in such "broken by design" things like...

Who said that I don't need it?
I'm willing to create a really basic and primitive Unicode text editor - what I'm currently missing is a complete rendering engine and low-level functions to obtain font attributes.

Of course, in case the OS would offer me these things I would use them instead of spending my time developing core functions.


Quote:

There were users which asked me to add UTF-8 autodetection which would be a mistake IMHO,

Which isn't possible to do with a reasonable effort, AFAIK.
Okay, a text file could contain the BOM - but almost no UTF-8 text file contains it.


Quote:

[...] now you want its "cleanly encapsulated".

I mean with "cleanly encapsulated" to separate multibyte character sets from single-bytes - what you don't want to do - see next paragraph.
I think that a lot of programs, which are currently in use, will no longer work in case everything is designed for multibyte character sets. Maybe you have some good solutions already that I can't think of.


Quote:

Currently I dont see a valid reason to handle UTF-8 in any way different than e.g. ISO-8859-7 (greek), when the system default charset would be UTF-8 this should be the system default charset for everything

That would be fine with me but I have my doubts that this would allow old programs to run entirely correctly, which are not using the provided AmigaOS API functions but those static linker libraries.


Quote:

[...] just because its a mistake to announce or deny something when you are a developer which signed an NDA...

Sorry, I didn't think it through...


Quote:

[...] why should I accept suggestions which only suggest vague terms like "encapsulation" or "sandbox"...

My "suggestion" is no meant to force you to do anything - it's just a wish from me because then I would cancel my projects in case AmigaOS4 would offer sooner or later those functions I need in order to write this text editor, because I guess it will take again another year or more to port a render engine that also provides low-level functions.

FYI: I've written a Unicode code point and UTF-8 support library - that is separated from the rest of the system.


Joerg

@Jolo

Quote:

I'm aware of this function but currently these function doesn't help one who wants to use Unicode code points, so I wrote my own. But I now understand that you will change this function in order to support 32-bit character-codes.

Those functions were designed to work with the current system default charset,
currently all possible system default charsets are 8bit, with future multibyte
charset support more charset drivers will be needed which dont handle code values
above 0xFF as "undefined in this charset" anymore.

Quote:

I'm willing to create a really basic and primitive Unicode text editor - what I'm currently missing is a complete rendering engine and low-level functions to obtain font attributes. Of course, in case the OS would offer me these things I would use them instead of spending my time developing core functions.

The bullet.library API which speaks Unicode exists since 1992 AFAIK. In OS4
the API was already enhanced so you can simplify/enhance the already existing
applications which use this API. Currently I'm not aware of a single application
which is still in development which uses this API and I'm not aware of a single
enhancement request so from my point of view there currently exists no need
to enhance it again now.

Quote:

I mean with "cleanly encapsulated" to separate multibyte character sets from single-bytes - what you don't want to do - see next paragraph. I think that a lot of programs, which are currently in use, will no longer work in case everything is designed for multibyte character sets. Maybe you have some good solutions already that I can't think of. Quote: Currently I dont see a valid reason to handle UTF-8 in any way different than e.g. ISO-8859-7 (greek), when the system default charset would be UTF-8 this should be the system default charset for everything That would be fine with me but I have my doubts that this would allow old programs to run entirely correctly, which are not using the provided AmigaOS API functions but those static linker libraries.

To repeat it, the current OS4 implementation allows the user to switch the
current system default charset to ISO-8859-7 (greek). Which e.g. implies that
there doesnt exist a character with codepoint 0xFF anymore. Which e.g. implies
that all old applications linked with static linker libraries may fail when
using a statically linked toupper() function. I dont see any serious problem
with that. The user is always allowed to switch OS4 back into
"OS3 compatibility mode" by setting the system default charset to ISO-8859-1
which matches ECMA-94 Latin 1 which was officially documented to be the charset
of AmigaOS in the past. And for now I dont see any basically different problem
when replacing ISO-8859-7 with UTF-8, it would be a new option which the user
would be allowed to try it out and which he would not be forced to use in case
it would not work with all his old applications. When font smoothing causes
problems it can be switched off and when UTF-8 would cause problems it could
be switched off.

Quote:

My "suggestion" is no meant to force you to do anything - it's just a wish from me because then I would cancel my projects in case AmigaOS4 would offer sooner or later those functions I need in order to write this text editor, because I guess it will take again another year or more to port a render engine that also provides low-level functions. FYI: I've written a Unicode code point and UTF-8 support library - that is separated from the rest of the system.

Is it public?

Well, such a library is possible since 1992, and missing since 1992. When I
added UTF-8 support to locale.library, keymap.library, TypeManager and
CharsetConvert I thought about creating a library with often needed functions
but found that in fact I needed a slightly different behaviour of the same
basic function everytime. When locale.library detects some invalid/undefined
character it has to replace it with "?", CharsetConvert has to handle invalid
characters with an error message and undefined characters with a replacement
text, keymap.library has to refuse to use keymaps with invalid encoding but
to silently ignore undefined characters, and TypeManager has to create a
different graphical representation for both cases. A web browser could
silently replace a missing Euro sign with "EUR", other applications should
eventually create an error message.

I think this requirement of slightly different behaviour is also the reason
why on other operating systems so many applications (GhostScript, Python) use
their own custom charset conversion functions and tables instead of e.g. the
public iconv() function. OS4 already contains iconv(). Didnt look at it yet.

@TetiSoft

Quote:

To repeat it [...]

Did you already state that in this thread? Then I missed it.
Anyhow, thanks for clarifying.


Quote:

Is it public?

Aminet/util/libs/UniLibDev


Quote:

When I added UTF-8 support to locale.library, keymap.library, TypeManager and CharsetConvert I thought about creating a library with often needed functions but found that in fact I needed a slightly different behaviour of the same basic function everytime.

Sorry, I can't follow. Do you speak about code points that exceed the 8-bit limit?
Normally, no code point on its own can cause troubles - Unicode code points, as you know better than me, cover the range from U+000000 to U+10FFFD. The transcoding of a code point that exceeds the 8-bit limit to its 8-bit (ISO-8859-x) counterpart in contrast can cause trouble. And I agree that there is the need for differentiating. On the other side, in case a code points would be taller than U+10FFFD a replacement code point (e.g. U+00FFFD) should be returned, which indicates that this code point is invalid.
Or do you mean an invalid UTF-8 multibyte sequence?
In that case the Unicode Consortium suggested to use signed values instead of unsigned, since (currently) no code point exceeds the 21-bit limitation. If now a signed value would be encountered (-1 for example) either a replacing code point can be used or that value (-1) on its own - as indicator for an error.
From my (yet small) experiences with UTF-8 strings I know that the most errors occur in case the string is not encoded in UTF-8 but encoded using ISO-8859-x - and then it is useless to try to decode any following multibyte sequences since they are anyway invalid.

Anyhow, a clever designed function can help in all cases where one and the same function could be used for certain tasks.


Quote:

I think this requirement of slightly different behaviour is also the reason why on other operating systems so many applications (GhostScript, Python) use their own custom charset conversion functions and tables instead of e.g. the public iconv() function.

Maybe, I never looked at them. Frankly, I don't like iconv() - since it is a real monster and it requires lots of RAM (program/data). In addition, I would only support those mapping tables which were published from the Unicode Consortium - and leaving the rest (the exotics) to iconv(). Don't get me wrong: That's how I would do it - it shall not be a suggestion.

Back to the roots:
As soon as a proper transcoding function would be created it should be of no problem to use replacement character codes, replacement strings or even error codes. It depends all on how this function would be realized - and I guess, because I never looked close enough, that iconv() doesn't suit for this purpose.


My apologies in case I did not understand your statements correctly - my English isn't the best...


Joerg

@Jolo & TetiSoft

Thanks for a truly interesting coversation. Please continue if you have more to
say on the matter.

And no that was NOT a joke.

_________________
/Harry

[SOLD] µA1-C - 750GX 800MHz - 512MB - Antec Aria case

Avatar by HNL_DK!