@kilaueabart

Quote:

I (for JIStoJi) and others who wrote Japanese handling programs for classic Amiga managed to sort of fake 16bit character sets. Is it really that hard to do for real?

All programs which I'm aware of (please correct me when I'm wrong here)
which claim to add japanese support to AmigaOS do basically the following:

They preload one or more fixed-size, non-scalable, non-antialiased, japanese-only
bitmap-only font into memory. Then they call SetFunction() for
graphics.library/Text(), the replacement function tries to guess if the text
to be displayed is japanese or not and when its japanese it tries to guess
the encoding (Shift-JIS, EUC-JP, UTF-8 or whatever). Then the text is displayed
with the preloaded font.

For OS3 this approach was already a hack. No offense, of course a somehow working
hack is better than nothing. But with OS4 this is even more of a hack because

- OS4 has charset support. It knows which font has which charset, which language
requires which charset, it knows about the charset of catalog files, keymaps,
country files. You can switch OS4 from cyrillic to greek and it will automatically
use the right fonts to display it, immediately, without the need to reboot the
machine or to re-configure all fonts which shall be used by the applications.

- OS4 supports antialiasing. And like OS3, it supports soft-styles for
automatically creating bold, italic or underlined fonts, which all cant
work with a 16*16 fixed-size bitmap-only font. When those japanese-patches
would support a scalable Unicode or japanese font installed with TypeManager
or TTFManager and use this font for japanese and latin and cyrillic and
greek and thai text, either antialiased or not, they would be more Amiga-like
IMHO :)

For real japanese support, we would need more than a Text() patch. At least
a TextLength() patch would be also required

The main bottleneck is Text() which can only handle Amiga bitmap fonts.
In the meantime those bitmap fonts can be antialiased and can have any
8bit charset, but using e.g. Unicode (which no longer fits in 16bit) for
Amiga bitmap fonts is out of question because of the needed memory and because
struct TextFont has the members tf_LoChar and tf_HiChar which are 8bit.

On OS4 you can display japanese with TypeManager because it doesnt use Text()
to display text, it uses a builtin custom ReAction class which uses ft2.library
to generate a struct GlyphMap and custom display functions for this glyphs.

@keisangi

I wasn't lack of the font that was stumping me. I had copied kochi_gothic.ttf, Kochi Gothic.otag, and Kochi Gothic.font (why don't the names match up?) over from the Debian side into Fonts:_TrueType two weeks ago. (It's 7.77Mb, but there ain't gonna be any small and complete sets.)

It was the notion that I should use TypeManager to *view* a file that I couldn't latch on to.

Now if I had a good Amiga news reader (I used to get by with NewsRog, but I don't think I have that anymore), I could go back to reading sci.lang.japan from OS4 instead of Linux. (But couldn't write to it... )

Next time I'm in Linux, I'll send some of my Korean fonts over and see if the line below that Japanese one says "nayka yulilul mekulswu issupmnita. kukesun momul haychici anhsupnita." (I sort of hope not.) Where did this "eat glass" stuff come from, I wonder?

Would you have translated "I can eat glass and it doesn't hurt me" into Japanese like that?

Bart Mathias

@TetiSoft

Quote:

All programs which I'm aware of (please correct me when I'm wrong here) which claim to add japanese support to AmigaOS do basically the following: They preload one or more fixed-size, non-scalable, non-antialiased, japanese-only bitmap-only font into memory. Then they call SetFunction() for graphics.library/Text(), the replacement function tries to guess if the text to be displayed is japanese or not and when its japanese it tries to guess the encoding (Shift-JIS, EUC-JP, UTF-8 or whatever). Then the text is displayed with the preloaded font.

I don't think I did it exactly that way. I did preload a Japanese bit-map font, but JIStoJi was my first (and last) fairly successful go at Assembler and I'm not sure I knew anything about SetFunction(), etc. (Unfortunately, I seem to have left the source irretrievably behind on my dead A2000 and/or some AOS format floppy, so I can't find out for sure.) I did the first version in AmigaBasic in 1991, and translated that to Assembler in 1993.

Unfortunately, I knew nothing about UTF-8 in 1993, or I would have added that too.

It identifies text as OldJIS, NewJIS, Shift-JIS, or EUC-JIS (in rare cases, it can confuse the last two), then calculates the position for the bitmap of the character encoded by each set of two bytes of the text within the font file, and copies it pretty much bit-by-bit to the screen or printer. Well, not to the printer anymore unless it has Epson-MX80 emulation; I bought DevPac and tried to upgrade the printer capabilities, but DevPac consistently crashed on me and I never got anywhere.

Anyway, I'm very likely misunderstanding something, but it seems to me like the problem is to get programs to accept 16-bit text data when necessary, and if I'm right, it shouldn't be overwhelming (so I must be wrong).

Bart Mathais

@kilaueabart

Quote:

Would you have translated "I can eat glass and it doesn't hurt me" into Japanese like that?

the text says:
" watashi ha garasu wo taberaremasu. sore ha watashi wo kizutsukemasen."

which translate by: " i can eat glass. and it cannot hurt me" litteraly.

you can see for yourself on the screenshot..
http://keisangi.free.fr/amiga/os4japanese4.jpg

would you have a better translation in mind ? :)

other than that, i wish too other part of os4 could accept multibyte unicode / asian language.. such as editpad.. or any text editor.

after that, building an input method is trvial..
could even be done in arexx..
all you would need is a copy of edict for kanji lookup..
you type the word, the ime can easily translate it into kana then all is left is lookup into edict for that pronounciation in kana, to see if it have equivalent in kanji... all you would have to do is input the okurigana yourself for conjugate the verb ..

like you type: ta be ru (taberu = to eat) the ime can convert in to kana..
then you lookup into edict and see there's a kanji entry with the pronounciation "taberu" you replace your word with it..
then you conjugate yourself.. like:
taberareru ...tabesaseru .. tabemasu .. tabemashou .. etc ...
all you would need is the prefix "tabe" in kanji..
you can add okurigana yourself.
that would do a basic ime (input method editor)

that what i did with a friend few years ago on morphos.
the program was called "charabia"
it was on morphos repository ...err what was it?

http://www.morphos-news.de/software.php?lg=de&md=text&sd=all

that was it ... it point to my website..
i deleted the archive by the way. but i still have the sources ..

basicaly it was jkff for displaying japanese
and charabia, done by a friend for me. (i owe him this one.. hi jah :)
and he didn't know a thing about japanese. he developped this, listening to my needs, and how i was explaining to him how it should have worked.
he couldn't even bothered to install jkff to display japanese ;)
he developped charabia while he wasn't able to display one single japanese char !
all was giberish to him, like: @#$%":^# lol
that's why he decided we should call it charabia, which in french mean " gibberish or nonsense blablah ;)

that was for the story ..
also if someone is interested i still have a old archive of "daigokai" software, which enabled the a500 to display and type japanese .. this worked up to os3.9... there was "ANS2" too see screenshots:


http://www2.tky.3web.ne.jp/~jpudata/JAUG/kezaco2/japanese-wb2.jpg
(screenshot from me.. long time ago)

http://www2.tky.3web.ne.jp/~jpudata/JAUG/kezaco2/OMAKE.JPG

@keisangi

here's some old screenshot i could find of charabia:
http://keisangi.free.fr/amiga/charabia1.jpg
http://keisangi.free.fr/amiga/charabia2.jpg

maybe not that pretty or comfortable, but it worked well.
and jkff enabled japanese to be used system wide.. drawer , filenames, menus...

here i used charabia to speak on irc with the author of jkff, Olfa.

this all story happened in 2003 iirc.

charabia "ime" was done in less than a week.. 3 day or so?
without specialy high priority ;)

as you see it had 3 tabs, 1 for input hiragana, 1 for input katakana, and the last one "search" was a japanese / english dictionary.. using edict..
so basicaly you input hiragana, and hit the --> kanji button, to lookup and convert your hiragana into kanji, using edict.
was basic but it worked well..

Last edited by keisangi on 17-Jan-2007 at 04:29 AM.

@keisangi

I can see the Japanese text for "I can eat glass. And it doesn't hurt me" just fine with TypeManager, now that I know how it works.

I suppose clumsy Japanese to match clumsy English? I'd have said something on the order of "watashiha garasuga taberaremasu. soushitemo kizutsukimasen" or maybe even have shortened it to "watashiwa garasuwo tabetemo kizutukimasen," which at least logically implies "I can eat glass."

Many years ago when I was still teaching, I did more or less as you suggest. I reversed edict so that I could look up kanji via romaji. Then I wrote an ARexx program that would look at a text file like this: \hkoreha\tnihongo\hdesu.\
When it came to an "\h," it would change everything up to the next "\" into New-JIS ($3$l$O in this case) and write it to a new file. Anything with a "\t" in front got looked up in the modified edict. Then I would print it out and use it in class. That way I could work at home instead of reserving time on the department Macs. Not as pretty as your charabia, but it worked.

I am horrified to discover that I seem to have left both the modified dictionary and the ARexx program behind on my defunct A2000 and some AOS-formatted floppies. Oh well, I'm retired anyway now.

I have daigokai on a floppy I can't read with my A1. I did copy over Kezako and Waa (a tamago of a waapuro which allowed editing of my class files) to the A1, but with previous versions of OS4, they crashed. Haven't tried them with Final yet, but I'm pessimistic.

@kilaueabart

i'm glad to hear you can finaly display japanese on os4 from typemanager.

i agreee with your opinion about the translation .. you're right, but i've just kept the main idea .. i mean, this text was about eating glass and not being hurt for that.. i kept the theme only ..
there's always place for various adaptations when translating from a language that much diffrent .. gramar and idioms, meaning of words.. everything is just diffrent. in my opinion, the most important thing to do is to communicate the idea.
the meaning .. after that you can always arrange and turn the words around, but if the idea, the meaning is understood, it's ok ..that's usualy what i focus on when translating .. :) but in other words i agree with what you said :)


by the way, if you want to be able to edit japanese files on os4 right now, there's always the option of using dosbox with njstar, it's am old dos shareware. i think you can find a copy on jim breen ftp..
hold on, i'm checking the link for you:

http://ftp.monash.edu.au/pub/nihongo/NJSTAR.zip

that's it ..
if you need help about njstar and dosbox just ask.


since os4 broke the compatibility with the old softwares that enabled us to use japanese on amiga, i hope hyperion will finish their work on unicode soon, enabling os4 to display any languages unicode support.. that will be great :)
better than the old patchs i think.. i just can't wait to see that :)

see you

@keisangi

Quote:

since os4 broke the compatibility with the old softwares that enabled us to use japanese on amiga

I would use different words here. OS4 is a PPC OS and still supports old 68k
patches to some degree, but those are no longer able to affect the PPC components. The authors of 68k patch software for OS3 are encouraged to check
if their patches are still necessary for OS4 and when they think they have
to patch OS4 also, they have to create PPC native patches...

Ideally no patch is ever needed for an OS which is in active development
because the patch authors can cooperate with the OS authors to avoid the
need to patch something.

@TetiSoft

yes you're right .., what i meant, was:
i hope hyperion can finish their unicode support so os4 could display japanese and other asian / multibyte unicode text.

i would like to use JKFF, but the author doesn't have an amigaone nor os4, so everyone is just waiting. hopefully things will be solved soon (about hardware and os4 availability).

but ultimatly the best would be that os4 implement support for multibyte unicode natively, out of the box, so no patch would be needed.

now excuse me i need to check my mail i have to switch to my unix box ;)
(a bit of irony can't hurt, can't it ? ;)

(and yes i agree using os4 is already a kind of miracle .. but we need MORE O_O ! always mooOoore ;)


hehe ;)

@keisangi

Quote:

i'm glad to hear you can finaly display japanese on os4 from typemanager.

I finally added Korean today, Baekmuk Batang (batang.ttf). The Korean translation is "nanun yulilul mekulsu isseyo. kurayto, aphuci anhayo." That would be "watashiwa garasuwo taberaremasu. demo, itaku naidesu."

Quote:

by the way, if you want to be able to edit japanese files on os4 right now, there's always the option of using dosbox with njstar,

I don't think I want to mess with anything like dosbox.

I have one more option. In 1981 I worked for a company trying to do J-E machine translation. They had no direct Japanese input method. I spent a whole year entering Japanese text to be translated in 7-bit JIS code. I got pretty good at typing things like $3$l$OF|K\8l$G$9!#. I could go back to that!

@keisangi

For what it's worth, I would VERY much like to see Japanese support on OS4!!!!
There are options for so many different languages, but not Japanese. Japanese is not some obscure language. I am trying to learn it myself and would find support for it on my Amiga very helpful. Other systems have this support, I think we should too.

I really don't know if there is anything I could do to help, but I guess that the more people that voice there interest, the higher the likelihood that it will be. I will be waiting on the edge of my chair, which I hope will not be too long as it is not to very comfy here.

Seriously though, I am very pleased to see someone trying to do something about this. Thank you!

_________________
AmigaOne G4-XE OS4.0 Still proud
Amiga 4000T 040
Amiga 4000T 060
and a family of little A500's

Interesting thread, I too am looking at tools for reading and wriring Japanese but my platform is currently MorphOS. I have been using JKFF no problem, I am about to try JKED for input.

Sorry to be go off topic but about supporting Thai...

TetiSoft> Thai fits, but its not possible to squeeze the Thai glyphs into topaz/8.

Topaz 8 is too small. I tried to squeeze Thai into a 12 point font ( see http://www.lyndonhill.com/Projects/amigathaifonts.html ). 24 point is readable. I think slightly smaller fonts could be drawn but probably only a few points smaller.

In fact, using my Thai fonts and keymaps it is easy to use Thai in many AmigaOS applications. The only problem is that the application must use proportional fonts and respect font kerning. It is also helpful if the application is aware of what Unicode calls "non-spacing markers". BareEd is one such application that supports these aspects of Thai (and other things such as an implementation of the standard way to render illegal character combinations).

I would be happy to support Thai localisation on any "Amiga compatible" platform, with the caveat that I am not a native Thai speaker and I don't have AOS4. ( By the way, I am the originator of http://www.lyndonhill.com/Projects/thaionamiga.html ).

TetiSoft> For complete support of your language in OS4, you need

TetiSoft> 1) An 8bit charset with all characters needed to write your language.

Done. See above.

TetiSoft> 2) A MIME name and ID for your charset.

text/*; charset=tis-620

Is this what you mean ?

TetiSoft> 3) A mapping table to Unicode for your charset.

Not a problem but I would need some specific info on the format.

TetiSoft> 4) The topaz font in sizes 8 and 9 in your charset.

Maynot be possible. See above.

TetiSoft> 5) A charset driver in your charset.

I need more info about what this is.

TetiSoft> 6) A language driver in your charset.

I need more info about what this is.

TetiSoft> 7) A country file in your charset.

I need more info about what this is.

TetiSoft> 8) A keymap for your keyboard and charset.

Done. See http://www.lyndonhill.com/Projects/amigakeymaps.html
May need to edit a bit to be 100% standard.

TetiSoft> 9) Catalog files in your language.

I could make a start. I know people who I could ask to help verify my translations.

regards

koan

Last edited by koan on 22-Jan-2007 at 01:28 PM.
Last edited by koan on 22-Jan-2007 at 01:23 PM.

@keisangi

> so basicaly you input hiragana, and hit the --> kanji button, to lookup and convert your hiragana into kanji,
> using edict. was basic but it worked well..

I have been trying to get charabia up and running as an input method.

I run JKFF and then have to start adjusting all MUI preferences to try to set the GUI to use the JKFF font. I could never find the one for the text gadget at the bottom of the window with the result.

Asides from the internal support and localisation, I think you need to write an implementation of something like scim - anthy on Linux.

koan

@koan

Quote:

charset=tis-620 Is this what you mean ?

Yes. OS4 already knows about it.
Quote:

3) A mapping table to Unicode for your charset. Not a problem but I would need some specific info on the format.

The mapping table is already part of OS4, it was created with the BuildMapTable
command from the text format mapping table from the unicode.org website.
Well, in fact it was created from a modified copy of the ISO-8859-11 mapping table, unfortunately unicode.org only knows about ISO-8859-11 but IANA only knows about TIS-620. AFAIR ISO-8859-11 is TIS-620 plus the non-breaking space at 0xA0.

BTW, http://www.google.com/intl/th/ only knows about windows-874 which is not a registered IANA charset, when it would use TIS-620 I could use IBrowse 2.4 OEM for OS4 to use Google in Thai language...
Quote:

4) The topaz font in sizes 8 and 9 in your charset. Maynot be possible. See above.

OS4 comes with the "Fixed" bitmap fonts created from some freely distributable X11 fonts, the variants with TIS-620 charset are available in sizes 13,14,15,18,20. And 12 or 13 pixels seem to be the minimum possible height,
both confirmed by you and the OS4 TypeManager when looking at Tahoma, even
when using antialiasing smaller sizes seem to be not readable.

I've choosen the restriction "You need topaz 8 and 9 in your charset" for backwards compatibility only. Old applications often use those "ROM" fonts as fallback or default fonts, when OS4 switches the default charset to something else than ISO-8859-1 it tries to preload topaz 8 and 9 in the new charset and
asks the user if he really wants to continue when this fails. This confirmation
could simply be skipped for Thai charset in a future version.
Quote:

5) A charset driver in your charset. I need more info about what this is.

Thats a small library which contains the locale.library ToUpper() ToLower()
IsDigit() etc functions for a specific charset. Example source code can be found in the OS4 SDK. Without OS4 you probably cant get that, please write me a private mail when you are interested in helping me to create a thai charset driver.
Quote:

6) A language driver in your charset. I need more info about what this is.

Dont you have LOCALE:Languages/thai.language in MorphOS? A language driver is
a superset of a charset driver, it also contains e.g. language-specific sorting
rules, the ISO language code and about 50 basic localized strings, e.g. the month and weekday names.

Like with the charset drivers, the example source code is in the OS4 SDK, I can
help you to help me to create LOCALE:Languages/thai_TIS-620.language when you
write me a private mail.
Quote:

7) A country file in your charset. I need more info about what this is.

Thats a file in LOCALE:Countries/ which contains e.g. the international phone
number, currency, date and time formatting rules, ISO country code, measuring system etc. OS4 already comes with LOCALE:Countries/Thailand.country.

Quote:

8) A keymap for your keyboard and charset. Done. See http://www.lyndonhill.com/Projects/amigakeymaps.html May need to edit a bit to be 100% standard.

That looks like a "Thai Kadmanee" layout. Thanks for the pointer, will try
to create on OS4 keymap from it.
Quote:

9) Catalog files in your language. I could make a start. I know people who I could ask to help verify my translations.

OS4 is translated by the ATO (Amiga Translators Organization, http://ato.bitbrook.de). The coordinator of the OS4 translation is David Rey.
Ask me in private mail when you need his EMail address.

Normally its a big plus when a person which wants to translate OS4 is able
to run OS4 for testing the translations, you may be able to install the
resulting catalogs in MorphOS but in most cases OS4 will have new strings
added which were not present in OS3 so MorphOS will probably not use them.

@TetiSoft

someone with an os4 capable machine could help him to test his work.
he does the work on morphos, and i test it on os4..
if you're interested koan ?

btw tetisoft, you probably know this already but, i wanted to say:
japanese is writen from left to right ..same as english, no need for some extra support for vertical or right to left as in some other language ..

let's make os4 support every languages out there :)
(with japanese first ;)

@koan

about charabia, i'm not the one who created it, i just helped the developer by telling him about the theory / needs.. how it should work.. how it should lookup the words and convert them ..etc .
he did all the coding part. if you need info on him he's the one who made the "kaya player on morphos.. his nick is jah, try contact him ..
i don't use morphos anymore.. it's been 3 or 4 years? since i sold my pegasos.
i'm soory i don't remember what particular settings i used at that time.


@shadow_demon

thanks for the support :) .. i think you're right,
if we show our interest maybe it we help making things to move forward :)
btw, if you need to ask something about japanese feel free to pm me..

Last edited by keisangi on 22-Jan-2007 at 04:30 PM.

@keisangi, TetiSoft

I am happy to work with you on Thai support, will PM you my email address.

Of course, I would be most happy if Thai and Japanese support can be modified and transferred to MorphOS. If OS4 hardware becomes available I may be able to afford that too.

BTW, I am not a MorphOS developer, yet; but I can code.

regards

koan

@keisangi

Domo arigato. I am just learning japanese and really don't have anyone to talk to if I have problems. It's all so far so good, but one day I may take out up on that offer. Thank you.
- Shadow -

_________________
AmigaOne G4-XE OS4.0 Still proud
Amiga 4000T 040
Amiga 4000T 060
and a family of little A500's

@koan

fine :)
just as i said then, pass me some of your work, i'll be happy to test it on os4.

@shadow_demon

sure, well at least you know there's one amigan around that could help with japanese related things.

@tetisoft

could i ask what's the plans so far about os4 internationalisation?
i mean do unicode multibyte is planned at all?
if yes, idealy, would it be for os4.1 ? or before that, in a boingbag update?
how support is thought/planned among the people in charge?

thanks :)

Last edited by keisangi on 24-Jan-2007 at 07:46 AM.

@keisangi

Quote:

could i ask what's the plans so far about os4 internationalisation? i mean do unicode multibyte is planned at all? if yes, idealy, would it be for os4.1 ? or before that, in a boingbag update?

You can ask but I wont answer, sorry.
Quote:

how support is thought/planned among the people in charge?

It seems that until now I'm the only one who feels in charge for supporting
more character sets than ISO-8859-1. Currently I'm still waiting for the first
non-latin set of catalogs for OS4, when this would be available to the users
they could demonstrate the developers better which parts of their OS4 programs
still are missing OS4 charset support. For example, the CygnusEd for OS4
doesnt support editing greek text, except when greek is the default language.

@TetiSoft

i see..
would that help then if i'd create a small japanese catalog for some part of wb or any apps? i could create it on the pc, using unicode.
the result could be displayed using typemanager, in the meanwhile.

i could create a small tutorial out of this, inviting others (say greek, korean, chinese, thai...) to create their catalogs too.

what do you think?
i'm trying to find something to do to help.. give me some task please ;)