Subject: Re: RFC-1345 "mnemonics"

From: RJ Atkinson <[email-masked]>
Date: Mon, 17 Sep 2007 07:48:49 -0400

I have been unhappy with RFC-1345 for many years,
primarily because it purports to be accurate about which letter
is at which code point in which character set encoding (sadly
the document isn't accurate in various details), which has misled
some number of implementers through the years and caused avoidable
interoperability problems, and because it claims to be "international",
when actually it has a very strong bias towards northern European
languages.

Earlier John Klensin suggested:
 > If an offspring of 1345 were defined as having, e.g., only
 > European languages using Roman-derived scripts in scope, then
 > I think there is more than adequate expertise around to review
 > a proposal and sufficient stability to not be affected by Unicode
 > changes, and that there should be no significant issue with IETF
 > sign-off or even standardization.

	If that were undertaken, then think the charter (if any)
and the document need to be made VERY clear that the scope is
only *European* languages (e.g. explicitly excluding other Romanised
languages such as Vietnamese Quoc Ngu [1,2]) -- and only those European
languages that normally have a Romanised script (e.g. not including
Cyrillic script European languages).

	As an example of why the scope needs to be narrowed that way,
RFC-1456 defines the very widely used convention (VIQR) for expressing
Quoc Ngu within the limitations of ISO-646/US-ASCII.  RFC-1345
pretty much botches non-European languages that happen to be
Romanised.  This was raised as an issue with the author of RFC-1345
prior to publication, but the author frankly was not interested
in hearing any non-European inputs.

	Folks who haven't been involved in multi-lingual
computing might not realise quite what a minefield this whole
area is.  Approximately speaking, choices about how to
represent glyphs/letters and how to encode them usually have
some inherent linguistic bias.  People who use languages
that aren't optimally encoded in some representation tend
not to be very happy with the non-optimal encodings their
language might have been given.

Yours,

Ran
[email-masked]

[1] Chu Nom is approximately unused at this point.  I only know of
1 book that has been printed with Chu Nom in my lifetime (obviously
I might have missed some, but it is clear the volume of Chu Nom
publications is remarkably low), and that one was careful also to
include the Quoc Ngu form on adjacent pages so that non-scholars
could read it.  Folks who can read Chinese and are interested in
linguistics might find that book interesting; I'm not sure how many
more would.

[2] http://www.nomfoundation.org

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: <[email-masked]>
Date: Mon, 17 Sep 2007 14:34:34 +0100

> some inherent linguistic bias. 

In addition, many applications of encoding glyphs need to consider
direction. It is one thing to transliterate another writing system into
ASCII for the purposes of providing a readable format. It is another
thing to have an encoding that works in the other direction, so that
some form of ASCII input can be accurately translated into a set of
UNICODE glyphs. The former (output tranliteration) is not so hard to
achieve, eg. Any russian speaker will understand "Ya ne znayu chto
skazat'" and any Chinese who has learned a roman alphabet language
should be able to understand "Wo mi-lu le". Of course "Ja nye znaju shto
skazat" would be just as readable to most Russians, and most Chinese
would struggle with long texts that lack the accents that official Hanyu
Pinyin uses to mark the four tones. I think it is conceivable to supply
an official output transliteration to ASCII for all Unicode glyphs and
much of this work has already been done by language groups, for instance
TISCII transliterates Tamil letters into ASCII.

Translating from ASCII into Unicode is far more complex and probably
impossible without introducing inter-glyph gaps and weird accent codes
like RFC 1345 did. Many of the input methods that are used for typing
foreign language glyphs into a computer are actually "Input Method
Editors" that have inbuilt dictionaries and help users choose one of a
series of possible glyphs. For instance, Japanese can represent a single
syllable fu with 3 or more glyphs. To choose the right one you need to
know if the entire word is borrowed from a foreign language, and if not,
then there is still the choice whether or not to use a Hiragana glyph,
or choose one of the Kanji glyphs borrowed from Chinese. There are at
least two Kanji glyphs with an ON-reading of "fu". In spite of all this
complexity on input, there is a standard transliteration for output. 

> People who use languages that 
> aren't optimally encoded in some representation tend not to 
> be very happy with the non-optimal encodings their language 
> might have been given.

That is the key to this whole thing. What is the use-case for these
ASCII encodings? If an encoding is not usable by native speakers of the
language using the original glyphs, then I can't see any worthwhile use
case at all. UTF-8 works for any machine-to-machine communication.
Existing input methods work for converting from ASCII to glyphs but
these are not simple mapping tables. And existing transliteration
systems work for generating readable ASCII text from glyphs, although
they may not be fully standardised. For instance a Russian named "Yuri"
(transliterated according to English pronunciation rules) will have his
name written as "Iouri" on his passport because that is transliterated
according to French pronunciation rules since French was the former
lingua franca of international diplomacy.

RFC 1345 should be deprecated because it misleads application developers
e.g. the Lynx case, and the work on transliteration and input methods is
being done more effectively outside the IETF.

--Michael Dillon