/usr/share/xemacs21/mule-packages/etc/mule-ucs/README.Unicode is in xemacs21-mulesupport 2009.02.17.dfsg.1-1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 | --------------------------------------------------------------
Unicode definition on Mule-UCS(Universal enCoding System)
API and configuration manual.
Written by MIYASHITA Hisashi (himi@m17n.org)
reviced on 2001/4/13.
--------------------------------------------------------------
o ... License
Mule-UCS is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2, or (at your option)
any later version.
You should have received a copy of the GNU General Public License
along with Mule-UCS; see the file COPYING. If not, write to
the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
Boston, MA 02111-1307, USA.
o ... Introduction
Mule-UCS is NOT dependent on Unicode itself at all.
This manual describes unicode definition in this package.
Mule-UCS realizes Unicode encoding/decoding by unicode.el,
un-data.el, utf.el, un-trbase.el, mule-uni.el, and un-define.el.
unicode.el is a main part and defines fundamental type on Unicode
translation. When it is required at run-time, it will be automatically
loaded. un-data.el, utf.el, and un-trbase.el define
conversions and translations at byte-compiling time.
Normally, they are not necessary at run-time.
mule-uni.el defines private character set for Unicode, which is
mainly for covering undefined characters on Mule.
un-define.el controls default configuration and set up the run-time
environment. mule-uni.el and un-define.el are indispensable modules
at run-time. Mule-UCS embeds all information required at least for setup
into un-define.elc except for charset definitions.
Threfore, you only have to load it at the startup.
un-define.elc automatically loads other modules if necessary.
Mule-UCS Unicode definition(Mule-UCS-Unicode hereafter) provides
various coding-systems for external representaions and some APIs
for other Emacs Lisp libraries.
Currently, un-tools.el is only for feeble autodetection on UTF-8/16.
You don't need this module on Mule 4.1 or later. This Mule feature
have more powerful built-in autodetection on UTF-8/16.
After all, the minimum configuration of your .emacs, site-start.el, or
any other start up configuration is only one line as below.
---
(require 'un-define)
---
o ... Notice to BITMAP-MULE on Emacs 20 users:
Mule-UCS-Unicode defines three new charsets at the startup,
which are mule-unicode-0100-24ff, mule-unicode-2500-33ff, and
mule-unicode-e000-ffff. These will be introduced also in Emacs 21.
Because of these definitions, Emacs 20 consumes all unoccupied slots for
2-dimension, 1-column charsets. Hence, BITMAP-MULE cannot define
its own private charset named `bitmap'.
BITMAP-MULE version 8.4 or later, however, can displace the existance
charset in case that it cannot keep `bitmap' charset. If you'd
like to use this feature, set bitmap-alterable-charset variable to
tibetan-1-column or indian-1-column, and load bitmap.el(c)
after loading un-define.elc, for example,
---
(require 'un-define)
(setq bitmap-alterable-charset 'tibetan-1-column)
(require 'bitmap)
---
For detail, please refer to the manual of BITMAP-MULE.
o ... Instruction.
(1) basic features.
o ... Configured coding systems
Coding systems defined by Mule-UCS-Unicode could be sorted to UTF-8, UTF-16,
and UTF-7 categories. All of them follows `unicode-basic-translation-rule'
to translate between Emacs chars and UCS chars.
By default, pre-write-conversion and post-read-conversion is enabled on
these coding system conversions (mainly for Thai-TIS620). For detail, refer
the `post-read and pre-write conversions' section.
The followings are individual explanations of each coding system.
(A) ... UTF-8 category.
utf-8, utf-8-unix, utf-8-dos, utf-8-mac.
UTF-8 coding system. These differ as to Line Terminator.
utf-8 means line terminator is undetermined on decoding
and is LF on encoding.
utf-8-unix means line terminator is LF.
utf-8-dos means line terminator is CR+LF.
utf-8-mac means line terminator is CR.
utf-8-ws, utf-8-ws-unix, utf-8-ws-dos, utf-8-ws-mac.
These are basically identical to utf-8-* coding-systems,
but these encoder adds UTF-8 signature at the head.
(B) ... UTF-16 category.
One element of UTF-16 is 2 octets. However, its byte-order is
not fixed. Hence, we need 2 sorts of coding-systems, big endian
and little endian.
And all encoding format and syntax on UTF-16 is defined by The
Unicode Standard(#1). To begin with, the term UTF-16 was introduced
by ISO/IEC 10646 amendment. UTF-16 is implemented in order to have
compatibility with The Unicode Standard in encoding format.
Therefore, we have to follow The Unicode Standard when implementing
UTF-16. As a result, the line terminator of utf-16-le and utf-16-be
is Unicode Line Separator(U+2028) instead of convensional forms:
LF, CRLF, and CR. However, coding-systems of convensional forms are also
predefined. Thus, you will not feel inconvinience on line terminator.
But you have to take consideration to autodetection on line terminator.
utf-16-le, utf-16-le-unix, utf-16-le-dos, utf-16-le-mac
UTF-16 coding system. These byte-order is little endian.
utf-16-le means line terminator is undetermined on decoding
and is U+2028 on encoding.
utf-16-le-unix means line terminator is LF.
utf-16-le-dos means line terminator is CR+LF.
utf-16-le-mac means line terminator is CR.
utf-16-le-no-signature, utf-16-le-no-signature-unix,
utf-16-le-no-signature-dos, utf-16-le-no-signature-mac
same as utf-16-le-* but generate no Unicode signature(BOM).
utf-16-be, utf-16-be-unix, utf-16-be-dos, utf-16-be-mac
UTF-16 coding system. These byte-order is big endian.
Other features are identical to utf-16-le-*.
utf-16-be-no-signature, utf-16-be-no-signature-unix,
utf-16-be-no-signature-dos, utf-16-be-no-signature-mac
same as utf-16-be-* but generate no Unicode signature(BOM).
(C) ... UTF-7 category.
UTF-7 is a strange transformation format.
This is 7bit stream, and avoids to use unsafe characters on
RFC822 format. For detail, please refer to RFC1642.
All utf-7 coding systems supported by Mule-UCS-Unicode
encode UTF-16 format.
In other words, they can encode all codepoints defined by
The Unicode Standard.
utf-7, utf-7-unix, utf-7-dos, utf-7-mac
UTF-7 coding system.
utf-7 means line terminator is undetermined on decoding
and is LF on encoding.
utf-7-unix means line terminator is LF.
utf-7-dos means line terminator is CR+LF.
utf-7-mac means line terminator is CR.
These coding systems encode Set O characters directly.
utf-7-safe, utf-7-safe-unix, utf-7-safe-dos, utf-7-safe-mac
These coding systems encode Set O characters
with shifted format.
Other features are identical to utf-7-*.
o ... post-read and pre-write conversions
post-read and pre-write conversion of Mule-UCS-Unicode coding-systems
are controled by the following variables.
- Variable: un-define-enable-buffer-conversion
non-nil means the coding-systems defined by Mule-UCS-Unicode
converts buffer contents by calling post-read-conversion and
pre-write-conversion functions. By default, this variable is t.
- Variable: un-define-post-read-conversion-charsets-alist
This alist defines post-read conversion of coding-systems
defined by Mule-UCS-Unicode. Each slot looks like
(CHARSET . FUNCTION).
If read text have the specified charset, the corresponding
function will is called as post-read conversion function,
which must return return length of converted text.
If applicable functions are more than one, these will be
called in the order of the alist. However, the same function
is never called more than once during one conversion.
- Variable: un-define-pre-write-conversion-charsets-alist
This variable is the same as
`un-define-post-read-conversion-charsets-alist'
except that it is applied before writing.
o ... predefined Emacs Lisp functions.
Hereafeter, we use CHARACTER for data type.
Because characters are represented by numbers on Emacs,
CHARACTER is identical to NUMBER. But on XEmacs, because
character is first-class object, CHARACTER is not identical
to NUMBER. Therefore, we use CHARACTER to represent character
data type.
- Function: encode-char CHARACTER representation &optional restriction.
(Note that char-to-ucs is obsoleted by this function.)
This function describes a character in a proper representation.
Mule-UCS-Unicode registers `ucs' representation for this function,
and it describes a character in a UCS codepoint.
For example, (encode-char ?A 'ucs) returns 65 as a UCS codepoint.
`restriction' is an optional argument for the future extension.
(It limits possible candidates of representations.
When `restriction' is not specified, there is no limit on
candidate representations, thus the system select one
proper candidate arbitrarily. This facility has not been
implemented yet.)
- Function: decode-char representation object &optional restriction.
(Note that ucs-to-char is obsoleted by this function.)
This function convert the specified object to a character by
interpreting the object in view of the specified representation.
Mule-UCS-Unicode registers `ucs' representation for this function,
and it converts a UCS codepoint to a character.
For example, (decode-char 'ucs 65) returns A.
`restriction' is an optional argument for the future extension.
(It limits possible candidates of characters.
When `restriction' is not specified, there is no limit on
candidate characters, thus the system select one character
arbitrarily. This facility has not been implemented yet.)
- Command: insert-ucs-character NUMBER
Insert a character specified by UCS codepoint.
This command uses the above ucs-to-char function, and if it
successes to retrieve a character from the given UCS codepoint,
insert it to the current buffer. If it fails, cause an error.
e.g.
If you want to insert U+4E00, do `M-x insert-ucs-character',
then input `?\x4E00'(without any quotations).
o ... Simple dynamic modification functions.
Mule-UCS have dynamic modification features.
Owing to it, predefined translations (but they must be declared as a
`dynamic translation' beforehand) can be modified dynamically at run-time.
By using this feature, Mule-UCS-Unicode provides a simple function to
change priority to convert between internal characters and UCS characters.
However, because of simplicity, this function supports a narrow range of
facility on dynamic modification features.
If you want to make the most of dynamic modification features,
call directly TAE APIs.
- Function: un-define-change-charset-order &optional LIST
Change conversion priority of charsets.
LIST must be a list of charsets ordered by priority.
If LIST is nil or omitted,
unicode-basic-translation-charset-order-list is used
as an order list.
By following this order, Mule-UCS-Unicode encodes
internal characters or decodes UCS characters.
(2) advanced features.
o... Font encoder
Mule-UCS-Unicode supports the font encoder for Unicode.
By loading un-define.el, 'unicode-font-encoder is set up,
And it is associated to "iso10646" fonts when your Emacs uses
XLFD to set up font name. (On the contrary, because Meadow
does not use XLFD directly to set up fontset, this association is
not used.) Such associations are specified in font-ccl-encoder-alist,
but this variable is NOT effective on XEmacs.
Sample configurations for setting up Unicode encoding fonts in your fontset
are as below:
---
(set-fontset-font "<FONTSET NAME>"
'<CHARSET SYMBOL>
"*-iso10646-*")
---
<FONTSET NAME> is a fontset name, and "fontset-standard" is a
default fontset created at start up of Emacs.
<CHARSET SYMBOL> must be a symbol of the mule charset that is displayed
by a Unicode encoding font.
For example, the below configuration,
---
(set-fontset-font "fontset-standard"
'latin-iso8859-1
"*-iso10646-*")
(set-fontset-font "fontset-standard"
'cyrillic-iso8859-5
"*-iso10646-*")
---
set Unicode encoding fonts to "fontset-standard" to display
latin-iso8859-1 and cyrillic-iso8859-5 characters.
More simple and easy configuration is setting x-charset-registry
to charsets displayed by Unicode encoding fonts as below.
---
(put-charset-property 'latin-iso8859-1
'x-charset-registry "iso10646")
(put-charset-property 'cyrillic-iso8859-5
'x-charset-registry "iso10646")
(create-fontset-from-fontset-spec
"<YOUR FONTSET SPEC>")
---
<YOUR FONTSET SPEC> specifies spec. of fonts by XLFD, e.g.
"*-fixed-medium-r-normal-*-16-*-*-*-*-*-fontset-standard"
if you use 16 dot fixed font.
On XEmacs, you should manually specify font encoder by
set-charset-ccl-program on each charset.
For example,
---
(set-charset-ccl-program 'latin-iso8859-1 'unicode-font-encoder)
---
Before you apply this setting, you have to set an appropriate
Unicode encoding font to latin-iso8859-1.
----------
to be continued...
(3) supplemental features.
o... Supplemental translations.
Mule-UCS-Unicode has supplemental translations for other conversions
than Unicode Consrotium's definition. Currently, the translations
for JIS X 0221, JDK and Windows are prepared.
You can semi-automatically use this translation by the following
configuration.
---
(require 'un-supple)
(un-supple-enable <SYMBOL>)
---
Currently, candidates of <SYMBOL> are 'jisx0221, 'jdk, 'windows,
and nil. nil disables the configuration for the supplemental
translations.
For example, the below configuration,
---
(require 'un-supple)
(un-supple-enable 'windows)
---
enables the translation rule for Windows.
This feature is realized by the dynamic modification to
unicode-basic-translation-rule, thus, if you have already
modified unicode-basic-translation-rule, this configuration
may not be able to work correctly.
o... UNIDATA parser
By loading unidata.elc, you can easily view UNIDATA of Unicode
Consortium. A sample configuration is the followings.
---
(setq unidata-default-file-name "<The PATH of UnicodeData.txt>")
(require 'unidata)
---
After this configuration is enabled, you can use
M-x unidata-show-entry-from-UCS and M-x unidata-show-entry-from-char
to look up UNIDATA.
When you do M-x unidata-show-entry-from-UCS and input UCS codepoint,
you can view the corresponding UNIDATA entry to it, e.g.
by inputting "?\x100", you will view the entry on U+0100.
When you do M-x unidata-show-entry-from-char,
you can view the corresponding UNIDATA to the char located at
the current point under the current environment.
|