The Unicode Standard

The Unicode Standard is the specification of an encoding scheme for written characters and text. It is a universal standard that enables consistent encoding of multilingual text and allows text data to be interchanged internationally without conflict. The ISO standards for C and C++ refer to ISO/IEC 10646-1:2000, Information Technology--Universal Multiple-Octet Coded Character Set (UCS). (The term octet is used by ISO to refer to a byte.) The ISO/IEC 10646 standard is more restrictive than the Unicode Standard in the number of encoding forms: a character set that conforms to ISO/IEC 10646 is also conformant to the Unicode Standard.

The Unicode Standard specifies a unique numeric value and name for each character and defines three encoding forms for the bit representation of the numeric value. The name/value pair creates an identity for a character. The hexadecimal value representing a character is called a code point. The specification also describes overall character properties, such as case, directionality, alphabetic properties, and other semantic information for each character. Modeled on ASCII, the Unicode Standard treats alphabetic characters, ideographic characters, and symbols, and allows implementation-defined character codes in reserved code point ranges. The encoding scheme of the Unicode Standard is therefore sufficiently flexible to handle all known character encoding requirements, including coverage of historical scripts from any country in the world.

C99 and C++ allow the universal character name construct defined in ISO/IEC 10646 to represent characters outside the basic source character set. Both languages permit universal character names in identifiers, character constants, and string literals. In C++, this language feature is independent of the language level specified at compile time.

The following table shows the generic universal character name construct and how it corresponds to the ISO/IEC 10646 short name.

Universal character name ISO/IEC 10646 short name
\UNNNNNNNN NNNNNNNN
\uNNNN 0000NNNN
where N is a hexadecimal digit

C99 and C++ disallow the hexadecimal values representing characters in the basic character set (base source code set) and the code points reserved by ISO/IEC 10646 for control characters. The following characters are also disallowed:

XL C/C++ implements the data types uint_least16_t and uint_least32_t to process UTF-16 and UTF-32 characters in C and C++ in conformance with the Unicode Standard. The data types, also referred to as u-literals and U-literals, respectively, are the string literals required by the Unicode Standard to specify a UTF-16 or UTF-32 character, and were approved by the C Standards Committee. Previously, a UTF-16 character was represented by an unsigned short, and a UTF-32 character, by an unsigned int.

The support for u-literals and U-literals is similar to that for wide character literals.

u"s-char-sequence"
Denotes an array of uint_least16_t. The corresponding character literal is denoted by
U'c-char-sequence'

U"s-char-sequence"
Denotes an array of uint_least32_t. The corresponding character literal is denoted by
U'c-char-sequence'

For example,

uint_least16_t  msg[] = u"ucs characters \u1234 and \U81801234 ";

String concatenation

The u-literals and U-literals follow the same concatenation rule as wide character literals: the normal character string is widened if they are present. The following shows the allowed combinations. All other combinations are invalid.

Combination Result
u"a" u"b" u"ab"
u"a" "b" u"ab"
"a" u"b" u"ab"
   
U"a" U"b" U"ab"
U"a" "b" U"ab"
"a" U"b" U"ab"

Multiple concatentations are allowed, with these rules applied recursively.

Related References

IBM Copyright 2003