Tutorial 17: Shady Characters
| ||||
| ||||
|
|
Character sets & character encodings
One of the most confusing things about HTML internationalization is the difference between character sets and character encodings. Once you grasp that, everything else will be pretty simple.
A character set is just that, a bunch of characters, in the way a human would understand them. For instance "A,B,C,D" and so on is the character set of the letters in the English alphabet. The character set of HTML is fixed and cannot be changed. It is called UCS, the Universal Character Set, which is the character set of the Unicode standard.
UCS is touted by many as the be-all and end-all of character sets, as it is supposed to contain every character ever used by the human race. I am not qualified to make a judgement as to whether this is truly the case, and I've heard occassional grumblings from various people that UCS cuts corners in various alphabets. Nonetheless, if you want to conform to the HTML 4.0 specification, you're stuck with UCS. But more on that later.
In UCS, as in any formally defined character set, characters are in a certain order, so each one has a number. This is very convenient for computers, which tend to deal with numbers instead of characters. It would be nice, some may argue, if all computers stored text in UCS (or some other universally accepted character set) with each character being stored as its UCS number. For various reasons that are out of the scope of this tutorial (and would probably put you to sleep faster than a crateful of Valiums anyway) this is not the case. The actual bytes stored in a computer might follow a different convention. The system of taking these bytes and translating them into characters in a document's character set is called a character encoding.
The character encoding of an HTML document is not fixed, so you have to supply it in order for a user agent to understand your document. Different user agents understand different character encodings, but several commonly used character encodings are understood by practically any user agent.
This is a pretty simple concept: A character set is a list of characters that may appear in a document, and a character encoding is a way of storing characters on a computer as bits. HTML's character set is fixed, and it's called UCS, which is supposed to cover every character you'll ever use, only it doesn't, but for most people that's close enough. An HTML document's character encoding, however, has to be supplied, so that a user agent can take a stream of bits and understand it as a bunch of characters.
This issue is further clouded by the fact that most people tend to say "character set" when they mean "character encoding" in the HTML sense. Which term is more accurate is another discussion that I steadfastly refuse to go into. As long as you understand that HTML documents have a fixed character set and a configurable character encoding, you should be able to make sense of the rest of character issues.
