Punycode
Punycode is a representation of Unicode with the limited ASCII character subset used for Internet hostnames. Using Punycode, host names containing Unicode characters are transcoded to a subset of ASCII consisting of letters, digits, and hyphens, which is called the Letter-Digit-Hyphen subset. For example, München is encoded as Mnchen-3ya.
While the Domain Name System technically supports arbitrary sequences of octets in domain name labels, the DNS standards recommend the use of the LDH subset of ASCII conventionally used for host names, and require that string comparisons between DNS domain names should be case-insensitive. The Punycode syntax is a method of encoding strings containing Unicode characters, such as internationalized domain names, into the LDH subset of ASCII favored by DNS. It is specified in IETF Request for Comments 3492.
Encoding procedure
As stated in RFC 3492, "Punycode is an instance of a more general algorithm called Bootstring, which allows strings composed from a small set of 'basic' code points to uniquely represent any string of code points drawn from a larger set." Punycode defines parameters for the general Bootstring algorithm to match the characteristics of Unicode text. This section demonstrates the procedure for Punycode encoding, using the example of the string "bücher", which is translated into the label "bcher-kva".Separation of ASCII characters
First, all ASCII characters in the string are copied from input to output, skipping over any other characters. For example, "bücher" is copied to "bcher". If any characters were copied, i.e. there was at least one ASCII character in the input, an ASCII hyphen is added to the output next. Since the ASCII hyphen is an ASCII character, the hyphen may itself appear in the output before this additional hyphen. However, the additional hyphen does not cause any ambiguity when reading the output, as no later part of the encoding process can introduce another ASCII hyphen; if there are one or more ASCII hyphens in the output, the last one always signifies the end of the ASCII characters.Encoding of non-ASCII character insertions as code numbers
The next part of the encoding process first requires an understanding of the decoder, which is a finite-state machine with two state variables i and n. i is an index into the string ranging from zero to the current length of the extended string.i starts at zero, and n starts at 128. The state progression is a monotonic function. A state change either increments i or, if i is at its maximum, resets i to zero and increments n by 1, then goes back to incrementing i in the following state change. At each state change, either the code point denoted by n is inserted or it is not inserted.
The code numbers generated by the encoder represent how many possibilities to skip before an insertion is made. There are six possible places to insert a character in the current string "bcher". There are 124 code points between the last one considered and "ü". Also there is one position to insert a "ü" that needs to be skipped. That is why it is necessary to tell the decoder to skip a total of + 1 = 745 possible insertions before getting to the one required. Once the character is inserted there are now seven possible places to insert another character.
Re-encoding of code numbers as ASCII sequences
Punycode uses generalized variable-length integers to represent these values. For example, this is how "kva" is used to represent the code number 745:
A number system with little-endian ordering is used which allows variable-length codes without separate delimiters: a digit lower than a threshold value marks that it is the most-significant digit, hence the end of the number. The threshold value depends on the position in the number and also on previous insertions, to increase efficiency. Correspondingly the weights of the digits vary.
In this case a number system with 36 symbols is used, with the case-insensitive 'a' through 'z' equal to the decimal numbers 0 through 25, and '0' through '9' equal to the decimal numbers 26 through 35. Thus "kva", corresponds to the decimal number string "10 21 0".
To decode this string of symbols, a sequence of thresholds will be needed, in this case. The threshold starts out as 1 and the weight is 1. The first symbol is the units place value; 'k' with a weight of 1 equals 10. After this, the threshold value is adjusted; in this case the threshold is again 1. The second symbol has a place value of 36 minus the previous threshold value, in this case, 35. Therefore, the sum of the first two symbols 'k' and 'v' is 10 × 1 + 21 × 35. Since the second symbol is not less than the threshold value of 1, there is more to come. The weight for the third symbol is the previous weight times 36 minus the second threshold value; 35 × 35. The third symbol in this example is 'a', which is less than the third threshold 26, meaning that it is the last part of the number. Therefore, "kva" represents the decimal number + + = 745.
The thresholds themselves are determined by an algorithm keeping them between 1 and 26 inclusive, meaning the last character of an encoding will always be alphabetic. The case can then be used to provide information about the original case of the string.
For the insertion of a second special character in "bücher", the first possibility is "büücher" with code "bcher-kvaa", the second "bücüher" with code "bcher-kvab", etc. After "bücherü" with code "bcher-kvae" comes codes representing insertion of ý, the character following ü, starting with "ýbücher" with code "bcher-kvaf", etc.
To make the encoding and decoding algorithms simple, no attempt has been made to prevent some encoded values from encoding inadmissible Unicode values: however, these should be checked for and detected during decoding.
Punycode is designed to work across all scripts, and to be self-optimizing by attempting to adapt to the character set ranges within the string as it operates. It is optimized for the case where the string is composed of zero or more ASCII characters and in addition characters from only one other script system, but will cope with any arbitrary Unicode string. Note that for DNS use, the domain name string is assumed to have been normalized using Nameprep and filtered against an officially registered language table before being punycoded, and that the DNS protocol sets limits on the acceptable lengths of the output Punycode string.
Examples
The following table shows examples of Punycode encodings for different types of input.| Input | Punycode of input | Description of input |
| The empty string. | ||
| Only ASCII characters, one, lowercase. | ||
| Only ASCII characters, one, uppercase. | ||
| Only ASCII characters, one, a digit. | ||
| Only ASCII characters, one, a hyphen. | ||
| Only ASCII characters, two hyphens. | ||
| Only ASCII characters, more than one, no hyphens. | ||
| Only ASCII characters, one hyphen. | ||
| Only ASCII characters, with spaces. | ||
| Only ASCII characters, mixed symbols. | ||
| No ASCII characters, one Latin-1 Supplement character. | ||
| No ASCII characters, one Greek character. | ||
| No ASCII characters, one CJK character. | ||
| No ASCII characters, one emoji character. | ||
| No ASCII characters, more than one character. | ||
| Mixed string, with one character that is not an ASCII character. | ||
| Only ASCII characters, equal to the Punycode of "München". | ||
| Mixed string, with one character that is not ASCII, and a hyphen. | ||
| Mixed string, with one space, one hyphen, and one character that is not ASCII. | ||
| Mixed string, two non-ASCII characters. | ||
| Russian, without ASCII. | ||
| Thai, without ASCII. | ||
| Korean, without ASCII. | ||
| Japanese, without ASCII. | ||
| Japanese with ASCII. | ||
| Mixed non-ASCII scripts. |