Character Encoding: ASCII, ISCII and Unicode

Hello students, welcome to Kwickprep. You type your name in English, then a message in Hindi, then send a smiley. The computer only knows zeros and ones. So how does it store letters? Today we will learn character encoding with ASCII, ISCII and Unicode, including UTF-8 and UTF-32.

First, why do characters need codes? A computer can only store binary numbers, not shapes like A or B. So we give every character a number, called its code. That number is then stored in binary. Just like a roll number list, both computers must use the same code table, or the text turns into rubbish.

Let us learn three key words. A character is any single letter, digit, symbol or space, like A, seven or the rupee sign. A character set is the full list of characters, each with its own code. An encoding is the rule that says how those codes are written as bits and bytes.

The first popular code is ASCII, the American Standard Code for Information Interchange. Capital A is sixty five. Capital B is sixty six, because the letters are in order. Small a is ninety seven, exactly thirty two more than capital A. The digit zero, as a character, is forty eight, not zero. Even a space has a code, thirty two.

ASCII has a big limit. It uses seven bits, so it has only one hundred twenty eight codes, from zero to one hundred twenty seven. That is enough for English letters, digits and common symbols. But there is no room for Hindi, Gujarati, Tamil or emoji. Extended ASCII uses eight bits, but even that holds only two hundred fifty six characters.

Pause and predict. Capital C is sixty seven in ASCII. What is small c? Add thirty two, so small c is ninety nine.

India needed its own solution, and it made ISCII. ISCII stands for Indian Script Code for Information Interchange. It became an Indian standard in the late nineteen eighties. It uses eight bits, and its first one hundred twenty eight codes are the same as ASCII. The upper codes hold Indian letters and vowel signs called matras. Because Indian scripts share the same letter order, one table serves Devanagari, Bengali, Gujarati, Tamil and other scripts.

ISCII also had limits. It has only two hundred fifty six codes in total. Switching between scripts needs extra control marks, so files were hard to share. It cannot hold Chinese, Arabic and Hindi together in one file. So today it is mostly replaced by Unicode.

Unicode is the modern answer. It aims to give one unique code to every character in every language. Each code is called a code point. We write it as U plus a hex number, so capital A is U plus zero zero four one. Unicode has room for over eleven lakh code points. It already covers Indian scripts, the rupee sign, maths symbols and emoji.

Here are some real code points. Capital A is U plus zero zero four one, the same number as in ASCII. The Hindi letter ka is U plus zero nine one five. The Gujarati letter ka is U plus zero A nine five. The rupee sign is U plus two zero B nine.

Unicode code points are stored using an encoding, and the two in your syllabus are UTF-8 and UTF-32. UTF-8 uses one to four bytes, while UTF-32 always uses four. So capital A takes one byte in UTF-8, but four in UTF-32. The Hindi ka takes three bytes in UTF-8, and still four in UTF-32. UTF-8 stores English exactly like ASCII, so old files still work. That is why UTF-8 is used for almost every web page, while UTF-32 is simpler for some programs to work with.

If your course uses Python, you can see these codes yourself. The ord function gives the code of a character, so ord of A prints sixty five. The chr function works the other way, so chr of ninety seven prints small a. Encode turns the Hindi ka into UTF-8 bytes, and len counts them. It prints three.

Let us revise what we learned today. Every character is stored as a number code, in binary. ASCII uses seven bits for one hundred twenty eight characters, mostly English. ISCII uses eight bits, adding Indian scripts on top of ASCII. Unicode gives one code point to every character in the world. UTF-8 uses one to four bytes, and UTF-32 always uses four.