lis 508 lecture 1 bits bytes and characters

lis 508 lecture 1: bits, bytes and characters Thomas Krichel 2003 -09 -30

Structure • Numbers – Bits – Bytes • Character sets – Coded character set – Character endcoding

Literature, no need to read… • Norton “new inside the PC” chapter 4 • http: //www. danbbs. dk/~erikoest/bb_terms. htm • http: //wwwinfo. cern. ch/asdoc/WWW/public ations/ictp 99 N 2705. html • http: //www. cl. cam. ac. uk/~mgk 25/unicode. html

Information • Information is best understood as “what it takes to answer a question”. • The simplest question has a “yes” or “no” answer. Therefore a bit is the natural measure of information. • Term first used by John Turkey in 1946. • Concatenation of “binary digit”.

Usage of bits • Computers are sometimes classified by the number of bits they can process at one time. "32 bit processor" • Graphics are also often described by the number of bits used to represent each dot.

bits and bytes • a bit can take the values 0 or 1, thus it can describe 2 possibilities • two bits can take the value 00, 01, 10, 11, thus it can describe four 2× 2 possibilities • n bits can encode 2 power n possibilities. • The first chips used to process 8 bits at a time. It become customary to refer to them as a byte. It can encode 2 power 8 possibilities. • We can use binary numbers just as decimal numbers.

application of bytes • IP (Internet Protocol) numbers are used as the addresses of computers on the Internet. • In IP version 4 (the one that is most commonly used), each IP number has 4 bytes. • It is represented as x. x where x is a number between 0 and 255 (why? ) • how many computers can there be on the Internet at any one time?

decimal/binary numbers • • 0 1 2 3 4 5 6 7 0 1 10 11 100 101 110 111 • • 8 9 10 11 12 13 14 15 1000 1001 1010 1011 1100 1101 1110 1111

Many bytes • Larger units are – Kilo byte is 2 power 10 bytes (=1024 bytes) – Mega bytes is 2 power 20 bytes – Giga bytes is 2 power 30 bytes – Tera byte is 2 power 40 bytes • From ancient Greek words for "thousand", "large", "giant", and "monster", respectively. Terms date back to the French revolution.

Hex numbers • A byte is often represented by two hex numbers. • Each hex number can encode 16 values • Written 0 to 9, then A B C D E F. F is 15. • Conventionally prefixed with 0 x • Use Microsoft calculator with scientific notation to convert.

application of hex numbers • Media Access Control (mac) addresses of hardware that allows access to computer networks. They are 6 -byte numbers, each byte written as 2 hex numbers, e. g. 00: 60: 08: F 5: 20: A 9 • character numbers that you see when you are inserting a special symbol in Microsoftware, e. g. powerpoint.

Characters • Much of the information processed by computers is in the form of characters. • A character only makes sense for a human user of a minimum cultural level. • A character is not a glyph. – ligatures

Information in a computer file • A file is a piece of data on a stored on a computer. • Any file contains a sequence of 0 s and 1 s, like 10101001111010101… • For a computer to make sense of a file, it has to know what type of file it is.

executable files • Files that are executable are files that make the computer do something. For example the file starts a program, say powerpoint. An executable on one computer may not run on another • Non-executable files hold data that is used by an executable file. We will call them data files. Example: powerpoint slides file.

text files • Many data files contain textual data. • Textual data is a sequence of characters. • A character is an elementary symbol that has some meaning – alphabet letter – hieroglyph • Example: email file • Text files can be read by many computer programs.

non-text files • Examples for non-text files are – graphics files – movie files – sound files • non-text files are not very important in library settings – there is not way to organize information retrieval for non-text files. They have to be retrieved using a textual surrogate. – traditional library material are textual • will talk about this later.

Representing characters • Computers don't understand text, they only understand numbers. For computers to be able to treat text, there must be a correspondence between numbers and text characters. Such a correspondence is called a character set. • Examples for characters are – – a c ë €

Legacy character sets • In early days, computers were a lot less powerful than they are today. • Could only deal with the characters that are most commonly used. • Such sets are – ascii – ISO-8859 -1 – cp 1252

ASCII • American Standard Code for Information Interchange • 7 -bit character set. There is no such thing as 8 -bit ASCII • 95 printable symbols • 33 control characters (0 -31, 127) • http: //www. ccmr. cornell. edu/helpful_data/a scii 2. html has a list up to 127

some ASCII control characters • • • CR (13, ^M) is the carriage return LF (10, ^J) is the linefeed FF (12, ^L) is the form feed (new page) BS (8, ^H) is the backspace DEL (127, ALT-127) is delete ESC (27, ^[) escape

ISO-8859 -1 • ISO-8859 -1, aka ISO-latin-1 extends ASCII with characters that are commonly used by the western European languages. • It is the default character set of html. • Positions 128 to 159 are not used. • Cp 1252 fills these with graphic chars. It is as Microsoft character set.

This is not enough • There around 6800 different languages around. • Some of these languages use characters sets that are not finite, i. e. folks can make up now characters out of existing ones! • Setting up a character set for all languages is almost impossible.

ISO 10646 -1 • Defines the Universal Character Set (UCS) • UCS. contains the characters required to represent characters used by many known languages, even the likes of Oriya, Telugu, Bopomofo, Runic. • ISO 10646 defines formally a 31 -bit character set. They are represented as 32 bits, i. e. 4 bytes, or 8 hex chars. • Not finished.

Unicode • ISO is a inter-government agency. Slow and bureaucratic. • Industry has come together to work on Unicode, a 2 -byte character set. • With some minor exceptions, the Unicode characters are the some as the first 65536 characters in UCS. • Much better documented standard.

Unicode and legacy sets • The first 128 characters are identical to those in ASCII • The next 128 characters are identical to ISO 8859 -1 (Latin-1). • Unicode is well documented and the Unicode book can be downloaded from the Internet. A must-have for the serious digital librarian.

Politics… • Does it make sense to use Unicode rather than, say, ISO-latin-1? • Many commercial pieces of software have data files that contain character data interspersed with non-character data. Is that good?

http: //openlib. org/home/krichel Thank you for your attention!