Breaking Classical Ciphers Cunsheng Ding Mordecai Golin HKUST

Breaking Classical Ciphers Cunsheng Ding Mordecai Golin HKUST, Hong Kong 11/2/2020 EMB project 1

Agenda for Today • More on breaking monoalphabetic substitution ciphers • Breaking Vigenere ciphers 11/2/2020 EMB project 2

Breaking Monoalphabetic Substitution Ciphers 11/2/2020 EMB project 3

Monoalphabetic Substitution Ciphers • Assign to each plaintext letter a different coding letter to ensure correct decryption. • a b c d e f g h i j k l m n o p q r s t u v w x y z • u v w c a b x y z q p o r s t e f g l m n d h i j k • Today we look into monoalphabetic substitution ciphers (plaintext) • MTCUJ HA OTTP ZSMT RTSTUOEYUVAMZW LNVLMZMNMZTS WZEYAGL (ciphertext) 11/2/2020 EMB project 4

Frequency of Letters • Letters in order of frequency (highest to lowest): • ETAONRISHDLFCMUGYPWBVKXJQZ • The letters can be grouped further by their frequencies: – – – 11/2/2020 Very Common: E Common: T Next most common: A O N R I S Less Common: H Less Common Still: D L F C M U Rare: V K X J Q Z EMB project 5

Frequency of Digrams • Most common pairs of letters (digrams), in order of frequency: • TH HE AN RE ER IN ON AT ND ST ES EN OF TE ED OR TI HI AS TO AR OU IS IT LE NT RI SE HA AL DE EA NE RO OM IO WE VE TA TR CO ME NG MA CE RA IC NS UT US BE UN CH WA SI LA AD LI RT CA NC SO LL UR EL RS EM AC IM PR TT OT WI EC • The most common words in English are: • THE OF AND TO IN A IS THAT FOR IT BY ARE BE WAS AS HE WITH HIS 11/2/2020 EMB project 6

Ciphertext from a Monoalphebetic Substitution Cipher • • WNWHXHPBJIHGXHNWFOUBXHUX QIJCODILNDBOLWONOLJBHA, UIFMGVBW MHBPXJOBNT XUIFIABONODWDBYHWXXGGMOXY, "AIHXWJX NDXAIIY IFYYWTOQDXH TIMUIMFYYIXUIFIAT EMONVXUWMOX BNQWOZMH. " NDIOXBHDBO LJIZXOOBIHGBADN IHUXDWPX WHWFTSXYXUIOTONXGO LMJXFTZIJ NDXBHNXFFXUNMWF UDWFFXHAX, VMNNIYWTNDXBJQIJC DWOWHMJAXHUTNIBN. NDXTJXNMJH NIW ZIJXONNDXT DWYQIJCXYBH, IHFTNIZBHY BNUDILLXYYIQH; NDXTDXWJIFYXJUIFFXWAMXOJXGBHBOUX WVIMNVBJYOFWON OXXHYXUWYXOWAI; NDXTQWFCIHLXJGWZJION NMJHXYGMODT VTAFIVWFQWJGBHA. WOYXOUJBVXY BHNDBOOLXUBWFBOOMX, NDXQIJFYBOHIQ MHYXJ-AIBHA WJXGWJCWVFXOXN IZNJWHOBNBIHO: LILMFWNBIH AJIQNDDWO JXWUDXYWH BHZFXUNBIHLIBHN WHYBOONWJNBHA NIFXPXFIZZ, NDXYXPXFILBHA QIJFYBOVXUIGBHAYXPXFILXY, WHYXHPBJIHGXHNWF LJIVFXGONDWN MOXYNIVXFIUWF-BSXY WJXJWYBWNBHA XPXJTQDXJX. NDXBHNXJUIHHXUNXY UDWHAXOWJX, BHYXGIAJWLDXJ EIOXLDUDWGBX'OQIJYO, W "KMBXNJXPIFMNBIH" NDWNJXWUDXO BHNIXPXJT UIJHXJIZFBZX. NDXTLIOXNDJXWNO VMNWFOIIZZXJ ILLIJNMHBNBXO. DWPBHAOXXH ZBJONDWHYQDWN BODWLLXHBHA, NDXWMNDIJO IZNDXWJNBUFXO DXJXDWPXNDJIQH NDXGOXFPXOBHNI NDXNWOCIZ DXFLBHAOIUBXNT HWPBAWNXNDX ODIWFOWDXWY. 11/2/2020 EMB project 7

Our Task Today • We shall break this cipher. • We were told that the original plaintext is in English and was encrypted with a one-to-one mapping from the English alphabet to itself. • We can see that some spaces between words and punctuation were deleted, while other spaces between words and punctuation are kept. • We shall use the tools at the URL: – http: //cryptoclub. math. uic. edu/indexmain. html 11/2/2020 EMB project 8

Step 1 Compute the frequency distributions of single letters and digrams using the online software at the URL: http: //cryptoclub. math. uic. edu/indexmain. html 11/2/2020 EMB project 9

Frequency of Letters: Comparison • • • • In message X: 13. 0 I: 8. 3 N: 8. 3 W: 7. 5 H: 7. 0 O: 6. 9 B: 6. 7 J: 6. 0 D: 5. 8 F: 4. 4 Y: 4. 0 U: 3. 1 M: 2. 7 11/2/2020 In English e: 12. 7 t: 9. 1 a: 8. 2 i: 7. 0 n: 6. 7 o: 6. 3 h: 6. 1 r: 6. 0 d: 4. 3 q: 4. 3 l: 4. 0 c: 2. 8 u: 2. 8 • • • • EMB project In message A: 2. 6 L: 2. 4 T: 2. 1 G: 1. 8 Z: 1. 8 Q: 1. 7 P: 1. 5 V: 1. 4 C: 0. 6 E: 0. 2 S: 0. 2 K: 0. 1 R: 0. 0 In English m: 2. 4 w: 2. 3 f: 2. 2 s: 2. 2 g: 2. 0 y: 2. 0 p: 1. 9 b: 1. 5 v: 1. 0 k: 0. 8 j: 0. 2 x: 0. 1 z: 0. 1 10

Most Common Digraphs • In message: – ND, DX, BH, DW, JX, XJ, IH, HA, XY, XH • In English: – th, he, in, er, ed, an, nd, ar, re, en 11/2/2020 EMB project 11

Step 2 Look at specialities in the ciphertext (Since some spaces between words and some punctuation symbols are kept, the most important thing is to look for special indications) 11/2/2020 EMB project 12

Looking for Indications in Ciphertext • Read the ciphertext carefully, and write down any piece of valuable information you obtain from reading the ciphertext. • Let me know what you have discovered. 11/2/2020 EMB project 13

Looking for Indications in Ciphertext • In the ciphertext we have the following: • BHYXGIAJWLDXJ EIOXLDUDWGBX'OQIJYO, W "KMBXNJXPI-FMNBIH" • What can you infer from it? 11/2/2020 EMB project 14

Indications in Ciphertext • In the ciphertext we have the following: • BHYXGIAJWLDXJ EIOXLDUDWGBX'OQIJYO, W "KMBXNJXPI-FMNBIH" • What can you infer from it? – “O s” or “X s” (not consistent with the frequency distribution of single letters) – “W a” 11/2/2020 EMB project 15

Step 3 Looking at the frequency distributions of single letters and digrams 11/2/2020 EMB project 16

Looking at Most Common Digraphs • In message: – ND, DX, BH, DW, JX, XJ, IH, HA, XY, XH • In English: – th, he, in, er, ed, an, nd, ar, re, en • What can you tell from this statistics? 11/2/2020 EMB project 17

Looking at Most Common Digraphs • In message: – ND, DX, BH, DW, JX, XJ, IH, HA, XY, XH • In English: – th, he, in, er, ed, an, nd, ar, re, en • What can you tell from this statistics? – It is very likely that “D h”, “X e”, “N t” – This is more or less consistent with the frequency distribution of single letters. 11/2/2020 EMB project 18

The first Guess • Combining what we have on Page 15 and Page 18, we first guess the following: – W a – – 11/2/2020 D h X e N t O s EMB project 19

After the First Guess • • ata. He. HPBJIHGe. Hta. Fs. UBe. HUe QIJCsh. ILth. Bs. Lasts. LJBHA, UIFMGVBa MHBPe. Js. Bt. T e. UIFIABstshah. BYHaee. GGMse. Y, "AIHea. Je the. AIIY IFYYa. Ts. Qhe. H TIMUIMFYYIe. UIFIAT EMst. Ve. Ua. Mse Bt. Qas. ZMH. " th. Ise. BHh. Bs LJIZess. BIHGBAht IHUeha. Pe a. Ha. FTSe. Ye. UIs. Tste. Gs LMJe. FTZIJ the. BHte. FFe. Ut. Ma. F Uha. FFe. HAe, VMtt. IYa. Tthe. BJQIJC hasa. HMJAe. HUTt. IBt. the. TJet. MJH t. Ia ZIJestthe. T ha. YQIJCe. YBH, IHFTt. IZBHY Bt. Uh. ILLe. YYIQH; the. Thea. JIFYe. JUIFFea. AMes. Je. GBHBs. Ue a. VIMt. VBJYs. Fast see. HYe. Ua. Yesa. AI; the. TQa. FCIHLe. JGa. ZJIst t. MJHe. YGMsh. T VTAFIVa. FQa. JGBHA. as. Yes. UJBVe. Y BHth. Bss. Le. UBa. FBss. Me, the. QIJFYBs. HIQ MHYe. J-AIBHA a. Je. Ga. JCa. VFeset IZt. Ja. Hs. Bt. BIHs: LILMFat. BIH AJIQthhas Jea. Uhe. Ya. H BHZFe. Ut. BIHLIBHt a. HYBssta. Jt. BHA t. IFe. Pe. FIZZ, the. Ye. Pe. FILBHA QIJFYBs. Ve. UIGBHAYe. Pe. FILe. Y, a. HYe. HPBJIHGe. Hta. F LJIVFe. Gsthat Mse. Yt. IVe. FIUa. F-BSe. Y a. Je. Ja. YBat. BHA e. Pe. JTQhe. Je. the. BHte. JUIHHe. Ute. Y Uha. HAesa. Je, BHYe. GIAJa. Lhe. J EIse. Lh. Uha. GBe's. QIJYs, a "KMBet. Je. PI-FMt. BIH" that. Jea. Uhes BHt. Ie. Pe. JT UIJHe. JIZFBZe. the. TLIseth. Jeats VMta. Fs. IIZZe. J ILLIJt. MHBt. Bes. ha. PBHAsee. H ZBJstha. HYQhat Bsha. LLe. HBHA, thea. Mth. IJs IZthea. Jt. BUFes he. Jeha. Peth. JIQH the. Gse. FPes. BHt. I thetas. CIZ he. FLBHAs. IUBet. T Ha. PBAatethe sh. Ia. Fsahea. Y. 11/2/2020 EMB project 20

What Shell We do Now • After the first guess and first replacement, we should check if there is anything wrong in the partially decrypted text. • If YES, this means that some part in our first guess must be wrong. So modification of the first guess should be done. • If the partially decrypted text looks okay, we should try to get information on the partially decrypted text. 11/2/2020 EMB project 21

Anything Wrong in the Text: Page 20? • If yes, tell me what is wrong? • If no, look for further information in the text of Page 20. 11/2/2020 EMB project 22

Seeking Information on Page 20 • The short phrase: “t. Ia” on page 20 • The second guess – It cannot be “tea”, because “X e” already. – Hence it should be two words, and thus “to a”. – Thus “I o” – I o – F l – Y d • Then look at the short phrase: “shoa. Fsahea. Y” – The only possibility is “shoals ahead” – Hence, “F l”, “Y d” 11/2/2020 EMB project 23

The Second Guess • The First guess – W a – – D h X e N t O s • The second guess – I o – F l – Y d 11/2/2020 EMB project 24

After the Second Guess • ata. He. HPBJo. HGe. Htals. UBe. HUe Qo. JCsho. Lth. Bs. Lasts. LJBHA, Uol. MGVBa MHBPe. Js. Bt. T e. Uolo. ABstshah. Bd. Haee. GGMsed, "Ao. Hea. Je the. Aood oldda. Ts. Qhe. H To. MUo. Mlddoe. Uolo. AT EMst. Ve. Ua. Mse Bt. Qas. ZMH. " those. BHh. Bs LJo. Zess. Bo. HGBAht o. HUeha. Pe a. Hal. TSede. Uos. Tste. Gs LMJel. TZo. J the. BHtelle. Ut. Mal Uhalle. HAe, VMttoda. Tthe. BJQo. JC hasa. HMJAe. HUTto. Bt. the. TJet. MJH to a Zo. Jestthe. T had. Qo. JCed. BH, o. Hl. Tto. ZBHd Bt. Uho. LLeddo. QH; the. Thea. Jolde. JUollea. AMes. Je. GBHBs. Ue a. Vo. Mt. VBJdslast see. Hde. Uadesa. Ao; the. TQal. Co. HLe. JGa. ZJost t. MJHed. GMsh. T VTAlo. Val. Qa. JGBHA. • asdes. UJBVed BHth. Bss. Le. UBal. Bss. Me, the. Qo. Jld. Bs. Ho. Q MHde. J-Ao. BHA a. Je. Ga. JCa. Vleset o. Zt. Ja. Hs. Bt. Bo. Hs: Lo. LMlat. Bo. H AJo. Qthhas Jea. Uheda. H BHZle. Ut. Bo. HLo. BHt a. Hd. Bssta. Jt. BHA tole. Pelo. ZZ, thede. Pelo. LBHA Qo. Jld. Bs. Ve. Uo. GBHAde. Pelo. Led, a. Hde. HPBJo. HGe. Htal LJo. Vle. Gsthat Msedto. Velo. Ual. BSed a. Je. Jad. Bat. BHA e. Pe. JTQhe. Je. the. BHte. JUo. HHe. Uted Uha. HAesa. Je, BHde. Go. AJa. Lhe. J Eose. Lh. Uha. GBe's. Qo. Jds, a "KMBet. Je. Po-l. Mt. Bo. H" that. Jea. Uhes BHtoe. Pe. JT Uo. JHe. Jo. Zl. BZe. the. TLoseth. Jeats VMtalsoo. ZZe. J o. LLo. Jt. MHBt. Bes. ha. PBHAsee. H ZBJstha. Hd. Qhat Bsha. LLe. HBHA, thea. Mtho. Js o. Zthea. Jt. BUles he. Jeha. Peth. Jo. QH the. Gsel. Pes. BHto thetas. Co. Z hel. LBHAso. UBet. T Ha. PBAatethe shoals ahead. 11/2/2020 EMB project 25

Anything Wrong in the Text: Page 25? • If yes, tell me what is wrong? • If no, look for further information in the text of Page 25. 11/2/2020 EMB project 26

Any Indication on Page 25 • Look at the phrase “thetas. Co. Z” – It is most likely “the task of” – Hence, “C k”, “Z f” – This is consistent with the frequency distribution of single letters. 11/2/2020 EMB project 27

The Third Guess • The First guess – W a – – • The third guess – C k D h X e N t O s – Z f • The second guess – I o – F l – Y d 11/2/2020 EMB project 28

After the Third Guess • ata. He. HPBJo. HGe. Htals. UBe. HUe Qo. Jksho. Lth. Bs. Lasts. LJBHA, Uol. MGVBa MHBPe. Js. Bt. T e. Uolo. ABstshah. Bd. Haee. GGMsed, "Ao. Hea. Je the. Aood oldda. Ts. Qhe. H To. MUo. Mlddoe. Uolo. AT EMst. Ve. Ua. Mse Bt. Qasf. MH. " those. BHh. Bs LJofess. Bo. HGBAht o. HUeha. Pe a. Hal. TSede. Uos. Tste. Gs LMJel. Tfo. J the. BHtelle. Ut. Mal Uhalle. HAe, VMttoda. Tthe. BJQo. Jk hasa. HMJAe. HUTto. Bt. the. TJet. MJH to a fo. Jestthe. T had. Qo. Jked. BH, o. Hl. Ttof. BHd Bt. Uho. LLeddo. QH; the. Thea. Jolde. JUollea. AMes. Je. GBHBs. Ue a. Vo. Mt. VBJdslast see. Hde. Uadesa. Ao; the. TQalko. HLe. JGaf. Jost t. MJHed. GMsh. T VTAlo. Val. Qa. JGBHA. • asdes. UJBVed BHth. Bss. Le. UBal. Bss. Me, the. Qo. Jld. Bs. Ho. Q MHde. J-Ao. BHA a. Je. Ga. Jka. Vleset oft. Ja. Hs. Bt. Bo. Hs: Lo. LMlat. Bo. H AJo. Qthhas Jea. Uheda. H BHfle. Ut. Bo. HLo. BHt a. Hd. Bssta. Jt. BHA tole. Peloff, thede. Pelo. LBHA Qo. Jld. Bs. Ve. Uo. GBHAde. Pelo. Led, a. Hde. HPBJo. HGe. Htal LJo. Vle. Gsthat Msedto. Velo. Ual. BSed a. Je. Jad. Bat. BHA e. Pe. JTQhe. Je. the. BHte. JUo. HHe. Uted Uha. HAesa. Je, BHde. Go. AJa. Lhe. J Eose. Lh. Uha. GBe's. Qo. Jds, a "KMBet. Je. Po-l. Mt. Bo. H" that. Jea. Uhes BHtoe. Pe. JT Uo. JHe. Jofl. Bfe. the. TLoseth. Jeats VMtalsooffe. J o. LLo. Jt. MHBt. Bes. ha. PBHAsee. H f. BJstha. Hd. Qhat Bsha. LLe. HBHA, thea. Mtho. Js ofthea. Jt. BUles he. Jeha. Peth. Jo. QH the. Gsel. Pes. BHto thetaskof hel. LBHAso. UBet. T Ha. PBAatethe shoals ahead. 11/2/2020 EMB project 29

Anything Wrong in the Text: Page 29? • If yes, tell me what is wrong? • If no, look for further information in the text of Page 29. 11/2/2020 EMB project 30

Any Indication on Page 29 • Look at the phrases “fo. Jestthe. T” and “VMttoda. Tthe. BJQo. Jk” – It is very likely that “J r”, “T y” – This is consistent with the frequency distribution of single letters. 11/2/2020 EMB project 31

The Fourth Guess • The First guess – W a – – D h X e N t O s • The third guess – C k – Z f • The fourth guess – J r – T y • The second guess – I o – F l – Y d 11/2/2020 EMB project 32

After the Fourth Guess • ata. He. HPBro. HGe. Htals. UBe. HUe Qorksho. Lth. Bs. Lasts. Lr. BHA, Uol. MGVBa MHBPers. Bty e. Uolo. ABstshah. Bd. Haee. GGMsed, "Ao. Heare the. Aood olddays. Qhe. H yo. MUo. Mlddoe. Uolo. Ay EMst. Ve. Ua. Mse Bt. Qasf. MH. " those. BHh. Bs Lrofess. Bo. HGBAht o. HUeha. Pe a. Haly. Sede. Uosyste. Gs LMrelyfor the. BHtelle. Ut. Mal Uhalle. HAe, VMttodaythe. Br. Qork hasa. HMr. Ae. HUyto. Bt. theyret. Mr. H to a forestthey had. Qorked. BH, o. Hlytof. BHd Bt. Uho. LLeddo. QH; theyhearolder. Uollea. AMesre. GBHBs. Ue a. Vo. Mt. VBrdslast see. Hde. Uadesa. Ao; they. Qalko. HLer. Gafrost t. Mr. Hed. GMshy Vy. Alo. Val. Qar. GBHA. • asdes. Ur. BVed BHth. Bss. Le. UBal. Bss. Me, the. Qorld. Bs. Ho. Q MHder-Ao. BHA are. Garka. Vleset oftra. Hs. Bt. Bo. Hs: Lo. LMlat. Bo. H Aro. Qthhas rea. Uheda. H BHfle. Ut. Bo. HLo. BHt a. Hd. Bsstart. BHA tole. Peloff, thede. Pelo. LBHA Qorld. Bs. Ve. Uo. GBHAde. Pelo. Led, a. Hde. HPBro. HGe. Htal Lro. Vle. Gsthat Msedto. Velo. Ual-BSed arerad. Bat. BHA e. Pery. Qhere. the. BHter. Uo. HHe. Uted Uha. HAesare, BHde. Go. Ara. Lher Eose. Lh. Uha. GBe's. Qords, a "KMBetre. Po-l. Mt. Bo. H" thatrea. Uhes BHtoe. Pery Uor. Herofl. Bfe. they. Losethreats VMtalsooffer o. LLort. MHBt. Bes. ha. PBHAsee. H f. Brstha. Hd. Qhat Bsha. LLe. HBHA, thea. Mthors oftheart. BUles hereha. Pethro. QH the. Gsel. Pes. BHto the task of hel. LBHAso. UBety Ha. PBAatethe shoals ahead. 11/2/2020 EMB project 33

Anything Wrong in the Text: Page 33? • If yes, tell me what is wrong? • If no, look for further information in the text of Page 33. 11/2/2020 EMB project 34

Any Indication from Page 33 • Look at the phrase “"Ao. Heare the. Aood olddays. Qhe. H ” – It is very likely that “A g” – This is consistent with the frequency distribution of single letters. • Look at the phrase “oftheart. BUles” – It is very likely that “B i” and “U c” 11/2/2020 EMB project 35

The Fifth Guess • The First guess – W a – – D h X e N t O s • The second guess – I o – F l – Y d 11/2/2020 • The third guess – C k – Z f • The fourth guess – J r – T y • The fifth guess – A g – B i – U c EMB project 36

After the Fifth Guess • ata. He. HPiro. HGe. Htalscie. Hce Qorksho. Lthis. Lasts. Lri. Hg, col. MGVia MHi. Persity ecologistshahid. Haee. GGMsed, "go. Heare thegood olddays. Qhe. H yo. Mco. Mlddoecology EMst. Veca. Mse it. Qasf. MH. " thosei. Hhis Lrofessio. HGight o. Hceha. Pe a. Haly. Sedecosyste. Gs LMrelyfor thei. Htellect. Mal challe. Hge, VMttodaytheir. Qork hasa. HMrge. Hcytoit. theyret. Mr. H to a forestthey had. Qorkedi. H, o. Hlytofi. Hd itcho. LLeddo. QH; theyhearoldercolleag. Mesre. Gi. Hisce a. Vo. Mt. Virdslast see. Hdecadesago; they. Qalko. HLer. Gafrost t. Mr. Hed. GMshy Vyglo. Val. Qar. Gi. Hg. • asdescri. Ved i. Hthiss. Lecialiss. Me, the. Qorldis. Ho. Q MHder-goi. Hg are. Garka. Vleset oftra. Hsitio. Hs: Lo. LMlatio. H gro. Qthhas reacheda. H i. Hflectio. HLoi. Ht a. Hdisstarti. Hg tole. Peloff, thede. Pelo. Li. Hg Qorldis. Veco. Gi. Hgde. Pelo. Led, a. Hde. HPiro. HGe. Htal Lro. Vle. Gsthat Msedto. Velocal-i. Sed areradiati. Hg e. Pery. Qhere. thei. Hterco. HHected cha. Hgesare, i. Hde. Gogra. Lher Eose. Lhcha. Gie's. Qords, a "KMietre. Po-l. Mtio. H" thatreaches i. Htoe. Pery cor. Heroflife. they. Losethreats VMtalsooffer o. LLort. MHities. ha. Pi. Hgsee. H firstha. Hd. Qhat isha. LLe. Hi. Hg, thea. Mthors ofthearticles hereha. Pethro. QH the. Gsel. Pesi. Hto the task of hel. Li. Hgsociety Ha. Pigatethe shoals ahead. 11/2/2020 EMB project 37

Anything Wrong in the Text: Page 37? • If yes, tell me what is wrong? • If no, look for further information in the text of Page 37. 11/2/2020 EMB project 38

Any Indication from Page 37 • Look at the phrase “asdescri. Ved ” – “V b” • Look at the phrase “challe. Hge, ” – “H n” • Look at the phrase “i. Hthiss. Lecialiss. Me” – “L p” – “M u” 11/2/2020 EMB project 39

After the Sixth Replacement on Page 39 • atanen. Piron. Gentalscience Qorkshopthispastspring, colu. Gbia uni. Persity ecologistshahidnaee. GGused, "goneare thegood olddays. Qhen youcoulddoecology Eustbecause it. Qasfun. " thoseinhis profession. Gight onceha. Pe analy. Sedecosyste. Gs purelyfor theintellectual challenge, buttodaytheir. Qork hasanurgencytoit. theyreturn to a forestthey had. Qorkedin, onlytofind itchoppeddo. Qn; theyhearoldercolleaguesre. Ginisce aboutbirdslast seendecadesago; they. Qalkonper. Gafrost turned. Gushy byglobal. Qar. Ging. • asdescribed inthisspecialissue, the. Qorldisno. Q under-going are. Garkableset oftransitions: population gro. Qthhas reachedan inflectionpoint andisstarting tole. Peloff, thede. Peloping Qorldisbeco. Gingde. Peloped, anden. Piron. Gental proble. Gsthat usedtobelocal-i. Sed areradiating e. Pery. Qhere. theinterconnected changesare, inde. Gographer Eosephcha. Gie's. Qords, a "Kuietre. Po-lution" thatreaches intoe. Pery corneroflife. theyposethreats butalsooffer opportunities. ha. Pingseen firsthand. Qhat ishappening, theauthors ofthearticles hereha. Pethro. Qn the. Gsel. Pesinto the task of helpingsociety na. Pigatethe shoals ahead. 11/2/2020 EMB project 40

Anything Wrong in the Text: Page 40? • If yes, tell me what is wrong? • If no, look for further information in the text of Page 40. 11/2/2020 EMB project 41

Any Indication from Page 40 • Look at the phrase “Qorkshopthispastspring” – “Q w” • Look at the phrase “thede. Peloping” – “P v” • Look at the phrase “colu. Gbia university” – “G m” • Finally – “S z” – “E J – “K q” 11/2/2020 EMB project 42

Then We have • atanenvironmentalscience workshopthispastspring, columbia university ecologistshahidnaeemmused, "goneare thegood olddayswhen youcoulddoecology justbecause itwasfun. " thoseinhis professionmight oncehave analyzedecosystems purelyfor theintellectual challenge, buttodaytheirwork hasanurgencytoit. theyreturn to a forestthey hadworkedin, onlytofind itchoppeddown; theyhearoldercolleaguesreminisce aboutbirdslast seendecadesago; theywalkonpermafrost turnedmushy byglobalwarming. • asdescribed inthisspecialissue, theworldisnow under-going aremarkableset oftransitions: population growthhas reachedan inflectionpoint andisstarting toleveloff, thedeveloping worldisbecomingdeveloped, andenvironmental problemsthat usedtobelocal-ized areradiating everywhere. theinterconnected changesare, indemographer josephchamie'swords, a "quietrevo-lution" thatreaches intoevery corneroflife. theyposethreats butalsooffer opportunities. havingseen firsthandwhat ishappening, theauthors ofthearticles herehavethrown themselvesinto the task of helpingsociety navigatethe shoals ahead. 11/2/2020 EMB project 43

The Decrypted Text • At an environmental science workshop this past spring, Columbia University ecologist Shahid Naeem mused, "Gone are the good old days when you could do ecology just because it was fun. " Those in his profession might once have analyzed ecosystems purely for the intellectual challenge, but today their work has an urgency to it. They return to a forest they had worked in, only to find it chopped down; they hear older colleagues reminisce about birds last seen decades ago; they walk on permafrost turned mushy by global warming. As described in this special issue, the world is now under-going a remarkable set of transitions: population growth has reached an inflection point and is starting to level off, the developing world is becoming developed, and environmental problems that used to be local-ized are radiating everywhere. The interconnected changes are, in demographer Joseph Chamie's words, a "quiet revo-lution" that reaches into every corner of life. They pose threats but also offer opportunities. Having seen firsthand what is happening, the authors of the articles here have thrown themselves into the task of helping society navigate the shoals ahead. 11/2/2020 EMB project 44

Summary of Part I • In case of no spaces between words and no punctuation in the ciphertext, it may be hard to break a substitution cipher. • It is very hard at the beginning. Special signs and short words should be investigated first. • Statistics of single letter frequency and that of digrams should then be used. • A guess could be wrong. Avoid wrong guesses. • Start and end of sentences are highest priority. • In the project assignment, we will keep some spaces between words, and some punctuation symbols, in order to reduce the difficulty. 11/2/2020 EMB project 45

Breaking Vigenere Ciphers 11/2/2020 EMB project 46

Vigenere Table a b c d e f g h i j k l m n o p q r s t u v w x y z -------------------------A B C D E F G H I J K L M N O P Q R S T U V W X Y Z A B C E F G H I J K L M N O P Q R S T U V W X Y Z A B C D E F G I J K L M N O P Q R S T U V W X Y Z A B C D E F G H I J K M N O P Q R S T U V W X Y Z A B C D E F G H I J K L M N O Q R S T U V W X Y Z A B C D E F G H I J K L M N O P Q R S U V W X Y Z A B C D E F G H I J K L M N O P Q R S T U V W Y Z A B C D E F G H I J K L M N O P Q R S T U V W X Y 11/2/2020 EMB project 47

Example of Encryption • • Key: h o l d ho l d h o Plaintext: t h i s t h e p l a i n t e x t Ciphertext: a v t v p g e k l d w d p b e h d h Demo for encryption using the tools in the URL • keyword: hold • keyword length: 4 • A Veginere cipher is a combination of a number of Casesar ciphers • The main task of breaking a Veginere cipher is to determine the keyword length. 11/2/2020 EMB project 48

How to Find the Keyword Length • YBRWY JFM N QCGYFR GIL SUZJX WJMFJ. NUJ VVL VBDM VS NUJ HRNAUGIEMIBI WBSMGFHGQS GJUFJX UNG. FTGRYCZJM GMYL TZSJLRI BVR U PMIVHY OJNJJYA F HVHERQ UAI U QNGR. OYFXY NQQNDM GTIX YBR SCPPYY-FZGJL NQF, VY QNX VVLARW. NUJ VVL VBDM YFOTMYQ FHQ QUHLBRI. IAJ XND USYYE OYFXY TWUOGYQ YBR SCPPYY, MCF KUGMYE YIBP BVR UFNXR FHQ XUVI, "DRXMR, YBBXY OTSF FLR RUXNHTKOA TZ LTO. GMYL YBVSE LTO QTH'G PHBB NUJ XVRY VX QBWNU RIEJ NUFH GMY ANWXJF. " WJMFJ AENHAJX NSX FFCQ, "IIA'Y QBWLL IUQ. N EATQ JMCPM CF BIEYB ZTLR. GOG NZ V YIBP NUJ XVRY, GMYL BIHQX FYIC IIVSA VY. MB KUE N'PR HIYQYPYYQ $10 IIYQUEX. " 11/2/2020 EMB project 49

How to Find the Keyword Length • If you can find strings of letters that repeat in a message, you can often figure out the key length. • The distance between repeated strings is usually a multiple of the key length. • A good guess for key length is a common factor of the distances between strings. 11/2/2020 EMB project 50

• • • • • Repeated String Distance WJMFJ 315 NUJ 12, 129, 138, 93 VVL 132, 9 VBDM 141 GMYL 204, 120 BVR 147 OYFXY 81 YBR 141, 78 SCPPYY 78, 215 FHQ 63 YIBP 159 LTO 12 XVRY 93 GMY 150, 54, 42, 78 QBW 45 IYQ 9 11/2/2020 • What is the keyword length? • Let us now get back for the decryption EMB project 51

Here is the original plaintext • There was a little boy named Jesse. The big boys in the neighborhood constantly teased him. Sometimes they offered him a choice between a nickel and a dime. Jesse always took the nickel, after all it was bigger. The big boys laughed and laughed. One day after Jesse grabbed the nickel, his father took him aside and said, Jesse, “Those boys are making fun of you. They think you don’t know the dime is worth more than the nickel. ” Jesse grinned and said, “Don’t worry, dad. I know which is worth more. But if i took the dime, they would stop doing it. So far I’ve collected $10 dollars. ” 11/2/2020 EMB project 52

Summary of Part II • Vigenere cipher is just a combination of Caesar cipher. • Finding the keyword length is the main task. (We have the web-based computer program to find the repeated words and their distances. ) • This example of Vigenere cipher is easier to break than the first example, partly because spaces and punctuation are kept. 11/2/2020 EMB project 53