MD 5 1 MD 5 Message Digest 5

MD 5 1

MD 5 Message Digest 5 q Strengthened version of MD 4 q Significant differences from MD 4 are q o o o MD 5 4 rounds, 64 steps (MD 4 has 3 rounds, 48 steps) Unique additive constant each step Round function less symmetric than MD 4 Each step adds result of previous step Order that input words accessed varies more Shift amounts in each round are “optimized” 2

MD 5 Algorithm For 32 -bit words A, B, C, define F(A, B, C) = (A B) ( A C) G(A, B, C) = (A C) (B C) H(A, B, C) = A B C I(A, B, C) = B (A C) q Where , , , are AND, OR, NOT, XOR, respectively q Note that G “less symmetric” than in MD 4 q MD 5 3

MD 5 Algorithm MD 5 4

MD 5 Algorithm Round 0: Steps 0 thru 15, uses F function q Round 1: Steps 16 thru 31, uses G function q Round 2: Steps 32 thru 47, uses H function q Round 3: Steps 48 thru 63, uses I function q MD 5 5

MD 5: One Step q Where MD 5 6

MD 5 Notation q Let MD 5 i…j(A, B, C, D, M) be steps i thru j o “Initial value” (A, B, C, D) at i, message M q Note that MD 50… 63(IV, M) h(M) o Due to padding and final transformation q Let f(IV, M) = (Q 60, Q 63, Q 62, Q 61) + IV o Where “+” is addition mod 232 per 32 -bit word q Then f is the MD 5 compression function MD 5 7

MD 5 Compression Function q Let M = (M 0, M 1), each Mi is 512 bits q Then h(M) = f(f(IV, M 0), M 1) o Assuming M includes padding q That is, f(IV, M 0) acts as “IV” for M 1 o Can be extended to any number of Mi q Merkle-Damgard construction o Used in MD 4 and many hash functions MD 5 8

MD 5 Attack: History q Dobbertin “almost” able to break MD 5 using his MD 4 attack (ca 1996) o Showed that MD 5 might be vulnerable q In 2004, Wang published one MD 5 collision o No explanation of method was given q Based on one collision, Wang’s method was reverse engineered by Australian team o Ironically, this reverse engineering work has been primary source to improve Wang’s attack MD 5 9

MD 5 Attack: Overview q Determine two 1024 -bit messages o M = (M 0, M 1) and M = (M 0, M 1) q So that MD 5 hashes are the same o That is, a collision attack q Attack is efficient o Many improvements to Wang’s original approach q Note that o Each Mi and M i is a 512 -bit block o Each block is 16 words, 32 bits/word MD 5 10

MD 5 Attack: Overview q Determine two 1024 -bit messages o M = (M 0, M 1) and M = (M 0, M 1) q So that MD 5 hashes are the same o That is, a collision attack A differential cryptanalysis attack q Idea is to use first block to generate desired “IV” for 2 nd block q o Can be viewed as a “chosen IV” attack MD 5 11

A Precise Differential q Most differential attacks use XOR or modular subtraction for difference q These are not sufficient for MD 5 q Wang proposed o A “kind of precise differential” o More informative than XOR and modular subtraction combined MD 5 12

A Precise Differential q Consider bytes y = 00010101 and y = 00000101 z = 00100101 and z = 00010101 q Note that y y = z z = 00010000 = 24 Then wrt modular subtraction, these pairs are indistinguishable q In this case, XOR distinguishes the pairs q y y = 00010000 z z = 00110000 MD 5 13

A Precise Differential q Modular subtraction and XOR is not enough information! o Let y = (y 0, y 1, …, y 7) and y = (y 0, y 1, …, y 7) q Want to distinguish between, say, y 3=0, y 3=1 and y 3=1, y 3=0 q Use a signed difference, y o Denote y i=1, yi=0 as “+” o Denote y i=0, yi=1 as “ ” o Denote y i=yi as “. ” MD 5 14

A Precise Differential q Consider bytes z = 10100101 and z = 10010101 Then z is “. . +-. . ” q Note that both XOR and modular difference can be derived from z q Also note same given by pairs q x = 10100101 and x = 10010101 y = 10100101 and y = 10010101 MD 5 15

A Precise Differential Properties of Wang’s signed differential q More restrictive than XOR or modular difference q o Provides greater “control” during attack q But not too restrictive o Many pairs satisfy a given value q Ideal balance of control and freedom MD 5 16

Wang’s Attack q Next, we outline Wang’s attack o On part theory and one part computation o Overall attack splits into 4 steps More details follow q Then discuss reverse engineering of Wang’s attack q Finally, consider whether attack is a practical concern or not q MD 5 17

Wang’s Attack Somewhat ad hoc q Consider input and output differences q Input differences q o Applies to messages M and M o Use modular difference q Output differences o Applies to intermediate values, Q i and Qi o Use Wang’s signed difference MD 5 18

Wang vs Dobbertin q Dobbertin’s MD 4 attack o Input differentials specified o Equation solving is main part of attack q Wang’s MD 5 attack o o o MD 5 More of a “pure” differential attack Specify input differences Tabulate output differences Force some output differences to hold Unforced differences satisfied probabilistically 19

Wang’s Attack: Step 1 q Specify input differential pattern o Must “behave nicely” in later rounds o These differentials are given below o Modular difference used for inputs q Only need to specify M o Then M is determined by differential MD 5 20

Wang’s Attack: Step 2 q Specify o o o Must “behave nicely” in early rounds That is, easily satisfied in early rounds Restrictive signed difference used Most mysterious part of attack Wang used “intuitive” approach q Only MD 5 output differential pattern 1 such pattern known (Wang’s) 21

Wang’s Attack: Step 3 q Derive set of sufficient conditions o Using differential patterns q If these conditions are all met o Differential patterns hold o Therefore, we obtain a collision MD 5 22

Wang’s Attack: Step 4 Computational phase q Must find pair of 1024 -bit messages that satisfy all conditions in step 3 q o Messages: M = (M 0, M 1) and M = (M 0, M 1) Deterministically satisfy as many conditions as possible q Any remaining conditions must be satisfied probabilistically q o Number of such conditions gis expected work MD 5 23

Wang’s Attack: Step 4 q MD 5 Computational phase: a) Generate random 512 -bit M 0 b) Use single-step modification to force some conditions in early steps to hold c) Use multi-step modification to force some conditions in middle steps to hold d) Check all remaining conditions—if all hold then have desired M 0, else goto b) e) Follow similar procedure to find M 1 f) Compute M 0 and M 1 (easy) and collision! 24

Wang’s Attack: Work Factor Work is dominated by finding M 0 q Work determined by number of probabilistic conditions q o Work is on the order of 2 n where n is number of such conditions q Wang’s original attack: n > 40 o Hours on a supercomputer q Best as of today, about n = 32. 25 o Less than 2 minutes on a PC MD 5 25

Wang’s Differentials Input and output differentials q Notation: “+” over n for 2 n and “ ” for 2 n q o For example: Consider 2 -block message: h(M 0, M 1) q Notation: IV = (A, B, C, D) q Denote “IV” for M 1 as IV 1 (and IV 1 for M 1) q o Then IV 1 = (Q 60, Q 63, Q 62, Q 61) + (A, B, C, D) o Where Qi are outputs when hashing M 0 q Let h = h(M 0, M 1) and h = h(M 0, M 1) MD 5 26

Wang’s Input Differential q Required input differentials M 0 = (0, 0, 231, 0, 0, 0, 215, 0, 0, 231, 0) M 1 = (0, 0, 231, 0, 0, 0, 215, 0, 0, 231, 0) o Note: M 0 and M 0 differ only in words 4, 11 and 14 o Note: M 1 and M 1 differ only in words 4, 11 and 14 o Same differences except in word 11 q Also required that IV 1 = (231, 225 + 231) q Goal is to obtain h = h h = (0, 0, 0, 0) MD 5 27

Wang’s Output Differential Required output differentials q Part of M 0 differential table: q o o MD 5 Qi are outputs for M 0 Wj are input (modular) differences Output is output modular difference Output is output signed (“precise”) difference 28

Derivation of Differentials? q Where do differentials come from? o “Intuitive”, “done by hand”, etc. Input differences are fairly reasonable q Output differences are more mysterious q We briefly consider history of MD 5 attacks q Then reverse engineering of Wang’s method q o None of this is entirely satisfactory… MD 5 29

History of MD 5 Attacks q Dobbertin tried his MD 4 approach o Modular differences and equation solving o No true collision obtained, but did highlight potential weaknesses q Chabaud and Joux o Use XOR differences o Approximate nonlinearity by XOR (like in linear cryptanalysis) o Had success against SHA-0 MD 5 30

History of MD 5 Attacks q Wang’s attack o Modular differences for inputs o Signed differential for outputs o Gives more control over outputs and actual step functions, not approximations o Also, uses 2 blocks, so second block is essentially “chosen IV” attack q Wang’s magic lies in differential patterns o How were these chosen? MD 5 31

Daum’s Insight q Wang’s attack could be “expected” to work against MD-like hash with 3 rounds o Input differential forces last round conditions o Single-step modification forces 1 st round o Multi-step modifications forces 2 nd round But MD 5 has 4 rounds! q A special property of MD 5 is exploited: q o Output difference of 231 “propagated from step to step with probability 1 in the 3 rd round and with probability 1/2” in most of 4 th round MD 5 32

Wang’s Differentials No known method for automatically generating useful MD 5 differentials q Daum: build tree of difference patterns q o Include both input and output differences o Prune low probability paths from tree o Connect “inner collisions”, etc. q However, Wang’s differentials are only useful ones known today MD 5 33

Reverse Engineering Wang’s Attack Based on 1 published MD 5 collision q Computed intermediate values q Examined modular, XOR, signed difference q Uncovered many aspects of attack q Resulted in computational improvements q Overall, an impressive piece of work! q MD 5 34

Conditions q For first round, define Tj = F(Qj 1, Qj 2, Qj 3) + Qj 4 + Kj + Wj Rj = Tj <<< sj Qj = Qj 1 + Rj q Initial values: (Q 4, Q 3, Q 2, Q 1) q This is equivalent to previous notation MD 5 35

Conditions Let be modular difference: X = X X q Then Tj = Fj 1 + Qj 4 + Wj Rj ≈ ( Tj) <<< sj Qj = Qj 1 + Rj q Where Fj = F(Qj, Qj 1, Qj 2) F(Q j, Q j 1, Q j 2) q The Rj equation holds with high probability q Tabulated Qj, Fj, Tj, and Rj for all j q MD 5 36

Conditions Derive conditions on Tj and Qj that ensure known differential path holds q Conditions on Tj not used in original attack q o More efficient recent attacks do use these q Goal is to deterministically (or with high prob) satisfy as many conditions as possible o Reduces number of iterations needed MD 5 37

T Conditions Recall Tj = Fj 1 + Qj 4 + Wj Rj ≈ ( Tj) <<< sj q Interaction of “ ” and “<<<” is tricky q Suppose T = 220 and T = 219 and s = 10 q Then ( T) <<< s = (T T) <<< s = 229 and (T <<< s) = 229 q In this example, “ ” and “<<<” commute q MD 5 38

T Conditions q Spse q Then T = 222, T = 221 + 220 + 219, s = 10 ( T) <<< s = (T T) <<< s = 229 but (T <<< s) = 229 + 1 q Here, “ ” and “<<<” do not commute q Negative numbers can be tricky MD 5 39

T Conditions If T and s are specified, conditions on T are implied by R = ( T) <<< s q Can always force a “wrap around” in R q o Can be little bit tricky due to non-commuting Recall Tj = F(Qj 1, Qj 2, Qj 3) + Qj 4 + Kj + Wj q Given M, conditions on Tj can be checked q Better yet, want to select M so that many of the required T conditions hold q MD 5 40

T Conditions: Example q At step 5 of Wang’s collision: T 5 = 219 + 211, Q 4 = 26, Q 5 = 231 + 223 26, s 5 = 12 Since Qj = Qj 1 + Rj, it is easy to show that R 5 = Q 5 Q 4 = 231 + 223 q We also have R 5 ≈ ( T 5) <<< s 5 q Implies conditions on any T 5 that satisfies Wang’s differentials! q MD 5 41

T Conditions: Example From the previous slide: R 5 = 231 + 223 = ( T 5) <<< 12 q Of course, the known T 5 works: T 5 = 219 + 211 q But, for example, T 5 = 220 219 + 211, does not work, since rotation would “wrap around” q Implies there can be no 220 term in T 5 q o Complex condition to restrict borrows also needed q Bottom line: Can derive a set of conditions on Ts that ensure Wang’s differential path holds MD 5 42

Output Conditions q Easier to check Q conditions than T o The Q are known as “outputs” o Actually, intermediate values in algorithm q Much easier to specify M so that Q conditions hold than T conditions q In attacks, Q conditions mostly used MD 5 43

Output Conditions Use signed differential, X q For example, if X = 0 x 02000020 and X = 0 x 80000000 then X is denoted “-. . . +. . . ” q Also we must analyze round function: F(A, B, C) = (A B) ( A C) q Bits of A choose between bits of B and C q MD 5 44

Output Conditions: Example q At step 4 of Wang’s collision: Q 2 = Q 3 = 0, Q 4 = 26, F 4 = 219 + 211 q From Q 4 we have: Q 4 = 1 9 and Q 4 = 0 10… 25 q Note MD 5 that Q 4 = Q 4 at all other bits 45

Output Conditions: Example q From Q 4 we have: Q 4 = 1 9 and Q 4 = 0 10… 25 Note that Q 4 = Q 4 at all other bits q Bits 9, 10, …, 25 are “constant” bits of Q 4 q All others are “non-constant” bits of Q 4 q On constant bits, Q 4 = Q 4 and on nonconstant bits, Q 4 q MD 5 46

Output Conditions: Example Consider constant bits of Q 4 q Since F 4 = F(Q 4, Q 3, Q 2), from defn of F q o If Q 4 = 1 j then F 4 = Q 3 j and F 4 = Q 3 j o If Q 4 = 0 j then F 4 = Q 2 j and F 4 = Q 2 j q q Then F 4 = F 4 j for each constant bit j From table, constant bits of Q 4 are constant bits of F 4 so no conditions on Q 4 MD 5 47

Output Conditions: Example Consider non-constant bits of Q 4 q Since F 4 = F(Q 4, Q 3, Q 2), from defn of F q o If Q 4 = 1 j then F 4 = Q 3 j and F 4 = Q 2 j o If Q 4 = 0 j then F 4 = Q 2 j and F 4 = Q 3 j Note that on bits 10, 11, 13, …, 19, 21, …, 25 F 4 = F 4, Q 4 = 1, Q 4 = 0 F 4 = Q 2, F 4 = Q 3 q Since Q 3 = Q 3 we have Q 3 = Q 2 10, 11, 13… 19, 21, , , 25 q MD 5 48

Output Conditions: Example q Still need to consider bits 9, 12, 20 o See textbook q From step 4, we derive the following output conditions: Q 4 = 0 10, , , 25, Q 4 = 1 9 Q 3 = 1 12, 20 Q 2 = 0 12, 20, Q 2 = Q 3 10, 11, 13… 19, 21, , , 25 MD 5 49

Conditions: Bottom Line q By reverse engineering one collision… o Able to deduce output conditions q If all of these are satisfied, we will obtain a collision q This analysis resulted in much more efficient implementations q All base on one known collision! MD 5 50

Single-Step and Multi-Step Modifications Given conditions, how can we use them? q That is, how can we make them hold? q Two techniques are used: q Single-step modifications q o Easy way to force many output conditions q Multi-step modifications o Complex way to force a few more conditions MD 5 51

Single-Step Modification q Select M 0 = (X 0, X 1, …, X 15) at random q Note that Wi = Xi for i = 0, 1, …, 15 q Also, IV = (Q 4, Q 1, Q 2, Q 3) q Compute outputs Q 0, Q 1, …, Q 15 o For each Qi, modify corresponding Wi so that required output conditions hold o This is easy—example on next slides MD 5 52

Single-Step Modification Suppose Q 0 and Q 1 are done q Consider Q 2 where Q 2 = Q 1 + (f 1 + Q 2 + W 2 + K 2) <<< s 2 q o Recall that “<<<” is left rotation o Recall fi = F(Qi, Qi 1, Qi 2) for i = 0, 1, …, 15 q Required conditions: Q 2 = 0 12, 20, 25 o This means bits 12, 20 and 25 of Q 2 must be 0 (bits numbered left-to-right from 0 to 31) o No restriction on any other bits of Q 2 q We can modify W 2 so condition on Q 2 holds MD 5 53

Single-Step Modification For Q 2 we want Q 2 = 0 12, 20, 25 q Compute Q 2 = Q 1 + (f 1 + Q 2 + W 2 + K 2) <<< s 2 q o Denote bits of Q 2 as (q 0, q 1, q 2, …, q 31) q Let Ei be 32 -bit word with bit i set to 1 o All other bits of Ei are 0 Let D = q 12 E 12 q 20 E 20 q 25 E 25 q Let Q 2 = Q 2 + D q Replace W 2 with W 2 = ((Q 2 Q 1) >>> s 2) f 1 Q 2 K 2 q Then conditions on Q 2 all hold q MD 5 54

Single-Step Mod: Summary q Modify words of message M 0 o Alternatively, select Q 0, Q 1, …, Q 15 so conditions satisfied, then compute corresponding M 0 All output conditions steps 0 to 15 satisfied q Suppose c conditions remain unsatisfied q o Then after 2 c iterations, expect to find M 0 that satisfies all output conditions q Most output conditions are in first 16 steps o Single-step mods provide a shortcut attack o But we can do better… MD 5 55

Multi-Step Modification q Want to force some output conditions beyond step 15 to hold q Tricky, since we must maintain all conditions satisfied in previous steps o And we already modified all input words q Many multi-step mod techniques o We discuss the simplest MD 5 56

Multi-Step Modification Let M 0 = (X 0, X 1, …, X 15) be M 0 after singlestep mods q Want Q 16 = 0 0 to hold q First, single-step modification: D = q 0 E 0 and Q 16 = Q 16 + D and W 16 = ((Q 16 Q 15) >>> s 16) f 15 Q 12 K 16 q Note that W 16 = X 1 q And X 1 used to compute Qi for i=1, 2, 3, 4, 5 q o Don’t want to change any Qi in rounds 0 thru 15 MD 5 57

Multi-Step Modification q Compute W 16 = ((Q 16 Q 15) >>> s 16) f 15 Q 12 K 16 Where W 16 = X 1 q Problem with Qi for i=1, 2, 3, 4, 5 q o No conditions on Q 1, so it’s no problem Let Z = Q 0 + (f 0 + Q 3 + X 1 + K 1) <<< s 1 q Then Z is new Q 1, which is OK q Do “single-step mods” for i=2, 3, 4, 5 q MD 5 58

Multi-Step Modification Have Z = Q 0 + (f 0 + Q 3 + X 1 + K 1) <<< s 1 q Note that Z is new Q 1 q Do “single-step mods” for i=2, 3, 4, 5 X 2 = ((Q 2 Z) >>> s 2) f 1(Z, Q 0, Q 1) Q 2 K 2 X 3 = ((Q 3 Q 2) >>> s 3) f 2(Q 2, Z, Q 0) Q 1 K 3 X 4 = ((Q 4 Q 3) >>> s 4) f 3(Q 3, Q 2, Z) Q 0 K 4 X 5 = ((Q 5 Q 4) >>> s 5) f 4(Q 4, Q 3, Q 2) Z K 5 q Then all conditions on Qi, i=0, 1, …, 15, still hold q MD 5 59

Multi-Step Mods: Summary Many different multi-step mods q Ad hoc way to satisfy output conditions q o Care needed to maintain prior conditions Some multi-step mods only hold probabilistically q Multi-step mods have probably been taken about as far as possible q o Further improvements, incremental at best q Best implementation: 2 minutes/collision MD 5 60

Stevens’ Implementation Best implementation of Wang’s attack q About 2 minutes per collision on PC q Finding M 0 is most costly (shown here) q Algorithm for M 1 is similar q MD 5 61

A Practical Attack? q Wang’s attack is very restrictive o Generates “meaningless” collisions o Not feasible for meaningful collision q Is attack a real-world threat? q In some cases, meaningless collisions can cause problems o We illustrate such a scenario MD 5 62

A Practical Attack q Consider 2 letters, “written” in postscript: rec. ps q auth. ps Suppose the file rec. ps signed by Alice o That is, S = [h(rec. ps)]Alice q If h(auth. ps) = h(rec. ps), signature broken MD 5 63

A Practical Attack q Amazingly, h(auth. ps) = h(rec. ps) q And Wang’s attack was used q How is this possible? q Postscript has conditional statement: (X)(Y)eq{T 0}{T 1}ifelse q If X == Y then T 0 is processed; else T 1 is processed MD 5 64

A Practical Attack Postscript statement: (X)(Y)eq{T 0}{T 1}ifelse q How to take advantage of this? q Add spaces, so that postscript file begins with exactly one 512 -bit block q o Call this block W o Last byte of W is “(” in (X) q Let Z = MD 50… 63(IV, W) so that Z is output of compression function applied to W MD 5 65

A Practical Attack Let Z = MD 50… 63(IV, W) q Use Wang’s attack as follows q Find collision: o 1024 -bit M and M with M M and h(M) = h(M ) q o Where IV is Z instead of standard IV Wang’s attack easily modified to work for any non-standard IV q Now what? q MD 5 66

A Practical Attack Consider …(X)(Y)eq{T 0}{T 1}ifelse q q o o o Note that “…(” is W Let T 0 = postscript for “rec” letter Let T 1 = postscript for “auth” letter Let L = …(M)(M)eq{T 0}{T 1}ifelse Let L = …(M )(M)eq{T 0}{T 1}ifelse Then h(L) = h(L ) since o h(W, M) = h(W, M ) o h(A) = h(B) implies h(A, C) = h(B, C) for any C q File L displays T 0 and file L displays T 1 MD 5 67

A Practical Attack q File L = rec. ps q First block: W q. X block: M q. Y block: M q Display MD 5 “rec” 68

A Practical Attack q File L = auth. ps q First block: W q. X block: M q. Y block: M q Display MD 5 “auth” 69

A Practical Attack Bottom Line: A meaningless collision is a potential security problem q Of course, anyone who looks at the file would see that something is wrong q But, purpose of integrity check is to automatically detect problems q o How to automatically detect such problems? q This is a serious attack! o May also be possible for Word, PDF, etc. MD 5 70

Wang’s Attack: Bottom Line Extremely clever and technical q Computational aspects are well-understood q Theoretical aspects not well-understood q o Complex, difficult to analyze o Not well-explained by inventors o Must rely on reverse engineering No “meaningful” collisions are possible q But attack is a practical concern! q MD 5 is broken q MD 5 71