Signal Reconstruction from its Spectrogram Radu Balan IMAHA

Signal Reconstruction from its Spectrogram Radu Balan IMAHA 2010, Northern Illinois University, April 24, 2010

Overview 1. Problem formulation 2. Reconstruction from absolute value of frame coefficients 3. Our approach – Embedding into the Hilbert-Schmidt space – Discrete Gabor multipliers – Quadratic reconstruction 4. Numerical example 2

1. Problem formulation • Typical signal processing “pipeline”: In Analysis Processing Synthesis Out Features: q. Relative low complexity O(Nlog(N)) q. On-line version if possible 3

The Analysis/Synthesis Components: x <·, gi> Analysis c y Synthesis Example: Short-Time Fourier Transform 4

x(t+kb+M: t+kb+2 M-1) x(t+kb: t+kb+M-1) * * = = g(t) x(t+kb)g(t) x(t+(k+1)b)g(t) fft f ck, F-1 ck, 0 fft ck+1, F-1 ck+1, 0 Data frame index (k) 5

ck, F-1 ck+1, F-1 ck, 0 ck+1, 0 ifft * * = = ĝ(t) + 6

Problem: Given the Short-Time Fourier Amplitudes (STFA): we want an efficient reconstruction algorithm: q. Reduced computational complexity q. On-line (“on-the-fly”) processing ck, f |. | dk, f Reconstruction x 7

• Where is this problem important: – Speech enhancement – Speech separation – Old recording processing 8

2. Reconstruction from absolute value of frame coefficients • Setup: – H=En , where E=R or E=C – F={f 1, f 2, . . . , fm} a spanning set of m>n vectors • Consider the map: • Problem 1: When is N injective? • Problem 2: Assume N is injective, Given c=N(x) construct a vector y equivalent to x (that is, invert N 9 up to a constant phase factor)

Theorem [R. B. , Casazza, Edidin, ACHA(2006)] For E = R : • if m 2 n-1, and a generic frame set F, then N is injective; • if m 2 n-2 then for any set F, N cannot be injective; • N is injective iff for any subset J F either J or FJ spans Rn. • if any n-element subset of F is linearly independent, then N is injective; for m=2 n-1 this is a necessary and sufficient condition. 10

Theorem [R. B. , Casazza, Edidin, ACHA(2006)] For E = C : • if m 4 n-2, and a generic frame set F, then N is injective. • if m 2 n and a generic frame set F, then the set of points in Cn where N fails to be injective is thin (its complement has dense interior). 11

3. Our approach Recall: • First observation: Hilbert-Schmidt x Signal space: l 2(Z) K nonlinear embedding Kx E=span{Kgk, f} Kgk, f Hilbert-Schmidt: HS(l 2(Z)) 12

• Assume {Kgk, f} form a frame for its span, E. Then the projection PE can be written as: where {Qk, f} is the canonical dual of {Kgk, f}. Frame operator 13

• Second observation: since: it follows: 14

• However: • Explicitely: 15

Short digression: Gabor Multipliers • Goes back to Weyl, Klauder, Daubechies • More recently: Feichtinger (2000), Benedetto. Pfander (2006), Dörfler-Toressani (2008) Theorem [F’ 00] Assume {g , Lattice} is a frame for L 2(R). Then the following are equivalent: 1. {<. , g >g , Lattice} is a frame for its span, in HS(L 2(R)); 2. {<. , g >g , Lattice} is a Riesz basis for its span, in HS(L 2(R)); 3. The function H does not vanish, 16

• Return to our setting. Let Theorem Assume {gk, f}(k, f) Zx. ZF is a frame for l 2(Z). Then 1. is a frame for its span in HS(l 2(Z)) iff for each m ZF, H( , m) either vanishes identically in , or it is never zero; 2. is a Riesz basis for its span in HS(l 2(Z)) iff for each m ZF and , H( , m) is never zero. 17

• Third observation. Under the following settings: – For translation step b=1; – For window support supp(g)={0, 1, 2, . . . , L-1} – For F 2 L • The span of of 2 L-1 diagonal band matrices. is the set 18

• The reproducing condition (i. e. of the projection onto E) implies that Q must satisfy: By working out this condition we obtain: 19

• The fourth observation: We are able now to reconstruct up to L-1 diagonals of Kx. This means we can estimate Assuming we already estimated xs for s<t, we estimate xt by a minimization problem: for some J L-1 and weights w 1, . . . , w. J. Remark: This algorithm is similar to Nawab, Quatieri, Lim [’ 83] IEEE paper. 20

Reconstruction Scheme • Putting all blocks together we get: Stage 1 |ck, F-1|2 W 0 WL-1 I F F T |ck, 0|2 Stage 2 Least Square Solver 21

3. Numerical Example 22

Conclusions All is well but. . . • For nice analysis windows (Hamming, Hanning, gaussian) the set {Kgk, f} DOES NOT form a frame for its span! The lower frame bound is 0. This is the (main) reason for the observed numerical instability! • Solution: Regularization. 23