《陽明大學放射醫(yī)學科學研究所.ppt》由會員分享，可在線閱讀，更多相關《陽明大學放射醫(yī)學科學研究所.ppt（40頁珍藏版）》請在裝配圖網上搜索。

1、An Introduction to Independent Component Analysis (ICA) 吳育德 陽明大學放射醫(yī)學科學研究所 臺北榮總整合性腦功能實驗室 The Principle of ICA: a cocktail-party problem x1(t)=a11 s1(t) +a12 s2(t) +a13 s3(t) x2(t)=a21 s1(t) +a22 s2(t) +a12 s3(t) x3(t)=a31 s1(t) +a32 s2(t) +a33 s3(t) Independent Co

2、mponent Analysis Reference : A. Hyvrinen, J. Karhunen, E. Oja (2001) John Wiley & Sons. Independent Component Analysis Central limit theorem The distribution of a sum of independent random variables tends toward a Gaussian distribution Observed signal = IC1 IC2 ICn m1 + m2 .+ mn toward Gaussian No

3、n-Gaussian Non-Gaussian Non-Gaussian Central Limit Theorem k i ii zx 1 Partial sum of a sequence zi of independent and identically distributed random variables zi k k x xk k mxy Since mean and variance of xk can grow without bound as k, consider instead of xk the standardized variables The dist

4、ribution of yk a Gaussian distribution with zero mean and unit variance when k. of i nd i y How to estimate ICA model Principle for estimating the model of ICA Maximization of NonGaussianity Measures for NonGaussianity Kurtosis Super-Gaussian kurtosis 0 Gaussian kurtosis = 0 Sub-Gaussian k

5、urtosis < 0 Kurtosis : E(x- )4-3*E(x-)2 2 kurt(x1+x2)= kurt(x1) + kurt(x2) kurt(x1) =4kurt(x1) Assume measurement Whitening process is zero mean and Then is a whitening matrix x E D Vx z T 2 1 I ss T E s x A T E D V 2 1 T T T E E V xx V zz 2 1 2 1 ED EDE E D T T I T T E

6、EDE xx Let D and E be the eigenvalues and eigenvector matrix of covariance matrix of x, i.e. Importance of whitening For the whitened data z, find a vector w such that the linear combination y=wTz has maximum nongaussianity under the constrain ww wzzw wzzw T TT TT E E yE 1 2 Maximize |

7、kurt(wTz)| under the simpler constraint that ||w||=1 1 2 yE Then Constrained Optimization max F(w), ||w||2=1 At the stable point, the gradient of F(w) must point in the direction of w, i.e. equal to w multiplied by a scalar. 0 ) , ( w w l L 0 2 ) ( + w w w l F 1 ), 1 ( ) ( ) , ( 2

8、 2 + w w w w w w T F L l l w w w l 2 ) ( F Gradient of kurtosis ) ( 3 ) ( ) ( ) ( 2 2 4 z w z w z w w T T T E E kurt F w w w z w w z w z w w w 2 1 4 2 4 ) ( 3 )) ( ( 1 ) ( 3 ) ( ) ( T T t T T T t T E E F ) )( ( 2 * 3 ) ( ) ( 4 1 3 w w w w z w z + T T t T t t T 3

9、 )) ( ( 4 2 3 w w z w z z w T T E kurt sign T t t T E 1 ) ( 1 y y Q Fixed-point algorithm using kurtosis wk+1 = wk + Note that adding the gradient to wk does not change its direction, since Convergence : ||=1 since wk and wk+1 are unit vectors ) ( wk F wk = ( + -1 l 2 ) )

10、 ( w k F wk 2 3 3 w w z w z Therefore, w T E w w w / wk+1 = wk - ( wk ) l 2 = (1- 2 l ) wk Fixed-point algorithm using kurtosis 1. Centering 2. Whitening 3. Choose m, No. of ICs to estimate. Set counter p 1 4. Choose an initial guess of unit norm for wp, eg. randomly. 5. Let 6. Do de

11、flation decorrelation 7. Let wp wp/||wp|| 8. If wp has not converged (|| 1), go to step 5. 9. Set p p+1. If p m, go back to step 4. 1p 1j jj T ppp )ww(www xmxx IzzVxz , TE 23 3 ppTpp E wwzwzw Fixed-point algorithm using negentropy The kurtosis is very sensitive to outliers, which may be

12、 erroneous or irrelevant observations Need to find a more robust measure for nongaussianity Approximation of negentropy ex. r.v. with sample size=1000, mean=0, variance=1, contains one value = 10 kurtosis at least equal to 104/1000-3=7 kurtosis : Ex4-3 Fixed-point algorithm using negentropy 0

13、Entropy dppH yy )(log)()( y Negentropy )()()( yyy HHJ g a u s s Approximation of negentropy 0 21 ,c os hlog1)( 11 1 1 ayaayG )2/e x p ()( 22 yyG 4)( 4 3 yyG yayg 11 ta nh)( )2/e x p ()( 22 yyyg 33 )( yyg 2) ()()( GEyGEyJ w Ezg(wTz) Eg (wTz) w w w/||w|| Fixed-point algorithm using nege

14、ntropy 21 ,c os hlog1)( 11 1 1 ayaayG )2/e x p ()( 22 yyG 4)( 4 3 yyG yayg 11 ta nh)( )2/e x p ()( 22 yyyg 33 )( yyg )(t a n h1)( 1211 yaayg )2/e x p ()1()( 222 yyyg 23 3)( yyg Convergence : ||=1 Max J(y) Fixed-point algorithm using negentropy 1. Centering 2. Whitening 3. Choose m, No. of

15、ICs to estimate. Set counter p 1 4. Choose an initial guess of unit norm for wp, eg. randomly. 5. Let 6. Do deflation decorrelation 7. Let wp wp/||wp|| 8. If wp has not converged, go back to step 5. 9. Set p p+1. If p m, go back to step 4. 1p 1j jj T ppp )ww(www pTpTpp gEgE wzwzwzw )()( xm

16、xx IzzVxz , TE Implantations Create two uniform sources Implantations Create two uniform sources Implantations Two mixed observed signals Implantations Two mixed observed signals Implantations Centering Implantations Centering Implantations Whitening Implantations Whitening Implantations

17、 Fixed-point iteration using kurtosis Implantations Fixed-point iteration using kurtosis Implantations Fixed-point iteration using kurtosis Implantations Fixed-point iteration using negentropy Implantations Fixed-point iteration using negentropy Implantations Fixed-point iteration using negentr

18、opy Implantations Fixed-point iteration using negentropy Implantations Fixed-point iteration using negentropy Fixed-point algorithm using negentropy Entropy dppH yy )(log)()( y A Gaussian variable has the largest entropy among all random variables of equal variance Negentropy )()()( yyy HHJ g a u

19、 s s Gaussian 0 nonGaussian 0 A e )()()( yyyJ g a u s s Its the optimal estimator but computationally difficult since it requires an estimate of the pdf Approximation of negentropy Fixed-point algorithm using negentropy High-order cumulant approximation 23 )(481121)( yk ur tyEyJ + Its quite com

20、mon that most r.v. have approximately symmetric dist. 21 ,c os hlog1)( 11 1 1 ayaayG )2/e x p ()( 22 yyG 4)( 4 3 yyG yayg 11 ta nh)( )2/e x p ()( 22 yyyg 33 )( yyg 2) ()()( GEyGEyJ 0)( 1 yGE Replace the polynomial function by any nonpolynomial fun Gi ex.G1 is odd and G2 is even 2222211 ) )

21、()(() )(()( GEyGEkyGEkyJ + Fixed-point algorithm using negentropy According to Lagrange multiplier the gradient must point in the direction of w w zw w ww 2) ()()( GEGEJ T ) () ()(*2 zwzzw TT gEGEGE ) ( zwzw TgE c on s t a n t is r .v .G a u s s ia n eds t a n da r di z is www zwzw /

22、) ( TgE Fixed-point algorithm using negentropy www zwzw / ) ( TgE The iteration doesnt have the good convergence properties because the nonpolynomial moments dont have the same nice algebraic properties. www wzwzw / ) ()1( ++ TgE Finding by approximative Newton method Real Newton method i

23、s fast. small steps for convergence It requires a matrix inversion at every step. large computational load Special properties of the ICA problem approximative Newton method No need a matrix inversion but converge roughly with the same steps as real Newton method Fixed-point algorithm using negent

24、ropy According to Lagrange multiplier the gradient must point in the direction of w 0)( www J Solve this equation by Newton method JF(w)*w = -F(w) w = JF(w)-1-F(w) 0)()( wzwzw TgEF Izwzzw ww )()()( TT gEFJF EzzTEg (wTz) = Eg (wTz) I Izww )()( TgEJF diagonalize Izww ))(/(1)( 1 TgEJF )(/)( ))(

25、/(1*)( zwwzwz Izwww TT T gEgE gEF w w Ezg(wTz) w /Eg(wTz) w Ezg(wTz) Eg (wTz) w w w/||w|| Multiply by -Eg (wTz) w Ezg(wTz) Eg (wTz) w w w/||w|| Fixed-point algorithm using negentropy 21 ,c os hlog1)( 11 1 1 ayaayG )2/e x p ()( 22 yyG 4)( 4 3 yyG yayg 11 ta nh)( )2/e x p ()( 22 yy

26、yg 33 )( yyg )(t a n h1)( 1211 yaayg )2/e x p ()1()( 222 yyyg 23 3)( yyg Convergence : ||=1 Because ICs can be defined only up to a multiplication sign Zt kWl 6zYwLS70VUlMf i9aJ$l 8PszmexuuTUWa95nd- 3Nh2rH2- UShC)#FrJJQgxSuVd*oSz#eY5dSQVJ&nCf 3P! gvc9) r1GRcDDY83lEBXbfLcu6ZrqyrTmMEID0MgqQ1%5dFD

27、C#ZZb70e+QQlTxPXa+MB#1m4Hinkr41*ymaIvjTLEegnbuU&-j RQF4qVOr br psY0gKq5v9Yh0Woj i+b41B$xCxD8z0-Q#GEw(PwWy9wsQqK!sACXNw1z7BuzsXvFDl bvxE4j &l- L+LpRAeMRS+XCxAcKHQNcK70OyNJVr h4Cl40gw) 7eF%0#bER7nQKsmSEhy*kj5(gegYTOkfnMKFT!cfNj #24j+$d01% bz&Tqur IWYMBOC7+iF!u$HFt 28a6DYkiCOtakZb%bNmcQjlTj xxXyRGei59P

28、bMEH+F*15qEa78*XNmyLc8+E- )grI Iv! CQNyRk-mpjS3uhPqo1PL%nr7MN6*Rm0 x97bv#qIND#7OR%1TW$5%VpGJ5IUe87MCa9*r 2hfzdR5EyBweI pJ7l%%*g!H+t) wF! !4- 1nqA6+5xt G$9kXBXAsUXL0vvrod)q9lVms7E&J$BCmDqyS%Jis!cuR8LA$Y2&MOwGNTf O8- AL9Xu) roP&aMcG6!r hSir E)V0hQXVxTri EQr*#7JVMqT8$30A!i fFE*aN*-7zdcJzgdX( s88x9yRsLc

29、dLy) i( vT) DeKRf O6YVl 7E$rSgi0CnxQvHsb*ZXr pf BNRsw&tJBO&ddA&5PkaEO8gxpBzY33wO6tr SnHOpytgdE64Cm($Qtku2DspTvFs jLqfVaa#0U)mHK*LbX*RN+WmTKL!*wdSxmAmWoOwJG4*#GMGel HGpan7*UoADimbuo9qUI *93xMSNI Jof SoQ#JwOkY-F*wT3Y9%!&hmuU#$jkGvkUvHRP-R8aw%YGpM$PQ7h5zgDkAZ) K7NHwtcvaS(Y18h$Qxqm) sGMwbgu#3FP&SEY9- $7

30、N2JPzM+mKMqFw9jopyHUa$HzW8RxgS%0aeqcVm1ga!n%JXNH5U3Xo% J- m kZDDeXTi V6U#lkbf NhKCI EuQb1Kowf n467TTTw(#5$KKgQwQ#f6OWvi&9DZRZ% CwmVGpUClSEt- *) Q&ej P8MSMToQi1%f c1ye! eWtYiLqQ2JI z7akd*vaptW7G&WGG1e-n*APkUJl V+Dp3nDvswf&zd$T0UqqehmU18t+-W7ENp!Qi XqMD4zvH9DP0n3fV( VKNa6WqTh4DsDw*PlFg)U9nvcX8tl7$kU3C

31、r8S!r O+8tyFY41oxQl v) XO+y+5-sz4Y0y4aBlbjKMABO1!L1&NNa6A6NqmzMu*XBH1d0ys6KKI TR3v36qaG+m38QHwoI %K**) &I+xG666*4BEih&#I d*HvCQ75%y-nGxix&MAxJU&yL$v$WsO2a)h)3i8QUFP6( RveD63#ngvtx- G*hbo( 9zX1JXQxq0BD! U$lwNDFseO+I C( D4jKp26vP3R+ESI gd5lJ59tDKYf 0hj5zbQE9RR2mg-QV+mpTe- oI rnsiH63f fo(oLP1h4acJMRbio

32、RUU5+XF$lz8! vxLMe%yG%8$%Yr cEdAZC$h%ex1ccX$40s%i xAPu*ZQXXr mGjM9z) uCA3%sH#Bi U$$mC!xX22cAW5mbEj NBsVLmxwTsqx$w%aAemV% okOsEBcjFPAX6! -9UKFm4n!C#hl V9KsiX%r 7If SPz2X-g*6GV28m HI Twlk#+r Gm)joTH7nNsC+gpw%T4Vg6Wih$%Eh#LsDQoxvdiye&P8cK+o! r5mLEg) 6YVdT$6$qNlXtgKZy% jM9ACUnU9pbupt(1i) 8vQ5MC*Q!dlh4mU

33、J ygLH9hKLoEu4r GP( GGegj#ZG!VYPx7- 5RemhpB5zu$!m- N-Cvn*F*iFF&b5I7#f 1BxBqNa2il zbIf g2%ehoW+fPxgK4iRZ$f 8&-G5T05o6T3ZZRe!yYr +- M0*dvqZCB8V0I nOVSBWj Ccgtr &qa%Nvv8DTI7FSe3E)- Bl +OVmEaeiq+137AEnyRsSjxCrACrByAR81xdf Eso4AMkg2*aAM) +&Bi -5FSCWWYJ5xoR#35- q7c1! n&GOt Jq94H5w08t - % y5z( ANM+DgODG504

34、brzRq**it bkNN*1U&) O+*R#d9lBb+&6Rhll fTq#Dkr7qxCmt 29UpO2XDaLTMr*POiZimiqC8FGKBXI z) PuM1r63O3TB0EH%TCB3vyr wqVt $*do54)yk2v9Xc#G+TGat rHs5- uQf fqq$HmcWEQXYxrhQ%7XLzxDGDJVUv) Qvf EtpBltU6Iwvkze7t7rr RTNzI PX%e5(I t) 2FiVl 4cnLvr nLf lY&!oqdDZJeZSoAqk9H45DguPEm!t 0vyV&%%c21!Y2466Xbfj (ov( uSel+C5Jz

35、Tpy4HP0B*mapPSN- INV+5*aZ7px3I z(BciEOF( #ls! o0GovD4#eYhxGDIXJcEr mvtDYh6V7+bdxwxWZIij l3cpkXsdDUfd+-xTf 9YL& *2DV6TaDbQmFdYe(QceamuX1OCmaVHGCqP5B6A! In3nGZOsswH7Zj2*( IBG3TLUZ4Ter Sw$3nG$F%D*gj 8mo! G1*X&27gDV9cH7&!WA2SL&6EQk+Xadr PRWX$gq&YanldU01pvI6t8%R+$#lJa23m)+sqr iz*1Jt-2Cq)21NW(zpI#nLaBj$%F

36、JqYRWqzORlVd5ow32AARG!*3&Nt T0p2rOo$Md65W)hkL0DI AD6u$WAmCzC) 8aHp7Bk4C8gIHAL5xKj3#VE- Fr 3gTu( aiBACBw9DXTu4+iwSTaQtFYI HlF77gMbZ+6OKI2!xaCqj or NSaXCK04oosEHnf0IbgtynN0)+qDztAycRoVuf ZG2( 9$f 8nxuvBDK1NMT8g4&paZwkI OT)ZZWgU7EJI#b$t uaO14hTr tL! Ku- hC&D8(9Vg7pFm%WDwf z+5a)! ei r#5m! as- es#- 0me02

37、I6S) 0gz(j 8c7$aziPchI BCe#G(p2C- TyeJI -&bHhago#j YBlG1#Nt 5r 2zAI ZVu%NQr $V+j $AefXGj pT( OJW6gwvnYk#&&WuEUSt) PV7X0KWY( wLQI#1f 08- CRgSKlN( xwm) Sk7qG8xYk5SIerpt&ny$Tf IVxpnKzy7W(yq1h8T! 8l T6FCcj $RzWqGA9WZGovd*- Mm gSe%dpSN( (v6FFTL3gAeVjpPxBi1oGZNq+D2$FXy7m5t&$q9q2r (QuXbeT347LKCikW6gLwKKGdw

38、P#zf #yn(i pFd%mNl Rcf OmnJuRveQPMMxxP)whD24o-*cs%nuR3UnIeElvLoi exfBZm#q6&G9G-W5qi4+lc8NZ) sz*FGL+UAw)qAqpcy)kae9Io) dL46t NIZ70kE*E9-t S8) *O*d+m0Q)uR#RQojw1aYmKxV1lFqj MU#Tys$g3OL8kAZKzCxruUGUCRH(QD8xkZOlSr! NyX)pS$i0$k3mcOKvHHbL9xqrYj p2VS#NpHFZCh+%bAA#mwZYl RP8u5V+)l VQiS1O) uvd6z23c7dQv+8ySJT#

39、GYZefgECxNBklD2l1ZzH! +$7vhY%U! &(3eQAiX5nFqfQUNw#G%uw0pucSlTQ( 4SXxFZj&aQBA78- X1XWiereH769uWsCJPNSc6L9Sj1Er )- 1InT0UwAG&Qys#l9K2tV0z3FKl$qJRjZQa!ShT2S2( Z#UCbCf4RB8#42s$KnIdRF7KB4Ft Zj(hw%&Z+Mf (id( gO3DvtW) jLnCj N8FaRwUrXX%hnC) zD)JEVi JnrYVE&) 3HK6KE- RzBUObn3k% ycKI im$NabH9Z0zf 6bhh%G#( B*go

40、- qJy9U9WRNMu$+$w1E9wl Wd8R4uaJG$hf ObpAWq00V4+T4c4oVb4ehFNZt L)H*i%f z4isVUNI&nCf2WMpdy#L0Fup0PI WBP#0WBn1l ZUbQuKO6&Y- !r 4hI %QWW6f ys7O6SqO66)l mSO#Q4-P3kBa)Q0vdWmH21WUt ng4L$ZalF2gIFtz2!j V+UNZ1GrpuIs++93(*3bGYWzES!Ge2uVooAA0OhnIWEEx-SzZAavvVyV6VIA9dbh#5r bsJoF$QLNm6NrKcbOdeT8iSgyqUhq4Wym3uV5zN

41、*L%rPr D5- #Gd+vR2wr O$5-kv$*I JpxBBRVsh)hMkoT0*k4mBOT0o4H6Nr jNK10 xp$%SVse%j Qjf #E#8ohyQM-ou$wTAo) 3geN0zwf o- ZJB&r eI 0yaYH&jeU8l I( %SU4G&uLxsgFN%E2%KcEqYThNMiSCzr tn(( BYBGqUy1j#aC- MOvpiN1cAk#dHG-PGaqn(j *Mjx%IX! -e(bHAFsg)+yY$(c98W! 0Ho+$pOi oku3A8e66CS4UeNTaz0NrsrRWi RB! lVTQ$Zl&bh2aIhe6xb

42、r yyeuwm!muYbj 9NZx*mM5LO3Jwj u2FmADT9- 1Ry$pog+TPpEQn+2cmp7VuOPyMcPMGzVIBsFTc*Pj xI 4r r9k%$vZxHGS5H- $11Htaxv$RkYSo% K7HW6v2fbY)j fNw) Bmi VId&2x0Q-Lz7Gx8Erl j(! yBn%t 6b4WuLS&! 3ZH8d3r 5M! zu*Kuoa1pkb- & 6QC6nr t zTn+K6%yA&vhT1eFl u0g6JEF9(MI81xr xsXw*) fp52&t c$h%DZzp0lD5id!0jkXfHy- 7RnPQ( 4SWz)

43、 U6OpBgZ+VAgO&-cey0CBEVJPpPlc4CgesF8519N8T0Y!s%KXEQ2pZlSR#XbzDKzk04tXCqYX4I Lw( JP41$zVfu6$&WCt T*f (x46ZN&GI ki g#KK(#RFXcgN3YQQ!CvUXRMcF*Rj ZJ( i8q$xPA&f VUb% d5yS$jnBS5ECdz1ByhAmCBPmZsKg( eC6jhD6o4(HqshhM( bnPuC8f&BD*+v310RMCMs1$nWkhoiMiPWJF( 7n-pLYmSYhXQbJW8z96( %YVUiLf ahjMqD3vTFQMz+05x($DFds&wPc9)Uggs6!&5o6Cges(i DgW6H9U*Ao1d(+uwf- I 2L0sj CNr bq06HQ714VFNxhouPQ9HWvcY!W(H6S&Ew#Axq2TCLR43