From f4a21e387ffb0a864b8bb9822716fe64aacddaee Mon Sep 17 00:00:00 2001 From: Abhinav Arora Date: Wed, 18 Oct 2017 14:06:30 -0700 Subject: [PATCH] Design Doc for Regularization (#4869) * Add initail design doc for regularization * Updating image links * Commiting the images for the equations * Adding computation graph images * Adding section on computation graph --- doc/design/images/feed_forward.png | Bin 0 -> 32247 bytes .../images/feed_forward_regularized.png | Bin 0 -> 46036 bytes doc/design/images/l1_regularization.png | Bin 0 -> 1157 bytes doc/design/images/l2_regularization.png | Bin 0 -> 989 bytes doc/design/images/loss_equation.png | Bin 0 -> 1589 bytes doc/design/regularization.md | 103 ++++++++++++++++++ 6 files changed, 103 insertions(+) create mode 100644 doc/design/images/feed_forward.png create mode 100644 doc/design/images/feed_forward_regularized.png create mode 100644 doc/design/images/l1_regularization.png create mode 100644 doc/design/images/l2_regularization.png create mode 100644 doc/design/images/loss_equation.png create mode 100644 doc/design/regularization.md diff --git a/doc/design/images/feed_forward.png b/doc/design/images/feed_forward.png new file mode 100644 index 0000000000000000000000000000000000000000..d312371a04c26aa6cd196e0bd1f51becb425180b GIT binary patch literal 32247 zcmeFYWn7ir*EI^bC6z`+kl1viNW-RE*mS3el!PGNAt@y(9ZGjMNJ>aZcY}0yowe`t zzn|wh?>Xn=`|b7Thy7#2b*-3d&N0UrD_BuM>M`bXOe7?v$1>95%1B5kX-G)OQRt}P zCsde|Ye+~GNHXFgs?Pd5DHzU#5?A*IwA_>!Mv*G$^SM}<(4Y{szso52FSiDX6vboO zA3${RvVuy3SSfOop_$&x22`j%v1%eBoexArv~<~g?Rq2MiBfDFcr_Qj;5YhOS8rPH z)_f|^V4PHJJuHg&tI}7PkvVN#*6`;spmm5fN-MGBQ)KX=!OIht6?9zns*|QXC~ykvdIR zuf3l{dPhM%5rL;mIuiAT)1o|n{yITJN$KPD!BYFS$@}+RDNge)e@czS`Y7n>=_5;8 z?zHXf?fW(cQrrZ%xXwDMq{1Inym??|CJu%xNhribhb)Hi3<_rQc$edewHFFYvckr- z$JL3sp#6IPl{$WMmBVBIzqv|T;l-)sZ47l z>$}d{@2yc3a*kWFB*QF{Qc|p}#r#UGk5+qLfrq8A#&7RXkZn*YiF+?|gzzQFhdoCz z!UKP(#F09wn1XJYC&a)_|G96+JmNA)(f4q1ofmQPZNNXr>gh(CBhp{`KqEGXZ}>K|lVXIAwQK z*)NxffkF{e{moBd_RR&IhcvtgtjA6F7h;a(e&g7eabL&;>)po-wcsuSWY@ba?5J9! zv9gE64tfKqGcm&ApTXA2OA#ggAVJ^noDAH*v0 z*?)il4I5wj<#kZ(?)^vw&sDV9mTMli}dTc#)3DT)l%Y?#mQD2W6gFkMp@#rURsx z%b{%fp6gh+4|CZ>Z<>CC2%m2Bn-ci1lS%Ov;)|c^5Ih(Xf*BkZrZydumc}L_A)&0V z&q_>8tR{3mZkRmzLcqC{S-(*ecDYKhIbBt-pi3`&E7|$wj0zm%7MMw>FSYO?F*OfN z62*ubxALG+-aTLLaB>g>nb%=!G=DwWbt|vjs2$aOs{F?h2j{*VTN8D>p5wPre2MbHLjl^H5&P0Cw z3XRuy&AYi?xL5atAYc9FUXWH+R*t=J->ch*@soP0YwJS_#+;~myH!vEF}X+r+RQJRicrv`;cN7qD&+Nb6Yk;SEoK1OSle$>4Kg=yLYF(HK0y7Td ziY~L~blaC>0dZ~lbA#hAMRluXZ*DrneIyXSt%)$-*;v^4= z_vXvgJqXjrx7f5vQa}MomkfeC7|@Gy=-L0~L_Wp*Y${xvFa?8b5vJvr`qhH$6BVt% zClk!P#DYlq$P233Xp%)2Ck{H#J9Ef7c6CrYb3Cr&SSp_Sq=SO9tYR-M&07B#N+Yh~ zhx*tZKj+{iIEZ*hu9fYO`0tO$tqv!Dm3Tk>jf6^Ehy$W>%abqPeRYJ z1svti%OlMXI7wNu{!<*UaM5MGi?u2wUC;MexEI_jn02b(TnaC>50@ASX_dbVe)Nnz zJ<(yyhYIpd=zP|`;AlA@OUI^3tta$)9xnhxhJ>K9_`S0876-a!hzZ684eT}r&iJt3 z$U#3}sNsCxEtVcX>^WyXYTrWRLG8dd4yoXcf^00dMnxBmFQmN6T8=pvl{-AIKQ9rc zW&cHeE`s7o07i7B=ZZw!*V`M@uo(3 zeK!&HJit-OJplPt0R?gjMuMdG)d(vl+>8DtoDlaw&fWJxhn}|fG={XStSn&!b=+80 z0gg?}ZS~3Kh*VFD$F*bKR(?SYldXaKp#&&!ioz~q9x3#4Zr4*+r`tc=&i7&zjNO6&QMeeqf=aEKHi z-P=3wA^m4lB*C7-VPOyp7(T(bW0TjxMJ&_)1glRC+{-#TJ9|KcF|air_?9=qtY>)` zSXk7`ySlsk-1eL3z)^8klm4C|7I+o=^qQy4fN)-|h+D17Cuc6pC?Oyxj4*9!sb5>V zl^!LVFr6?}(t*3LiXFM7gHEBr3#1?A7SCJlByC+?<`jOXIFK_&gR^vMtjjIN7!k?W zz@@Cx^ZxdnNnd&Asf0A0Rpv$I!C#JVo8ZuI<0gush~TovBKMCk)u?>gsx%-|U>L;@l%si&FiR3-RKznV;QnWYMW6Q>n$<-~=Z zyGyp)a#n-kb{&~)Qsc6M$_<9Qv}HYREP_%W46oE93a&66jKianRBiWZm*y!7$D%0Z zejVAkrLWAqy@DxkwVVpa3g?SXd~<02Cp<%o8W-XdgsF%*Na6Y=LYx3uuaoZV)3Pw0 zz>mq5sbW3;&op~M`%@J;<^3?@u16-sf{~4B?nlA8c0Dw*TAJljUks^}Gi?@{n~wHD zNQ{(nW*33c*_z~0<1(P&;epXbfzeq~qSS}LnJWxj)?$wiyUCf*Z57c2!?NLN#W&p8 zP!9l6dc&lARwcY$bcKCrJiZqeGRsIyM{4N1{J`Yw*a#tFqNb(IR=;V~9}%VEK*?5i zmqg+#phorsD9c;f`T1-T^E@je?`)05^4L~d&2Y)U;pbDk!-u4fiv zo~6kraTQ&;P`G@wBYS&xxsg`lN=7z*xYCV2yRz{yBATPU&Fb&qgmd^odyu*3L zAr~#~6J*&X^A)^~>`nP}g+pRllk2lxl|9F9WGdafJfalNzQ@#TaRRfS)A~H!8%tMFRC=f7U*UWGxl`H^=<|uIbG}d!~Dh@1fUa{ zJRR~2bZ5kD_##d4s1;L?I_ab!7(>TBFNIO4572xpP-5DmS9~FlC;>u1SASw5fpYSn zSN??y} z;F6Pj?X>zl2!2e$^*a5NJ&~58A_a&d76v9JxmWQfm0iqWZomH!RQf^s!P5) zkZj%jbjM9cm$SPTZ(v;!mok}*BXpw(8Ystl-Qo(X1x+q?dN3vtqZKh~Lmlv`dRLo^ z`74AJyL&Z?O6m^4PvVbJl~kk-5&sSNQ+I4fyZH*Gz*}5BBtvCdrNJLc{xO*vezREE z(a~WA!ZNhx_z!>FY!bJPzPhsVAARxW&~t1)af+f!=W$=DxTR;`p?F~ z!a|?Wr)zX)q7% z!DQZw<`;2`Wuq{Q(cCQ1Nl)+E)wu@!(pSODd6|s>ul1}MBGkA_h<9}k_JViCEiy4O zT9d=votygm`)iK}_~C`r;*>lscUP7@zkjo&@jLAuPEJlfts5m{$>Kh7RCW&-h@^@1 zP5~HwI*)tmNd*B<-;(X4H*okMBO{}Qin@B7nWLZD_43M!8~~|~Ya1II&H|}40=n;F zQ#k@t2-DX8qG^kuyhnsyTx$9Y>U13B0do(Ya>5rN?zhmRC8E3^Vld-I`9NPOh`$dG zy2^OY09BEIkcBlcPqHSy0TIlPh(X8*IBT!<;)f{k(V2V*@jeDie-0ugh?0cbX^Pvj z=obd@NPMR+d{Xub1gjw8*)q{4o1)&!Do?*K`$L%J7cuaNS)hlB>M_B;9^twC`oofS z2)cE`G|~tEW3?Pn4m|H@p%{sYauTc5u7d6JPYWX6gs|x5%+SSvF!Wv=T9ZTo8l4OE4DnOskODWU;X|4+dv~K zXkucrP&>CbD86csH}&>6?T7__kT#gT(#+|)6;Ef5%XL6ply@qm91sHDSyB=DW8Y;6 z27kosKOgxa7r$CDz$XWi@pqYMTF_}j%gV}>m6Rw!Gi7(MDALo{7rozntpEd)?}g}>UyK1GbOT3KZzIjivlC>9fcU7Er0cJ`lyic z(Swj|Sg_Eaa!C1t9)&u$LIIyWyp<=g{cy0%xZAZWjO1xjYO1HGJZroh*M~_8fapHE zU+>l&jlhL^?ryJtl$I*?_Vv-QvMPXW_^ws)-VQ*r-nxxcrlYWj6x_lnC!1piW)C8Zo7qYfmzYvL}U9Sc%1U<~4i6?XVTVZvxRBX!?Z zaunzW6FHfRvU3Q0k7G6Tqmq+JB|hnY>?Hd0L++{hL~%;8nGY|a3NqNg{kJo>i~?TpGEpPXVa!rOu{Qd@QMCu+l&JFvBmpGwg;}#y*t+Rdz3#A! zBJgH)o~4b^>TX+`>;-qe0lqgfmi^g7Nx|*S3jpSt@weXmw~%j zWRfwoQt>ZcL5FON(b3ff8_tvn#j^pF0%PM>3*Q|3Hh+xrPn*ibg07WNR1$9M!lajh z*e~#d=7>4o|MEtD@D4PHV*(im#q+l9kN29~&Ij^xbDgDYLbPx}gtQ}F?rtnj9rx#(Z2RI^Yyizw z#IK2mkqZV*-0jN%b|A4_$BYo54kWo<(EI-^g$vzQW+YlV!$?nd+4mW3rc7IF!T6 zDk?NDUn&89m=4fl@n65U7E>WTyhJyUUHW+GujO!hD^S3>% zcNXR=&7%tqTAE*RbJv)i@6DA32M61LGT1^5wfv18e5Mr&fL_&A=(yxdWKg_+gA_V2 z7N8>9H~`LPQrn7Fh2cd(DjCXN^tC zMjd7=Fwq|k5nWOkKI(w#o(O-zPrCJ6Li0~o@EK^q639fl!(YTkA6Pu2o%R1TzzdK9 z8Jxtk_059EYjy$1TL2=++1S{;%s}Ff1)we13J|8w7TD==574lGfsXoPto0jR-B|)q z%O>CH6AR<@%l@bI;c?{+j4(*jnD(GKbV_uK5N=GcA+-3Loww~8K`;1 z!2#E=wKeka7b~|AeU?Kh7C`#@_wU=WH9I{z=zW!ij+xU1x~5Y{U<95e)F%iJkQ|KD z^$wN~97^c0iK{+L{fWt$@epa-JQs93Q+)wzn;*n}!AHvLxbw`~$O!6VMma?F^5sh> zFzp^th-<8-1>DX)_`5N&GBETI;&ot{Kz+19jM~{odb?n}bJ(4BSfWdF&zybd-;^3g z!X3BZc`bjLGxi724XfXy74UBeL`tJrn=h7qLD%qjDfMN+Dedm#ufjHg?Pag%=(<17 zIoRS)Gc}!#>wJxk{oL;FA@Q!NNxOUeFUZR(@I>ZN$S3N>Pkb705?^ofaTae>48oYu zU3{bW)a7{&!-rl18mi=vztFO0^O{}Vl~$E}WfJ6udT4IRwz1*;@8^L;=T28sl0RWX z`c1AP-uf)i?HiBq%S>>@O+TG->wxB5({7>8-qa6p)=cVD=>pa-w?=XzSc^0(-iMt4 zT!U8uNbLevlPkt|WBF>nGZCY$#{lJY^?0y0U88g_=%lm~13^e9jzwo2>F4rpDsFPK z*Zp0>MD6b#w}zVdU94pAuEqWf2!_9(aQBxlUrrkwcV#6^H0=V>Sy)-ei}nzFd>sxM z-B5eoIWsS>76sqfk8>`o>DuF?qvDRSjC#)Shf&_a=sYO{R#Q_`yOLqgO`ru|I8E03 zzrAufTHR%hFK8;`Ec?WSXo%mI46mC}uor?sjKs0(C4T8@i!QZY64h$bZn090!cRM> zUS6jfd+^$@)qB~bFK+4WmCNu@*$3lKc+%3+&gD+~^X;Gw%G@*5Q&(3X2AO;#E-nuC zOs{ER{0dOko1wT~a1=ZW@7AMeIyl+)y)aTC&qhE|Y`OO85v2j9xewRO~~VKZYTEG<0+}vsIQFnM61k0hnlk zSTmL8D#{>j$cu9YYLlha-~h0GeL|!2mI?*L*xlQya+8$1tBDrIqTc@g7!f~IH5aT? zvwTL_l23=4=kfaOrnA81YV?}W{bfWlAhumq)ruZDe|*ZO&x~N429@CVcemW+e4qY` z&M_JLP~(dH^ISuKTH;sU2ARoEj&lGC8wckF-zSO6SB4F6&^Y%fiJ;I@Mqwx)P|c> z4*10%x*gX$vvtR?nwdG#i4e_a7~*lQKUlOYHJeNa$xIfi6BkcgcFOx=JZ6LRGv;-l zDO4z#B{k4JKikn3z9gJCXIU+bXirn);q)uWJ zv;jx04{&~|zy*IWDKK~b%nEV-Azf=x;c~n#B_}5rdt3B|+7Jx|@V>58{QUgech~zR zo?wv?+qpZnRG{(oqGDxVasxB70c03&2x6F$#V-AG)Qbqo?o>L8(ZDIt#1aDcG-yT5n*vzU$Net9iV%TC^Yp{PX@Psoc&fv zOL{kaS3~x=$YoJlhZdW>%4`^g{E?neWL4EhM7z8@`Q~sLbNeI6mBaS9)o{RLl9-vE zN(RkC7eLi_>V%vlVUxd2tLDl}9Za%m*3x%bBVu4?R*gH0s=W_<`L7qimWC}gkHsi< z1pp~ovi{4&z{i#ZX%`QUCJelYBD%8(=d$HVl{9q99h?n&GMCk;rTdF+Aq6wZbo&BO z-lndO{$@p}GhspT(9TQYBr-&47nebJRI6p(WCd~LPX#FYzj2B1V3P@G&V$v;C!b(` z+Hyb(!#gWl|NPYXY`12ii)@czI*bcsW%E72*hp&D*_b!L0N6ZT`lT!_tbwvLdz=@b zoqwrhF_s^@ILeQWzUe@Z;$SFg64{iX^wuaCJ94YVymxrLD z)xQ&7idm3VUV#?PRvC8xD~c+taJHsW@4sZSbS59$u1o(Vww9anN2EN8=6+|D{Ab}y z?LSQ4Cc_2I%b7XwgMx$e=vi6A0kw=S<1RY%vkwCgvn*#0o`)&yF!RN6s%f3!s_6AN6^sNSE4(AT1x#C`PvjKH#&35>|rNjK31xh=-ae(9H%{Nj7-QIyc z1(hF7sl6i)Nzv5K&JN(dwf!9WZ6;6%EJZheM^)fz zjntrq*1uhTELFe!&S;&=dp|#?`hl;HGi)q->D>)r8j5d>(4#KLGz=JCaz|N&z|lpS zzgcR$qIg?UyNr1Wpi}Gy=sgOS9hS3S-f-`z;c(@17v$y4Sh~W-V)4dcAJAGo|Mao7 z=(^a9#yOV`x%+V{KT&d*3=!C-@u|mn=2_G5Tmj>{4cC%@Oonqw48KNqHG`nHHl1lRt+DraoWxv2je{>0b7|9OfXJgx!1x%3U59S0&o3wC=VA_Erl$QRF z9}bNEzyjKV51F?tY0CZ(o5uia+Fg?L$LQGkeBeJpxQ}I{a3uLdSHmX%qZ{5wl)#9C1x_n(jbqRPt5?tVCvwWwm8&qp5K{G;AzlY{ z+<94qs(ilY+4~3Z2Kxas1nv}w^p?Q@IEVaTlCb#pt z&Qw1`j|zM?%ODYAGSU}R=i@pyYIjZRb9+D%tcRLK0lF_|^0P-O`mMjeP|QByy)1sO zJeE3pHtAaByR5?Y6;#qk6y871K^x7b*dF)j1)nN!l>Tuw!GrwqCh255Df%ZJ^X zJD`WK?a_1k!#o6mxXD@TuSFlU ze1W`QtqpVX>`?jlD4jVyeE#c^-f_aBqHQM#9^o1wq3)#(P1JS~V)?R)JX@YVYs!fG{~*|`OJw^7f` zU-#wADHQ;~BlxrBx0eyjbGXk-NB~xzQ!J;zXAeL^8Neao`p_B;hvm&a@}*(k*>?p6 zF?Nl!f^OE?4nR{ZYdxiNSJ=g(9}yrGb+#NEY`7b4D1Fa*5O{)3`{)09txh%p40jp&sg8vfY_3GY%A+yAEzgLiZ~-3B43E zerqrdiq%89xv0{KV*pVPfzrSQJqk@t*L;GJ@I^Rhb#*li(G68DQ~p%cTDCPx;Ms1Rw*@8ovm4Rr@o1-qMo1kz%V$qeOj_ZQT|ddm9Rst>RHA*Jr9 zm?ZlN2s8#=@H|z#pZ5vUt>XyPo4PaCaDD(>YPq2%NwKkNm9D2-bw7Umz?rU(o~rc* zNd@)G7o14pcZl}NhE_iDQ}Oe20(FXceSoCpU9Mi_%;0UqAJgNw-3M)77WJ)N$-Sd5 zSI0*e0_0E~3qOCu95COt8MydcB=g$SViQ|A)#>h1y+bMBd0U$~}C51o0@DIE}(b8oxFRKv0fq>$$|~TqUYb zY=4=}nVOPW=asPR1Mjw#V2V+6_pn+&X%LE%Z{T2J4N{@Mu%aNp>SZWIA@%&E;Z75U zS}Veejt$vWmJ_li`c0)d*A{qMrW20kdKn1OG>klr)~A#pC@jol*%yoT*%=@*?#Oob zad7B;CDH&C!<%~$BA6<4bKu8p(A?O&f^iM(ESbIlli2A?NOX!Bp><)(vOGv9vw zHftTwjPmy^G|0iy2kGGvx3x=%WUc!Ri%~Ffaylpgi)VDsn{Qud^VOI}YHk2fi~{C@ zrzN^|<&(RP#^oCc^JdS@7(r8-y@mZ8SVIW@0%|Az)7I$bDo|g3>4NmlG<%mM`6BcdyzT=M{KO+2qnhFaIP9--0_#v9=%bkkB80VCO z;`b&dJuEe|%D+c~U2W}$LHfbswweF3R^#(1hwziZ(a*nBxCG?n_s)RT69a)&M9T60 z4S-kj@={A6@xKBd$`JFB>`vgS5mZuFKg$hLO&m~s{>~LK>V4>QX%u9!nr za7eKLz<~YyH^?$3r(0v??|Q!af+me|eC3dto`uB&yKK*oH3sJ^62_=Rx~Z${l@&OW z6CiojLpeA&*1+z6K{$(wqnJyN$^f9&YVq_iv9`7z1`jj_R@H@p!9gz>@$vFdX<<0n z7S})Tw5SdBQM43t>|2Mkq)IIQd=K9EBgYoD352#+0+|Qd2>T>ZyQ#Uj)PV~AJvUbx zG`I8zo~`YdQ>JM|UMIs+v4E-nG+JVCFBsof6%+WVe`cz(zZz1{OBzlF*pB+@($Z2r zpaz9k*(~VuIqoEcG|li7`tQC{$Wi%^=z-gXu@nhxWgWYYOi}Emzfl7NN^{(RNVErZ zt0hwzb93_<9CFxAb@z%$@<3Sik6!^22y(Ea!5cMFcoJi=d2Il=PW1rmaj&edK3-l~ z`ML@b)oTuW<;+q&kO1Mia5sV{I&TU{E3T9L%Z5o%W) zRO!H%vb#N2Fp&*{3Gr6Ob$if>?SR7weyXwW3BEo&~T9lHoa|M+VcSIJizbOHU;uF?qXmspNE8;Oe;A`^2nXy+tgB!*&i`kt4pF} z)3xHnZZ-s=eiasm@Z>yb>+D|wrW<1H-a-J?#k|QGw@-*zXpp_oZ56Honb8KEKpcg& zTErL$sYpVq{~pD~WNaV+2l|0k( z-im@pOb=ZVe-{^&OP(JEeWfN2ZdT}JS0tiFfixlMZJ%h|egbwwenH}OB;*lIBoxYu zmxmi=4E-Z5jvXrfHK-#bNJTv4QI+>+aBl^u2lm=PJ?Q(_`b2HWfC3NtWri0#p{Yj; zcC~p%CGeMdYSM6#Na~=#;yv5{oLv}=F9C3X7n98RVf5=t#@985Z zR}kz0xQoeL5GJfK5T`k0gj%Q=lYZbW#^Uez&RwUo#HK=W=~{VToQkz3BZ1=lub(9K z>;LqV?15wAAk~X!Ero;x0A{&I6 zKcas=DS6}lKIP1GH?NiAgq;EjWc+()Dg}(>wHbR!X6Eu4Xrd8QQ2Ld83 z=e>9ZCMM?Uc1cUz?CdO}eIknh2=EZp)pq6)q~{h`DC%ia@An}C_7&%mw>vAzT-9*i zoUk>^>OWP>6`w~o-CKaqpvZI}S!+Sow}LD8SwFaY!dDMy!8y>H2L|nM>Yxpg&ES>J z3A}Q{`Iws26v%8tzVRucA<~|&zm-Z+FZSXSU&{RpP7BqUot+)@1fvhJKzNAz#$mem zsZQ6FM2A2w!V$$W6%0e?pJ8zBt|W)+wtQ5O3U3%P&|QjvfOM<)7jCX3ZO(R zH8pjuv9U1@nUJS@eBatMEtFqkRR?d%x+A^9{F~BCusKee#i3ta_G~l`JQmsV2#!B0 zpbaY2(DCNnikjEC=npR0z9HkaTNQfWpMbZaLBNu1e)4yC4OAr8-j%~ia~l!v!1m!R zr7X#bCr_SK0~|Y5cD6es2S`d4IREZzbHKyC22E|ULeBT>Y}vPOGe-+F;Nj##tf2ZX zJ2rH#*&$3Z0ONW_k$c@JRezXTp0Bnd@r(mgs=fu+mu%bwQqLvcWqoc~P#noJCnLsG z4s5gH8@#1yKlf?*?!LN`JdW`lyHor!`Al-?=(v&WtO7K6l}$HCeGbo{>#WP|I6^Ui z;N{=B-TD93=ovx1s&^1IGRgoPh|GA2fx)+=B%MkCD<*--uLjsm4e+C?>eb%)`DU7v zeQRp?@Oto8;x}RhD8IUSUm!92RY)#@5sa!IH|o2)-;@C@ntIx$M{if6b|3VY+Mu+? zy*to-nR)TQqMu1%jPLpNi#?OI#AJPKtp`|d-vMjOLl+kp6Hsi4@m%!9u>*diN2?Rc zAOHHUr)Uz_xlz~Rdn$?=1jwi+wF%4v$K*cIqb9bD1$4U6-qF~uwU^IM?^ zz=Zz~eZIjG^xm&PSY?GiVUd-Uk@>do*pL(Siw%#P*$!2FOdy2x{k=u?qQH5H5!9Xh zLXPpo-6(Bj5)}Xu*iLsmxGTfE`8_&@MdeNAho=>jHT*Hw6}JL87mrMwpIv`=1D0TA zZEX!F>;-?dU4O#MGN8nAR(Daz3fV4tze$5Ny#-v;OMt53F|o0NjL=vSLX%Uew!4gS zK6a|znx(*y{RdQNU(U*0)4xh3bpK31k z$o*736W2MlBi4B?@=nrRbN%&viiYmT&o_Y4Q7Z!O0xE1zb<0ygrBcW1+R40=PDocz z`fY9_o@J4$;+|-zY%Xs$S-KPoV&`^!FfDDo!W15@wK0(Sr0(-eCA6WixIHoj#6Ps~i&e7GY>+WMVP_zL_Jh`*W`l zb}Ilafq19#XJmv5lSVmq#zetwCX)rk zM}%=}TYR+zS-^U%Fu*If5R(?)ab^)QZ)#M9Q@+P(BXaSqu|5yb7^|S=za*aOeG)A= z-W|Wn+1X^{$lp?yRK<8WNO4br+);Sne0`^z!(o+3s z#-;@$GfDVh`QW3KPUFpQQ}Kx@Lj|{X{3^s21!>_sfBUZn*&R6j(e5f`(h>ySs=t6p zsr~+)T16a}ZgimM04FwOX>*P%)GPM_HsmG@(MZj(fBrG9;nOBGDDMUk!M#ATu&lBJ zrmmw&{$6^7 z`6grex-}9HPcDJ(nJCcTMN>1wn6|jiV^+(!9u`PC825B}>M&oE6-* z$8*y*2-EHc9F310oOnq0!DXB-qt4ruJLtcLgypc>frSm`xWOTu*iKFHS=~@iu*g9d zzc3caVUTo7t7iLPElsG|E@}f=LxXT19W}v*(XZkn!4hX($fGBt* z;QGk>=x!tv2|NO$1zN zf)fdlN^dQ+v@EZ6z=jhyo3Hl>1pJz^e&y`(O2Bq7KGzUJPY}$$!Yq~fdG>hh;9s)J z>EyF6zvCM?Be0hbK4V*>qox*^0=mo+Agvhia$%WjfMg&l0u|pHICT8*IK}UKAKxl& z+7vg*RsKPw%aLZBM=2xI>~fl@z+!jS(sO^i?mklAtbNtVE4(Q+7(7biO zJ5Iarpn0Lc5pZT0xizdR@!o2gz)f%;(Bl?s8KllK2-7HT1;So7w*PSm9pzcy89*8N zd^Gn!ed_^+w5PT(D%!fw>yZ(k0C-Tf78X@Le?LwO10{sv+qZ8B!!9H8Ch(c$_l~$~ zQ!+-7@GJZux2SVpQTUM`1!b8ZRYfqoAfneD>D}X9#j|DPe~7&vlQRhfZL{Sz%+`^ zbFM_cmhe}rT}aiZ`n0KN{gnA%FTnY{%G=u%AaFatLd4J~BMJscoTb4n->Q7W@##Qt z!Aq2ett;-Eia!7yW^HG%i<{%Y0@2G+2YIUb9~QsJXE}?BXzsWe6WnP%$e+P-mOg#} z`dn-dUX%zft;}Pcld~BcxrOE+Wja|>xW7^xe-qbXaO@BJK|u`$oOw99PRwUsL2!X= zE!-J>1ejGN*=Z>(4i_$*az^q-{YG|cP0RtI6EZ0=tLSxPj>G(r1Jzy7IEr_-h?sKF z{>I2xuzs3JQwi_niNI9Ntesp+efQHD+gpA9L5~nPC#gT%3KAvo3#QADH20pIpKo>T z0>M&!s=>GCt>obppwYR@D=I21SOYI+2854+B}N)=ReJsmDUXio&@D)@F}W=k;dmj1 z_42x396VmiA6ayLbg%f(qNPUDd%m|-pTl3{XXDR6G_psW>&hGKp$}n1hadjP>Un0L6%|R`@UakzxhcI zaMNpd5-%v9gmDQA2^1#HA>@DL`UfGc8a`) zGOuqBy~m&6xKyGErLx#v(uS&{E%}T~OEhKUg6b`RU}Ifqq?pd81>o+sb^h4T?-P)& z+D*XBs3lJt_KpqMr!A`xIX?R|x$6E$hK|8Y!bNV9L^*!qjV_U?!CPj^$5PSPk{tX9F~|&Z;9>sAbn__cu9= z9ba26zmT#$Nf?<=B>y9TuunZ~!^bcx!OdF)46w=_iM~BgtS!H~mA&@hUG6pxnb-F4 zfPG76PED;*Goyg(NvJ$WCIW{2oWP;0?N<7n%XU%NgkOrjpk_

`E*o11uHRpCa7ndF@-pdZH2O73WuQYT+T-i5lHsIzjjyTbVNZk5 z-B%>$B@Mwv<#nL3Q?)P2V>&6{%Stv~$X$Y*RKoXDGwBz&hX{X9`~hwVc0#K45KcYW#qD~5C3NAimbNnS&_jaYF>Hf3@&_5O)~|YtvC))X1;U77zBq=U>lK9tBx|%5a@C(BEFBc z;f>bjpX|ga+?yBhEdd;9_qQ%OZpNZkv6Q48z$+GG;IuQTbeWJR=b*ST?I+{2n)1pd zMBwJ2UAE3UC~RyJEsOTX0jU&E+fj%XI{AT5Pds~+Faw0pe7*bS(Q(;6$riUr_NG5|iqTruh`oYDcl>xuqlf@7595>_0Vu3uI)bJ@(-V~=!SP|&Ho zJ$rIL6w*z3hSUN;Y;OqB*LpoMk}fQvKtJ5Ks&mH?LW*S)XwWBaurbK(XTMN~ z&g>$qgxup}}-wPmZQOKfrn@khTu2S8LW zpjA+M3--3fwY*=tU)V~#D+Vt6E$vctH<(n&FCfY)xz)C&j)x4PK7LL*UuG;-U*Au( zcNw~i!_%3+xBJuE-!{TOP|S+IO%jxR-b)ZRd3%^R+@$wg1r=VunPI~MjpB{BN&$yK zLVtl*UELwKwcm~RcAHU!dHCCzNBZ1M!7bH%eD{pO+0XUR*6UU~dp^x-N9O_YDx2QmJ?T{RLdsH0IurG&d$!4z-f->hI0e(V9Am23fmLsbgDW796HT6dl#vny5AwB~POA+jBqW3wwp{1OVXE%rDSc>68Tsi(mT`Iz z-hIEwE>5aJ%^6V0$5q;dd0Ez(UwokN6Heohdd&i(WM9hvjNjGu&Z;e4cvdJmPJN6o zZlWyk5fP;k0f+8So~9r5QQ$I9KD`^b`)!JcN%nu=C#MV=+4DKVr17c?TJ$=lVo<0g zGp~BdiKasz_ULR%I%L7ns`xOO;c7xhXH+oF8N-P9;Y!K+GsHyCP7Z= z6XWHNCW(ick2{I>6fLVBqeh=3nP1TtMt7~oetY*}R6^zjw}mFkUnY3zf`@Jr=KodN zS4LGCZEF)7*mOxNu_={O2?6O238foRM3hF97LbwWbpnp?zrEoN$e$&Kz26ubn;S`~iemUm~L4F}~v-tRV2*N2Yc0SwM>-1bge z$%Co8)~_rz^FDin{V7ZDaMfFu^vr&Z{v!1Z7;tyq3v(Iu|OBXR9O9k@q3}+*3Ee> zbG?F_Ml6%G_Lsh-Xz8Rl45WDW)%-56WMRv~MX6QkpfoFkzbaU8pVl~Y_lp7gl3(wn zdr%H#D7mrVw>Rv(UuLJL0T!;sd7*;g1)-EK4A7l}HoF1%F}&r9J>;A^{jqr68aEY@|E*kW|@*}uazZjJx-hP-lx9BOiecQFn2Ah*nKPT*yKy^?3zCKQ7eS8mosA3<85a`9UvSN+w(rJXml zsl(l~T2nLfPrTlg53PK&&L-6*K_i622P|%9rz6}&0yf5{egtm$On07LNzoGItxa6@ zq4=7#y#ugqe6mx^0~FRo$B*7y&t9 zd{@DYsmrHx4GgTJSmoaBx_+Q6tXDA1LXh#sHq)P_ti10Q*%NUH^ozFej@RGCIgKTc zvdhO9S;?kj1L9ZX05X#&H;KALxH%d^r;;4qb`QFw{N5R|PtgDn zXw5PID6`iTy*YZXh)|Am@M@2=6Yumb&(67(A1a|r`F2gRIb5RYtI0j$pr~`F{`hcS zT5p!hOYR$I5E+TS0P23FS6W)x-Zw>mVtN89r`NYTYvmqyhI4xRHM+1$1&m6P)-OlS zVL9~Y^fCe0#VCC|d@;{Qe@`;*K9KE`TmkaQly6l^enoy?`&Be{28SgXdl{otf+&aK z*qeu?BP?G$ehCoQlmFqc{<C>FRP+Y0EJcq6ws@_vD(>&CxGvs3!4>*~P@FyBw<g(E{p>;tUH1@qUAaap30Ao$l|MP?ZJdNM$YLS~?#^xLSBT%fK!&3+>hVjk z|9VM)SartpDO@96F z2t5|-dF>F?m{JFvug?Hf$+P#kMm~?Ru-~y~KaIg$vkamLpkWqVc||Jf=oF6{;=4<| zO<+ZyUb7i>ITy3)3ZC9&upHUR2=7jYQheGFJ6)2Ds+dJ~FK!v7@21Vz#ZZl=GZB+wE1c%J2aJ-KfK z{|M}vpZ&OhAw_FNa8aRI)(kD-HYJR?HqA=?XMUyL-o1iVsvHumtH!wo5+C}WgbGq* zE}Bh_#y`I+#8`eB#jHFc8W@npY!zfc<3;rCMjivvHB#Kb{f50COvw30fUmS6B-`(? z&hI@z((ONUDK;M`gMiTGU!&zbl!@+{d^ebs{(&E?ee(G>Q@L~&d2FO;e8xH2eeE?L z%gMj4%=~K)rYbxah!$lS z(51oezTxS~)6UF?66E?menwzLYm0z=igh<;Y>clxk>^=Vkv%Q9(3=_J62@qLT~yDB zAtJ%)9m9|vy8=cI6YQcMjU>w+vd()%dVqq8zm_$?W?n$9n>pph3!NRVMD+0O`>}9F zsR%6J6u_tMr}er8RUbZ+(!EhJusvY300#vAwo?RbaOPp&G0K%v!%^UapWe}bA;6t% zEuON05_4*@d(8u$aURSiqC-ezx)w0I{E1)~_usK}U zIhJ6aUSYI#%$_LUD`Ok%@9{Kz{)x)H^Ks4F9i7Oh{*Kh|g1-xSFbJJXh;HO`?gSok1qMrVl)&v-vQie5dK*_u|g20oyIGLd?VT;n06E;l(XeCuri9XC%(z{%X<&LUn>%c z${BTWJI*;zJ7E3LQGN5SlUKp6u4LY#?{(9I(DvJ#VJ!B&Vk`ER&MDz&5SFxfPzMIM z4Fm)Txbh~b;#@#Wg;nAp?DOwW94oKn_O(Z-Rp*SYzmd~S5kxbI2;U>K)ePy|$gOi6 zOAI+hs#6!cy90;1C8LoKLvf$H_=TjqYT8X`QWxEZjYSu++1-@}KJ$(8BwiZ?CV!01 z&Sc!~;{_|?Zd=FB$bDo=bnx~ZzhVZabTwe*)Ro+cYdKo7n65mqbB42UF0!kmug%KK z*rDZ}BY|}uw8u!xw<&Mt&r=fwIt9G1V0wDN9Au-+%I5{OyR#qI){20XmB)ad!Iv7h zr8$UpbpyGq0x)dB-0$O!Z@iUUd75`>fN8^NDYEAJ4)B9jAVklbPQPkOkjqsdN?NYY zCk5#b>9_M@A8m339jVx3)3UO z3#*ThiyL(ObGT=#KTZ&Klq?@Y#>dOcYkEyvdv!jw5x@A7Q8b%R&ecKM_QabnkwK(9 z69#KN$u6Xl+KbtTbNNNfq#9Z|v>W6w&mI>@vK4v?IoI|8STjaj9ct?^T%4W+dE>C+ zz#IiBdpYlJ*qZCtiV-%X3feq{&1n94+U)aE$`SNz0PWB$uWcylf9j+zTx1wHi%oFH z`BBy7wWV-*vxds{r0v1hJ4uk!Pw&8=P3ctA&`9p9bex(yr}Kv6-tlvx!nPi@*5pzU zy%aF7cCjuQTVZCyh^5Egsm=zMY*bjzb$R&qNzV#S@(O0-);TggbUz09%{CFN@uf)4 zN7;Qwk?+n+mDp_Lmv&um;r?|LTxCC)|7wWYf443#>Y<#>qE0d(r)3WM9A)u9?&fZ5Yh$K~Zlee< zpP7x@(>9`XND~=bMGjfh9dx?2dCF-z}=DuT%b2#F>h=+M>%Bm{pd`z1_X#Q9iMSb%=71$rdA$kQAS%}>_ z#Bv3)aTwEu2U@@!{|X$U=dipvh%3sUgpR*TNbs~!Qc^mEOfQp_qXzssoPeM+hkw@p z2XtnjDo@xOk6T~)b1t>dY>ZpHKLHGU_U@I+1|Tv9(TKBf7JtcS!i9@ZqOAf*a1)AR zDM(JJtaD4za=MN(35<1Zcp?(9aN1>$fVU^`!;FZV%Pqh?T`}Vy8o6EGt5xgyCOdly zB$iz4AtS^CymXzy_v>x+f$vq+)i+AfQzJyg#CQ+MivAaw1)rA`l42+D4FmgsgKyBS zxbDMdSM4Xa=^6tQlL@!joFqsdJ}0m_wfS_ADhR6QXwtdi30ei~fIvD1wnRh~){obm zrBDA&C(2G;q_MVl;xw6Z6tFwYObiW&;o$Z5r_oAB0buZ5m0T(7i=o373ArgKW{%mt zcS-=?j^tDLM?h_+i%=Mlg#=T0yZ^7KO_7;5_jI7@3q)Oh&2|X-1zhugK);;!lIm8w zdUd=4P5|An2-JPB|1Z^B%z84}MpM3G8FjR)Vpc)lf}I5b?nP*5Xf)K+)pgq2+o>TH zl=mkv5;%LI;o&@R^6m7gwUt@I_i&dV;Her455hiIV%|;UM+&8z0ea}_=eh*dYtWM` zMD^r{oNiW9T?^AA3)>v?Wfo`oE_bs@Ib-b20aDudF`Vw$cfy~v@1GbbfE0bcz$7Bj z!*!0J6UQfXZ;6Y7bmQ~#O}`C^9OjvWG+Mi)KG{faGnSd8LDKBpAJrJC=;8)z$Pn9? zmaddxj=aVob%w}bm3=VRnowdRz7*`2JRg0rWs@g%!&EwscR>r_pV3UD@Xt-h`<;3- zIts=>w%9oU&Wk5-*tb@SQREMg{)dKfB!1g^W{Tg+kLVGvn2qse$y*)=qS+tDgOd7g z;4}L#md)toGPjdJuT=u5_PzP-+qcKza58)Rj%941Z1{8qz%9Omij5>Q6Vn0avoB8Sdhu(vD4`3(p?QbvZPK74;+ z+miApS}zjROF_t*gahD;2Z|`V%g4TYgw%*4HvIg&3JM1{X}nKSyjKyaD|Squ;v5@s z<>NPq@HQ4ne*toQp7|IcnR=b2dvoa0N!MjjkSL6SB}0@m-6=ermt}xxh)^^2E$-@ zgLO?9P>XJ$%FMOOsXV1gKXe8l^4qL60CZG(cwD;0fmM{Pu5ey)E+M+7fT1*An20xp?j?d-Rh*QKFWG|Lczz_Q@O8JTt^GOl)JLm+;Nx`$jza$(^`;T_T z!ds2Jf*(W!=O$-H@KSlH{@=Qe0i&hNhpzMlO&rz@Xkx*WzLH4;m?>mi4wBQ$vX zmqo=>6Fq>LS?CfiptE$mh$5Mx3ut8*Q}Ae`#Q!)YKc`$B3$`3hP_w z7A3uhr6r(h#6jVUfPihf7*!iPpq$R}D`k;LId(Eo4u0}yu4jHv;d8gZFB$MD{Am6| z58%J_Ue(jwTRq<)~t=)}*6j3Y3zK5{B^dI#aQ(XWSY<6#;krdJ@1u57XMXT&|cOy|k!>91u_nL^mity#ob;fxOclZeD zL29w@PBU*R(kB@Rl0AkTOC%L_7ZJFAb~63`t(5rZGRl+-NeKy~oZ!g?0FcfHM+fwqLBHw>v`qjotMlMEF%LW=DwN+qC1GR33sZ`w$U2G&Gi61H07iH zttr=PQ*}iSZjSKX3xB7bE_6#on}bbeW?YdK~Y$UErTvNiuf_fKJoA!rXis@pnWy3x2MF2)^Lkn#9ORDjWm~$nxLG z8q?qUuOz>;-NV8au zAyA+_kU~Y2M29!dA$Ti{9HbBD;!GUGWJ8;&#la~x1MTH_(WM&fiN05wHP7GI|I64! zvpI!?@+KkiEtvlVyj44?jSj(gmQ6TFi1i(|6ftl}g|h$tRxfW308%dj*=qFbA1tO* z-8mwiaEPxILGRB(N*PTTP#A$QQjm@Xjqhx}+Stf2S{x9e&?>vrYt_Q!_&%DbC@W!l zAqiB`dCh8cm7UO8PNwsjT||AmqC!Nq{7bQ1&)y2L{k{j+l~E*Rh^I&dI}cL z@AWold{_UehTOoGl0x5!zzmG@ChTVW3?}eQo1MCSk^j?l4Oj?Ap=n{`^Uyjx^4E~g!70Y7) z(kV$9@sBxofni()T`8Fm@(eQ}pMPR;Fd$PZ`0mi?3L#1^4fmJ+F`PWdez)gvmjJfW zUF#ywZI1-b8^_$+DFFMb)nnGSBltO9EqJwMVx8o}t+=j?Gf0n7+=l6V1HWrgh1C$b z8X*w`6O;2uvjg3eCu$#F4*FDjyY@fJeen`VDgU+E_E5y&FKJ^INmT3B@`&p=0O)nx z8qp!)fF5P9`P4_}yNrzla}Ur9|KOTmUTtGJU|%(I zT`vlI7Hoj#-QrbK!rB9BV;evg7zUlxnktBH#211p^a#iv>9OvE%3!e-Y-!Xd9ClWk z;@ao`F0_0^7%$;SDK9gm@f@IokC9D^ou_Bb*F~>i)yiO}Z|!d`#HC-j-9Zlu5V7+q zUYaFhU^fx~Vtvir-1klBq1~(C9_g|WkN(Xam?a%C7+WV7dWQX%XEx#LY5@c){?Wgv`E!N^l4UOu$EU(73WL|=JgdWDDT;7{t-$dNH}L3PAC~b`^#>wl;mz@IlXs zSRuP+a}tDW9-5m^##V_7^}LHeS+gpQo?uf7hHT12#nB{7Qw_&uAXdtiSq++a+76eD z>pb0h3Z#7xtB_EBy%}=$-x7)=insvd44|!?b(IH8gBsP0f+G5ad~dS=5Bmk!?dSd} z{|?~TfOoH z>#S~$bAU_-<-&_|I)C*Mm=WddJ@DpE3FfjjkB)t9VU5sC#>AZB%+~?&-k4NSAmU+5 zK_rb7QB%&J&d5H*)2suT($6vHbYd>C1^|?ePH@`0M+K2Rg=qUS3#aRV~a~Zqf^goWzOn;quekVK#J0hwn=H(0M&Jg|Cnj z%Rf8-jUc|2u}bnm)W`5z_cg08okzzj6U273-%~!VH3-z%QHB}P(n$1Y7*Re|BD${K z!fO;W>?zyAdf8PJC3~t_@kXh$IwyXA0LGA_%Rzy^7d`WwR+of3LpE22Qag_l?*ZP* z^^%XcW5fc9$AJsB?XQ1Z|5%X_oKo$gTm%&G#-7%qKzE14hJ`*8C`hHl{VbkCKr?g< zuf9g6fQ5~sxS-^>YI`;J=bdLLz|2R1l9>6;qTjg!IFL3KDeTGi6`zfN@q!%!ffoQ` z{|ug#PyZmU246FobbxhqckmF-s_p7lvji9xB65R=3@=OWK!(5?1ibXAYbI?#oc^hv zfx*Kklt@vY_N_qz5-nN!)0H6J0Y;{dH z@Kv|_rYJ2k))vtBzN8HP zKU!@fKrWwI2a1P@V8)t*#0EnV>nCf4WCMz!`o2UOoREiMjj@hA8Gw7ZzNrNcin_kB`2KA8ZIw*&Ij8&LvG)8Aj_p{Zf>5p`RU0ES~@y)AW>}v*EulCd2Rk&J6Pz;zIkk( zO?@U4sR)*TAR&bmwNmF3icn2fcJ`aIKoe(9lJI!vHVx{{$Ym;U$HUv7jO#s~wnfdHqL%_?!3HGqB(hg_ ztb4fF|7D-6=1*Ul#?Vf`o5EJ=Hs1~$t{+u)OC8t+gxX!Om)N9(4wYdQ!EUy(b2uF& zBZt}?^n%%33vqX^_!NY{{IHEEmkP2SedAbt3{z%(f2qjC>=*#eh`k?oxm@txY@ zzf}fq))4mTH!?tMZw8q;>87^79nW@`2Cx8x(q9W&!-n`1F3kR)4>) zQI%8rMG44%#Uvy!8r9s-9k4m_p4T%l)3Y!z*wMnQXI{ij&phcRXP9oYTv^ztahe{y zd2WpaArMqESNSTqP?`$4jJmuvjE4TwfpY2cAe`g&*>_uK%H86#*2Qc_fhX;5eCg#olmr-3w zV(j~74~iU+I|o3MQt17ytRqBJVgQL7WfV3a*`2fu8>|w3{xr#usXs_u;sQ!x$-3wu zm%TS=9unxwUn-}zR<^;)3iF;VI+Cm`oHu|e|468kYV-8T&@)2~Bs@=t%UI!gE^(iQ z%)s}GF^AGFu+q})&)DSiO-A>(*o*A;Y1n;)EFm^>*%~k~nfU71va>X#bsW!!u{K}e zIYPRGLM0#Tn_$Ug5~;sC-k)(0BT$57!vb=j#=u~d^q;f9Mri`WH1XLzfCGMp zlLQVPp7zCAkVblY&r_cTt^f!QJyd>=J{^Tj0n0*I>*J%n8-7HYXssa-p%31A?cqbH>gc$XxcbC2SoNjO*UO_@&R7S;N+L^;|IdS= z+i=j+8*sLuHT$f7Y_7CSlp(1~5 z)xnogwZSy2as)PQ8ol!pE8k_{dz0Wb1H5@z=)Ly>GP|3;kw4jR z=xBOOR?TYedssnsWMNxfrKXf$0j|9hfX#k1BIHf2CBX&?6)soTA$GbBFbde>KZl&W zima^c2b^_Js$?IZ-ESdzz#24eDR}T{*cOf0%^>JYI_p~$*mS$4p@)heEN89b`xP4S z0OeT(=v1EkSK2=3&H+#0cHzv7)si(Pc_B&EIpb=wc=1eE4!TJ4Ly{ zR0s~4-hweY(gJNe!#EwHVVciQV3+;T;C(3NN|4ygpvLpF;E%Yz{wWzooOL7Dz=pM0p*sn zr^$|)Ech?Ysih~vh|BPR)ixna>BCHCG_pWQm3jpvOXA^lE-6{a_PWc)D{bzs0>R|= zmjXpV92w$=m1Lr0W_I-3J8y!}XcSo>hkP)^4}BQSit*k-kd#UiBuDKC61@8$n{OVS zn8*VH=;21+!3#n7f&oavAA6X=s*s^m)X?a>Znh-#LrIL{WZ7!t1a`~BCN|+4TDmJK zwZ?nX5LTq$=I5KTW0-K3zx14VJgZZz-mU0QH_-IFX;b>YU#6s_ym-bTLg-CdTc9-#6 zUGT^B;GrZiJ?0Jl=P+<^uNt-a!~yYS=-sREb!ot*)2JDWcma%{;;r=1Rnl!yRPT(nK>-?kV4vuD)<$k;f@my=&6d64CM8X{9NG3LJ_GcNK3@ zmZtuoiDpLL&xNLt(#WCmQD`*tZ!JyPeieUoe<*1XnL~}>MR83$@t?{(mYFruItc0T zb@4x`R24>M_kK{U66FOY85>C+#kq*kB&ulm-+)q;I+}y>%k+rW&AY0~NPn@T(5Xr> zPc$*p35`UQ3_KWDgv4br)8~FONcN6^6PrKb#nDf5dPX&XLgjs}l`>qvk?F z18lf>6dKX2mWih4k=YfhxPKq9(ih|*zEev6$%*p>ybhOI!Uj?`Y#Jz_Y?Xw`KO7oX zE%f(!_^_r;eHKm64bKag97E|H+S5cf+nC_803}=*}QXTNt1U&)IaKAD5%gP{I7AO>51;&1w4(o%m-`Fox z+>k3&dy>YfA$f#oZ zyCQGLLxz>d{m1;gx~#1KpJc2yH-C7Pbvg`;7L7imV&hr(8i}nO913G%VpPmiZfoJ> zT+_=cMa(f9lp#-~c$v~kHBEK3tXme#O*QfM$s7(`{i5h6jk?D1s3l zaDvfX2tIF*ndZY*4&cV_H&J|j8EHf=G(v|*xq&kDYGd!TDpn*(a^1%U@;&&y*D2&_Wj@hy>TZFp2hGKaarmZy@I2aHo=~ zo;xYW3O;YmXuXJbH(*U~dV#e#LF}Kx1`n9Stnn^3prHsrmSqORY ztM?-&v~UU-q5phqV(5vHVL|+|nT4@3DkS(C#{oFOXMvh6kCSyc)jnd`-29>;DraaRy$|OY@=*S(2V)1^gC5UwxfG9uR-lm>xCw6cl2r>DQPnz;PK78WCmwXjxxo#^0b%ErnClxi zJ73>=381pB17x-hl&!v6d7*k}p9~{oj*fu2f8q7*G%1Quad#2&)ZcYG=KX@H5bnH# zD;BCjVJ!=epJ`_yBY}t>L9QY^IJJCE9m`%d^br!-pkNX(M2HKjZQxlErMx6c1?vn( z`qmflxN@2)66f69+@8OBHL|)vT#_v5rQaIGA^`V%1VeN$k558RQNoWcWOe=lveown zOR;Hh%Mc%hvLZ{NBTn5eaQ8{k!JPw?3S$+d{Q1EFz69}0W9iZzv+}#KB#c5T0CY&` zK^Ttfz%5mUt9BLUP(7jX!-U9;E`Y(Ud~IzhMH$NDUU5pwJ`)5j+5!9s5jx*sl#m|+ z{D5;7#^!ktdMcocf`lYHa*E!(2v=7qj)mT5_ftCm(HEK2$6yoxQ?{QFC=@~-|a0TK#bilO?RFh36C<6 z36%M<9?hZI7(G35eYfA{GLTV6R;iv#ZR85jKsH7T+&Ld$SLcJY>|FtIhX`Qhn2Xj> zoWT}yraNUL2gSSxbH1s@`I7)g1Fvh*MG4NSBgbB^U02^8`?cLIeU{}kB^->lrpTtU|lVTfd6gM7>(tb`>K18Bo(FQXyLty>|ve2A}_@8u3)AkWX7@k zJ6CO`=@?hE2~&E*N~DqX#R8Glqq|>J4tXdHu-w?aN4tX-zg`9WQlwhSGy@IY6~v*{ zgA3U&%p&=mjdPJe`*O2Q=F7mWN5_8#sWjkPgnQ8`(<2zlE65V4r-Q$&Zz1VX#<_^s z1EDHnMv=QB2Mf*a@#SSX)`4Q>RpcxUkR7&iw;G(X!aES9y&D!s&Amu0>$BzcyN2Y2 zL*x*uE=IViPuTz?kBrLb2PwD8g0GX@U{gJ+dgM-Ep2Cu;0uVL;SI>kZPPXsZ< zg5X(h4@R03m^2Dv3q8L?A;EsliUn~*qR>^-{Rz?g>W!gTYO9+=il6Qv_KF*(j^^2~ z_&5dZ(+lv0U2=FpDPrHup*5a$a+~D->csAs{HP?V?M_TRKNr}^QJF?>Y%M(Pfb2cG zQ|yRO9S5&eldC?4OdV?MnCVwl0j`L&LcNSAv?x2w z6i#<>9h5OLE&<>(NcL$OcOP?6=pa>Expd<_GIzbe%k0?MGGG@GDbUCerkL}wA0aEF z19~aJ=(1;|%zGS4=suhG9e?*=J*Kc{681-2COZLq`H_ z`%U+h85x~75KCzkoIX-&6!Ygos(PB24`~|oG7QgK7!jIcoI7A$K^Nrb)P_P9iV+$D yHAn(yBhIJhg^uU3M8>ZgFKZsDF(J+|d0ZNUQy34n09@#frK+f@P$*{^@c#gkQmLl^ literal 0 HcmV?d00001 diff --git a/doc/design/images/feed_forward_regularized.png b/doc/design/images/feed_forward_regularized.png new file mode 100644 index 0000000000000000000000000000000000000000..677e99bfd9f8e72ed9fe4b27127af2ced202f447 GIT binary patch literal 46036 zcmeFY^;?x)*DegWB&AE~Mj9lgr8}h?q#Fe35@b;VN=PH!Ai3ynQ55NJ>F(Nd-Ou}c z&-?EE9s4iX_i_Kg;acmu<~8QH#yHP&oHIgAMHUmC1RVhZ0aIR1N&^7_sR983aTygE z{7dCCVHyMkDg=3{SFgQ{cQequ2(-@6RRx8p$t7xNA(mw7$gibvQCuj=#B*eBg^<_> zn=^FNrn=iCmJ+|lEEkYw8azT&)dQLT|3FrV7uAbs^I&mcU|E4@hTr&1Jvyw=3 zIJiiWHW@E0Bb71)2{E(1Vg@*?H1Mbp@u-6PP}N{=0dWXu&vQIRQh8svEQ#widzL>T zfy^r$4Hduo*QHpI2rBwWueonpRY9hm)ca~%XVSvZz$zD0+C|iA3X#gqyOv9qxjh_m?9Pv;z%;IOP_7>y?AG; zzKQ&6@dOp=kLYlWAh&qs1eL#dUrrgof5Rz=h^~_xi!&>Wbd8HDSF6#o&5ywPPXOeo z1i%tIk5|hc9A+0W4zg$nroA0rD0XTrgdUE?X#g%z`fZ|_75 zV)n9#s3_l!sc#n3uZV_+?k`h$?ayCk{7{r$3*u6%A=IV=Pu8n$nI$J~eG&8_=s1*3g%|HI zHmG~~(&zT_$jHsj?Y4Oo(padP#qPfOV}9?gpep|>_i3rmc-bzzTOHq~=|)k5*N2n&kM__{)|;+2(qcC^HY&1bgRc61u(B1n@D9s;%b;5# z4X%pRu9GLIY{SHIm*x!~`}FBkJ*BW0XD9kTB4!NT9P&)Pqk%fOS_Wx-j!#>7*$F`f zwd8P!Jk>6QhDM*sxLfo`*=Q&PY{aSSYTup3kMx&E3=VB3%9)&I66-DAClv0@^AjiG zhfxsj@DPMB`{zZ~N_ePZ!BCJ&lZV&>_B_Ohm-QPP8=j4aT^J+dhY0#N3%&>P+S)|^ zMvqMU6I z8(m6`lRXGd-3F{$=a*!M8rT93I~ z$oOI)EjXNcAc@yo=n-Qf&kbL4$%&(YloS+ygG+~li1dL7Bp&V;+_K+)!KX2z5)N5R z;I&CL|1u?qS0vfN@$340)_Hh6*@V4e-ZPEW5T@q9mu4cMQ)$cw^6L1vBFCV|q|xSN z-N1J>y#U(P#}n|)A91y;>{1Hq;{D#CM~()O2iJ*gGWK34eD>lE)X;l9@p#6*Qr^L# z%tYDi{%X_m@95VTAUjh8Tyxy^5|}jCmcuEve=r{`kS~*vsuEN#pdMYQi-YT-Bc~WW z5kW+s9Obscq{}8oY?rq8U%tNHsl0J?ov(LHbC{}nh0|AP{WDX+#HszxQ?JT2*zbJW zx=!q5LOxQ*IrP#u3IxCI5~0R-nOc0AgyKh87$k%qC(g!E38t;3C2we${=1^9OM;1= zJ!xo2VQ^re=&rF)#=`m(ic9UjMaEKcEIq1At%w72wU(U)I7Aav#Y1-t4H5IPwI z!BRRzF*FFPJ}F*B$qmA-4z;NDk957+JWZ9C(tM6HbbGT6IspY6{$N@8ChY-k$AgTM zw)gUk8FWJL_6eHj_ca@y8~bIc4QU zQUB|K!21)*by=KB6aTYBI!4Ax^hw4nKaJ@;j~_2L~t#tQFHhRWvGc9csP6}gB{?aIydgn{#*$^9|0?RZJz?n0}< z@3FBRRGrKEF0G}HLi>la{dED>OvbI4Vu8+FAe)Ft)jSzrfGE&*XViTP;zw$NYCCrk zww0)goW0)rz;+2Ue>8EvH@klFQ!JT6#3!A~P#s=y))yb{8%)w_ggw5jjL-5c9kBhM zYqN_W0=+gs{?JlyGHkY(&xL-?_CNY~SLdM)hj3$Y`~kl^1^{sHla>Oy+$&UeJKa zTtQ4P&OkPlV9O>+Abq9Ej`DK+zCt3utniB@lBxRzrc$>0@-x9ohYIzcC`NT_8x&Kx{e66=7}o22yn=CkU@}$D!<}Xq+)8hfj5xFtTvU;aXXPt zx>QH%yQvY9D4I?n_9;TeYJco-JgqDN7lzLJzKI+$Z`V|cpb<@));bBCji*W0C*y^(}3z&k1! zJwBo*5R`2{SyAWh>NXL>zfiOWe}D0qG2}^4>b`a6*sQ<)=muYh$>irDJbPWE0%mqH&DZF$s>cH6TrwlX{l9)7lm(c?1`&_!^-Anrc z3Jn;Eo9@(eJe!0dvz}6pRjFUw81x$vNbqNH3F1^zR1`v#*V9WeM+;*KIxp=fb#9@4 zkBq5td@%J*lD&EN#k%eW=+iPzZC6(9$0Y$}&lP=37W!gH*FPz;t8Zub@}gUn1~A{F3N5|pQR)2S(TSTE1ua63vNj^|r%d{1%vDRS|r5t7D$^5khYs;RC_ ze=KEX`rA##g?if&K6azGmyif) zxjUadJ|5(DKso5Jd8fvWRC_uO z4%H*cu)xwLb-YuYHuMR-+i3s>vQFxq7mfcs9XSB`RjyhpNV83icIsNUOU5hR|J+cS zOUZ!_&5Pa?^+@5@wnzFGY9F+}42HpKgkMfJeKJ4ORdf-GmQDFR#$fZ)Z}!+J zK)Aiy%*qdZxR12&NPp>`p!#sXc6Zo~z5X-szD9#+?g`E#60~3FP45mWZZn8~bY){XMp*nw+^NA)nPqrij56KcAyN zc;ScS_3zhummdkFL!40M^z`&TefjeIr`Utv9`qHg?R>@_kCb$F>G|=%Wl{_57#>M zAMAG~%IyR9RgI;d2jK*47Z=)9hFe~=-R$F&sUR$)^CLMHm0ic=yPOI7eaE2!ogJa` z=mj39KyVdQCsO2icX%>rOx+mYXi8J+(aL}$I&OQ*RMYlJE4Syg?&m- zXMOhUnZi?c6a28!%DgW>O+Z(=$Fm5)M(gZuftpj~8b{4DUYjaB^}=#^7ifJ$?<4x5(ho&jcfwc_YjU8>E?%iBZ2rckP?Zbq1c{F`1FNwuMgSxwk@fM?XEG&_%l+yXqe zP06vrfS85G<2dscPD1`gX|8BKTeke0(g=9&>edgY{@6$j1&=3>963T4%n7z06W>M~ zKLfI~BN%Xd$^P=?#7!gsfNxWT-m&`|fcLB14 zCnP*Yt8XbjtBbXPsQRlEo@n5{G+j9|8^crq6?ExQ&qloX7myYoo(Ysj zsUW$K7X9x$@{4?rL|MSqdh(~_WSvHr<*rm13}*e80w=@$ol3nstoge<^3Ahf$j zW;S-B*`S`(dqH7mbtchB_( zTTWaRM9t^-8$CK4ZzRL9c*;lh;}B1=$|scGD$){HeTqGK2lK?hn&0r6Kzy zBh7I!)PdJJdD;#Sst>=d(CNgkKv*ene_|_5@HM2G46DBiq?E}-i5APdEKz@R-?QyR zw;9_)c^R1xwRYnQK)Rr3XIBA|L>>&Ldwg;>$oU0t)-yG;{Zf?$|^4K2N zk8K$;(KPlb0jbCf-0TA-4c{mw_~U-m+!&9OIQTO@KK?<~v_nIN$Aq3%cR02)A5e$3 z0G;S)ooaB-Dl=-<)-Ka^J&odMg94(iI5a$*MF0|rJ%RWW0@`2spu*)X`OlWm0R1Ul z{t${u6P%6qwrX|{J2@!{s~O-I`|6sSRWrSDp-#MJn_oFYS0|hOfLT?(Z8XG=98mR{ z>^=tK)-6rP*x_wg?{_OWORLAV@RiSYMf&W>H!H@doVu1N?Coag<8zJHn4wbWFjFfp zdb7(t-RNF;3mUrH-%)BNX68?TU&66eZ7q-OSV2s>+Yr<5I(wa$7jy1#Vz}b61et0o zuBuYA0{jus4%W2Uy*3jmE4Mgse;NhwAkqM_Vbg>1toGY-nhr-4$y$i)BcX`zxv=d* zi?{6=TNzF%=;|$N8yn0aXE{;qpNw05fDJw%|(*<{$sBOQ70jOV?h?yVy zw@mfG1|^Z}d$LBN&N0&{60_=30hOs@%u)Asme&fJUiF)ZIj_~27jFh|p7GeIb=XXl z>Ctku-&PBQXLXx(Y#0RlM;KMc#|9LJzUXJ~DPsuY3m;WP!?z3~37+(^Wb7nK64)}c z@DG@@hzAg$k5Sl8R?L6VG4}4ovM)+CHG^x3WrmF!3`$?JQ3r|G^u0Jn`^jWKlHsDR zzjWUis50w@H!cs>s;X^c-@UpDKv7uX^=-1>JS7GqCoYC0jHxP?#5yPr>L(`8QwMak zOGzp=Hb=c=cB2a81}CGMjghoT zPEE!2B_Sb5&p1sxe$6+(YuA4Lnuq4KqiG{h{xI>DY`xOSlHhsMNNpLO6O15bA4?%p zF-~02;iXdFOP^{lyB~jMHaLsmhJ)7z1>E)G_ zqW}E)1K?~5%#c0!LCE-R;Pnxq;DagLD7SPh-D-dlC>3v@ClzFXG0JFl_V` z7{IlHFqCAY8nWOT={GxtSpg+}pJclp{RKou$4_y2(k=lKsB<~fb~a&9m!t#ZOn+&6Lts!=f!k-LKG>S`7t(7O|XGUjvCOWc3HKX0J(;Bi?Rc=}7W8fb~z zEiEl8{x|2nfZVjI>*`*XLCg3N8Tq&r;Fxewye4gfamhuX4RCRN9mfHd9*fqsPl`{> zY|B-jCQV|J@;yH?84p#{!xld5P$?X{VmygO^%X~=rX5v~x_4@r?YDaKhA#RkSI+df zjtLup#MGr#0i8fvNSrkGRZe0v@O%pjFClyK4jQhvg4A*M1S&`%3UG5b{kWi)s7m{b z?GJ1Gi+(kp+(dllS~@zie+tJ*1ut9r`uZlOt2}xMbqLB$+9?X{k3!j;5w31*XoFT$ z4_MgQDR|9$pucx_cNswSi57UzW>CN6wYfT+#HOgCtZcOF)@XI;wF$*m{KE&jkzIHu zIY==qzBI?5fZ#c7N9bNYi5);RXmH~77P@B)d`5Kxt(g$wZt^@HyVs>Wan< zlId38cYD;2L48sk&ykMOZZfPeKw?Xtj1Q7VeKWmx{`fr#wn_?@g$z2M?lTcHi%I{B zd7WyXvu(WO;V_G1x0nU1`k#WJj+ApJ?MCpTrIS%oKD2{=)t_Hj$Rg6R`ca6Eiwidi zd+gL}r6)qe*qB{a^eXT72GAbiAXm%Ma-C9oO?K>UC?-83IU11fcT;L>YfFhajMFFD z0|Utwu~gM}CM%QGb7jDwfUwmQz~i>i4=BEWCMA|h&NPD%qLrw)xLnWw{>nY@1I1wV z*N+ULcSBhZx7M9~5o(#xe1800-M+q0*rJ!>O`g(I^6}04O{H$jU11OG92{9i@BCR~ zAx~76-U3AJV`a4=ub9m47!s#trHop~3%J-SKo4R<89z#;*6eG}!}j=45Kk(I+z%~T z%Y12r}5mgHr1_&cu#F=rnSNJ zV5Xu8y-(ggf6^q-2)w;jM{ZTnvO4X9Dbid3ym*J?s>tco;ByNv$C+ANonlu5pn3#i zbp58v3DU!-!JumSnelF+`0a87tfCSghmje^`ZS0iolC@sm9IV6MLzE9n(Haq;KiLF z#79)d9K`XMZN~k6bW89li-|lJC0b|-;0&Gz3-h%W{T~%Z1RM|wEig%Vs9B zvB-*0mw{BIp8TSfu-a;bq_bFm`zr)QQ!*6NgyHq5e=#kiqZq{6K&M=PoB#z)S7do9 zow7U49N{G*!H*Xx~YOG_)i93`Lkm~dDdM0;TH+0lbQ4gs+xLnWV=jEY7O1IfZi!~W17g~Ke`dsUcNk^W*p*L2zg|CW=O2g4=e_GX9 zFt7LiT;mJ>>r---!Wv&w`a@>CLm>Spa=(W}%TxIRX}?Ch!Z0Odqch~&+bs>fW^J*C zZ*pily56oiNp{{Od=9#UC0eBn5d$<9SdR^)k%IB%l$8A5E_DRwy1TpQI^mmZYil3P z)Y%g!B_}W93%oLxNAiHvWSO`iM6{*R;H|O3#j@*=g?EeopAr)C?0FEiV4`P#G$Z4L zy$-X@f5pVc>XbEqMgR5wFA{*dFD#H%VhDl{oK#ePxKYTMc0>#?JXXxM2C26JY8PFK ztE6RBM!(E=l?=he#Ds)X!g#jMr2ZXw1Q5lS*m4~~5dB)4qiL{cpZj%=Kz04(loTdV zG~LS+HDVb`y9kq1$v7HqejOnH{H!71x5&fIThZLojoBJ zd<$1se~h=jAd*AEr5go>=3SvuiW@Z+X2b+UC$Ctuu$O%JJa;CPfyhcf-@LQQUVyGs*I8U#d_38HuXVrpa1%Hbdw2k#mWGPx z)f=PhJfdp<8&6a%0QAC-X?xs7faZk-YMW=1c2AcaRW;R#rFeH%)dsRhTt4ZFDRkq*jPhOVmFGq&s&Ec6_ zSgbc6v}vkr4*mR@4E{2p@ZIGyLR&teW^IJZMuo1LfQsS^+Nj~trX>v`3y#e~%N5JZ zmoN3+&3ou%?2E%P-G97P5MBkKYlp$G{qEf0YL)WgIRJ_IEBLW>Wjb9OBabnN2CTFE z3qos7_?PeDxjPpU5@LVXnsgmiT$JdPl$8-k5fv2`sYRBRlzi&xk?QgPb0A-;W1)wI29co}w5lN= z{`oUjE7!WHR~&Y}J+6pFF8GFISrA}_USbioW%M1lQtk4Rd8c=pxJ(#Nm?~+4P2dU6 z5EsOS_Gjsd3$Iyj!ug^PPviXtt7FPk(KtZv!VYi!VSInG7WqPs0`mGhs-Mu}z?z&F z+lIl6DKFipP&pWV60kBBZT&fvDs}t-DM~`se|rJ8X7cn6LFj$FffkrwR3zsrE8fNR z3s&$ow2~!1KVN~O?CP478%(gguHd3%@17@%^t!rL≻!9U8;XLdRCNnLgTQW9yiF2qn@ndCnD^@{icj`3PH>^P2MEvL9{!@SS5~ zax%-lAZR?M8P!yr-v)65w&Y4&dVlsuKcmGKIaSNJ6}Wx$^=~a&j(ZOj_ZUp1#_wSP zjhA6SCLZ-W!p+8aZLOAk z=q`B>-x>Y-o~j9fPjjm-7~(P7pNPM(4S$ip?Ex4pyaOhgnEEi$;{nUD`Ze%o$grjMPYr^9xL!6>%aZT9VM7_l*quCoZ5 zjzigN{zzk>k3=r+jH!wl8T`~S@uxQe?uD@n02@)({awKeT~jW4ziVOx7g~}y6d8kq zP(!&(5GYijy4mw#Ek0*<^bq1(6O+I@52JU_xl6>wHA9F~pyFc%${3z|Gl?IQ+ph;H zqe`3N!34$Vcq7eO0rb?wl_lbUVYY^Wp+pw@#|7FjWAIW`dmWkEPL!n(5EBnC-kh9)v}d z<)#3lMm{h}{RLp<@jgmb{mPhVP9l%b$EvmATK1G zLDDka&9w)%J1n&5!D)oLOEBV1A11y)F)Je?SYWNfqRb6&$Tt4Gb)vi6PD!UT@t$`lMzT47zHcTCiiZeRAg3cA8 ztatf#QLTi-kTh~NGRFMjda|8Bar@!^LTnue$9W6jUyZWLvoMyKu$xCJC#6}Yz5Wk( z7dw|fX|UNU7Hm!~{v&vLp`8v43{1hSe^RXfArGzVEQkiiinu!S#Ww%Q#DbbOdsP7-UUV_(- zXey{C9s(?k3v}IJb#}kyW$vi8 zLH8~DhrAzIXdW*OOZo@u>b?N2uM~D54(s|IIY3!rNid2 zhYG0{!VU%pRKOJ*g?Pya_KbRs@Oagl*w7^yZyPJ;=|iz>4^0`Np*NM?xA;=&C?fFI zjGJgT7vp!I19t~Tm&MuMtPY>Ulv@9%VSx&wFH^`{NMWU~kV8hO5V8L|%w7y`MxdZ+ zzdepRE}X_I69y#j^$M)S2?^Y*^H;3swM`!M_Btj$0#f#>V`oqgCnkpeGP%n`)`PHP zKOnR_#74`aykyd%<~7h$2v|x|P0{wo3S;q}iN~y>FKY98mNLJHHxa2taC$V=5f)0k z1_!=Z%_P#|^JQm%jI)?OMQhu(A->he&BQ?Cn0`0sDx0x?w8m<$4?9^o7RW(Rh9H-hP>zVujxEss`Q{^sZsBb0KW+_KnV$>7|n}I(7FgD>HuZ+ zxDI}VV%n$`R(`Gcr+B_4-)Z+6tTypIiYc`WDM(9|rI1JX`(Ws$kHP+8DZg2(RD0a1 zp}82EaE0;tN?u$M;N_EG$dmt_t||F7br+!bSiZJkB!OMkDR(z+O&vbA|LBwR!|2_- z7F>J}OioT#<&mVElEFI+7Y90t37`UlC6v&`tQUV;px3UKhi28>|D7^st!FNfp&M*BzD|w_u{8C#%wbo zy5BNYrX^Wt0NUOpA|Nm%m3#2UgrgL!xFB!T|FGL)Y=~b-_j~j6-RXj@-ku2jL>GZ6 zxsXs4S1<`IWoBVHay0C-8(Xt?XM9Qh~! z^jnQVGmt_$1B&ppgD+^cmYV)X-&~Qt+SNPp2K?2$T#g~-SHV$7Kx-?vPIvtRMrC+N6e^!xXE-LK-7{Y5GOVuMBGT&7A1oEH9GkoAGj3_f16O=dJ{&lU z3c4(H`~q!Tow!thg{9PMZ>A1oczF2W+bLCzyBR&0@oCSMMMIGOBO~!vuqa3f_bReC z_Qa_p<47ZXBOC8V9$i>msRxB7BM?0uQb~vCj_V>oHtX)@))Yf2wqS^fol^%C;@%Hs z|FAlo-a`#M0WfT1VSJ4CCA?!=fQYbgjA{)N2yo#EL5o1U(X5O9gp-M~{{)H=VKgcS zMf3*4`{hdLa?QwdM@u0&swWT?Wfhf$CcE*H0;sRu+39R28mdQl-F^A^&G3C#GSFwd zyleRIP?I{!$%Q=AYaD0!d_g`ZR^tp=9WHgkwPQ(LV>4UnAUBA#tf71!&{umLJb;_Z zIRq+9&I&PBMPm4G%p0=XV5i?6(E8*lCxYm1a?}8F>G|ioNHU<74}7y8p~Jxm2J)pW zP)gHxV?o+antp88Gscy>#SXlbl=3%T9=irTV-SoHbnR`WZ+?h9+~#Hl-e2W_F^2D> zZ5}Y?vB&vuO2Rg?{i)no6wngLQK-dhv54a9l6I(s-k<`|@!=^lUkI50#U&-xmjT=| z2|O@gJ!^;*IzF%gsj(;6<-?G~0E-k>5XO5?TB4b^gY?M+j`K}t<6uUEdw=$+QNH8T zf8-JLj~aCfsbYbmbBX^FN7l$oPowTQ|C7L?xItcfzw?L3;p6v$Cd?&?%cQPF!p-Pa zn<=%-sxbU-OC*QDHN(`eQbJuhiUk8s5zeK*pnaO;+ML(dA-*2Ax9aYlGeiY73PQIua+X{Vlwe$Cx%sCGproA3x>^L^eGXTLQ z5vY$7OxVR3^J=@4uWw{?(mSSsHX{8qQ$!Vj1LIVmuB0Id(bQ&Eps|~~JE8sam3BD!1KK?U zSX>e)E==r?kB*Ks^z@iq5CvS8g8TsbsEl8( z0drlAtaq3WQT@>U?3j7Tny&c(J9sKIG&G!rh2z)2B*EBXJ&)Gp!TPT z_UH7%$v6l!U)KI5pe^ zWI`>=(Rvp`6U9-Q=L1Y!U5|=zX=M_utgZRCrfaNqNtcSN0d5!oQpZOQgL>ANyu2RY zhSyU&S5s+4S=DC|&?k$?eLUy7Acu#(u-e+UQ$Pze{+y9PZnB6}D!XWAW>jmVzJ7ak zV)5zoXPqWPc3qQz-?kJR=@x$frG)yu9 zH9*y$y2M`W%`yVMl}N~h_7oU!gj~QLDBs#*5CH^t4pQ@^=wilO`?GAk; z7t*%0vMMn4JL>(`>U%N4YmV9m^ZUQd0yc5tgVe#ckXX`9Vd0^&1+#{CbM0OpttQvm zjStNOzw{KC-{p7UVX&wG#ej~NR||~6@=8h+BBP?{#l%cZJ%TOIz%Yf$8)Q=H$;pYM zaw?Da?AF_gwGYvwSWguh0ie5{ai~{J<980QR99DT0I)Ox$d5)qbTl?CoE6 zfzCT6=jGA8Dq;&Q13eo*c{pE+EZ820Pq#DNAJmizPq1{a> zw7@kF_y6{&o2AK7)k0hXF1A`Y$^VQ1+^*SaP^vQ7|6>p};SNZhp|UdIkxEL+4h|0E z&O0-N#Kg%anwmBJTW_6+HiDU|6fX=Vh7XHyfBnvh#C;aRi|`Ym3mVEEyIbQ z?HVJPkU*l#euP+IOHNgdx(Yw|us>A=lnE@PVwx3#cxGHC!uSmu4Jw*J8azfCEL8IpkevUtcTLkDj;zj~7Of6xq#in#sPuXVgD-JQsB;-g?y?_~y%qUFG< zpt)Uq@#1e#J0!T(rL4vpcwao~(3cmt22LKqN0zSfi~y%zahYPsbv%cdetk(t;62 zVZ&_9(}cw@`Fjq9>!@Em$)EYt!G3^)3`vfZlW(CPlI#VgOxbjI(Yoq@#UctoH>YPJ zK5vz*tgLQRtTUOw;UaG3Nia=jHVL@OUt3?-VmE|UF@W2p0CZrPR`AH7h#vKeKe#T= zKi8eG2iLXfPp}uP^(%kfqz$Y8AO5N@@@0HHRJy4DyTdBK)57G5e23f4KwJWE%!(lY zHEK$mzu}k}g+#>`=OcbSkGih$Wfwn@e_6X;GVFw0Y^Y?38g2Z@Xa@B=b6!EY=rF-_Y1yzy!I`M%W8`<;yR zgDHy6s3XVE%hAD@^$&iQ{Wf#pM$wReBriMX;Jl3O1JF_MN0%^!1w!}r@v)A@OyC=zvI4H<|FOI@d6e3NdD^9(o4)?cM=%NC2Yp?Lrh2-koAP&o?3f@rlN>pE z!(YLR#3u(RZJ&-m#oFsL#t9POuiMYycNQ6Mavd%Xj*RTNEr$_rbGAks_4ppNG6D@* z`_N-AD>E}dQ%`R?5s;d)q3_?7T7fIq0)P-U&^((kB|ujH-yp$cISmE@f%4A4-)_sV z9{VhI3)p`2>$0L1wqmAX0`>$IwyIK%KLT^{KL3|nl8&Vrt%w^TiwvMZBW-YXvK0@a z$Q6T#fyF(~o2lsN=um*ic$xumrl+N)CGhZOb%(pLz=@o}YRln|fC4mht-Fa()2K$s z1x%5cjYj6!l0SQkYL{kt3>IJvUjU)0XvG#nXV_BpbG@<50oM%QY$VD_5nG>7))4zkAd0ASSbevOWBV;VxYU~2;s@b*pv9KaapzHAz-Jf zvHt!0cZG3_m#aP)1$1kbJDru8v>VIBkZ8Sr{rYs~SFWt{uX58>VuGab^dqjPxnbJg z14jWkzpe<5d5ttQXf`w6W$w<@DQ^9ZBxI=y)&=M~AIQ4(j1jf90ch> z53HOyYBXAnS`)R50fw9RBh2VJ4_;v}F1ElC!L#q*zkg0*H=0>%kR}|IB7dXRHMe}h6sg+j}TyKeV$1wk_h(MLL3U%Os-m}aFf9@;Ej~9>>e)Oy~ZiR5w95ilHq`Y*u zdS9L{=usjJjww}xA$KD%3W0gLaXhu;hXG(&S_1~GWZ*!P17bODliKvuStknR0PmzB z?bTH~il00<%0+DK^$HC8bq!R_%)ZV6o(Vr(FA24>;Hz%vN^`9-&LlYYTw=X=bb zD{WOX)&wsc|Edm1FOyISkaLj!PfG*+-0Pa^>N%$d-+wt8woe)IuwP=hW5`b`-NI9x zj+-W1Lb9oV8X1=wS^F0dKy-1xwJ~VH>`o+MK&fo5HtR-i1aV9>>1;Lq{)G!Zf@3l# z+M2e`N`%X@FGG=6`1vm;_fWx9tB^*`?6FLUc)&K3ln_DIao9D>*NDm5LbG+j3y*5w z0oi2|h%h;OYmZe;Yn{`)VA``gJ^wwiTVo(QARI+DZy}pha_hCliLNR2(X5Q{Zh>IG zhyF}IY>ndt*8G(k-yFfLo;?D&*rN~SU z8dmu+W6#MTV^-NMd0H1--kI?P)5ep5aXaZR%J zAj{kP3Q1d40A>mt3;ha;ioVc+2BKXKaEc6hJQVxo0`?Q0zyNy%xyjRp#Q>JT+f~Z< zcoonW2rQ-BJJ;Y{WsXN#!f$rpR9XMx%nL{kbNdZ&2_`%MFINewbsh+kEPLbHry(JY3@S{pjJ5FtM z-&~0a3?JLtbsECrS@;WL#X7)}XY^tg9;-N&aB&RmtyJwW7Ab~X!@F;~B}9{f4>yZ$ z{p9cdoMWdoe83_v(bZWdHV}&eywnBL&oW;1?3=XcWk;6+2BL50W8&xWm~pjju$XlW zg6hi26h~#>ZQ{ws%G`M`aCppL;95(;+4}M2EkMW!TVG$l z(9IT_VGli}w&v2NKKPhI;nqU=^gICPH*Yy+SeWX?d^7*8=mRhx$pS?@VN`cY>2dq( z%R*5gcI3>DmZIxl7JMXoS^a9c#JB-WnNm2-LfjJG-JK55#-j4scHVVN`h7K=veF;R z6mZGdY52ipDB1X1IC-9%p5F2VoWlFG^qIhMYapH$by!1NL*uuORK0G|1GWEv9#m7e z$vg*MW%^nvL*FWQe?l|S=yAE*6nltxWGGyLxyQgf{;350&xcSJKZ65M=mpomlIdE* zZLBz5I)<*&dk5c`Fjk&bAOj3i8Y6QEM#cjGK5o5hQ)6|*FSk{l-xQp5&CrB>Fc0I^ zu=@X`=9!;DW=;}4rQIW*|CX9_p!_oJ$RJFGCfxoUz=JK}QiAO_e>fVXn$D+^cR(K| zR;UM$9n_qc=kFcWnmq{(?!Nh@QtSW{P@MbFqSnFrElrgP2@QXBFrbIqCjqK>xt^ez z(2c&bx@tc>rtd{97fUY0;6fhMK@JQyj-ow*7?` z3!`6O$49x{3Pa%UJ+{sLO0Yb*SO~;F`-_9chvt4isHvG-w<_A_YapqF7@5lvQ5D*_PVi%A-G9o<=PjB~!pMp%({Rp43=O&wXBj3i3 zdpsKcuE^2)n)I)s?@{AsX6ah*+zYqJL%|C|<*9uXfaYu+VflR@v3gryV-E8Gql#ci z-=xapPd~H4>rrW|E7%Mwv;DFZgebIHLsGHzsE_o87bgu`Bwium*9sR^XF1qPlr|Wr zCmx%6gguwh(;b7KY=fVe!>xm#t~FH=$5~yxZ0=rG+kQoWn7&Zwxu-AvoR+iM6T?8%qLc1xejM2MaezH;si3FyY;=PB`R`Gz@p077gWR!UOcZP1V4 z0)SV*%P+nxW0Q-eD*OMKR7&56SxGI=fMe!5;616{4(vLr%evnHuqlORBVPDQxL8Q- z$QM<2ZmKI0Zn|;!GsO1%IgHQH;{z-+PHgSgvAs|?HghP*nOtJYbbjgUX{kt3WB40Y`!C_Ns-aw_o@OBwj@&!Byk3nbY1I!j;F!C z^{ExWe25&Q>BDCFJq_^@@}E9LtUSZkiAEjx*;XZ8fzSqE>GSqINm#-d`%=dp_wm{- zMjJ-6)cT68LY^;ePZykaF1xncQCtiee4mEH>IB@03S6KDKG>Bz&!iDaR4-u~-smxX z@$qk**m}FNtP6~#Y38dX9FtFlRdp%c?m5*cop6||viVLW!cp+cMD-~J{=AQQ_L)Ih zV?y{h=#fRjbEaQqCFID331Yq|9mZkY+#%SvfDz!3i?`3wXuzkwR*H*{F7)pgu)FFD zjx6!*F|syy3Yjzuo|9ICp_^S`*d|%m^OA71KkdltOL2&fUEE|pL%@sycHYmav`kli zog|F94kq;Nb!Jt5Bg8{j<{`X$#W%0YPk^^kk4hmPC-n&~wcxw`vRl##pQm{)+xn9B z`ah3sb}a`deNOc^MKN?#U6)&U&w*?hqDqYO$%Qv4)=+XVtEIQMH{|y64H9lq6MDyr zo$0cV2f!oAU^ZCnnuX!4c6Te{1O=8fwTj;Xc3nrob})h#E#Kqv;~M7nnu~vXexL&p z2KsmGY%9c}d(@lus^g?10NqrkyYBD3AuX@6wb6NTyLdQ_qfK-6hTM7mdd67fBxQ-3 zIQ|ClBO$0Qx)Cbm0N?h@(VY1)!IO8+sib2IVJ zp`uUH(;)c8utVGSk&TN;GDlsT=In#o6Xsy9Vn;|*;R>86^ZY_7KH7_<)>DsEjEbM~9 z)2R3p9{RnA!_?s4o-2$1t7R4a zMPA2weGCWKZ0PmUW#HY}WZ*ykC|dRQEDt`xHW{J{U1Ht97(E#-S*NBghILKLyX=hs zw--{LkA&Br_==4-K{_sQwV$5-{VCTrAhQ%;+QjP!?Irvw^VC$iROEus*$(aT$(+tb z(WRZ&C>93aMpyeYG#S}T;dX%1hmv2=jrU5C+F^=?yW&b{&#wk08^8!(A$0d^DUX^= ztxfChE`KP9pxBwSUXGMmLj01H15 z+I|4oAU@5xZMw3hgpy?lX#kKf79%Jw4BKl_TtMT~Pk6)T~fu!1|z zGzJx>?)_u1P}q>yPA9l^pL^9|&b%S_2eScagHhvZT z^lzb9tBv&E0MioQa$)wN$D%pIowJFD2(z zk0_&Kpkq;aeW==-$c9~Ulul+bVWubV*tfp@@FxOC-#IxXgS#UP(m223OyV~G9&doa zARX3NF;XGj{NE?1=nGTZWMfG3ThzUVF({Q=UUrm})~%E{qEeLvwU@mgG-0B-tdyq< zi37|ETPlW<#$%gk{?=yS|orcL_4a+WQ~xCnS`NZ3OqML`8DYiyYC;>3cv%hc5!- zGOgDrZ$d40d!Gz(TcZ6$Sv+ujv-Hn<4`aMT*foFeG4WZ83Sl`iAe`pe@BhW#dqzdI zwC%biCFh(GC1(&pvP1<*N)wwbIY*J$t$*$wmM1a|d$zjyEV z+hdP2&X4os{Me2)Rv4>$&gz;~HLITIzOSb{XepA@PFLT>3`Yw0vSm%p$;7+$-R~t< zzRgbqVjJpOsQcR+VX}Ll?TILGQ7fxYs4)G#=D*NwxjfBnHgSp(t6zz~K(BY2f2j1( z!9gLN*afF(JEPSK7$tGR^RUN1 zzRs}3yfwKL78Y$+Ja;*>Fofkde>>v9?i!#KQ`T5|jFSC4L`!fzA^5&mzp$7w+L_;k zTRVW!#H=!q1YQUJp03NwDs zPFX4uDJVJ^y6@_~Gf7p1Eh>~P8GgR`nIj3oH5FlU?9M!$HKh9X#Ao4!?JsBDD+f$qXC@HCSp_Zj^c%W-pGh+E0R_a!x>W@RcT$yLrYcf z>rU9}v<;{0KkAYvA4B9m#9;4vbA#UyG2uN0(c$PjDbWl9E*v)Ce2T(ljq#oj4&7-9 zS=;5RBX75USoS$hgO|H4LNI^lJ_L@~ZPp0z|5ZKWL0Pf$g3wC^X8lUt_E}jq(42Es zpzfr@q~FyOLg3V&UHXa2W$?eRMq6-%B(6BS(AJZ7`2XT^V0!1|(wPh6>i2YmhkImH(|7@ zaPXo4C)3!c6!Dpr%!Cg5N=kz{`{%15fz$H9Kc*H)L#U;@f5q8%FANmr=SFCGKCmahlGWT=Qq> zHo=W{r=rik8qk~mZOaYi`uowB+Cbusi(~US*7HU7`e$Wb?qDC|F``5G&?>tmYE|sw ziyt$|^mSmdqKzLkc9qkl^ZJ>F-TaIsBNZ)`@oM_7L7J)eg$&z$*J7p*Ge>;7`J-+p(b!*2S`!HN8IF=>A7_u>HUA zhX>bNkIM?kmsFc3P_8{jf9moY%vK1zr0+{N8;Pl#@6)Ha{}UZy#YC))Dgdolio8ms zbV4QS=7*GGo1oH$lLF~uER!rIk(a^DvrN6c@*_k?-vX&8E_2sn- z2?~em9%ABS>v$-+6ukG-Ua%NLc89**cvuxo7*Dq6HINdQ;H&%`WgZ(m9LpAam^=fz zWMd3u)6K80uBuCfCyf31W!p}Fs02BuoROXq#*jpyXh^>kOCSp(HJ&tqYLVjUdZ}5* zl3&t4UO@(D=HFp6>grnMqN7jYdY8LZc261i&;PAj?zE2`rWM_)znyCg+zMSlWm+d? zWysNIByie~{rS`B-$DLM6QDapp1oK^kWx;Yi`VDEnF1k^lD6a0sN z=^5a&8gpa!QSES{eI z^-9WhOzWww0d{TY*@!0vWKX_#t^>357?E%+%OdXBd+>@@%-(Dt2?#j(WC|ihiAKA0 z7p5`Pc(_$j>-_A$vcGkQtT7Ng!#R$=f<5QY%Ih@A>Uth2rtzfx!^6trnyIJ3?^d0D z*ky)m!4V5zoPWk0{XCoiYMlD(i^In7{7S`&{QuGdu6F*sh+R~Zo~M|U8;SEyox(&T zHrke|)A-(f-Q`q(=Piw8tBScUSJv#*sf64-3i@6ii8+exC|drs`}Nl=q)fht(Dzfw zY-phxEjd;gqkU@^an3wO!1K$OB?mZ1;UVfQ-5aLH*yjbGvB#~Co>H$X#1bF;E-gzz zm4YB8U+opKq<=oW0Y8&*rfCEjI$X6Nn+@gBuk(64kiBU3yOA@xb)8P*3iT)%ys>+0)QFiA+YqXqu4H zCtteBrv=YuU37Jc=P+fpCDV;)g|uQ3`6do;$07j{61i3DWZXNGO9b&TYMZ#zcrG!~{sX zlSaSJ%HWQFUW(ZG{nzVEhG8;LkO+s+;wz#gEV*Zt`0%yg{Rf00{69dX@(i}8j1Y!jFl$xAy`Bi&5F!9kND^gl%r^rqaz4T7l?l`rdxB$v>eMm(U~0Ndm> z#QGmn$5?T4g&c?63c>%>J7Rknn(|724u_^L*k0`noq~{dXVpJrnr+uc4p;pqrlRfs z+Sa5-9Gu{|fjFBKjY4}SOik!jsND^J`20~smailDL7YgdNjX1=nm&=AS+SM`GA0Z3 z1{xY{TAxwzLkSDHxaLTM1xm^Hlp+&p=Z{CA+lX>EoEa?1ZhC^g^NfrP$B1eQ9&8sJ z5E^O!`rF;J(Igv8KD~G)`Ia{g$R|&n$%_wRm`tl+grHRG6H^!acS99g-F(AzDmmEM zHM)uB_B))ykKF5;ia(>x`_A)w*HYuZDX9f}-%>!|=t+A5<=3T>ha=5CLQ zuVTf{Q>bP{3Q!QJUj})|Onib%ZB?XK6~uJ_i|JyMxuz@c25;wql_o3kZpsTr5buKS}91^z0)fv}lKO^?SLD0)wd7$JIBRZfDVsa!)orwmrOmgJM_Lm}UzwaN*gi+JVvd z+O*hTj?K!n-E(iYdb)4Q7%UrZzH;5<>!&q?yotaH4!oLjWC>BJmCqx_ady9GPuk|= zta_=RJyOglD)4`R#7q>vJzTCo(qMhh4oTFWy)RzCYCMtEJnF%)S^%&jL{KbV@g-nP z$oll@-H#uID(M)cPaa`7oRmIaWcrVc9k$S)8Gmw9!#K ziFvoxHV=i^+9-Veb~w)G^a`+)#i{C4xtFw;>;$4&2cNS$Phta9&7{?3Ky5JL>tRxX!l35JkstnU|SmKs24wm zsBo&4>w~~G3BlmR)iDr)B)(Rcgb)`J74#RYMy!Z=9j%J7xvpIR##}t`wa@F8mN?S` z`6`mr<1tbb?t>z0HYI&ds)%dG^Lx2?&rvrDj`cJcX z)H(hWDE?6+Lq@Le|CSRD803F<3x+4_Var;YQ#Eo#Bz!V0(wn%=VvG~|-En;&-s^$7 zJ^59&Yt4Bww+u(;@^hBwyiJsI@6&UlZy<>nPR8_xqO9GUB^qQVbm1d@ z8Qf$QQbAEN4gW{2D1D&iW1~%wcC`SSeF)%JL;=TaKtW#K`ZMA$PoQ#gD@+p-KV{>X z{|M_IA!p5HF+{7<2JGZX$z>!XP$=He<-jJIPnD4w@iFJjYkpI9s2_}MjhlYY^V$!D z9rG1qs5H|uGnX*7WtGGn=4upxIN#lOuEyd!u&PEI^3LE9Uvd1{k21N@yorxydK(8Z z_-B4zvKoLp1%&JZ(JaQXDX?NDkG}(^@y&0vPLm!5+^t0b03hb{=Dg?8qen>$Qof?t zTLGoTfIXrNM2dqzbNE4Zc1D}|&)b$IsV6}0U<%BH!>cMuKor_gvA5KsQ^jEdnxO>b zBL-`(O<6<<#qy7&?PK|^6~^n{iU0ht`1?k;f$Hbw}_tw~(fu3dw zfS26!8^-%Q&;HMHQ7n}YDJfg##LKn7*^fOkwhVVM_b@UtItFlzbzr<&X6NSi)s~<3 z;d|iiDnB76i1@5;op~LVhLlWPCGgG9oC1G#@)1+)f-5>Isu`)H zssr+sA3&MlvptfToQ_5Jd(A73mS{Q1<#iybq@yy3*0yfXB0C)*E%}YZixH~lo_Vbtg>3x0?|s7Mq|fQ2BL9G1G|{Sb zyZldKk|$+naTw4fXoGdsgEGO%T~|esCoet1ojie;AUxj(=s}Bt%1aw0B+mgipa6uC z=5@AbXJ<<;FE49=Iadb?h5DyVZ&ORm105+w z*2;LPywl?god~R**jKKw$=2{T+!hvo4d_Tk)4FR-glkcm0nvaO5XNWR{&&XwF?T8; zH*ReJa|Rbs{-!u|9b7H2SiH{apcDw$z{oQN5;6iHKX@$L%J>)-YbTa-Rj{i7o$y{O z`$8P4`8a#-wFBGgsL;OzVA_bN+ufs={}O=N-CNPF9TgdDTp0j5px{0a$5UcKt7~}C zLlH-Cfttn~XYz#qj%VhAYtb=GK1~ zp=qEr7CA$nuT4A$$}S!-gWi818j6ty6w%~F!J!ZQYs8nLe0-HiIE1i^i_7SkWG8pN zRi72LGz> z2A$&q_t8h94F9+9qwpY6=*T`(1;2ZmPUk!dT#G_yxOsIoMqR?V|jU4zu z9ny66f1COLQE#TDp!fY-3-Et?_WxJov|~#qR!%;w!JcC#Ic0#Fh2?|x>xWM|qD(A1 zuzPwU#90)gJ|tSq@OAXxktRTx1=Kol*Y7oLj%f9 z*LSz-?+K}yRejFMY*vgOJ(=!9OXV>^o0?+2zKFJ;WB&7J&~>lyW5EmG*1x5B%$BbI z*jlO{gZZpN$S1CZrU}q4Q@g7(83|du?{2PD(v=vKi|EC@+hEkPFHKgY5_}F4%y(x7 zbGv1~lSR&vuXE~2V5i1_ssH_c%rFZo$x#9@`sL@?j#$rYEeHH0e$lZ(z!)iLFpDK| zRnXsdp}baY{v&){*nUs*u~p!+#iqisZ$GA0FMe6iX{HC*L7`?Rr-x{EiY`Q-EO}YSdIY|?0;;O(FUjX)r zAJWn|!E5ss1i}I~#2>P=`M@438EC;2kpid>K;GFQCM~@(4Q%U;0I{b`{C%o6@EdRG zH)7IT}c$mPQ;;dmw57xylV$r7p0>=b^ zn6{v2l?f3BaWG4;HBEZ|jJ*;B)4l_U4wAsF)Bz;OqKyk}1+A@rUl3mjz>8>ZcG9W= zCGM`Ee(Kl8sn4~Ehu^V6lkuZ1?321Q9v0L#tjYg+A3d_nir|FW{R7?9dqCorZd4yy zo000Q{jC}FMYn8DFyL$bH{%xJV97Fe$}AA>K}NErrCC8-qHu82TSMOeM+?Klq?sCv z;p=FM@7a}2->}L}X?-6Y?UXXl&Hd@(>*zT>W|2NYn$#u>qOia zI+&39Z=eAHHm#(~ORpCyDv2No+YgLw%s{=A4D7=?h_hWb0?hUO8P$K;+lETOK57mK z!*4p;h7D|E(w>xa%r?FL5zMb`5EV`>hW<9N!jRaM={})7D@lYbns8i1KL+uOG52;^ z`K^eaMLTUK&`SEJ01vf8BD%2VT=r|OcKZd0$g7k(nTF~jq0B5IB09unl!opJr~jKz z(2P8fe4U*6yfYB}4~R_+f(_>C2Ipb>bp^8t(0Tq0qjeO0*3*q9?zjlg13CV`d+uV# zk+$iO`y8xAMa?y@>?NR`5)Y%~fIw_*n}*avNp{lxzx4d^2Lb{=zLY5ICx47?t9kXI zq?Ft){ezi#A*Ynl010U?IVIbbuy9S^@1%X^8UuOHsGdFk(TLhYHWH1pijIiqd0THc znScB!o7O%J?oM{g#28Xv)h5&C@_vRk@}b% z35PPP75C@LF-zY(0v*U~&=_TR?YW6Gu+`+}$N>M^CVfKc@bsqjRMu2=bK3H}@|x|` zy8(X%{C`ZsVB&~u0YPu|SDF%5U0r}^Vn#}p0U?-Owg!yrW*1;0kC0&>1&EX|d|Flz zXDv4m424_(RlNt0x0yVHK$2~wU)&JHbVC}ylYR>-1U+ zuA}V)r7}->oRz=|lRk#hxUeL<395qlA=<6`Fuz zFGjd~fEVq&?sQ(iSteh+?1|a4mq4S^q=pnkMbNo_n%O~c!8R867C6^-a(D`JBV?E`ela)$ zbq(PSPy~P2-=!kY0g!i4+(j6GU8$eDJu4RnOibWSy$5*m7m6%ZgVa_T31?txsJ*RR zfQ8s;h$yGhxHUQpo7+xxU;g!76nA}uqZA{vSKSwe4Mkb{drr&m!HHA!^4Uu$9fPDV z6(O3UDT=DRDI_sG_u#o|@7dg(b#k`xx>f+$$jlu_FL3d^B55cp-XKR&N4vj7iN;w2 zzaOHKtiZI6;V5z3L&|8-;qUihpP>rC!l1yvx*V+IjvwKkyNg2T4AvJskoq_x2zNvW z^3Bm@fM%1cg&%dMgxD2aKp=>>mCnM~EG%nH(R_++N(|2=8LC{p*SadM55C@73|2{t z(>9_g;t!NE<;To9Fzrj)zAMC)W4nz) zr(dMxROz1L@ZpMgfosB9+0yxDD&thFhGcGV`S9K}jC{Tx5$}wDupHs7Qd1K{n;}r* z9NWD|O&eVpP5JOVFS>iG*ysA)iE56Mu~W*iRA7V-w$4RGlcLdAtb(ECpKnaN%M=vhtOeM z+nH}aI9_WUP8*ayS)dn3@!rCt`RRvmuqIZE`OU_#4l5$I9^;nU3WX9%wD`GkR0=nS zF7hUv;2n?c-Iw{HMc3qapnidu$+ASh08=MR%MRzyk?c5Z*s*#5Z?f}Q_=lYc@FX)< zkG(tr9{IbuRnNMemm+~Ka4o%2+EO|MLc}9Qx+*yffv?kFinH{AiDi-wrlR(s#Tfo- zw)I%?Dkv=M_+<3%CP?a>%6U7}pSupd6fOr(Ym;E$HmJkOQsMm~CNJfl;VUR($0(-_ zN!kRcuQrPDm1Ltf*gagB_gRV5`yV>4qi4I}U^4*4qv8shsj1ahRuvX$JEu|mJw2?q z;ri8DyBw_ojDbof>*ziR%TJv^Z7{Dq&l#uG8NWR^U%d4<>JOD3=g>roUK~4e8Oi63W9b2Et{-CX%cjK#;_liDJ-@`b#8wD8VE^tP3X+-R0&on4Z!x z*d0B|-1v6yZQ<{W`PyQgR!__rNSDJBb$M^^r+3uUp9_O^11m8J?S-vjU$3shoNVV^ zX+JpQXH-`Z1v-;h6H`)N+&FdIP^5qOu)crMu;3`?>nl;DmPs?>qE%W6oMKFTe1xMC zCozqgKxIznvsZl&B)i$rXN}6geRXwl$JH@7O=_qZ$;1Ut4&UQYal`u=Jka69oyo4| zr*83`m?1k90inApe5fMJSgjXXn=~dLo!U`TIj0$;9U$M7xsG6{{gPhs;X<*DB|pMy zD_t&}fd z!)9O^*no-&M#)&tVEt2MSk&*JD;V?BRPS2vl1Sk|h@IZiZ>gJ$4ufpp zGx9Nb_1BO63f;m|%NdiUlwmyjAvybg0ij$1o13V|QhXNgsNxP#2_@g?nZJW`X=&mk z&g4d-$Wv23TE)1oyf3ltIk=6DIMsg+SMH zo9|#WL^XG~goih?y}eTmNmH}f#{^Hhw_Y#}0mWdJX96&Y?;+Tvyww{GQa7g)x&WIe zQq^IA`w-WCZV{KNV*?UTpSR6_T+mpu)feeks4w|}C~oc~O|o))z>v}*X}Tqa_;)EE0T z+{WhHyRJf_@6(Gnw8Sr5_r&85OTY_FN9873<|)Q6)+*pP4FN=vaf9IR_fzRhL2?aZ zb`CNDYrvn>2Oi@wuto_yf<$_@1L|akXHQoG+9wY`KtMKd{_%n>7m%os{J3#ovV`?j z`(C;7m!@PUA%&kY@vCTWQQxj4*{|tqrWV*tzziHw-%B&-w$oisWTP1w^%)ZN5A#a| zi?lqp2_l!iizzUvMM0CsQkkYu=l@TM*&T%>FwVu z_@4nTl{$spZ5}ONn21>yk~g(}0s!#2Kx$&NHZZ2`nG`I8sa0ttsZlk5fHs^iGP>O4 z#`Z8mOq9L6#Ywrr*|53Jex*0tA5BfPtEOfo)%0FrTo?)XA#^7WW1Ud1yPIE>QE+&J zbK81U6shWLc$KKJzOEBI_s^Sqxh5x<8%Ejl3e?o!EOM=-|2i~Zi!lFfRX(QzPOxs_ zAEkx23OV*@a!76sGHSG%cWGStobP>X=s8{@us**PEGc+MvL+?lT$G~yx0&MB6 z9>IQ(Ea4rrb7;cb|3r!Ozm1Gh_9e)(NDgGM?k_%z4R<48;5Ldpbq8J0y|BPN%Cf3DR*fyjagj>=C z3GTU9KjaEO7YRB}a&s-~A*r`R?_=0~js;=RLO#M7PV?zdW+E}Pmtsd_a)gFX*Z**D zKZ9dyeLi;UEn?Sb?Hwp6w*h1!vYBEYt%;XFoj*RiJ?mv=mM3yLjs5s5YX9iyMc5j$ z#8oRALa`*BlR}YF)cqF`!@HlvPPk-aAQ*ASHKL|SG@t&8Tod63YI}{kDM#-9nwW2_ zWU(FOGWN%I|F!z|?fWlO;BlS?KjGV-C+G7!C2kGp2{uHI;ugDUUA-}YzWrn8A`L34 zFmUZX-Da{ov&BP;Vd>f;zCD*cq3!aZ7akZ0+cLO}e_?Fu|CVdu1BE{UWLMyynKwnX z%SnvI5eOD3e*sWI?|s1a!brIUWT&`hOS{L#cs+aCdBTFoTSLlI} zPGD*(Rb^^{U8h2fB7xUeU3&FxCI09iGuD02HzC}03>{!>>keE@quc_Tvw0^B)ecbM zZ(sGY20MScl7JDz5?*_n=QeTNGhMaKLqh`(vy`xAT{0=&MCzgwo$8xtNiKU#uYulU z*r+t7WApbYCJiI(VrPrfC~Q)G9+8d~Hzu$LEMN9FGbi$RFE5565S}hu* zXJpz-Nh&IC^Q@1?m6BJs+GC870N2by8w~rmn7XcbiKOCSC`uLRfj-Tu}WLCgWBm;0iHR6MO<7wXnRJr zoh~ldhYzOY&u0=G@VbOk+HhMs5Ukj@jrvBzQ5PCt!?qAtC&yMR_K>N=i6tIjFJ5M+ zFfJ^l-d~a=eJ=&?AdV>?oNTRoZ|wi{=vRI(_QPLKI#XLb{fQSfER- zKWKa=Nh0FnR_+G*IwtE0Wdv@2gDgK~(q%b-yA+W_~L#8o*+r#n{TTS<%gaT8X{1aVep{jcG+3eZd}Nba#~Yli4yV zEI~!G6&=;9wE%{HpvocWR`#y#zN=7~R3`XAvO%A!ZAf3e_gc~FXkoWr>thUjt*&6{ zSTWvtGl6~Jgb~1Scd0bE;=7@^dsN^gpbG25xLl8Nl!0JZYU@64GW4Z#yKrm{MLq3O zdtT*jp1I;*`z;~E-j5=RMF&ooJWld~a8akN8C!C3 zF_<4pv}pzVqu;VJxyqW#yPOlXKX_o%Mhpe(r8hNs#28{!h(W=cPLOVI`v^FCOdtjc z*ufDLqgWI|^e{j4%~f9v?wTv~%5`4k$O{&#{d1x55>N2rP$|agTgmF_?`7@Ou2LgO zPz#oZH^Q9Q=xyF`df;odYJ!*k*F4Tc0v*OySyO&T&tL~^cqDGWh&tKB=H^gHRW949 z@*xfCj9gsvCPlueCfDsJPI!^$2Q9sRtyip_7}Twnqgm@6;RDUI$R9h^w(>4kq8U5M z%eOf7jTX!#2@njB?%U-mZLmN{0;+c;SY_sYj_l}8q=G&ra^rH9m+BaLB+nq4fq~P> z@0)A6BZnm8SCbC&k_CJAiqY-4$0#1bA<|^#TIriapT9*1ng~uYl)x;Rf>=_cNxTf! zJlOIs?N=CP8}Kg&jbUJ)>!F7Fl@8%~&$OnLK)A+X;oQ}K4EhuKH zT1dy!2KGGKL#ao+9tWDGQ0TR{H>yBFxyqZJ+AQcX@}s+r?`*Ll`t~-Vg;YiHmOpTA z{gzwT@q7Zy-YR&YUsWi>??*S!69+PpU=_x3g+B!l=t_^|?j*f3010bzL_00C$X(%h zUHj+oBh(c;E8zAW(hY8;V($?h><49(bd^z&s+cl-6AoqUAFe{kog?5WE#W^m5I)@p zCocGv;>0U|hO!BPKbggN=K8~%m&?RJfQ<$rued`{Iu8r?me2EINDzI4cX(FU?q2CT zl-{*f>tp@1cjex(Rl*2*&ZMPzZ|}kUTIJr|=xhbx0G@?)o8=myDe#n6dmiJ*K{CH6 zs%W7uKeEobU5{RwSS=N1$S`3R)EDa3+e@F>Z=}4gffU}E)Nd?Y1ft70R%}-G(I#j1 za%j1jp*K-$hiZPeBZDw$gt6q~q?%{$@M(Q-?Ow;}4t{-VV8OjiXHbEv*Mr6FjG=^s4p7KI_h?+xOerhVfPhk z_Xo9;r9`E1ob`R!gMu}wX8mXRO}5nr??$9xvd1gc=KS95bBatNUw1AT=0q!|7y~X1 zoQ>&bj`W~c@qI7#hCV<#ZVzX+t`EzfqxmL}5-t3tU!PklxrC?O-p+N`pRTY#zuwVn z#xbwbpydu^SfHCk=^$zaT~Tmr-(>0!EG(;OcaBoKz&w0kcWRSc*1Z|2DW`Rq*R4G9 z4dwE?p!(%E&AnPXT9*C-u3Qzu)ipbd_hKt;3p&EgOcvOmlb4< zFHb;ug*fj-GK_o>xyA*RF{IM&*#m_J-mt6B zxwqN04eMh-e=kk|-xCDB`wzK}?Cxx87?-y3zv(_n#OB)Q#$A2?ZS_qzx-Mwm>?%>6 zJ3-=I4GS7IN#D%+06OK~$2tn>`V0pY4p|pJ@6QcT<{3o@QZn*HJ z_lL@JmF&sl%_I@+I7|~Y?DuhZRk5^=RfwyvjAhQ^ang?1?=10(1hVt8ivfY7Nfa(t zD_$)r^|~5gfAc;*n(b80H!-&&7Eg8Y;WWMt-SeG$L#e2b9rRt#=vkq5J`(*)wU`23 z5ur8)e zZ9n<&?I8#(SRF1lo%o{cW{5Zb3FX3{BC9C8N8h{McA+m-<%^&73ETa**^&l(57lTQ zTHG{AGpR=Zs}MuiNS0al1NNOgGXqEJ;Dq(Wra=e0U%QnCmG};&(Us2iJZ^NDyBdE5 zO`a}PE&MzqtMxRUAKlJwlrQCh?azdlZO}4=Yj<3%V0w*xQ+q1rS>}QB2`%$v13#rZ z&EJb#q77gSr@vU=nhVuzkJ=ry5MCb1*=3OxIklQDwf?@9K^^(g)?(4L>0!}iA*upL z!>38bJ?vn-IcTj#h_^8ZeA{VcKu}x~os7xpCMB3cDGj(hLX$&c6l z;{)Gick%8S>UB?@KP-f|?{Jp3XU;6QN6t;tuh70x;Kd{>w92xHbiPKxywSlVsFN?* z_jonGGnCHZ2MxGHeXd+{>-Ps7L@%92vh$LsjTbn-Ck~TUd#<_}z!A%-eKFOJd%hMx z6Lu^nstQN5{K%?WwZ5Ib=Eqd7#-R>(ZMzyf-6@?$Cz{ES{WR^+qOs~MeY6&G9ZM*4 zYO#eL-B$ef%j?Wz5r-%sfg8u=m#R<`A8C35|J2};+4gTOK&@z{5(Db(;a!wff4E+I zn2=&&lSJsZVxoAlz&v7O@zmeMC9PR} zXuyf9x3u+PQymoW=$H8o8X2GN3xQI*z$=UHR8U;ZSdB00VYZ2@v(mX(s5ClrKxWv` zzse=C{D%GWCE|VM-%l;=4hCG5Pj9B!nXk{d9KY0E;%dQbSekJ(;*UC&v?-2``%+ly z>zvQXGdHnyaNL_kkG7t3arSaXXVQEKf8k}mS!xocxAnTw3D%C5&U?STJ2`O5#*c?S zZM!(T%>CoJ?$bMuUFCMEdg|nJHqR_)izmz{!2;&aj*wUGh=a=qhD4oJNw9e z*9HWR9=s2l67flvY}l>DRD56OFTK|tt}F$cwHxTDWw`rxS?OMH>cc$*S9nWnx%J24 z>w886v3X3!lqNne+~)K7PHc#~0(j{ukYn-s)U%>>JsLY45*p380;m%&xt9}$IUqzf z+w!*ZV#SdwW@SLM`fuYttMjy;m4zQjKD?BH_s^YhfZ(YXkFJL6sX?Q@^by|8Pao?% ziG)ENKQcTr$3nS@s&YVSc62H9qLyB+Sx(|@@1AOyouBg@Agwr8jlGllU_HFReUH@; z^-{ma{a6Y9=jY?{io$It+{cJ6*O*Q0xBAg41`%jY)L~9{pXhg|zV6VAYqW?NZ~W*^ zHcRBCs;g+bX|VXx7&@vzNnT7)*hp+Bb}6D?e)R-KNVA=3)#iD-NxA#1v_{PLp+>T7 z_TtvCi}jtq`@NWYh|SkeGa-s_*9u?VpNA-nVtT(AzOK-({6sJOOu2`W#mS)BX+zv@`>Z(OkyY ze>|M^D$xGMVCMcRlHtO3GV?8beh0x(jq3W5qA`-oy(Z?SkU%gS;1VW z`Q}l@vw)W?|B(VPQd@s)hOS3Q3zL2P>!aXN!r6TNAA!FGC*9r+{7mOqUNwtR#qN66 zvYIX3hgdy*ed+)OGP@5b23~Y*L-ATj1iI(JVkE-i-h~@`G6+`Zf)w}ik%2>8Yhp8a zyPp)Rrk}e5(dpAJqm0L&IzvnMMU<;MzrL#L{4+&s+&9sY7>X7sdI0|_cBZm6pwsW$ zHo^!PYF>ibLE{twWND`cwdynzL5kUW-xPUN;Pc%{6sg|JROHjSZ2GfCUR}zsR^y!O zD@_Mpvzj-^chzm)rj&g(B6MRipn^TgZ`R zrPG9@8o&gxf9CIh`yHi0Jb>huH(Z5(X|XZ@CG z?_m{2IaaQGT>gCAqt1kzoh8RUsU@eG+-#1@T#uMa?<5Bx=ZZEe|1+l zv56qE$`k?M2EXDCS2Ugh;dYbb_1<5~+t0&4PL=B9gIYA8cHxQU$~3Y<8PKx&d3t)D z;}bC6CafhpxD;+iX?KH#H5B!$=rI!pPx@a|u%}@jYdku-w_B~j%^yN$yj#n^I58@1 zkfIl5fT+Jxq%on#F+~XdP@0~}o>sdWumk7MC=(OPC>8xJ_NaGW;h56wWbfK4oh|s? z8sxY#>32#_z2~|{uavXHrT63>k-^f1o3XDMzi>eP#7;ktx@3?S90ecN68<|Y!?3_Mwc?UQQ6UlYH5_f_ ztB9uoUe`;mC*|kE=@Y7b>%*9+2pJ(MzL{V`hQ(4}>yk18p0nGOx68*2H0H~R2e9ga zmEA>%dUw6pE;%@DgCpI8aytbk2@JI(F9~tbmp`lh45{^2!Q$Q{#!%_$1t`?$2k;F7 z{yY?1KLd5ivq^vk&)Y)rcfUM}!{hT3VTlroQb&*r{!Gjijsm*|L7c~k@E^>BGYuAP z-?nr7uZNY7pXcg{h1u)9q*e@XFZX2uJQsa&jz_;6h{}c_bV7bCa}3*^E-vjcStdRW zRmM@g3l6zu)qh|_Jk1%5`CV5+9X0dVKP{=6{H)LacvabtGkdX9-S8BKa`m#^MA~F- z$-Tcp>FyAUki=cmy3FbJSI)-Ue#WC%YodjD5IL|jHZzwkD-!B|)0%f?)`?|wpFBW} z&@OZJMlr-=mf$xxGVWI@N1^(aC{P*eKQf!O+9x?6vzaTBpYj@4=uK}=51nLuFS4+m z`HJ{qs}M2Doy!aC8`VhJ*t}q~*{2zIig0dTWenmueolBChY98`#9T&fpcYvM>AF;P&A$^ZWN zv&3dO2^BI<7g$dhHNVC2<%<`{&C*G`vF>PFS~Ljd2^jJ6oGr4|ISLuj5?)+UAgF{a zs+L%Vls+c+g;a1iOwDAAZEBh%MRoKCNVS+kDl?GKCk3(u0Z^>9=X{1c&%#>5Ia#Q?q;c&)}HQR^K2FjKW^@T90ItYVf~76x>%L(E&>hJ&5%S@OJ!l zRnj%GvGD^z;%fUj!8%dbJhq1qg9Qv~!NIB5{!B{AX|ZOpnWM>~(6^5oj0(LIq5rN` z7;y8xI#zN@f!_?Km z%QtF0K%Lz@1;|k!`Vc`_?sNEkltBT=5*IXbEanRYcjM&5<*+}2olXF)T5Sl{BNz0# zIL~B&tf!}UZTxHaF_Q_Im?de54HI}77R?xagyDL{tb4H|AzraMu6{NUDkAA3p zPE3uUo}2T+;D?g1JR+MM@G3+Kv8xybc}a61w;K-luwy zl4CibC^y>u_(EES{4H?=!J4;L=MyyN*RM#zMC`)!gYQ516p2ga#|#Yx4TA0dF9lIw zCsm=BI{e$}qW{ok2*zHPI3<26iC(WX+R&Eg*;If3umWjk|H(Gze{L(aNYfMuyNTvcr*;n&8$uLkF#tg7* zl@f8Q#LC-08hYx1val`LZk0Z;!yXnWJ%aqRv+g(xRAsLZ_w_61rCd4quO3p(v0_4! zQE5g2t1LY_J`t+jUeS(hf1K7kDV&eI3L_PU5#@%oPL`JIkWMc)DnEH0TFcQ{U2elx zs%nzTobZFaPRPe_3VhW$}fuxrL?k*_M9RuRj`8+2+2(qx&JdOLEi(0m-hn` zxTySK59+CggD}ge`aG7z-U#^}Rd(d5-5@SmAEG-^kgrL@s^>Lv_xih1B}qy6OW!(8 z0_x8r{`e>YdvZQ9LNkHn6I?!~w^y!oBg4JIff3r+I!cb-_{tb1@;QEB(JceoNQOI5 zKw-L3$bODz98LV~0hC+6uHO#d;x7myi*M9TsL6pvc&y+BX{E|DD3lHqByX#>2Rfoy zFyhw%x2FqW7!G3lluI3f(#;u2+dfuR)i7H2G=rk8=jDbi=f$U|?psqXdi|w;nrOrH zQnLxwU%pJ9R-{ybp$NJ6_U86Kj-_d^-LgTsd_2c0OumnYK8%Yq#?D9h;_B+Np)Quw zj&({Z6&aYv8om-W=H}IXkEu+|MhEe#X1(utC`>+MWt9){m$Fsp zDhLt`g^SXuDR)nwy0@aOz6e-C_Z9!xE6lEvt0>}NJf5KZ(| zk3T3lV1%j?y$4)Vt(w^E+8p{EMjU3;0reo8YZT3SZx=r?@t42p`Y|C%oVxOBLG|Ar*&DP^uFHn4HYEndh8zY*L{}{hBe#TQYL8+B* zewItcj`;W)LKp>Gr-T2?2P}A0Cs-+7Y=G#B=n>$D3Inwajm|t8ZOq(Qw-~6q_!k^P zra!+f9svf+Jxo;j%`fZb?**i@r0A_K_G$54Z6DHMQ1G2wMk7GnM)9NbX|a@i=N!f( zXl9!fO(uZwW7mPSQ^)Ye@ghiG-8HSI*;Twam@1`ID+`-*2B~%3YO{MF*_{lYp>Z#O zMN4Lb(j|DX{qfjEy5$aO2JO$Z7iE7Eh{W$OsR~JQnymGYqDJ491r~UXOCHbjG@XD7ZenX)Z0!WHo#m=-`d=q8XX<2G}O}idF&;j05s*{0Qz__ zRRY=F28iQ;^vukgr!ZlZ>87{yQ?&2O^|yN3{ai}RDyBB5-j~`qfF)v-LCQXcKq+8H z4kJTdrhDqju7EfShJ~TblrU$GN{7kBHWqCsdy_vjgkx0%wes z4ubA|rlk!3{>KJiz)Seh40JxPkp#XWhI|egswd37akP<<_nA2?Yp4BNPc(T6~J%ci&c?#p)m<$yT0>3a;*HZNrhAkHII+7N8 zhIakyc4^?j3I~G!sj6!+tchWZhN^Esyt|7YW1aUoVkl`hU|<;dB{3ltI+f`&D_C`D1(k^hFx7FD7hl zdBH7VI(6^C+Zr&~9uh|C?Bo2?dK%Dr&Dx~22+(>K{f~Uc??HVPQ(%$!wUpirS%>(5 z7&7RNU}SfcHDc#WOJq0eGux8ph13T~eF5q#&gqTG@d zO*mL4G!L>~Z)uM<{0fY!BBFTbat^NYcTqBrNsd(Nlv@=ye%PsjHVPyAeHi+7SRZ3i zor!6j^LXd>-V7e{Rw(h#Q@o&eoEZZ-Bef%SE_mt`xsmi3V|owKv|nb~_qAw?)6*=HTDkUc6} zxXYd)gh~>Toq0uEk}WHQjH6-XmywbAe@^3{*S)-6p8I;9=lgj+>v^8f`||ey={>1FpdjUSzV4M#1*q_A$7IIB0+PYMZEe|E|moE9?qG8DoX# z7%igk-*!;*kp;BB`v^x6nd8q&&iKF>h|gc1W<@;uw_Os(aP<86F+}3O?WV9KdgAkw zJcxV$wjYBfF%tX#z>J9ew_O32#7umCLKt!9pLTjnSQ0B}PnqOe1hV~?-Mp!=7_B9c zus<5OWpkjCY8yz>2%=J9+3g@wtH02ISJI*qLQDdzK3cT~Qls}b_b&E>^Qw*ce=JxFzDI7%$32I_#zI(9K9 zu*c*M&RVE7 zYP7=c4v0G-fcjnMLC(QfnhhXKWEK2H0EtAEcIvy3`m?q28X%$$bfj?}2)L=ka5*nXW zNGv!{y5qscb1W$8ya$3qdx4dn1Cf)o^WI2HYOz%-lluBEQZRmcTcB$QpUbASa=$Rh za@=qQi9LpMUU(@*g1v@7E6?X z7)r=Wvl;t7oG1}MVGy-%#9j5;w{D`~SeKkrls@oUp)pvi|H7rKGIU;>6x zs|YG^tod60782A+fn_IZxvAq!Es4B!8(3dO3i{y)FdqoKSkhMWI4XqJlt8Qj9yOq= zqf_Lg0@jk>n3Ono3Meoj^y+4J0aerKi&6t@Ufd80+E|PPiQnu&=+jicd4ja}LczVO zpqv3a0L{dDF2Aw7q}~r4*C)PN%1Z!DvoT@=O8T8LWsUdh8I5a`87STdwB>`P5LZcT zZP6U=ssa?`sG<7lTideiTj>Qrp|kiMA3(&h7~TgD^3TWs#dx$Cicw#oRJzc(b3!o| zc@)!4q)94Z8_|Vg55P8ElsJt>-5O+5a`BO=d@KUS2x?nA2bfeID3c2CYEqBRvJo|a zH?U(gY%UuDz(UADwd2fVksj8>m-c1^!-Q>30aZ$$O9FqH^0wX-UZ)uWSy;1{Ihc@AGDZOWQwfQGR1^>uLlVl&SjZrh8KOjy z3&k9R_(ED?I;06)r*5Zs`_2`+^za*=#Ks(rulIZWbt8yBwCMAR;x`Z;l7!%g3WMrx z`~(Dr)G!P5D^pBNK)SQnl{Lt!xDOPP3)<>9T!qy)ZpR z33#%M(pvW>Gk7l%EF;o<)Y=-vf4!hqbY%e5M=gjGRnvQ{LZmaj7)+?Nyu7|n7k9yx z_n937D29{_<~6b+nHR)Bq}d|Fij|Ev1mx(OM&8GwNQ>(^eQGI`9;)jQ`x&}tGsi^a z`TGFh^A}N5%q;~|j8->%3+UK8rj-2*Fsun>rWCXW)~|Y!wKp}jUqzU#osH%3mC&=P zRMbW?YP_pJ+ewLQBwWEO=(F?CQQ6xY(X&N2X0m^~6R!zEG;3Nk_R=0EN|BO&phS~n z$}rD+W2^7I9gZ!@lZypB0zVJ9fGq6F`#hp_q#587p#2=`?PN6l!wLguMaX-NjlGK; z^_cevbTMv|h>dIoXwn&0r7!G!LZmvnu*G2#*HNaO znfhtWWCK>py7DmWQ1hV9Bc``UYq?eHo5Hcuz7D)A#0`3jmOU~BB}X}`GiW|dR3gKS zfbR%4k{iN+8^Ei7ZEKsY=@fyJwoPv@l6za^4)Oi%PN$CAv=5vjq5WD<18^-i7kjL#{vWKi^CwOA=i1q*x_)6 zWR{6(^-+yN4|IhL%|*3A3@FjJSdPc6Q@{mng{&UdW55m|MqPnkHu$7_%<-(OdEbMd z9WNHe1-BL*U#^=9}lqucE*v*>d4Ufea z4FF4oOT>)P<7e^(_nw$Zyhq5x#msFf1w|S>c$#^=kgSzxsH{Die7)ye0)T2k(Zc9_ z(wlxDNFe`Nofh5(F3t={`k&RML?qCio$wyy#*)2{r?e-IK=z4-GwLO`ic(K$UAcmbGfxoFvxhg(@ z;Epf01RST8foZotJHI-EHF`4hV#_vwJ*W67Yt(em4<9o|xg;ts-)5d38R2Hu zxuC)jBMq}%Rk_x~p1^r3>BJHZ`n-?6QT36`c>|gk=A*6d@D!=JFftk_9p?&p`Q#37 z82mz_#=8kGd%J7GQP8$Eb#3V&Ws@fm2$S7Kt#hUXgs9vw8+Kl3h6RvnpaGwqb!37B zMed(qV!A%|W1(oe+R1tM#?2)ObC_oIRlCcytk)bl!b!5?Bi#6EOwEbr;@2ls8?XB6 z97&_!n4^+s+p&n_XViTpUnd_|s`Ne{+S`F=!K@ajRj2-f2p#P_01aZ)v*0}?_V%lb z@iP42STIu55Jrq=n@a?zTzFrMTz%D&5K2NIzmI4De|>V@2UPN|+pFSy#8MB|*p3&A zV`Z8kJ3r2C$~2=@(U;1KxG6`AjYoWm>gwpDn4#`os$6&7K4E)RTQH535GR2Fm-Pn+ zTPp%Vb2KxX6IdEfOj=s;R#@K|2%Twf7D<4rnO9+zL^K__-!d$88=5s8cBhbvD&hHO z8v%k9hkl6k@4mqZppe*Bty^>o0M&ecxrFgs#+>wslQ4`3+05EFWRr&PY*5Rb-3(kR z!++@=W-1RSCz%eT14n+qd3>pCi2LsYv~i8f8;HusN~j5BE~P3c&6{RYrO1(}=!qX} zkpOz?eIBWBg=P2-?=1)lgzXNS;~yQE-pB*knEnE4?}799RH4T#p#==&dblXJ1$Tn3 z-t`M){m+;-J*JZypmULf_P~^l78mjU=@oj9Nw%-VgxssbCVtz*ptkfGbNP<};J$o% zzqVf1JivI`D`%s4QUUvO{nc20<#hq?H#GR(QzRY}taiaOkd+@pc()3u{(YGLB0x7T2ua`|dEm?1Uvf`T+AiMBmIolf@!c}k88g%}@POoR zT@q5~JnX{$dy(PYxG1id5VlT({{-cl>r)H9OOYG8pSs@D1rt^Eh+l8#B{6vl(Yt4r z@#V25w)voTJNp6G%=2~(Kwa41Aw5{7b8CDc5GkPyZrB{}`LFN=LE6Zh&gkM^?n&em zfUiwtHsA+@V=P6KIjo)DY1!`%H%pXv${b8AkLUaxF*VaA{b7+HIXig@@`)4G;t-Y~V?L;z)B2jR4NQ`74379<5 zXEB#2s`g#g?VE#Pz31xQJd->1t-6=a46*^Tp`BeC5_l!&J9;gOT){RWzup7<#$~JD zsQTH3$Kuu&3XLXmVx&-kF|kM!tCMqcQs?})CXrHBN@BKZKjBKd->x#W_tkmTm!>pk z{0FbTW{I@02AO)7DoB`pnai2+*7XOrI35)TNR4-m&U^8DvXP^qV%{E}(N7<4>QnYGm)|sewzh>n?dNkF(P87+TkMt<{GtNulwO znK7rb=T+X9@7Y?Zs!fM_HgiW`>jMaB$+;j(bMvqeg@8Yth+Scj?elm~;q~8CqSmw~ zz?evRan)Qp_#UP{*9LjJ{HZe|*<+xEF`n7qe~Pkkopxo?AIZ%Vv9oF2o!5D_flI*+ zsYvK5g6(cXnd_O$g13SUzT0;sUv&e8UNctCtY1QZ>6sk^SK}UslNnN28BiPkZkhRE z$d1J<&dY9Q=A2t7HLJx4(#mq7cx7`!=&Lq^zJ#phSUGHyf{Gd|te;%%L?;bPp!?}> ziF#ANG<^3rKgD3EDHYtEL(c}vs|*CqTe}l>2DCFM@;e*|5}{W53(>k^m*Y(0>c8el z1h3VY4)R}AGs`KLk(#+h>X6H7PG!C`Jm^*H^W3g)q9uLKK@i%ICWNYYhhA3cWXuMj z!C3E%E-lp%yUuVVK;cL+*chp8IwdH0Z%ta<&A8^1;q6Pfr>)8QOHBY*Tm-tl$0}9n zFmWbn;pYQ9l>U{AuweSH%l9a)(&lDMB|5M4^YNY8-?}~N+2E+>m4?-W&V>Ntz1SK) z+U_6maR~(=o;3GqRyGr#bZM6JEV&j>SVoe>@!^+M&wZCr`kDQpVmJ_#^vg})!w31u z5$qf%-FoX-xr4QoGevQ#rpzLYldK~h&9SqL zrJZLw?I)Q5FcTqyD4UIq0rX!3)S`_qbprsIo(LnAW6bS_Dc-eZxn)+Ctfo94TnAf& z-o6{!-23h_RGPc{gZi^XP}HN@bK+JGPCbwtH>J8o?X-Np9t7wibG)+mkw z%m69-GUlL+i*D%snxNVnPcvZ@NBT_DZs~K?y-G*1g3N;24Duf55JwRA(LAVs`|*qU z25`19f|a6j*jdf-%U1N``ET~M95`|e#Fv{O^GJ5)pXLL{SP-~>URxz1e?$#x&LF${ zFj)Fp=n6Eo8?o7yj{In9CsAy32ia3@BQ*+XLKIFwBE`xwm|guI^MS%mN(JIlpgI(5 z^wYduNF*QlB1cm7?f3hMODWot$DMbJ);84MitS$$OIzT_tN3N93^Tsn+ts9l3S6)l zF@72=wLGUw(7UcT(DhtCw9M;+WStAO*GtAqy9*kX)N>!#Zdz}Hl*t(rw{0>56B5`z zV>Y_FS~K+@(H&|erV>E z>xp71Z(*;zRIghP1()+a&|P>axae#Roj-x9;Gq&iJ;bRX{%R)m2+GX&I_LcwhbibT zOrCJdc-&(-PVv_#;x`n=>R6y^nH`zH3E}CZ1JPd&mN*~1C$D)JYdCqG-}kyyvi@{T z;aWI13;xmdh=!zFs5knkec+t`GDNsIb=(swQ?lsS`xUS19JnlO!AR+z=@orY{OoV6>mTd)hSKzbvIHvJspWQCV%gN$-JY%+j%>oLxiOzhepEEq z4ZbFOa*UVu+qh#v3C&ZX`eJ*NFQvt)L;v`M5g5x++2s3Kly4q=Gkts;AV=}s{*a0$ z*B4Z2(sha>=I`G<^0EPXAC!qtAK&;MCexHkr~^pZnT5i#X1U2F5;};O7_!IWY1Wy5 z{u}3VBkKd#4SUPi{I-_Yle&TE!3Q|=){@NrA?-)4+$)+l7d|}4ZI4(BoO0cK<$Zjm z_mujR3yycf4~i~5?vf8ukac17CPV^7g`c})9s0cPG<4s5qQ3s%q+e-z=mv0kjx1P> z*0*nCNNcAD(8eE(KwqcaKdkNxYJVXq)jKahK)moZ?EAacNh0W+={(X5_lc;T6ok0l zgp39H=Yv6hE2E#A)-yFjn1=6ll7&AE&UoY3?$|tseW)Vs=&O&IuX4nnXR|DmvY`Ty z75vgf9qEU!N!%mT0?Xq4RYF9^lBm?d9(Gw{L*kD=JVOPe;mm#nuwj=+?Rc{Y##)1; zK5wGBB9}hC7+Km+n> z6QR`Hb~SEA(ZwZdwA#EyQ9ju46}UC|&2~|kK+h)^0syQ_@PbAwFw~zdQGo&RunKLu z?!2xYkpDnXh=t@D9F#ARB8)#5ki9AFJ)|$hhB;f|KjlJcF)_Q-#))ZX%8I<&GyZ@3 zE`$=Ord>4qv(>;m0!N|OoQ$?L-9RmIoWQfw8jF9mm2s=~`)RVc5l&CY*}Z<11d!BZ zWq%8Qy`5z=&szP_*|#m}kNP9gyM8l622%B~oyLPDJG|nkk|u5{^fYa6P9mwLWt)+8 zy<|CIDrC{%?`V5nD@aQ-q<>v@7M?u8%HKI4jq7`%ha+L)&@1UXb5f z!3r6M?3-I|P|L=IU@Sb%5=$~*K<)0c2Q{Wo9;s81Q8X{F0_A_2cW4XEaKKYODbl;}w zd=}yE;|`Vv4L;B1-4Z$MDUGR2E{HgfzN5NxUQfb?#3DL{Le!N*B*PO9Fw3$lYIJ^P zexj0@$CNgcxKsXk;~siAvb#0o;701#z9!}d43spoLa${j7EM=P2`GGdOBw9^GEQS` zoFhZl_4P`}BP$L$$l_uB(#Ni!XF%}MWjv>cR9a)0x_&{@lD1u-?$ z@Vke|d5H@QD;xf`Z2TvN;s{;rl{ARGsp6pRO5c!IK?F+`VT96GE(gyK7%|MMj^!&| zT$Gd4`}jXzpgJ=M5mIc=xg{Ufyen-i%41fnjo$JE`v`~7!!tSJqOxrnquCK|8%j&A z|JQ(ec&-7Q91qCPbMc3=*)s$*YJPEue5`rz_9{lMHV-L?DrvJY|Ep?aAdvON4;GC8 z;2ZO5(+WzN2BAxy3=jY{_q@5$J!Rj5C`Y>}7H<9-0LoC5j0p?^g-n*Ky1hRpa1=>c z95ArU!^v5tTPrv?a4^}XAAjRu{%BTk{LiO6_8==Wru(UZcF0q6GqL;luJkFTs@)qm zDrF_Q*|{i0C`Z+upMjf)KjLWW9?G`JX3^mI*LQxw>L1#qVaAF>LlQu=3C*dGn8qt_4C^(cb!BdNMEWja zT!y@7KHp<_l#IEzwfsg4{{fLec@akzo;Ut`OHSOC>r!T3Vmz%RMtSHwOjR3o>|9s6 zIjplY-FAMP%F+^ln-rhG(ed#mOLTShDjbuANn!1IO%#Er{}%yzXfs9T8P1cf$vu~T zs2twqyof)Z!=QfIc=44(9`O>R7vin^?PKZ=ua8Q;+78&J=VJv^lU*h-I zNzlqQhF$Z3+*5FmfhsHGX7cYQj~3FUzB^5qQ*MLZ-qz)$u#Rk0MQ^yygwq+;_=r^j zJj!Zp?6!dw;tM2zM3y!(NEY-dT)O%HKJi|#FE5@y-*&qpFWuFFckvkKYtUnB^y)G@J{Yt5czTG+_&nt( zAJ+85UxDPON5QTg!@Ps2oOH&NKaJf~R(8%vy{#!_oi{@lrjA+~DH_d>{d^VRUe$G} zlGEU_d&uQ+brfnfW^Em-84Zal?0QEge**g|7}~Tp{fZOVtlh&cwNG9XQ6G9Py}x{^ z_3|71PQtNLvPxF`SBw$+M`f}RAcE*dQ|mtpP#w;H3GYss%Yymx{JC>RTEvK)oKjN} z1J literal 0 HcmV?d00001 diff --git a/doc/design/images/l1_regularization.png b/doc/design/images/l1_regularization.png new file mode 100644 index 0000000000000000000000000000000000000000..e1b9c7a44f94dc027598a98da93ddb8133190972 GIT binary patch literal 1157 zcmeAS@N?(olHy`uVBq!ia0vp^JAqh(1xPSB@w`k2QY`6?zK#qG8~eHcB(ehe3dtTp zz6=aiY77hwEes65fIJ>L8=O=iFto34NS4sQC8zgol zZL&q_W0k+f-{!`DR$|?;@O;v7%l{?+?iUxb6zBy1C_lK~h<&gAiZ4q}I=9=Ww|C$B z(z)}O{!#0mxZ`X*r!O43Xw?D_hwvgcZnEQm{_3^8 zhsuuzp3b>BJ4`C;HZe`x{=39}X4vA`!`&H1&iM^4_8hGSGw)x`^IczF^Oskzq-f@g zvX4(9mQ*Q~2Ujlb{TyPy!%Wck!lzlcWq&W*$CWr88)?-|J0a9T$QQfnrF0kyVKW-cQ5QFZ_(%SKIZd`&t;Njbd;IBmqF;N^uXzJ zl-^Cy=c;i3Z8URtX|(-agGE(3<;B)l-u7Ht{UT%I&ilJ&`-ZEWc+&S};)`WBW_TR) zI(SBL)$uItOYgT`*yzDm?el+DPk3pdYNXnwWhT$b)S{?BioJpb$ZHDR?|+q6ek`!1gBe<%65)YoH=@UI-- z(}A&JkAB5Z?{VC?~8j6=l)#A9zPQvH14cg+9uCt|)UO_QmvAUQh^kM zk%6JPuAzahp?Qd*k(H5!m4S(_fvJ^&!QvgCPorqa%}>cptHiD0(o^qppavz74F$zk z9+^R@#ZLL9c`2EB=}!3-42H(W6-JiYMnG(4XjEdnGaRVe9IDzUwJbGsk#1QCO1w%bkL$378RboIRT%kq;7vz^X z=jY@X=^8NL)`>+XxH2~>KL==Ri0?Tkpn--M1{$In7?he`nv+snEgnpd2ep9j=@ UKlKVQmoqSUy85}Sb4q9e06^~6>i_@% literal 0 HcmV?d00001 diff --git a/doc/design/images/l2_regularization.png b/doc/design/images/l2_regularization.png new file mode 100644 index 0000000000000000000000000000000000000000..d5c2fcbc2ccae75ad083162e5a2dceb0210be298 GIT binary patch literal 989 zcmeAS@N?(olHy`uVBq!ia0vp^RY0uG0wfs11@y9k6id3JuOkD)#(wTUiL5}rLb6AY zFHoHt14Ba#1H&(%P{RubhEf9thF1v;3|2E37{m+a>rPdj=S$Y3w=^mS!_$R)@l&6x7ZV+&9dzo(01h{y4_Q|)=zksHMv-tGIFs>q;;7T_puq7aBJ%baLVf@sTyCX+9wCvCnbMwna{IL7}haC>*@R z8anY-o_M3Qxy84`Kl!Kjo;#^2_b~F@zTNZhecxLxFY72WF;TkewF=W((RnTl8Xwu# zwr2##2Ay|bR_fGoW~ROJX{VW=zc%MxzL>e-@`sD{cmDj6zOy#7db7GRb1QF`{Zx&D0x-jV@gGQj>Ev*9!uWR;bt}%Mnds%8~ z6~|h2jj~zKwA;5i8~=)*#;aYysGjatpL+ayc?~3k0Kkqi6n)eKYMz*j4Ld^>5V?yx)#z*shgRkDV32eO~6=Q`sLb>o4@a zu{~3_`Q8)VU5`~h&cAoDOr5EOYr4_BsxMEIr$4*a8~1DBH$CYE#{(y>yL5v2uI8Ja z-^GHq8>+puv~~NMrg7!tjSC-bTld_s@rF_UrLQ*vB0rntcsAtOA9}H6YPQ<~_7<^<Y-?eQ=H?$UF9LeQ+{b) zN@iZVQ+@@5g++z2p|+8!p|+W!;j^rZvw@1up^AM{%TjX~98>a>Qr+_NN^}kN46GC! zi&D#i6Z497{gZMs3rkZKf>Lu*6N^(74D^f)6Sy)39qU#@G2=t}`7MbA6+@$;*pk#>eIVYfj44$rjF6*2UngC+| BgDC(2 literal 0 HcmV?d00001 diff --git a/doc/design/images/loss_equation.png b/doc/design/images/loss_equation.png new file mode 100644 index 0000000000000000000000000000000000000000..14212ec8d36c803de96bde8a9a4b5591bd20434e GIT binary patch literal 1589 zcmZ{kdpOg39LIl!94k~pQPPD|*xGMx%tAIcTQV$N6tOb1usdz2(5VqBbxJadP9fuw zOV1hV6qY%pJj*SYM`Q~*b2&s8&hMP3f6jTH^T+r5{(QdQ_xttv{`F1uqk0*_EMNct z7*fcdGyquU1J&Dfp$(WE8k0bx%f$QO0pQVArCmq&6PdR;At51 zN@SCI_vKmi;`qFQAg=V0GjrHxDXzaUD>fdTYEt2IP6o0@u~xV;s4sfH{y%$VK=?V zsaQ|xD0ZLrSmCQj6qwna*UflTbE3Th?ST(pM+^QrsR%b%U;$X4s3-jvVZ z+1yqBTt;DG1;}PrZL+rqEWnPXlc_q5T*Is-xJtC7z%O>0I-$FL8hdX0X(=W6+-=ka zx@fsVeo?x9$rY@)G>J=7DH;&mMQK zYmR793m(!B9=55un>i$WUvT@nOQQ2s_nJ0a_j&5?$VoE(_K<8kr82l=uV2;q#}=t) zB?iiiGw!8`tu{}#MkZ{z(JTuPjj3uv zci(UG`OwPTWqK0UX`yJgNEN6w`7+d1*l6FX*)6#_AHX09_rl6MRpLq7V}HA1&rQiz zlo&G6`O0PGA!^Ve^xBw|R|9TpF0yk=et02H7c43pwmhE^d6E~HI^-EFvAtJg)Uw4q zK2}*&koj;t)Y{P2Te&G^9M_jqG+)gNKDHxAF3e4j9KmJB`^u`*(E+(Gd54B^ffgOj zAKQi6!9ef+($N>Tl)gO{Fjmd!$)htxUTY#9Vzq;$Pe!rJYnuA|Z@YOBhO7=(9-DDx zKYfw>341ExO3TGRMUL@BH)v?yuCfu?Nc>b#22rc%u75IbRi)py)Vg5&F-|d9#d>}N z`iX3W#3RBmx{!f8$!9ms$a6|y;h!lZN_9zfz?}$Rz1-B
+ +The parameter `alpha` is a hyperparameter that weights the relative contribution of the norm penalty term, `omega`, relative to the standard objective function `J`. + +The most commonly used norm penalties are the L2 norm penalty and the L1 norm penalty. These are given as follows: + +##### L2 Regularization: +
+ +##### L1 Regularization +
+ +A much more detailed mathematical background of reguilarization can be found [here](http://www.deeplearningbook.org/contents/regularization.html). + + +## How to do Regularization in PaddlePaddle + +On surveying existing frameworks like Tensorflow, PyTorch, Caffe, etc, it can be seen that there are 2 common approaches of doing regularization: + +1. Making regularization a part of the optimizer using an attribute like `weight_decay` that is used to control the scale of the L2 Penalty. This approach is used in PyTorch as follows: + ```python + opt = torch.optim.SGD(params, lr=0.2, weight_decay=0.2) + ``` + At every optimization step, this code will add the gradient of the L2 Norm of the params to the gradient of the params with respect to the loss function. This can seen in the following code snippet: + ```python + if weight_decay != 0: + d_p.add_(weight_decay, p.data) + ``` + This is a very restyrictive way of doing regularization and does not give the users enough flexibility. + + **Advantages**: + - It is easy to implement for us. + - Faster execution of backward. However, it can be done manually by advanced users too. + + **Disadvantages**: + - Not flexible for other regularizations such as L1/L0 regularization. + - Does not allow for different regularization coefficient for different parameters. For example, in most models, ony the weight matrices are regularized and the bias vectors are unregularized. + - Tightly coupled optimizer and regularization implementation. + + +2. Adding regularization ops to the graph through Python API. This approach is used by Tensorflow and Caffe. Using this approach, we manually add regularization ops to the graph and then add the regularization loss to the final loss function before sending them to the optimizer. + + **Advantages**: + - Allows for greater flexibility to the users of Paddle. Using this approach, the users can put different regularization to different parameters and also choose parameters that are not a part of regularization. + - Makes it easy for the users to customize and extend the framework. + + **Disadvantages**: + - Implementation requires comprehensive design and time. + +## Proposal for Regularization in PaddlePaddle + +### Low-Level implementation + +In the new design, we propose to create new operations for regularization. For now, we can add 2 ops thgat correspond to the most frequently used regularizations: +- L2_regularization_op +- L1_regularization_op + +These ops can be like any other ops with their own CPU/GPU implementations either using Eigen or separate Cpu and GPU kernels. As the initial implementation, we can implement their kernels using Eigen following the abstraction pattern implemented for [Activation Ops](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/accuracy_op.h). This abstraction pattern can make it very easy to implement new regularization schemes. other than L1 and L2 norm penalties. + +The idea of building ops for regularization is in sync with the refactored Paddle philosophy of using operators to represent any computation unit. The way these ops will be added to the computation graph, will be decided by the [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) in Python API. + +### Computation Graph + +Below is an example of a really simple feed forward neural network. + +
+ +The Python API will modify this computation graph to add regularization operators. The modified computation graph will look as follows: + +
+    +### Python API implementation for Regularization + +Using the low level ops, `L2_regularization_op` and `L1_regularization_op`, any user can add regularization to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support regularization. An example of such an API can be seen in [Keras](https://keras.io/regularizers/). As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since regularization is a property of parameters, it makes sense to create these in the layer functions. + +#### Creation of Regularization ops +There are two possibilities for creating the regularization ops: +1. We create these ops immediately while building the computation graph. +2. We add these ops in a lazy manner, just before the backward, similar to the way the optimization ops are added. + +The proposal is to add these ops in a lazy manner just before the backward pass. + +#### Storage of Regularization attributes + +Since we want to create the regularization ops in a lazy manner, the regularization attributes (type of regularization and weight of regularization penalty) can be stored as attributes of the [`Parameter`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/framework.py#L421) class. This is because regularization is a property of the parameters and storing regularization properties with Parameters also allows for shared parameters. + +#### High-level API + +In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) to create neural network layers. Hence, we lso need to provide regularization functionality in layer functions. The design of these APIs can be postponed for later right now. A good reference for these APIs can be found in [Keras](https://keras.io/regularizers/) and also by looking at Tensorflow in [`tf.contrib.layers`](https://www.tensorflow.org/api_guides/python/contrib.layers). + + + + + + -- GitLab