From a381d625661d030c7b08d48445c12a7890839b50 Mon Sep 17 00:00:00 2001 From: zhangbo9674 <82555433+zhangbo9674@users.noreply.github.com> Date: Thu, 30 Dec 2021 10:34:40 +0800 Subject: [PATCH] Add train_amp_python (#5449) * add train_amp_python * refine picture * add mv3 amp train doc * refine doc * refine code * refine * refine --- .../Step6/docs/train_amp_infer_python.md | 93 ++++++ tutorials/tipc/images/train_amp_guide.png | Bin 0 -> 64515 bytes .../train_amp_infer_python.md | 292 +++++++++++++++++- 3 files changed, 382 insertions(+), 3 deletions(-) create mode 100644 tutorials/tipc/images/train_amp_guide.png diff --git a/tutorials/mobilenetv3_prod/Step6/docs/train_amp_infer_python.md b/tutorials/mobilenetv3_prod/Step6/docs/train_amp_infer_python.md index e69de29b..31bd4dd4 100644 --- a/tutorials/mobilenetv3_prod/Step6/docs/train_amp_infer_python.md +++ b/tutorials/mobilenetv3_prod/Step6/docs/train_amp_infer_python.md @@ -0,0 +1,93 @@ +# MobileNetV3 + +## 目录 + + +- [1. 简介](#1) +- [2. 混合精度训练](#2) + - [2.1 检查环境](#2.1) + - [2.2 混合精度O1模式训练](#2.2) + - [2.3 混合精度O2模式训练](#2.3) +- [3. FAQ](#3) + + + + +## 1. 简介 + +Paddle 混合精度训练(Auto Mixed Precision, AMP)是指在训练过程中同时使用单精度(FP32)和半精度(FP16),基于NVIDIA GPU提供的Tensor Cores技术,混合精度训练使用FP16和FP32即可达到与使用纯FP32训练相同的准确率,并可加速模型的训练速度。 + +本文档主要基于Paddle的MobileNetV3模型混合精度训练。 + +更多关于Paddle AMP的介绍,可以参考[Paddle AMP官网教程](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/01_paddle2.0_introduction/basic_concept/amp_cn.html)。 + + + + +## 2. 混合精度训练 + + + +### 2.1 检查环境 + +混合精度训练的加速依赖于NVIDIA显卡的Tensor Core,目前Tensor Core仅支持Compute Capability在7.0及其以上的显卡,因此在开发混合精度训练之前,首先检查显卡是否支持混合精度训练,检查方法如下: + +- 进入python环境,执行如下命令: + +``` +>>> import paddle +>>> paddle.device.cuda.get_device_capability() +``` + +以Tesla V100显卡为例,执行上述命令后将打印出如下形式的结果: + +``` +(7, 0) +``` + +结果显示该显卡Compute Capability为7.0,因此符合混合精度训练所要求的环境。 + +此外,若不预先执行上述检查,混合精度训练依旧可以执行,但无法达到性能提升的效果,且在代码执行混合精度训练前会打印UserWarning,以Tesla K40显卡为例: + +``` +UserWarning: AMP only support NVIDIA GPU with Compute Capability 7.0 or higher, current GPU is: Tesla K40m, with Compute Capability: 3.5. +``` + + + +### 2.2 混合精度O1模式训练 + +使用如下命令开启单机单卡混合精度O1训练: + +```bash +python3 train.py --data-path=./ILSVRC2012 --lr=0.1 --batch-size=256 --amp_level=O1 +``` + +部分训练日志如下: + +``` +[Epoch 1, iter: 0] top1: 0.06250, top5: 0.12500, lr: 0.10000, loss: 6.64041, avg_reader_cost: 0.73829 sec, avg_batch_cost: 0.86344 sec, avg_samples: 16.0, avg_ips: 18.53046 images/sec. +[Epoch 2, iter: 0] top1: 0.12500, top5: 0.12500, lr: 0.10000, loss: 6.11681, avg_reader_cost: 0.73225 sec, avg_batch_cost: 0.90348 sec, avg_samples: 16.0, avg_ips: 17.70925 images/sec. +``` + + + +### 2.3 混合精度O2模式训练 + +使用如下命令开启单机单卡混合精度O2训练: + +```bash +python3 train.py --data-path=./ILSVRC2012 --lr=0.1 --batch-size=256 --amp_level=O2 +``` + +部分训练日志如下: + +``` +[Epoch 1, iter: 0] top1: 0.06250, top5: 0.18750, lr: 0.10000, loss: 6.73047, avg_reader_cost: 0.81649 sec, avg_batch_cost: 0.92645 sec, avg_samples: 16.0, avg_ips: 17.27027 images/sec. +[Epoch 2, iter: 0] top1: 0.37500, top5: 0.50000, lr: 0.10000, loss: 6.39062, avg_reader_cost: 0.71931 sec, avg_batch_cost: 0.83710 sec, avg_samples: 16.0, avg_ips: 19.11364 images/sec. +``` + + + + +## 3. FAQ diff --git a/tutorials/tipc/images/train_amp_guide.png b/tutorials/tipc/images/train_amp_guide.png new file mode 100644 index 0000000000000000000000000000000000000000..ef0a0aadabf3a5ba3497e2f6e76678b51b3cada4 GIT binary patch literal 64515 zcmd42XFyZkwk{lsf*>FW(uq<8snStOY;*xZ0jUu|X#yf00#T46Rk|QWMWhpYhtNAj zYNYqx5^5mr#&_>`@3YT7_xsL!@A+|On8`|3vKC{GImdX$GsYzTBF+OYX{l+d0m#Sz z01eUuKwJPkRrR)i2>|Hm0E7Vm05yP|j0r$NIwB)I0HO8(%0G?)fa|1J0D$5P*+0*I zA^-C%h5Z+bKaa^~et(hJ3}B=to1w6$Amas)Gm=p-k`dbhK+<*3k^Rwrf9ZE4Bd4G| zcb|H0r-Mnm-!t z??(5JMkHN?jC2jskI$3-=q^xQ_{X{b_5pE*G`Zx7lYom9WTeSN!3Y2W@VIZ`LV&;4 z?`zGAG54oR1yr+_%(#iclHA*^M7D4 zEzTtS*#{1xo|o2d0oW`*&^Theu>XkXGfs%jWyRsx+Cx?}GK$Z#ZJIK0ux=KAbIxFWhMR z{(Q&!WV#@*EbStzn7|FMq;LN99rL?9Qs~=N0jRNE>9Z=4oFRe$!~(U%XWt}}K~O;8 z{I(bcCYuyxzVNqga@3|O7p%TEcgtLG@Y7PLog382XoQ4{74euR@Deo6B3i(eJn1s1 z4e(`AP)V_XD~H2P$rCn1{Scztf# zrtmGpW-Z@vo=LZXfx^bHi}hiqNLA1Q5g@%kD}tI$yV1vz2-aC5sBeg|=#FpF{^G5x zNPHP+W5O3FM_(K*Rmk@GE?1y}tYaGT$o&PoxLth~IwaN8I`>=n*MH8=$C_+1fSeqb z1rRIT4ipMMUo^~c$-Hs3^mToaj2}R-G4mE#l@0Ys&_TDMw?YqFF~xBT{u}3l@_S1r zRA{2tlGyX>e(tVyVk%RLI@3!{)VwlE()4ebgHGS@)?}P*nT!?x7_D@=+h1LclxXr< zR@Qv(R^eKyP6J>$4dwC<)fR*rwq^^@1s#FKIz>a*MCu14;&{N)ai$eu=GYQ{>Z+Tid|$4pH&-taJz7P=nTyzz-+tm zG~>|tfMC^W(j?{V;n+8%DAA1Rok+S~(6Blf($)pP;{HLkGX>4iaczW>HeGYiy89@4 zStDm7{KVyaz^_a;CDA^H$Q@CL?ep)>Nz5G1U%njiJ(Jr7tk++P#~B`|@{F0daI7iZ z$Qv%6i}wPDJ?bx&N#7XAECXI>x`hK_26B{X7Slv(%?G}HzERTfs^DT zOLC!d>4jZaG8ggej~tSO^pbh2X+4~3R=&h_4t*Om0 zn&Q$|lb4oM22fo8@i_8vLVDa~dyz67?Z+b(e(Xoerkk9KiguGG#p2x7;WGM3FFmh4 zA?x9vH*U^kgga~K^iS%w__KS!ML!Kz=IsUP(ceDRrz_W`a{F|;5jy}GK*N?S8__2w zHghVO^NyO*Irb?+NUUC;ho|&`GoNIqw|;bv=|+D zj2>B!}oh_YO2$(Xe>ts@nyX=OboeygDdcMkWq*tatYoHj!A_# z&*;)oS)@Ad0W5m2JpENsW(T`Q1XwV*xzwldu}`J<4c-rco6MSSUimpY@i6(yiDY4^ zHYb%P-)ow(GF0zfO2&_>P_5T+W-GTsiRv#q0%-Am>~y_!PGrsiH^&UC-M8>Fx1oQ! zrSHl@WcCdAb=J6y5g@Jrsh)X7Z;155J8r~>BZavA1X?O!C7&fs6k2#2On{d8%Vjc7pWhd zS2m?fE-~K5?cPJZ)?94TKm)t>A_4?rlQkVdG$!#0Ph(wNJkJ!nmA+`Ktvrl*Y^g(6 z7_zx!I4T%N1UzoE7MR&yZYaNXB6v;xPF660KGCHJdf62gbF0(VWVm-_+itdZ?()Ga zJP~jewMG_{!@k5?k&E4KWIat1_keNFX}oMg=Y*wCB{4cMnRQ|K&7RPhS-5qLGhL`4 z0yw-saJe7AW>=MnfLggDP@>T{xQMxRhH%E%^ED`1dfQUxt?SA$V>L#9lfZ-j+9d4l z6bre_k`rF|?sUTNmCM_`z?9$;#a9OYj$1CFnJhR?wD{O})X|8*GJrE(W!*b5bPFJqj9JU-hE^}7DK(B$Jd@+YcS9aVybak|?$Q}cVl#xZ)43Q|j^twqu?t-@59pVO{cO^?l+9h^*_GvjtyxQniCC_M!yFJwAhiJ((D7nXYFQ zu_$TOf^4f*I+FXX)>&s8C@<3G-vqExlyK2Vc)UbQgg0m|*3uv3Z&qKwzISZL)<#1F zG+W=*2Cwdgo9mg~UgS9Xf&bO?wD<$xS^4|2uotH4g|9#ym-OgyN~p%Y z$Ohxr^B{NeUmKL?T|&`nO$_i>#24@&B z%ka?raZ7D7!|j{>i&?^@tJ?X-m6^87J}J5$A^lHQg1z)T;iu;$mo)Xi#op%&Ec|YB z$v5aIXP74q+1pW5|HF6cd&(hvyK%{W2@7o^oX`=!f*{6tr%Q7RjUH9p~e$rIwZI|By=SFOS*MaY*^^D~gezjc| z8H`_ndyJq!pOsX5uLTa6Z-}UUTRlAjsxljI=d}C@A*g+gxtI3X?V$`1&O*$HsY3bC z_DcOnatcrq{V$}Fvu7(TkGGx%V|KiUzL-RyK&=H{eFI0uIqH8-+~ltq8J)hJiFM!w z1^cicTOV;XRY{yaw!N;%xGkgwvkCB>apahb@MzTTvb+J6H$|>t4SR02Yj}rKBCcJ# z!_44p4=oa;cz1{cb!WpCBQ3?fQ-@)Ovj$$aE3(2`Z{8$7`WBAaS~Ttghq*yCk{4WB z@WQi1z@u{DhIIeeo{vIaG^P*2Wn!*9tzD^MTk^_|eN7OND-eF_C~oLERi-`gdEusq zL&RC_86ASIID+VKhS@kx)P_sddYN;^)z{+N zPh%?X5WbEGK4b+Z_J%>Vf0?Ois$QDYt|?#HxYA-oag5p|L|@)vG&xd&`v9NRfu z!Yk+$+v}8?Jfia)=~fzoFyB%r@{SdhqC2bI1|W`DdhZYm+zLH$PVf4qphg@VKa5<1u z=7(gCrgAHkopbhI5ri) zCA6UD927}oDg;~6b=*WC$iold;hC450@%E-pLA4T!spFTu~cwYosMXD=86RcbCtWM z(TgliWyTNz74o6(I-o_tum;7P%!>|hzJxjbdiaLCFH&nnz41wcmqAW3{T+}=EXooj z`}HVNG2FOpHO?I2C%%^TzE9*iAA;7*3HGlO?;JibQP%zmIO20X5kjDlbM9_1-^ETe_zdk{@y*L@in%^$Uur(S&gB{e+GLA zr(5giuRUK?dn-wYNn(eNUj5d~5&qX@_cWLB8AfDogI$1ULa_IPk_fGBl`EJ)J!*(8 zN-%8k`MaaA!-BNeKjlxZAZ5aT2C+1r2MJe~vt9x@rSehj?*3>u^)nTN?Sd}U^zwD)=*m~u?d;)c_G zoi&YICAW0&-Och_L|l@?o0%p|>uoS)9=$I1xxn-~>i|{+b)>UOZargt|Lr-b>d0=O z_1Mo$K)Q?wm}_CjiLB_Rmv&P9G)BXQ?|;}F?Z07Dv0|s{29iU}Dv9*oXronX)t-$n zI|QTdg$uz}S=@$FRk#Ww1lJ z8+Hy)QmmaG5aL>kHiSoCD}BFS@M4)(!}4;So5E}CGGwWbjmr-?@vP>uY=@0UO$eh_ zu>8xZa^LK4TaE=80uUmAN*j2wwth5u=D>D}G%^s!SI1|uQ(L~tL9%@e-0MDA=Z`Dd zk*~#H_23WbeubDP9=>AihK)winv|jGW2}oe%6zI)gBy%*))0hN9`*XyD2wc`=RPB? z&-oY_%dy~gUZL_{n@`29Pe~|3o=47C>!h1;f{}Q+zIWRqX9eps9kNN+tJIB*gUpBa z9o^7G0Lz=O+sp@S5X zYx-6OV-+?C-%NSr8E3mh1O!Dx{uGbKl_alDPRix^*+yC~ zYii!3dLi7#Gv*2JQzE?2U40tp$kCl=&*2p+QEVn{H`>RO2JIzkaDpAFL*BhI0BWJ_ z%j7WM_?dZmGfyI!yM%mqyPmw`^T*gE{o3v`@_@&O+yQ<#Z44tC)&Xbs0CL+8Z5*hH zpV%+1KFwCGej6Nmv$ycmB$;rpnr1hwY94Zh;c`Tp>k&R(rqXz2m9VwH>sxCmY$#l5 z1SvS!mk%JrNOU$SVA4I%2c1cC6R1G}N9x`LX_-y-)^Ns`yf{6HC~Bni^Fy4Ty_*V~n1ULZmsA=HPIplDGYb({ZV4pz2acu z@q7u#(PgSaX|@PgQSK3?72LLuO7ur0yQG<)F`yEz##na*@GYrcXb|VgpOB0BxY(_t z^=2cNL;%9mlh!)eEUyK%$zS1}>#(;Lh}WvIxbxs=%AHrP;&-1(M@x@4k-T2TUO05Z zMMe!tVCkqBkFuqUN%(ecC11UPH)NvbF%D;KOSWx~(ZmY42o95Hm*v&!In|Ph$+Qr2jcB9Whbv1_CLA;js zOm{P7Wu0WRx-1q~uSyzic{QJjHVI*J=JhT%H(AxA>C*Od2_lYHun&oV1lNNN!&rmF zhn??vD_3JyMMZt`JH&QlW@JD@c{je04X1;rhKw}u9wa;P@DEn_<6-u9WjX*v6>D^b z2zX!L96smOJN66OST+q6Msb|E9hZ&J{K9B=T!mC|d`~g}?h4>ky?}S?WPt`Es}eAM zaI>Y#LXKX9u&$Ywq~k6q8zNu8zWh-;3;L;oMrf-eK*)dM)OHCNP9Vd?Vhu_#Ukh8Z zF!BrEM^sL7_!KdrN0sEv!mRWm4DNZ(AppS#Zu@0((7) zsbB*7X75=7m zDy%9-6DLNBrZ}5F4dWihPR>-RJguZ=eWe<6Mw4;HZ8z3p!Jm2fP_vDwC9ujRw3oOSx1v79D|r-s^N3x7q2&mQ&^q5_#2Y84is-BP9Daj; zHaSZSV%Aq>t*5pB(W1VkK3)Ogo5#B>{-}dUE)1i7wKh9UegVggOONv_b9k`>80tq4 z`TcI4SB)H$ENPy%8n`7L0gU+?)Yf;|(kNUx{@5B7ztto(s=_~`rW2CRxshsib5ZkY zked`%f-!`Pt7HfK=G{{jP81<=L5HcYZIX-D$=Zsh?V{sPp|KK469Y3+NUd2rc8j(C~`l`%;-+g4u(gOW?VWzqR z98!1k&en=6HL4UVI*_L7!N_#?T9<-}p|*8gE%(;w1~r>Pp(~rWakj9mY-C~z%hepJ z3$GB(v~qAgklxt5qlKPshPK$fnPxh=f!Yb)kIHPW(Kr3Zda>~1Vtj@Nx7sl{5=1XL zx%TAgn8%FxxxTrUORdxQvi8g$WS#TveoGbtly6RwKTb_7o0;h5k18L@<;&$OrJP4G z48vn;+Jd*1e<5|Xr@dDE#R#@56Vc?}w+BC)DyO_50omBP@@*f&R}yF|DnWiU^bz&& zci!)6o#hNe6mC`Du(AR^?7PR{F{K7AQm^CXHj z>80|yfari;aQOfUf_=bbqnSq47oF}g2)DKntiMmd`GIFX3BXyNd=*Qn)+R(CMz_f= zDqJ5w?GeB3d&|;7)WZ2mg<1LLUT6T#NC3At@BvCCexNGi1xxOmn>IESkjUHTWA}G< zbv!4yg_@Zw@%~x~_V;8K%8b-ECBq}175G_=6_B@=~44R~mMzn$sPu|ps|IoxU0K=3UV%ngLguxNMc(oyu zP^Q6U*_qapRsgRNJ6&y;q3i*)4HRO@k6BubUjSa*s~Rr)1a;cF=EjmHSg0T0Hr*}7 z0)79+n=)n+c){E39!f8il)|F%b{G>qoROlet`1fdJ`v1&dF!QhpCo@;8gi$J1r@CI zDQ4%@6DMk%{3bTjur1|Pj{b882@j`DcM`aOJ^;Ft7V_Vic%FnuzK^IfA-X7Y@FfUc zfw!&^V~-!%z5o%B;l=n71?R$*3CBlg@xo z5_{Oi3pe%dfI!jMh4WdmSQ5>n{I#Z$F?d{NJrv=?H;(5 z+j_I4Jj?POO)4me0jGi&jus_t4(dtCElo2{ zgrZ7z8I&DXpckLy2x+!D3;=(AbkIUukp`c{ev91s7M`68p0Y1FMeAwRmWdP z75*x2e_ZdAZD{!2p(WQ4b#qi-%>7Qsk{_AbR$*-h?T_6ZRkbA4GCC9T2NTA$bO)TQ z57tNtGLnMY^{6xsFvbMo#%r1rm!&%LS~EgDf2}ny@m+-0TMGIWnQh1ethotAZ%!rEN?0vZnD}Js#am zC;oQ-r+|Fro<;a$svrnY@e2zEU3%Y=$_P-#x9yD?`8|DiU#f3IPr! zg=b=o#A;4Ocp9NsrebzqpPiP^!_Ef`&fZ45m(F-^f-!3DX)A?J-%UI&AcwxQYdpbT zvHJNA?Bs3TItxSvL{Wv}WMA-PTEDvavHkX0`Q^C&nTZdAjqVU%{jThQgI4o%@u(FE z+vMSjjD6_k)6BHB2JM)EoPqF+blxiPwHhK|%OJ+r|N1%ql!L{gzx$~_{Z*^N2ACc9 z3Lh)Oz`e<8DT(oO&tOL@w@8_b4@^3L!NocAS{kScrdXfW#TDv0ZZDE$*})X0dtavn zQ-yZ4Xo1GNR0J<*Gtz72uj6FUE^RU4bM!0TIF|*ndm`TZ(29!nsYfL?z;>JW(hnG{ zcngd_t1iOnwNxzB&K)tywZ%@+RRk(4z}6oY`@S>Z&6+2fA4jH?BPUk9#*dM#HI>mu ze4Cp(0Xm<&W#8w?i;CH@OcZ2Fn*~3g+0dQMnfJpKv~}OZH?gxvvskNGGx-8?FI;al zZh>AP+=OVRJV(U@I~R$W<0dm|Ie(c5XGDG-9k@vZ6l?i!9S7;4szWMX&Sh=L$%%Ro zR5rK*ajvS)!4#1f4PVUC&FAT8vl{kzYPDsxmgpYYLhmob0)b`8!A)$IEsJ}Xr!$!_ zrOew6DddwAtk~9Jo_k!7CTpg(s_BmbmCXod5(G~shd=ErYY;yl-nMWxi`iceNT>Ur z`G&~VZ{Z?8<^P$6B2`MK{)EBnf2T zB2ciz0$93TNg&zwQk5{ziCVRKdQ5K#rV`r$b2~j6^+@;$qfc4%6WO6T@||+$~%{?=dc@ zGx-BPRA)3SZP$#N41s(RUD?3m}=mm4*dsBUZA*RYRh!49>ZS#0}eKg(p zotqn;anrFGD~X=DCxYZn1sGQ>H%<-xeP|A%_sLuPfl^hvy|7;XG{bH~L>=3uvpg>X zQ^9CJj+v>MNqdRueJ;r_t=}D3?i^FRd%UFVYDv&ukTZFoTJ*s=Yx_|BC2NS#-2+2s zOAC7KmW&H1Slb-Kl($*Sr2FlcpB*H*E=7{-8GGKc_3Y~J*OuILENWg%CQpJJWNXRH z=e3xK#U93nhim@U*1i8|I?o$0jy~>YenxW#(ubBl_84w3^ap)tV0}|}zEM)bb)U7Y z(HVq%Ks~t}bMyfrcr6N&xd^1e-9ed0XNaLz7Vk4&owHwedG(87afVHTO(psZ5g?4$ z=sx^@?dh|~a9l)W16#M-W?cI|Rv6}^4$J?Ajf{vX7p)VHub8jW%gF-q@}Z4GZ8mc6 zF`QU5RPlTXnN(lAAwc@5DYB52@uOJYd$>=-()&r~(l-?pw&vqD0{l{n_bNp` zm?g!TVMAUD)+%<%Vt_2(`1lotiTT`N4|8`XJJ>J0$aNHgJF2bgvVjAQ+4aw!AX8lX z5ip(Sxo^xMoF7!`1YcXk5&;y>Aj}HzSJm7B+Mft`R}I9yX>uf7ccdT!?!3VcZ9&`l zVMM@P+Fj^sK=tW|P0($n-9i=*O-8xID$j#0p-cZgBe& zq<`5X2l#y)4E3WD@%<9O^jz%6HGc#SF}`Vv37v;^W-|W5M2|EFs5ClhAq}jqB%VjLUH>qWbk`197C}5Jl20M5DE?W2IAFDdv4KERqIKMl)%{0WB0OK2$#?yP zx(q}zStGJTvP4yF?K&({#tJxIyJd7AH%|HDg$`J;V;KEKJr0}}dS^5>CK_j$H>)kq z@N0Y5rhH_9C${B+YBMEWm*~u?Y%laB5ioL`MN+{D292##ca==1&l&bDr=;*IZOnEjlCxVzIdI4o=q{F zo?tS6Qhf?)RJbV^PJlwop&M9zlG?DD1VxLS-~e8T0mjvfCwo5TvIFJ4=Bkw3X-%J; zX`$sTbrH#X&DOpwF84$r@{uW!O<@nanw-&GR|`QAEjnP`6XOINCnhfoXN3}oiIVv= zJFzO};%9q}P9S?(@YT;!ZqJM-=vs|e7w3Jyz>p=FaE}eON9v2vTdK?)fE%RJ3fWUP>^;KptRz|@LKX;sNRGYuc__wVTsG=q_E2nPXOc^syP@IJ zw=mp~2W{oHRL;`#Xa+_rsaGN$&*$PS%oIv7XHT^PhI4wnK>G5zduKzdf+qziCO^l% z+KVTX#YJw?Z}tarr|V0Vdk$_BSQw67@E`{e2IxivbU7ZdE~(L<95Z1N8AO2Dwc*cL z#5z?!c#lS@gMfmqVX^Xr%t|1RfwUTyD*jaknK-pVK|BS#0lM`Hv^KX0y`%D+2=IPn zL;5uV3jdFzuKwpeJxKSohqgPkLVqe^I|x;61b>3yA)xHWx;<_cPA2voKC)t#E^RXY z%K6gD_3F`G>sadb`#vd@JdQhh69vZ!XbX6RXcHU6V3gCSYOFg{b^Iln^Oa?C)yh5q zWyUjL{YP9!)8l%to?RQ28!t|yuPd0ewDhU^!_K(9#rd;Lzh%qzxgF+*Kf9IV1si3^ zA{BW4tt2#v1pLzTf=eS)4;Uh#Oho}~82zJo*2yE;h%6Bh3AqdP5`CXpZf34rnRe&!NjPuNyUwK~<*ofe5ZYQ~!Co!cea&#I zw@yNOyT&oFG^O3b!AXr)y9w5fe=?gn``@x~a7n0UAr+onn+PCRNB*ChIELRQE{JfA zhmR1*I!y%h<6RCej|RB=eApD(zmL5#MF^09qvV!Ii%K6K2^g9`}L5;(ufG)_$^$%p$> zoL?GJ(;OKDUp6Hfuwq*x;GXO-^3*C#4tgCVJ#sZ|p3 zv4=0?35bpYeD*0Y)Vl^gdGmOi2#`@B1+tqI`+uSVGReq{%F}YJj}9LOwon->*mI#+ z*I0?2#mP{UTZ*EJ;Z-VLP+8Q;GgJ&y%Y9Ku6dHt z!UfDdK9Dbdr^>9~PxkkxtRh+QNYR)&__O*H)y#Hoh0DvT1(W8yYw!4vn>Fri#FxJD zL$2R}A%MR`_He31KuL5u5%9eeexFo(AWts0FRF|M<}61w&?%B2j4?7J_iterRW{&B z{l^YRnXG(1`NeZk6-y?s&u!z$Or<`yb25=#q6ofnX(k{3_#W802+6X?aK@aG+(tL@ zU}Jvn^ok4-aJCFRigE3GRZal-(%UmY~iJ$YJeKGj=uRw?Sw-5KdUU>-OS58wO5%~S} z@qlZmck}Xv2=ix;TVT5!P#O@}jQ(4=aL6Be4b2uKNdZyE8fok;!YL+8@!c_opKYHn zJbR?E@sVx*7?w$>+=EvdvXdcazqoGyrE$h=8?JROY17l13>;WKqZ1ctlLrqcr4J(T>Z< z9@^=OJTBEllM8J%3mUNgH=>o+2Q&&|{-;9qre|>JE!WDG`wX{BXRae`D@PCePVBZ} zS8F^tKi%i8rTFpsJ9=;ToPIuU`yDFNSM#d=j6E*ZqI;< zB~Whr5qnZ)rVRYN_b@mDQKY<^9?_Epst#GvH* z0)4sFGeD%QtYIX$C+?P$#8*3KvE6H)(NEKE0=7L~bayKp&3levI_0U1FIajzYz_z6 z!(?lZ6C7GAShSos3Sf6OFvcUmGWS6^D)koy_S2D~eWD*#>a>f($uE&r-!tuF-PME6 zQxaO40^$W>6Dl#oj^SUOOOux&pbrkGqEmaoeylKQN$zEx7Lv$Gz<;+Bk$=%k)_-!c7G5j)mgg2tVpfR~!E0J0d{T zE#_pClhdoD%2EyRSj3e;FPsnu_U)(-EkObG;!C2~zxwD6k zhw_b38x#;1Rze8fm;hpHM7ok-S zcxK3LZ{(a!1c-x<8A)zpICpEL7dmGT{lGEwH@nhB8rO^)vG|t5oN&ToXlo2|ye;=z z_?JJBK7*E!vJk~0!(;(LU4j(wAgf5Oq3USf@Nc%n@?UHT20ca4>Lr8`KBM*s58MNw zpJ~txlb5_zwiZU~1f!5<$$DchVqcy%DAOXKqUh?$=T9zh&cD34dzBtAkE|Y*XOubK zzY`}T#w%6$A*%bS|H4PLyUR9=sW=2*{}rm6*^!C(&Wvb;ZgqKOBSS%#)eJ-_w>TF! zjyegd>>*rr^o72!l;B#gi-~BU&719R;)7e|SI$JWdmdlbT>02j!4%Gx{N&{$HG5ew zxA)*w3y&~5+zhPA&<>+q?_r+eztpuT7?0&$on(5QClYGkT71E9X0qvX(6O6e)cr?j z`bIQGhjGBwrt%0~9n0ax+1AzZ-W!kuN)>Yf7Ms_!xQ3xZOQ<3UVip zF`h_01S`W!nfWmqPcNvw7#60z?a#d+#*A@kodktO7D)~tVjZPLxCgEU3PhG)?h!TS zKaqu-R|hGH&(fptO>ZZg3W`5nj5^+VId5Uvvnu&Y=__Bi`9gqxFxyt@rsHXwYyr)u z-pXq#?#naVW4n8P%En*M)TEBSL zKD-vWI5xJYJ9z*Tnwp(O*O~r0I0#R)8zL1qt++Z4gJ)Dc#)tsN2+jy#INS#P%GY8h zv27A_kAH-}4t&T)1o+%Y5nKKcxBKWG`_pZ%1!GD)D=PUo8smmmn|m<)O7?dtLHwhV z=+)Kci5a!h{*urXN(A-$?OQDjArz5onIR3rO%kZk7Q_b?dK5AeB9ViE>NTL1laF8K z({3KLF)2Odvl1rX|I*6+X=ae&?oskKq7XB=*OtLvIu0>g@PDio)0PCb9X=f6dt5z5 z$t!tFs^q$?qD^Z|+VKXs20ce>!~Di~%uc^NM?^K}<&KWSa8A*V*Oim{Rh{yMH(X1c z#+=ln8F8jRl5hwoC@0XAG-YtDh>`xN|Aai5u|XT)*vbFG4U`fh;k#^bv_v=Ve#hKN zBq$5_bCXoxlB?Gh_;-8`ID`#IOu-52;1JR@-YF&mL~i57NZ3>RKVeV4cJS#X<3lA< z@gBa@wu{-{?i)NNrhgFBm6{nY1E!hXoMkKxH5e@%dx_Mc6<`T6MSk=>zO z7})CJ=-9nH!EZyYgk-ugR}+{;c9-hb8Tc;JJ@c~tmC54|w`xio41Q|qFU{t6Xio7K z(BIuu^h0()r!o{l98Kd&FBX713>ek-Q0J@o&~Db0edsvJ`OY2Kg4`I=SrTow${Q*O$?3bH`s#41S{QqFe8o02SeA;V zPIg1xR7-yxE^n4`Pi5ub9A|(}KnobUD1m?PzK6dqd&W5nFY{&~Ja`Jdf4Cik@662E zByD~KoBMwKdoG!Te)UH|OPx!ETF|~M?#x@Bpc(}t)B^F;$V1luos(_F*^?NG6m)|L z$w3lqwbh47hmyahe`LA_gC8)j2o~nnahnRLQ za`hrJCwU%htniSZs@O{eud#P}7eCgo#++E8OgB3>K^MehTo!zIRj2+>1d?(j6K zTMMr!l55i*b;r28njKzPNsL_JrYLxOUcby|ezEaolP^XzKs=MqTMzpJBBvyakFhP^ z{;`Ku@4*V6vePS(LMONWmM)9Nf^}0&hG1K3h?TL()lCimV-$0+pLsF~UQ}a-egHm!-jVzRW!OMp``=iE*L+1~9UYaB z)@?5NKLHUY=s2I`O!}ICui66WPNO%A-`MOcG1EPh4IoTMJEO4syq6*=2y1b)g>0F?TWv$dCU6pkTQ|C;zHD4n z$A46E^@A2U+z!dg87P`k&9#_Os;(C+PF3hhNS~llw4P#-2wTH^5bOd222d#b%Ucqt z(^T-zhWM4~&Ji^m#iFrgsVkjQl99{Ey-((!sY}^DJ!!K#eZKj9g`Gc}TFtoLSGBr) z$}#gCm|ed!GRq))P8^ZtWI8@Za^v228_WsZ6;K}a9M$4X=^e-K6)+p>*!KZu4=F<> zoo4ERa;{U#VpiTZn3M_;=_Qs@3&*~W`4JAa3WoR?q5dm6^{(m7ZfTxHlQ`O&_KW#4 z-zuS0ZA5_csiz95;G;-rmIzQLSb;SOJ#*;OCFmKxl!Q)wPs{!dd|+9Jhya3C9dscU zZ7D#CyW?{5afmWI5;i{rrT&4X|8DdDm`48}#|(dUo6!F(&7 z3{9>@%VKFGlz}a0=C{S)9fzebt@;hE&djK0u#sWYn?D+JQ!AX^-+0<|4SgkIL7D!f zz3EzgbeGP^SNmOY0R@dAb!92-%gv9ZGy&uqEOTwA8GN+2D@qbh^NnK4^~?$?1D@1X z?w-lLSygZ_eqy(O%6 zXGx`R0d~dPvD4RX9C9OXa`RZlTJ<>MmXTdUN9W4)_sGcT+lR1G?}EsKn)%K7a?#pW zx{0dI39V`PKrc4i+3ZloTDYe~w?bs0lt#ztyJsUlz%QMSnDlu~ci?%*EmTI# zm(0tmSq|d`vSGAk%c4@JMSL0;?aFTTh|4V>L?Xv92$mH~VGLp(MCD!9)lxUJ-{IpO zMPkTgt1IqgCrm5W?MMv%l$Tt3mngW6agpq7X$_$XRSZg)fx+sXoXWJ0tKEiLo(roD zWphqcL3W4$>ho|?@_G%&zcybPdTe|Q`UNc`O#JYHIvn5-6-$cuU!AJ#ZhLk3yUedP zxX_zq@4j#1@=naMkhBRmtd3&KKWFF4L-Y0y&`V)cgijZfhng~O3eq0N>15AXOxHxS z@zzD9&(sM$nh^X+vzLdK^NHwaWp?$3+VWS_r*AiWpFEu;Jnvl0(HAm|YC>gm+uu+( z#Z=Zd$J9cf!Qg0CLhEfWLOchaZV#LRE-{|NL=XX1+=`@#Bq|`}1h$<)3Q}XQ0Ds~Y z`G^45oQ<fLwN6qf^$HF@2^5!u0Z?XDZ@iuzCtXx^fG3LW<9Oz<^ z%v6K~U58lZY;D2AFY0d0HV>=Dr*Z8ioQQK8%hr3o;C8=TyG`8QKU!OT@Suoa&Eli5q}K}yM{;Iz-=)+uj8WHR8hJy`Rch@NC>-N?4ys;E1iYy7o)n^ z^y6ONg7u&h^2gfmB~s5ieNQ$zoOEUVSrQ|85X)B8%PjjtN7LW>)p! zbL3?#5|-*#9QulrHusuxni~qp)Epp0zkYu54}ibFV@1&AvuR z+oOrK+mH@9{yJb-IFnRsV!8RL0{-q>{e-n3gRGg-wMrke!Yxkm<%v~&sn zt^ek>|0~YGCF>uY!Q%fdXYg0|B>j(a1|(PRe<`a7!inB9z{oEFIrJ7tWZ>+rFh%U9#4srfG}2`vT;BxY{IuFLYOy+iC~WUnR7cj!r!q zFYf5bp6MBR-Y{j~oUuXnqVwn7mX$lOMpWFBB!3RKH<8)wCLQ0$68AjQD~#&O4^+G& zYLmz+;*E`UcS&|`!C(Rc&20I&A>VYaUXQx5CD6ioN|m%{?tRZ;30%lgLTNiKCZYE{ z`h;*IcGT6P_W@4kRCovv2GZNcYc&MQJ+1KDQj9Pg-VD(+)squoxIu04OWOan0w!as z#n%jH(>R1^gVD*Bd3(;TxR)swAG&$un0XGHo(kk2zEl>9@@1tRXg=}A+ESsk4EuD#5 z1%t=?D;9+iFzr~u#g50{$7V*H9PPHUHr%$gNTGe4$=O6~At|3*3CBZCpc@Z7&K5IY z`-|)=l47?QN9Y-WB<^t7)=`Jb{29q{4JG5-SP7)GMwoX!0Ude>P5oUOe)PYnd+(?w z+bwSxMMVL@La$1bDoBw|6r>B0-h@!3iFAPgAyE)eKtitqDj-rL(rf5QlP0|rq?d#e z0)+V8KF>KbbIzGL>&*MjyVksav2;lm3$K0c{cHQWM8N*lYsZmzlzZX_%#$0~IpFG; z`{@YD&C7*5!)Jbx?aC3Qf6F9(M`mMfP@u6u8@+@1`ItyJM2!)ppXVwUN$IXh z$M%&X`TSlb6Ys2(=trJsT_UuAnTS4vsOLIv)AhcWIZ}U-T@&Juv}W}esoxPdAnJo| zToL!U92EQBDl-nTTu9;eq(~_wiAgC;m=K2|hWw>@K?_H4E9>kiQr50)b@(oAieyu3u#1 z>bk&xxE!40$#4**F> ze_jY;$#fWild-?Z2Hqm>2mC2#+#;!=v%nXCBTfcH&_t52nTD+vvT=eoF0u zTl!fxaLxhAZ;)-GRI6i|+&@aBsOl_M{l1r&zW>g{N*8+m`q#|*u|}8$?tEvok16RA zK@A3mGk~vJeduUiP{IV%Y?*(0EI7Qew=!L>X5zz&;d=vq)5S@_+?bm{4?%rZp z-e0{gFIb&sAFUcU<8NK}wMcASR`di-Jy2{S6wP>@Zh7dZvH5*a`J{{SHmad~hSMau zLpPikfOovhfxpO_k+kcT$0d1-Iznv$7f?A*ll7I}*D98&ap3ekLzcyF9)(mt61TWj zsT6}^CeYi@ZwWOS*8+@tq@{go+B#|ZLaU~q5>;7;2QKq$L2XDm^{>dAtD{?NXk8lo9|G^lf{xKb(w)tWF zr$NsE4|TyGx-;Mw3mJN$bmO!Dz2t=jod!OgJ8`!PUN_0Px z9kEo5+68?gDgaX-yJ*&OOEv1XbmraRdJ!Lg>@PB%uOkO35prgb?txAPt=2wOo0rRS z9_gE|HhaM`(i5USzM6_Jzk6?4=~gpFlnDwl#yZ3)i9QX;=oR}`>s#TN zMgVQH3X z7cAp;hcl^ip5Is|sR$$T7$;jthjCNk<}I+LM5sfCoFCuGC8@u&(Pib4>D&?cRyq;BaZ&Xa^|l zMb`d^Nx!y)k@`#dR^%O>JaVSQ+VW7f;me&UTu8pKAoD2cDO z{|r$E>(?sJuA+@K&aAc1%nstqm8yUCP}yJa%F$EJ@w!K3k-JeQEpR$Q9S+=PEFBR2 z0Rv)c=6WhUy1@}UU+AjU$6b|y(Mc9^^UElZezGJ@5u0`>I?=jF@fK#W_=}7aifES` z%`Gx_68e~-SP{g<&aw-tRQK4UsEX#Fii<3-gg?5t(Z5PmZ-%LccjM^q0KBa;rL*k) zn!5G4ijj=;n!Oq~@8pp-gSRXU@wK|&j=uxFyDb&(pjP&aj02#u4^HiWk>y?;+lUBi z-ba1g`p*=I@n2+WkV9$+A$bXYb8+XSgL9DNP_+=W>!}n)YYn5tT-J`t2j0Cs=vP zgQrLR?LdeR&OHSKR};MFMJT`{yp_Xtz(TjKULLjHt2NUF9+v7T^JC%QB)iw|R_6u1 zoclxr6V3b(5@nYthutLYUFE0jYC-4Wj8uiJCt#Y1iO0@k9W{# zYGu+0{dlohZ_-kG*eTQ(Fge#w0{Z)@z(jj{U&VafgO;L+zj8Mk@E+pY^A$G%RwPAnc_j`45=UVM#6ou3NIY-bw|?B0oTF+EdX`UJ^qRpZoZ90;V8|$1 zwHZsYf7`kiVfB5`x4tUiY4Ol(()dBX^@d@1=Z6EutX2rQGl?I;m20_%iMH{&ugV?$pZM#%g)lu zuZzwI-l>x%yV|_6ns)S^r+M*9N`Il9pm+BjV?TGblBC8wKJi2?AOBlI<>(yx|H$eO z70qaq9wRonZGhs@8>2BGooEv^I*>T-b>!*=P_+{rekt=A`ccc7uY?G5!E0e|*ylo8oUJXkWerA0Jf^C(KK zUeG4p==xys@S3~C=6ZqKWkW(hraJ~1f~F?%SUDdffh4q)S$R3M_~BO#dIp``ur?p{ zpX##|?9mdBmm5x`iC1yVrKCY@ZVqIkvd{UMa86p2I{CMu1g;?sm9yKu-dU+{)-_Qw zV{l$55|ddY+S>o}Fy73`I9?cj1DD%q#AWoe`1#l`vZVONJIkov11* zZ$q1Bw?t|yF*vh5^Uf}9&V(OD7(wmWI`1`yFEM!nM*IrHzQ;={-VzCVN z_Z?l;G}PDTu;7VnE9F^=@0V)OF*Z396I^tLLKef`fH>Y?CnnRISD1gze;x1bZhYuo z++Uo>^x128>Udme4KyK70x7uvB7;trz>YVZn-wRW~2A#CTryqnmd_}%}Qja9}VWeXJg|Hu(NM+CM?%#_r<~9`Tk>G;C!ll z8`IJAPIgq1-%2dM@n2$+pqypU2xRVXV!#5fXi~-iKsUWu)^E zlUC+pVd}z>e3x1;t@X7kR~Az2+>17~aQUR|+mjG|)XQ>YWD-kx8R5ig*hl+9)stve zc?g$kN-e$Zl$s&G4Km=sTyYh3o%3y`U|wZDU6xZ$@^rNMNQS1jlzS6EsRPn`HN&#g zNiv@D1TaPdS9u;{?G9>S_TcE}KN#CcUGt!$t2U>!!(Ty%Eby$)00<89T>r!3Ly=3z zk#I)+pLo>;qnNEN{$^kM`GPk2EZ0T z`XjwDW{VsG12E2(cOJ}o%xLa3A185A65i!nb!4;XT7F?O7%Awj@U(05kMGan0N!hJMg2kc~EY9H0rvx z)M2_OPjZVt*TpYkZ+*MC3K?uEmkzklPFwHW5ptlKPrL@HKhu)U0KhEy78ILeA}Z83BE{9 z!47|i%2Hs0i%mbZx;Eq!L{7Tez^Ysh^$`G-i)@o7^`}*@y~gz1xJE|A?`-U|!=#yc zg%Rb;Iu%2Y@g~le#dlXLjTP#<@{GE~{-#%tCOWJ`b}@iv!b*E87+tTBcZyFY#CH8( zr{7Cx7e!mME)(?!uD5aspBSutTo=l}Sy>(EDIK3R)BPxT@Ex%yc0BmvIlqL!v7i2K z;+2f_lmvgTKKV~oR?b;AZ}M0YGUOuG_CZoxn{aAEnsx@bW$vPazh_>jr^x4cwT!Jz zZSy;!_0vd^86TdnmaGq;HdTm=h!03I$^O`Q+D5rY zWb=i1m~ate0LF{hWReXN!scz6&F@h6WINq$q$({X4gYWxSr{eF@NQ2Mj68`n_us@0GbSJ_nW0ew)-|-sULcdW?Y@j{f0eiu&Dl6nNrq0=VHVB#xSGV zs|`b5jL^|m3?Iu4GyN7wj|kk5SosBcA=2n0dtiAT$5pO#A&$bnJgO6aT@?2MNkLgv|=v${6L( z*8je(Tu`uU_69>n9Y0{3b3E3b#SG^pA_%$O17VQ;euT0=Pf9#uwfWuG2<4{eC0 znsg_#J~$~mKE|Kw<^j|se08}RUATDXYVP^Er0pH25){^=GAWP2HKpg0c4mh3-DA;_ zeR(R-WMK=VlAyNP^Nl*hh3Q$PL`MwsMHa*31UnH1En+EKqFK)~2LogR1E=1KA`QT! zzC(86h-grhXgMO9kwk+CKUAV8IFLA8Y@8$)5YR8eLfZBSEvnJ?IQw}T{)_Al^p?o+Y-A;< z<6q;5Kw~qXv8@Tw(RMF4s~DfH9As$HSeo?k|Hh0$Ej)`$vJ)hYUW?D8v1i(z8LG01 zUdc{gu(L?5TSYwyGFfncop}*g*2Ng=Q>de6YVtf*8PjLDR^2gfEz#kj7C7bzp>wL5 zy@s3XL|@*RyI8(=IV|4Q%ct-p(73!ZmPgGrHle_NS8M6>EpFyZos$R2*OwB$u9uIn z=JD^zbWacZRb(^*p--kwVP~1$Dir*smfVnjbj5Cu_nZG+u=!&INQ)GxK^TndVY^d5 zEZ50qU022?`0__wz?tYz^$c_Pm)L0%mxamK>hM-+M$x<}Xe!}L+JV?ZJMP!VF9mFi z?vt*VdPA36Tkc7;`EuxZ-i>wPelS^H;~F>OazslJ9Zjpo#chx)>LqTMvo z@`BqS{(c_g8GnjSOGX16XRD?9gCR-hgSyC6SBp)2RV7}$6Yn4ADcWs+jc3q*Im%y7R0cvUA96~uHJoNqceHN%%(G(I(9 z)_nVeV#{h=C*5vAD~JDHOsK*OU5Sv#W$;leIu1FGtAGya-PUM=*Zct=JdO9q7cbf{ z^p_rSxF?5X$;(T_Qu?KK9h)t+&2era7zn#_EpLwRs3$ZE9F)Id?Of?Qnb_v%r!`=z zHW{pge{LP$xhn>aA&<9m!Gmcz=|rQ@6+{8N>QGuAP zi=XREA41m##=9_AyeME40xOtf-{LC)cFO|Bj24(#s#d6A`RtA*!x+e}ULIvmNLQ^G zE3&Dr3f|JrF^N$uUni$;i`AeXzwGOa3?k;UP;<@US{`?^yHAid@vK4RGPnVdPI z*x%vi#%HJ;V8qxV5OL^JIKVuvDZ*>^5rEOLB#ZC_=Kcn;%agy3me@=KfKVfH`LBy} z{~EyiXJ5(xOeO-M6MhsA;R6mJw_2$9-H0yvOb&IzBEs+0GIy6yklQ?6H$0PM^rn|0 zj@HobpwRT_IT~n97?fN0)mcR~xBEn#)m{15UeoMg`-ozuo-#-FXRVjD9#`Ofv?^rdUhBr@^cDi|q0Wwu%&rVg&NnwVWia67WGQK+L~SIbDWK5u_!4rK^Z(KB zjM{gfN(P{H5c{WMcN7#Zz-nDLB6*tB-T8ujA;^2WgK<9q{{5;)o zJ8b6!OmzCuNJiwTCuDtED+QdakhMW=M56FukVkI9ZfqpRfZ>39}eg~7QEYgP2;so@F;!=Q;xpnS&l92%3ph|$unf038 zHgc%3zr@9&9Q!UU^R53PeW^75P**a!?#`Q}%n}T2@qH|S_hT> z$Ye}zZ3H3Cowp<-VDw47hGukBkg~}mYvJlxwb|Y?OsJCECB>VrxeX0T@_{bTo$b?1 z45MWRuJ!h|krkZb$-L85grHe#WG<`7GQgHIIJ=fS^wD$^u1wkCtp*&>8n;u;s3Yih zx~9eanWXT{J1lw~Pe!mYd5}5JvV4O{Rj+Pf9~JA}`ui|jLhXDMrR+BrsWvHRyA+#jg?HAb*JDl}*5Uq|_w6O?N z1xSVS2$jx$K=*fmc>+fthdW9M5?aZ+;KY){$qB#VB3s5y(W#sMlR+u72>w5fF6eV40`>$E`n{?x5H4GfK#on?Tl7uOQ!T8_K;)^4 zBouY6-1qTj6EBcE?HpgqzS4A8{NiG*C{Tu^-(?sFEo)*4&-`pGKgIka8sFk2Zd(({ zQ|RGwpP#@>MdR{=)ZTHb>Y$bW+`Xr_5?r>mWOGnpH8t)XINDH;bbP;`^XyGe z;_GKJe$<0^ioX~C02fHGAXirsYCIu(w64^@G<5{Mb{X??vx((c_p77@JLTjh_x+2- zN$ln-2%gb8-^X3z`Fm9Yt;v>X5RRJD69tiKVYqh!Z@hU5%&4~$Hwb`A;oo$!ULt_-s| z-*cx;d^={->w%jX%_tDht7&zNfdV+k{6In>nKiiUEpD)d?vr|(d% zT~3MuH9P9&yQm*?y@berm@WT{bOr2px}x$wrYmUo`ut5gz_f(1_C^|Dqyslg>y6(n z^SPXvAJBHmwLDdYD)r_;%KMAioNXG-%p8U4_aR9?{a<~NsfBiX>a@=p`5eM;jV*o_ z{E@3!89%Jk--{xL@eh_~g#d3v5bYN>Qp?LenG^O@tINVKu%5_Zk~a(QYAyrEpEms+ z!&JL>GKr0lm`kKa8jMi9c9qPp7jJwVPuS((;*kC-Tu|xiKqYWbqw1QQ(5~AKfhcMkf`GB?(XbPpPS=MedS{( zo!)1|2lp$CfH#NEyr3X693jA6qfF~vpXnPLB;>k{xsvjx7x+(D#Il3pAYPQmV7xBi zbL;(uA6ry-}r*pmu2h?eNK?2U1~R9JhRtWto_0yINtkEEXBva zZ{qgWK3B2Ea7eyl?7k5_hVxvbJWPRbvcQ9Ld?7`v_^8OQQsNBz)+=Bj@Mh3do1YKX z!)i&;xJ;C3Gl!sFp@i@9(MUv=>6m@cFQkky__Up+=LBqdTC90{{$st+B5La@wsW3{ zbDrgIOv4yegT+Ne!LMhm#1YVsF`x5FV-wie94R<>5bog6hI;A6T}4D0FheZGTJcG| z5$_dI2e_|0$u94VyxBwqynduTJc)=P1d>?$7u-*&tV98O({m6pQJpBA)iQU?P!9oi zPOc)31Z&V^)XAh^yF{E{}#3~;Miuhmro#?+#-2e8|qAReF zJshXUAcTvcjUOUNExMXtW;s0oU+#O}!*P-R{MwC;r^ZL6(U@d+k9^#dS=w z6z|50U-n{C#%mUy3ZVFW2HJ*-YVx%Ib)^*5(>G+aJXE%Cx7jk-b@MF?|Hm@7@4Z?k=~y>qdGY$5;fVO$5R%eS^hjedE()~joE!}Lc-RwE~p@L|Zg#`8}b z?+z*P6GF=a`?R)O7AJYXG}O8MAyNa|AHf2NM*0MM0)8YTmnxQ&9%4 zDAjSi7qy|YZB2pJV3h%0FoKXy;@~|2eq{su036~O1sz>L5}?VOzy+!Tl>L0XDNqtD zq*~ySs=c}6vxUHC0&kcBe%7b+=#@?Qeo%=&3RqkYJZiUJ=%XvNYRV{f&h>3~GJ)tqyjitB0e=P3${!0H_D<;Ujj zOp_7d+m}`)h8nYjWlZF9v*y=FS`77!xTZYfG$9R0X_;PZT+>L@)AlOH5ns?<*dC)) z!=|PxRad2B{r)*ll|FL(vQ@_9ZK>ZRJ?218+zB^k(Xa}8C)9np-q|B0_3RDt71>*yA1jV*PL!Q}f%d8-=`;u>R~g*I z3=tnl$`@khYJU5QP9E;$c`3%BNUzt`W(`kd!p}9p&RSW~Jp8Z6W`9 zv|F+C&84fQhSq1Fct3*fb^sCRKgWW&M!~DnU5>PHWu8WKwPp@?w90Ye~Hx?VFE`LP6(j!FMN&WyWuE z+g)zD<{d_VVg+3!fUjOsCs}p}NKI@#8Jo#1M;leP&sw}Re34?xoMJw#*& zqQV2RhX(~HjiHv~@pZwhsLAxj#D?xm+(ByJ#H&)RHD4Gu+pB|2WotR;=ui7&bx%mc}EUwPA5Q9ur*Ol8lSsBFK z==}n_Z~RG~Rz`LB>29R9Pwu1=MccEMsNK3ppWh`U^&Y$6%}=Xg>R6M_%(H|tsqc03 zs#BLg<=oGDGP~{d57BQNE9=S= zw5bQH)YKmbD`mY!2v~Kl0VZDs_z?OE$RXod7VXYxJ*VH@Q|;C`bT?Jd+Ga}Dbbs`F zSR4)06Cu%QLh-z+LTi@P!hW`izlu{&pD!Xr$uESc@_Z|#cQ%8UG1 zFO2WDc=TH-hl~@>{1qP?1JQ)}?TjLr_iwaXb5u=vbkaT<-VUN8s>U~SP@;j^+D>!$ zS$1vx$kJCvwMA#o*B@r@UZD@4xe?5IOTlD7YN)RPBiRQT=J2Exw0BZu5G`G%m7 zd<{@R&u}KKt`~3>itPKV2b#<8c#@6c)Wh_p; zO()yxsfq9PsRxyeRp}TF%F@2#`eqO4(d%0kLDq=*kD#v%h@}7o@tO-KF$6MA;`FHW zHVOYy;cO$Qbz53TSqGG+zUr@J6B>4nZbe@pcO-kOfL7~2Z>q@6S5OYQzkZhe-_(`% ziOKn2WFWi4>{}#CB@l*Q0jNtG5nwjgo=3HDo`tS-%u&``x?^_2YO~i^Myf9^!A1{b zmL%@`FwP|LakK5xFl$&}JmonpM?B-4*uc;>DOxmT(Gc~IE2{#7K53q>!dqoOW4}=s zVMd$k=|X3BF=u!FowZ{w-27l|Rp^AZ$%;Msxa@V4Ba?5oKwXFD)AgQgb&L|%@^+G) z`t{)a0*kz5#Qm(Vhg+fZz_x)Vr{suQifG}Fb3q>;&c8AA$?W>Buu7{>(a))N&Sq-! z;XAk#F}ay{D=FK#X!X#0t@cxfIM6}OzWt?xZUkRcA|9e9X#Z@7@9-dMB{?0S9VY*7 zhcFeJ!YWlW8LB_pAZpu<$*${}oPO*-Y4?ze$dUyP`Ro)`8yq+Z zHQzsM`OiWbj4O8;|8R{YB9usf8`C5qfCWgxatDf5e{ptxi~!cQabp1B%u7hy&?3A& z#~6@N@q+=*U)DIhkam2m+!F3!U@zvof8>TeYF3WtRsd^;Khgv;8VtV(Wr*lbdtX;1jv^ zSx%(Mz*fnD{B`K@Q&j*+=C7MvPe%%<@FWd$P4Qia&lednk)@JL9qPc`v5+?NzPa|3 zx7T!Rj}O5dY)KoNhb%<%eOpgIN($Pu1!~QgST9bWxh3=TgbRHe$1yiq8tAPGYA9>@ zZn-fdGSc#Q$&7wzmO<&@^B*Gz0t)H!LUr12mYQ8-)fJv~Qc|&z>5}o_QHkHybW3cM zGo<`wbCP@a1mtz3r%A84fekKTdK?#`86fkY+#utSi+MSBa{Z4fH38^a5719Ik2!!| zcrL9Icq?`6n4E^T-%$TJ5kh1X-35WoDyc~D&K>nBI+p*WN@M=~1YyLtf18F2Pr?0& zRZ0|zD*1^1v@jT0l~#H(NH{vNY`#3UrGwL=dUZ|h_-%~6TlneO*@k|+*=cpXI@aKK zPb&TQo|OLYJ&DYl$E9WaQ9eYR2*ve>l6YZo+)gx+-q+{yb(6-5r0+jJB+d0bC=cUP zdfdep8K$gc&n~mkpJEr1_0Z&Smg?m$@$i*cs++sEk?!50p^z%w{9j}bNr4?W4k6)w zA(NJFoV(jw&Jk32z3K-7O1%un7`C%@peRn_&79mGRZU~gc8iM6306w1nnb7dPhjwg zgqJ5$F8)~jsblk@xBHKJS)qmd?hT6RY)29=qmVT>7Q&PhPXT5B405u{Kriw*NEIicqF;Tp5I+2@~QwQq*aN2RnSzv2fjF%MQn$Nhf23|v;BTa+KNf&BB2rAQ;_hxQ zw=SR~c8Wd)21W+a>Lm$$w2f2eve4-6?)m-B=+$phQ)3A*-;tYxa4P8HvF}C3dq21g zG8HD?L$g>54T)%oJwno)6gh6w;N{V*#7ukgk_*HCp=sx;wty+AL$h4>p z&#w8joa)s?6WuU@SI@ea%HO{@dre(xHH*%ojl@9+#J0AQZebhhpn(rd2Nx^GfbC~) z&*#28J=-GFV<}bt5WoK2@@kh*Y?6?v+_OwBY*u^LC9}0zjw&1|;v7;jv0h2EarAs> zLz@Lp%{$GjVf?+|W(WP&bL7pa%?PGy<5qng4=sTlJ^aPNyQq;WwRH8jQ^tD2gI;)EDRIxNwcr8rH5F1ROR8<52~6wxTq zLp?~lS*Ra`UcWrCCTI3=u0Q|8jU%fpmnA6XAH>FDa44vPqG{)4BNxss6bEVxS4TId zbuV8uZo%)sl&|GhnKz50xS4o3pPsg^E+pCF>LHY-g1TOh;>!)(W;QF7sM@J@Jh{n{DI5R$Gl2o=##lF~61f#}q6o7gP!vM1FU&nU>&x!l7O{Rn()qA7mNt4gmgQgw+Z z@nn3@JF5SZ@MY7EA`8zT6T3afrc7B}RSSyIf^fJ{^7&Y;!tJaqr&Pm|?D_Y#k=s4j zuSWY*Q0Iu9gYS`g(3OnAASahIA1w$v&yq zvf3Pj2>3xFK-3FX(rxxWif>K&H21ew5#~OZ+cF zYQ7&L8t_d8$y+Q6UAC+h>>704PDducUUIwcawUBK`)QethoY_bu?d!u0o!5UN=Qo) z?nsXs0ubsBBapO%vh~wOx1QQj2^Dg_^bMH(uy|4jtjpf;Vm^ReneGep6Jm>P=uc)@ z$?<$oSs?RBOiW0*2KBd*$u$=usqbkww7V?b6^KYm?6G|H$`{a_nFdn)do@uvYbW>hbYd(58=|zrEsg?j8RvvL_=D8bXK`&iIYv zBKtKv#kc$LXV>VQdT#PODI`t%XFl8ufvPbiUuvb2{wz(x!{;L`rU3vZtP>D(!muc^r9Zns)(jx9@bK3@}q` zcKhVN*Vz4q0=obM8Xh;W))>9{42T)+f7Qx1ssD$+g0wLo5~ExY(U8<51|dTH0V2g` zu^Rj)Q#yQMELaN6*mtJ(?$r+(XT~|qEbZ=rd2TN=D<#g1B@>^FN@mb5j1~r=R{K!S zf%@`2Qm(JsxeIo_Ut*=9QKOs}yYb?wm!RQ>1D${&QryeL!qS(6MbOALtybYNVF!;K zdd^Dwnla7lWr)>y`?E#|46ch;1~s>j}VEu*LXsgdv79!Xx5D+eNVso|i`$?&PGm z!7;oA@|(?as;|;UM&DJ{CgnnTNqtprMx%;(ezOKq+cMlgZ^f!N&s|gEc1B;fA(-Hl zhqVKHPA)WlZ5(>|QQB}WX6+Vex9=BOsD{u$@WnkO2lOL^5$2rSGVg_3ZJx+2-Req?Pa3%1BG;Qd>1EpQs+5QDSuQ&4 zXKNCCIBxRIX+#;@>md$SGG-0O?)JXh_|O{7t0_jj*? zKn4z&N#aC194!(C#vV4AGo*Se#X|MvyAE?6nSU}&jnod0O(~hFp1(8MnuytVRsuxY zR^6FY%mAVSQHo)-W<{3071LqR@~+#G=`<*lMd=dQK#N;2$PJ&7qSIqm)P|XR(HaZw zmSf=UDnI%tzecu*$%$w$<4hIwDp9Z-G%i)2O9$l{2Y1M{X+v%exqz`5Z(&PHuIFAHtt4{r=5UX5`9jLdB zTz%9KdMP`eh>*?7CO3KS-3U_8F*PsX(i1?8rDeq_ygN*tf69*+haP4}6yv8vk9Igu z_xVp;lZl5&(lI@$KP!3H1nA<|`svt$o}#9n0epQu;&5mva!T3b780~Wt~JQpQe_DI zi*|mQY-meE5Ep0^zUiwl^~kNr>spovKQ`j`^aR4aA`<1^Ax zy1*4Y2gZ?;OKm6RxN2OI@Zn$Yw6g@|J;X*e&|$oE@)7>k=Pc~VkfY%0nG}?Dq$id(vba`RBUqpdqoO z&kLfPH~Sir=d4kJxXxCMdZR+9y5aCjxGo1P?(9aH+LvIrlujqlc zv^a9i-O>dRrx{Tm+aI!1Z&YBprc*>;UK>1TU)yVK3tPP4XNUIeG~cfV)25Afdf9{8 ze#X!6x=8K>-3@}P^SK&O=_2n+EYw;~l;*>@hkVeHd5dm93_7%m$nCkRnKxseGsUA_ zRlOG?YtJdP+4LCezd|26j}F#{S}cy{T-HQ&%(tOTFbDAKHuYE~H@7F72SGdIhm(K}i%qEwhn}P* zY4k^+H`>uDsPFP@to)#jG*04klhwH`Al%4f*A8HwtP3O;NP3cAWu)W9y==}py~EC2 zLnPtbf-Dnjhu69qeE*P zHwTL=pcTbj-qllGf|Ci*+$OX4@h;D7LOPh$)Zl6aQ7m}z*4CxP>;?C5-|nQ!l5C?f zH@fu8auTPDpi$k{ZRPsRJWA&9sVsmiaYT~Azn2kZv2!6d{yU*D;F=v-JOkZfodxsz)eE#91CU|%0^09@6 zt+AietF3|VPKlh6-+o}ht!N6OLRngWOF+OxWD>1HXqx6H`|niWex|&Puka%4YATtm z-)ln9!EWN1-I80AuLTvf2hW&Ix7pN$%aVI>O!VJ~kz?Z!8rV=6mH8Qwj7LhBZ}g$# z8Mr3}-x@Xip?nE95jGZSiHVjbpuII$z&RJuj*%+FF*w;0@(Y>S9|gsYpY7yq5jr?&)jXK_*6I7F{gQa`v^2uNlaau5Q&Ah2z=UgRt>;aj zAPx#TRr>CGda{4kX-?BL)0e^M4{35=p}(pw+&fKnMk>SP{DpNzx#`-g1?%eLbLqLr z0nrKMpVzJm7K3r^1Fl^)aM{}N$%V8-ZR747Kb7RnnE>IC=WhnKAD`bto%akwlw36RJW%%}$Fnelx`c(Jg zI^y?C;2bS1S^MuqDqP}x?ZZg}ndqU988~=Z&>hK0gxyg$W`1mwQmK6cI)ur(|Wu@g`qWf-C`j2`x#G(T>%J~)xG}w=6tx-3yoNax9os44D*o?mDWa)S4 zNr-D@^()D9`TAV8Z9A+`Hja7|)L2Lx2x9uV^G=nVrnBRd$|=;M=i-Qw)Yd^DEV#Z0 z!vI>a*zjMy2|2xJ@@Xs;W(OEZTuTsR0985=8eqj?=BZ{iM?3|NV=i~J1*f4mgDQ{+hTc1-g6nO!ga8}^`td(XI zlPU%s7Vu{HKx(qec-MZjg6T|G*!m5%1I3gptGdAbS0aO#S_fTFxACv6f_!ug^Pu^I>#+$mn#~#QKehh0|i*= z7t(h$D0v+oJO}^5arHE(gvjof*UT5YsK_^EfHWJ-lTVvjV)$vr?7`r~NnsboA=WJ} zF)&OtIFToUPBU_WS3nim$@_!+-3zn}HW*D^`qX?F<|U())^F>68>4&?-@{-F82{dZ zb2fKVaU_wY(zy3qZmxJ@9mL@D0k&#`DqL)XZaA~5>a$5Y%$VTA*bbsZh3tqfef^Sj z+`V|feGd?bD=Sm&1AV*BKS+n#47~7Y!!-gNU3<$`>rYG|3;Vg)BJ zC-iL2yq$tSb7UJ^n9voy8rosdwqW0*C*^<#H=PEt=KFRbUGFZZHP=)vtb?gbATH){Eg2Xbq03_2c-*Cw5TLI zwtQkK^MO&)q`mzf*K^xul_XvACs#U$D!5z`tJ$&h>p~`=Lw7YgS*=Tyw8|Pk#1LNmIY;Dacu-(N%b6 zyK3n4ZU8%g7KH0H^PCte;kSQ$7&&kvHhcQJyNTE)-!}7YW5zGs4e9DSx6f58xYz%x zQS^rb0z1~)C^7OejH=JJMLs{f>b$K9`~$g!nBJ5yxMs~N91^=iSE%1Hn)g~$M{|#V zLTJvgIB({|!MLWfA=eu0ZtXVxQ6cVxq@4snQtK(vC71mIz!>So{4pF8_Srtx5aD1( zW{OdeLU@qA*CE}3d94yz8@czb>T-PaX?`V86Z8A|8DPLJP6P(*di=)V6V@DLj9=7wBC)zAdi;j<_g(QEJ+7oU^=IcF==DfQ zlacAbKfrrAzleO#bEjzAndIo{iI68>THC}&QSd- z&r(9=#cf!FXgh5L=nBlBa20)F^ZDof;tO^z-UaK+25g6vy81U6pnxlY%7dKYQIm0h zFkB1@h-)~5*C%IYXJOP+hCF*Q>$rc2^6{!K^7h45EIcDq7o zUJK+3(WVHhVlrHz`BRCa1FnVX*mo-lg7=!FR?#$){pY}iFqJU{=DT%; zxtZZUc|FDJ${vq3OfrusywI4U;1X+WxypDNao!9n{X)=s8yR` z9D!LlJ(u{E$Wb|%28&h0<&9R{%I@6Z^bxZO%iwcp|8`cBF0r__kR>?ZH|PFMKmKn) zj@7mjgBnL=I}Be^^)gm>xYD^w<$~M1MNd;x=EI6GU>Ymto8QW8fsTo-ZTu5-iO&vw z=#6JzyWh@G6bgl<-r=072LTHPEZD45MEL2Y^}c#8f)n29Gl6w@Yj4_gWNq7UAgk2( zkt$um0K789)BG)Rb#aS3?n|x1BV{kLx$~NhXmU#;l8op-JnslDB{s*FHuFp5n@3u& z3Q3+64lTAp*Yc%aoy>X%06Y`AqZhKYbU%AaKW_?jFpFY=I`vy5T4H@K>VOUim6E6* zAT#0L`b;&z149V2B+`3PQ=;k!aR)%N3_7T?qVm(g7-%uQ2YfIa3oipBe?WU)c)}u% zJ{6429#z6oKd8a>wa&65JfA3B>cpKpNGGpgxA>HLt-VvKYvLs4|hq;;9yZk?r_x7D>tc4&-ueQvYO}e z%h9O@rmg0|Y+qtXMH=Ww`FwsfbP*yW5@ z#?s4|1bEMNOgR`Nnh#&nZ(^q#E!=c5fvXguYl|BzDl02gErQ{8EGBO_&VM&Ui}!1t z3S3x#^Cyqow$>}#+P2MKeW9zG`R|zk844m18^CGDv?m<%?aYM>0LmGWff)I-8ThgAEgZ z(*CiQ>oA=n<@d!t)ViEy-bHguoOA(w!$`4A`gDQPiNRqvxmusdoDemb_-N7QEEdRr zcur}kDNu6-)WjR%c=N*XDB`;SKQvY!C_-tVN?K5$8M)352XK|*5bPaP+%ehEmmi?r zO|N!twCFE`>=yffm(?M&(m;BACPOb#;mR6X=X-Clh*BiRf_7DD^;pw zY`Oa~W1~$_tQHYsp?MhvwC&e{B?GMf5LUt~ZU{rk$`hFID8qR5#zkbTa6f)}{p%q#n=D;|@vl*BnT|&1|2X5M9!0g5T@0zxX`kSe=`wfr- zI|`2)B+bbs8BQ?xGZv13uxs{6cZymqC!2Z?Acif?cA?)$l>|CbuI1Z!TWIyZ()13R zrZt#H6S!C&szy0LsZHone7H3Kz~(*bs`r|k1OygODNbp+83-yhvR95 zQs4myM(JW^R_ONPpjeMc1GSg_5-+!QElF z>+O$Jh0Nc7bh}>xMcE7X?A}_3YI}D=DQZPh5ifx*;RFhX<4&x|b+@%B6wb6HH|`dR zWS4Qz@O4t~O(dzj>fS5wRg7h5F|s1u2Xb+KpM|Ch&#W&fW~&2%&*Q2BFYlQPb6wTE ztZm3$*(=4G3bLt3r78tW+pfiNVx{teO``{o#Jim3n#kzNUNip26=(_(- zgTosSi;qz!s6d0`vK?k@VYG)6^mHt1qxpQ|K@LM~amh$%|8g6Qh-~_yvb5JJDq3(zL{=0S)IJ%XxN@`5etF z0Zu`7eOdAHK)LR(b{Yp+`G_zABwh|EoEWN^+xu(E-a}a-plGaw;Ny?^eFxCFkP+EY zqK`Fd?iDbA!HxO=4{l%CAy}GvT;RZb8{6D8ix>FyZ)t>1$TYLFV=Xt?aDX)f?O=Xs z(v2o!zffM@g28h?!(Os$TFM~7)-N)N`{Ji}uFRn0MJ5i`SVRCN+1m>0QWVVK^PA3> z67+Zwl{$Na{x(|0W|<)n!(7y=_aQ9@9i)*zQe849tJ2tfB0Ti0;Cs1YGVNATuIe@N zAjw-?AhMy+VRQ-4M!*7eLVgQ;c`e^@9vu+Tc`~hO*IlOrYgT$uKtG-hCb`&_@Cm9ebv7o~DOf#CET4FvvZ@OhqqrqhkDF6{O)j~y8J+Yg34Y1% z?UhL*qqJt)o7rJf3xMSQ3a~!{0x_PgN4KKb%@k7j<~vQ{s!vvGQ>TwMYWv810i|6m zQ=E{83ZYkL9%!p?lL5tFS*#D?Vy4l`v7C=>(9F3ugVSYKuF}Xmi%IWZ%0_A+X-?cM zi8N4uz~M3FSKruG*jcm~06h5P$`YQXrE6Hg{9}As#v-5fq?_$*&atWch4wSN?TXc4 z)}LBK*V^zNpIUg!gxhP`a|;yckQw1&jnaX!f&_A7I&(!*38D9V5g9hDex1$iE%67? zOk@g<4CpVFL4&i}s$Jh@!jJWm;H=CV^|=muyyn*oRcn8_yDQt%aNFF0`f-MrKDkzw z4xN;4{>DPr_GjbnBOMO`qDpFPt6F++Qgod>RCK$k#>A)%P47||vY?it)Gtz6I%uQ4 zs;D#jJl(!j)M*EE^GqM7-NnYj?Da0>mCwj$b;I9-+b4vhvS2d`v@vb-nIC$|H?=9) z$+x&p&Y0#u#XY*=QUt8K3w?NE)-7xvIey2*tMe;=m2OE{{vhD=rZrM;!O8>h6Ga+R z0EIG5VXeCtj-ts>C9uF|+Zms~_~+nxkU|jE-{|}SQvAoW42+b2^_H8|`a6@2I0N)j z z>DX8+=}Pg;{Vmb1zCdKsY+*xVEYeY{NIHBCHesHpMI?+U z%ua!aTP&+y)XA}q@+oIFe&|gYpj_;exo%v;K6VK@R!@RslZB9DZXK1mbu)+0B-ks) z!~=3G;yPf&Zd@3X;w4WaQ1;VKvwby6#qfgT%X(i9Z~a}Y5KqL4)U#q z5$VB8u8x3rkZG1Wpr+g%$7OZRUUn9_ga}?}twRSDa(x-xK(dZ%8*|SWT})EwG8xFd zafT~|#<$%A>qVi`?@T%*C@f?)^lgnndvafeCw9q_&bfE^sX!Ax*5$t$Ftfu(X7puaBV?~;VIN_mKu}N zJj(K^6h*rl6P#_>vAG+pb$dJh=EYm@O2Wu^{E2Tna-|Ir-*N?%Z~6>po@_N^U-P8>VO>`p%ndYa%NaK9*7LdckmoF!UIvX_}GUdwDEVe(n)UBpgQ$+aW&yaB=}8TFls428c0 zAu<7x=LaYaI*%gIH3EPo{clCTK?m^xSC*!7Pd{Z5$wYP`dIo~>#2&AXquIY-@qFI? zw<~}^OBs72it4`!ER&N1H4+BC@>yY^@YJjaSj74Da`4cX*`Mj`F4Kt$!mN4b(rG&J zIs)1h?@}I>X#kzzE=x*;(_YSf+OTnj0d)4O5eaLg$7r~0Fr(#rKi#8`ksy*a_%s?m znGH-Atx-WV#`68AnJ+$*aeo{$ngg9z2j7Dl61!% z6=mWqAe`PGAroxJh~;WH$8xfM33J}0x%QfG`%<(B&m)P@Z9W;4z6}xCimewA?-S5Y zWw|E@RYZ_&be>#Cj2?v71hh!!8YArIzuW(efRlfZfSeuIvYwZ+)80N$a@an9*gQ*y z6#iRCYD`}_xL7WLV8{j6?%d+ei0BjCP`~v(%ETl(I{6>*uN8=Y4;BczX0(M z#J>buLI(pWMGJB;sK~B>-?%ik`CRe-7J;~MFy$L?GfsmJ@K^B6UF!=({bVi>sN{qI zt=CV&!moafPX91lhAvk8>eslZvOgMt+TqNIO<4jjk5taG;^78>cH=ZU4dK7rEP37B z$@4`-r|>W-BaXj#zB*Yt++!n~9pJo0Tj1Y)YiTNzak>#-t$`fuPl3M6WG+P4CRry&f^9cPLTx?TW8fV<&Nw81f1&Y?sI> znTHba6Rc6Z7`3gT#Ko?3a|#tF5hO+X_mjaFSpn-132)*M{lnZ$?%3AGD!l0Q{H zg<(+S2p^H4dKp~s_?P}I!kN>gOW!Q`54AdFLm8Q-D>=`;I)_qUD#J<%+lW~=ABJTr zbq(k9X{!prJlhO~#0C|1AIdt?&G+$&cUrC2Cr%Jmj81^C(Rm(tv);@R%Qp8hsFJtZ z3Rs@TD-T1$sue-GXDYS(9*_Qf2j4cFl2cs9GkebD#bTt8pKdULbkJAG7kL zS);YG`^ki-I~a^{VbtV|kFPJ99Rd_Ce-47+KURY`>VwkI-_XC0fJBSq7%64QwbZ>{ZF=5Fpj!zAy(DKj*DK*g;^C zJ7;ATFM-C)UN*o(8a>R9XV5wI-RcyNnP!K_>(><1W8JQ4y;5Z?=$;p8f&@a@2qr&3 z7c85NM5G_6C8rA--1s zF1~X7Rya;RX8cmQKzs@y>^w;ZQmaGM?FCIFQ%$9JZb2JaHCmX$+yZk#zgpiJiYA(> zI-9C&k3BrcdIP+MM@;a6`|JT0_&;pzoy(#!R%4S{8BDJZ4|WXgvE*!sT$81ZVPbl8 zyHjs2*ZCPD-T4Bq`@{CBqqbA zhIsuX9-*5_k?1!gZ(FrFP`=SvG?f!dR1-@&qNjM@*%KTHF zXFhGsg1LShpSWA|!e>T2h|74HJm~D3lzaNT`HoEfoR=dDwYsJ!QE@uC+9(#_EvYQM zt-tZlI(Xts@dAis_c0abvtPg71yO6m_kMs51ip#0G0fRUFQwBoxb)mV(esyNbhLNk zyK;+>D0X+DkWxPSgv_^eX&2Tz@hU~731?3~8zo-_^pTE@Pf33ObQ7S@7X-rg1*1!T zmLMUxNp%bWloprazBRE{jjUJN;C-Y<#^1FXl~gRi%k};Y*Jy9R0Xdeq!4pQf%OLww zM%!_cr;GIj2SsT=)AtRZ4=H5PnS7D0ti9FW2o-D^iIsbBqwh6p-p5(wn0+509v=dY zp*BEB>VNSYdc_max^7Pw{s1u-qE`w;_E`Yt!Y@{Fs29&r%Mjxeuu1 zwxK^jXje4B8p!t3!vA(zz`+eNqF4GHHU3!=1u>uwMLeDO0m6NT{ORVo+sC{=Kw8Bj z$L9n8d+)UVyE~zaZbEi9!56K7jYw28zK}BkFlZj#?^cva?pzF-xcGXln9=Mhd5V&Y z{Qg0Gc=~EGco5jjrymgzpx}+6@I+kZa%_8jPFL$S=xmvT9B)IRkx=K&J05T{)%&Mm zwG!lh=d5pi0bDaZvK}`Ra+@JmC|`PXDOt06xWE4Khicu-)6KVsl8-su-%7+@l*xNu z|9FeWy5uZv(xefKW{uVx6DdIp8*T)~N^LN>+AEHZug%dRIVaf(+dgmq#+wSSHrNQJ zcFuOr7ap=SHxt$11B(ql7^SJ#&a|6AuM;$JA_1}%3$|qQg)N$A z;1W1vc<}jm815Juhl zIvM5Z)P=E5vh7=|=9-kH*VW0^X=bQ|B6%l1Riw*jQtErN1yO|RT6|T7(@ls)$aKu9 zltma|XL(r{G6nF1%@i1Q@WrHiEV{#j8AC$XlwQ&pwl#D!uA?UOPBjRpWAR)Fs4foQ7E20~-F# zk>g}8giqdQ{zn2BbwI&x^(&;keSE*|zwSd17_k0IwfTBH+L_FH8sW`0j&#R~03w{X z%-ex3@G(2*4!g@NCcFxyk8|6ZxKte;de)Ad#GRqNY*Js|kTF4Kz1{c{k;iJSa!Hci zrCY$oqz$W>6MAU(+3pXjeW@B2`F_Q4;{I;WHPW|X;(|S|FG_)u&%+adt+x}PEWo}4vk7NW ztQgTN*WT?7V)B{$>@#0+nd!k4H7TW|WotZ83k5hwWUp4s%~bJ4>ui)Kkk%N`pqKe? z-s(Z-V(75RsaWcEZ(eLcv}2lZu5+&2b1~oUprw%|-l=xx6d?+BBdhPH=jFROI;<98 zZ7xHRP9ZuPM*1(~r{(+X0=!wsv#4NhhAF0fd8-niT;jLrB1a}Sos=1wv_aI{lx9og z?;RZb?)3-<=fje(59GV_Fx!`oo3?RR#KLdv$x)4*@pnIIaGuleo2dE$3NBgzpIqVm z0qT#^M`0z2s$f6J1`ES}PERqpfh=bslf*Wd5GQ#|_pY%mTHg5w=#xi20GL;k9_gpb*fBRV_i+f2pjhenay0T1lm966d9z0EYzK zu=`OpmqN1&@fO&M1~geMH*78?p!$iSUbNNi;#+at??Zk)PPmc4agq#LIWNa@Hi~4R zW-Nj8J6Lv#s<+ZY0K&6P?kbZ5K&%IL^)rAX6)z(S%SA1!GdESx)sIO5#);(^M!Y zUVr#FGIM7*Uc&NOj?ZUJO2;rqQ|^TxH$|LCP-!>`zPm{}@uK|aYXfBw*A>bUX!m+1 zy4qnXnz4wmGiR$HjONy8HZQV(y)+1x3I)yg4)B@x5MNc&eHD_ZvocZ=VwXQ?QVL5S z$$Arl`EWQ5)~I;5M>n!5D|1jL?YCcIYNYu2D_6yR5-A7rXXa*0%jIy7dh>qt_gthb zc%NI9kXHCFnfd(Elg+h_`RpSVSwqzFEOp4=Yyy5OVlZE&c@mC=nOpK+$ zC8VOj5mpl31U-ZI)d_Eyr6G{GSQJ0DSyrm(I%r>I z_FyQCU)zqaMj~%w)y3`QaV}bCk6houc=89x=-gk*#ig#lxXo<*0I2|dZ2nas?Ne#S z#{!?9T5|5Ld?#?YpR5pJj2>PB%_u8-U;32ar8P;X`@c(TGJZ>IiaNC2Y;A1i*H!;{ zTJv?c&>u>HyoB{^(o4R8_snd~gt_|4a56>0E;Or)h3<9>%-OkeVae`-_GCSEzR2-8 zY6Z6F7HaF}c6Mn@SLE96vtDCVQYP;_cWnh`m8T|ei?2GlNsz|Z7G!5NWRcno(&;C zcr}$m_Bz;~D&EH{V${E}zd4Kf)=$~QDjUy_|4{^!|4-7Reg~us5?qQt3x`t5>g2vN zFCVN`5!uY))_alKc({Pvki-FXN?(dFo5}$b)9Bk-Zgr5wTCvd3LPc z2P3MnaM_0!c=2tF&p*7r*Fm@D;BV{K?eyS9(fe0E+8RQ$VunZdRjrA0YZ!;f^m`MJ zCeS=i^H0Tn7c?&r2f$ZS%VS0cwSWO#B_{F3gHHp_0~J@kw9lx%Y<4BX7Ak&&e6Jt? z)l;#HB=K8eDnS5S(P(-Z%Zqo+G7voGdM9eE;HtQ^-N)56SYqZW|7hc&0q~yU!#mBR z@elV>SQU$#0jvASReFcEu9$AP0AFLc-K(0r#y3HZzR+J52zR@4AJs8CMrOg$5#ILkb!9YHE~NAitO%b%4`!m%k7Mu}bCn>DC8^ z9uwtnM@7e*uZ-K3m@FWBz{(%Y&h$yO8w<~t&Kqx&#eNQ^qo1_as!&zlgf0^;)8Dk} zq-Ac>=j~pyF~b~&G%&Wm(GNd?i@Tvo(BC8QI>g2zl;P$!Qal|B?9*|l3C*OM+uW#9 zI_p|-r^${3dy&fpv>POQYyK3{3 z8GD*z3O*7vM0uJO2J7Xq!=JN{SV9Rl%#OWPAQgg3_CX~fSGQ1Vk+7iNK&pT-YB1e)S2%K1zIx~IV$RkeKj8}lFGhXJ>a%fNM7fv zl+_F;UC6*efOZaa+j!5uSH1 z0@e83l{&AFEZj8Fkxc*i*ch9!)1kcOlvvW@zwlGCu_@$cYN+weqM(mnXP3 ziaqx!r}Y`5Y$oFvU3$vSAI5SkJ+Khl`e{#l(j-CkSiI zx87IaXewEag?xeb$A+-?hK30J=5HTup^sy>_^zQUS`3Ze%{W%yWtEt{xiN>C4fMWb z2!qmZ?@iyB)Tu>GNi}zhDd=|Z}7fRkMmD)%C$CZ-On!a ztONdt;|KYQ{?Hq@`HplDq?SB`iKAP$aAqWoy17b2+L2L?FhgzRXX54u=V06M&Y97H zKP2eS^y#AHm#61edy_8cwMc8($W4|H9{?Z{S)Cvz`_Ddx+2s? z-N5CY?Y;<8Blmh;Tycr+iEsj2>6JK(7(I@_PN%DvyV7O7w*A~C7;b%C8*EjoH+%)- z-G!GCRV%>T21i@`Z6nR|=^IH2k(x#*=y#8l7F0Ndwa}IzlnOjmQC6e%LTDe2>pN_4 z&06@06s@sZQT92rs|uxYTus>!wd323z`|@%{`!~I8Gbxm(!X?6MI^ht!VJmDpYi$V zCe5{}yT)Nj_#WgL9DKevjwhtnaiNA9wC3hPSx@Vwqg+{54|gfAs;hwKh~JruTzna* z_&w`=HfIyvPABWB-wA}EfupaR@4JY9bkaDH8KOsI9}J1lJER{_>=4rLj*j4l`Sd>i zAox0JG_`#d32&}4p#o&cN=ht#mkonH|i)Bl#}MY)aQG17((`0@^7mIa*c{U?I`Nz z2=OVpG#?!j`kK}^rWNTvKh4=(FM!v_yc}FiX_XhRQyO*@AKEzfsx>#7OrN1)wdby- zxeXxPlbIZR(Askw2_%8tnNT`*cn^o)mFoa0_w3ku_qb66|<}P=MQy<#1cHG?H@^N?4^c` zXowm!KCn5nq_QKgOr=s-hUCE#w@`kwR|aNPu%`LsS#KXtPjtFnbcb%Lu@hoHwgfKT z-_*Y84kCfNT1iLu7G@Y(>z9kJMm@?X%h(9zGI#;vzNiz?(YOMqzI>C`>WOx9f5@{< zn9_QAkCXVICr!fR$FWtku{njm5iE|OP#!99=HeIBReWLVi1%f5)c_ZE7rGiXX~Gyo z5nTCj6D6qqFtY7RH_wNhlJA@^mAnrlzTYv1Jx{WBvtuysFPjGD_61vn)1kP117mK| zRG0kflotqiE9#24iKzhOGM-uEYU^XB~LA()K4ZIs_O_@#^U zTFCRFN|Hq9r829+j_e_a_@t-`uX@XgE@8EfBYmT&gVJw%whBg-!?NeN^yo`Wq`5z( z*2Rzt-~IZF`I|S90xS(4hnvuaZ88|U*x)|+?4%K#KF5Xj_R}@?7lOXmjBGOR9n%;> zsrP#<4Y3Su&3RDcSufvKQ^Df0lZSmG5pc?ji_Ppiu8MIoS`O(vAbmRQP4B^3Y)&oE z5WdifWdy){948$0%qf%dePOb*f`Ny}(HOL6j^Ba7L$xG9LF>Vs3%hFMx2zv86&tP> zVQ*$Hnx+V@RjLV%%ap;4pq!gRS6rO?H)vX@?Qdk4<0wW;_0?S~$4>??zcNsU$|$@# zf4dj19lrYmv~~(17r?3w08ShN)-#jH+8VH@s^P65!u@}X4^@~BY$QYj{pn`T`C4Fx z5y7pgdz50{uS9pchY|zU6Mj9p|9*o1tDoQ>{}X(uKP3*`hCm}rUJ}lJXIPw?4DZ>P zOcSPh5Kd#NaGIJot-!c5U~bT_F9SNmElVlnrEYo~$gx@GD*W~n(A0R}))+m*(2!38 z!S1993-4XXE+{K|dY5fsT?Ir_4b!`@#0gvLD&s2MAE0Ql;I+2n) z`bQ;8oAym9w0m;Y?Z6B)m>`#pC~2LM9u$$i{p|SuJuV4muR3%TX^A(E6a|k|kPQlI zpj*k9^9X-Sq(_uAc?;m-m}&kQsTpeW#`!{qyD#tgm6eC)e}8=Osw(Z;kePyD5(mDT30K8TH34?&xek}R}RQUop z)6kqaY6jHIO=9@20#N4=Qz7iX1_PknA^9f+fW6Ya!XA-HG*JcW5BNH47W@DSNaMq= zpuXROe!~IJ{UKAzucBIH_}5-AhLyQ>)_&Yr=I3aiggvDi=9zr=gvr;atTqvklF(NI<{!owp+VCP`<}O#H3dQ0kiT&3{H($wT z(tYQ{RW#rb+7P<6L!O5|?uE~NSkg#KR9`GfUZ9QUdVj0qWV))bq~o?=Qz*CQxqBLX z_AjSrrftfnh2xXMu9g|J4J~B$$qXPxTo%yaH7Dz2#`&O*#47R$IjvWjj(|yszSvExqRyl{lJ^wJC!w zR@b0kEd%A}K6odl77vgfHol7CIBcAhHkAcn3J^O!UkHNAn z`Apslgiahn51qlom9gjmt;I3%$&Y?ZS+c5cRcZO(B)z^=O!KDIfw?fCo&#nG4c-r@ zv8qVEh64xN2=56f59;xR^>*4s&-G>8Gq#P4(C@w3X--yr^u9rr<=<*a`$>(gLKzY6pzh+)-9_mDUUjMCF zQGpt=v)$&h)9wrO<3_XF*@M3RR9Kjia)Mf)>$WY;=C^~@p;kkQPw}Ei89=_=;?0hu zep9RT{4A%!Jib)(s-n5pPUBg3VUU!Y>W2b-9y^&=a>UxnCNOQS%!1s@h2{hLiRkFl z{OnqU3x#B_35UUY`6>qe;9mH=<&Zb zYU2MFqo)5f+3HVYp;xBj!^o%URvKP&b>xsM)-M!wi(>W#Y16@Rwaa=W_Nh$bBgC?9 ze0Q5S8$MAzq)%u$(^Z6w09@p@2m?Pl#R9=?5MzkbwNpv14003#dP&T4bZYW$Rbh?!9QgLi_JS_FW&~=|jMj^}#1h$jsNGhDo zx#ba5(H8rT`U9cDh_@xL8_gmVt=H%uVj2C(vRP?PUP9Pf7(eRy3tSJ@5BXGOZT~WD zFX=!7%n;CQv1g$DnkWjbA!Em(C#B^R z&e}Kfm&d4MzQw5|_uF-EjaQI{VCr#&q27GPNGL9)RfKXHDTkG7fRb1D7_Z`U82-Ak6aH z5Hru|r@Xt8IqC;r@C`F;_=iZA`QSVAIIBi*ccWJ0?}jAg@F;RA-hX-dc4%vP zR7Hv=o&*P09R)0h?1f{9Y$He>Gs_Eu-0vYtS?;%jJv$gd?0io`_QxJjzPcy>97i%Q z3m{cFaUyNr+(|C>%=mD%coUtW2#j6Y0zM&W22NE~6B(}Ab zB(vI+%b`?6v!s=I`S(We+3d9vE>1ffHL$c>M3ef z(d8JE;@~B40$%AR&Ibc_PlWImSwP!~Xg-GYPysE8nm_?E-cz#ER>~{ludeD-~ZGZ{}95Vw6i7KA&Wct_Oj!7 z2Ztjx)AUlfL2xrO(r^LR=fY~_lyT(W-9;K$o6aY!-$hQPX&j;1^4ZZqO)t_lYS?|N z#%kMfIcNwIrN3V_H~@G;lnR{OuPe=#xT5tH0^sXgIBh1TU5Z0@txljcJZ4;f=)Zn7ZmDG<%WpW!4@7kX$^ za9oD8(k2AePbM>l#kKjak&3A5jujQ2dXJz1eCV7^>VgSH%ZyMYo7#dAEE z$x)*B0~8Y*85*-frW42HwDL>x@apjQ6jp-J{5Bmy07r)o%uA~eYJOBRB=k1@RChS< zs;qWcVoqN+$NPJvqsxgRfPqRPe;agsw{6vK-w>>kaH8+?c@-wC59~DP;NR`fjwjE~ zv+7lE`z;S3d3ZvYG55&E%W`#iEsb;IWOFPlsYK7;f9|izT9CDHDrtn{2@n+53~?-v znm0Kw2Y=u=zjAeC^l4~q_6D)zePB{r;Q25Wy)o~tOIB+$H^cR?CLtrzFSD*HVIv2d z9J}2tglZ+FlX@0|xb$|U#wET_tv%TZGH|$UT9t!$TTu4Jxvs0$;$FUv4~=bBR<5xX zZ$%$C>)RyX)%ysRLGXO-z122B9Bh68dv3g7*ze@R6$PuA=94qyDKU*LpY-N7O#Gb5 zn>*2Ua8jLONua{Xmn=Zu0$c5xfd$^9IctVUr)kY1&e>}ny)C**1hU;iy+Vr9I6&S^ zPTfG9cxWR3scrz+NGbRLTb8PJ9*)f6iq)7^IN137$N-S99E)@a0K2_1M2M_Cuw7mb zIo5XsZuII~ zWl0$tm<4&lM_$wY!=o&}%ETKg`&mIti^f~@uEWlw7*)?%p)DklCTMlI-IGCW6j?hu zswrSQb`tYbeZX|!|G!9mkmvo94D9*K`i8(;O+djz9i10f%5vvX{B9?FxqbOUGDcJM zKB?au?jy3KtWW&yEi6gGlTB`Yd7sj`Gkl~LnIA_Nf?OGI9YLhNhrE}5iIz?q-Fqiy zBDz>uYdkoGD6_SQcv-vieSle?{h_7fC5OjnN|8ro_3%CREAwCVp5J%*vY)AGWR+YE z5nnz`u7|!!Lflcd1!qaeBz;;tpn*-)l}frg-Qf1Yh{y04FE8nJaLVrLmQqu%%rFe3 ze%qsXu#sn5;qIW6D10EO$(0!C6zSB1OwU937!~5xoI`~dbWE;7{23}PKV7?_`{7~a zRFg#nH+O9l!?}dWThIpt@#XnGKVWA5%MBpfW~`dp*+*e+8gTPR8wE!mx*E0}dV5jc zl7bCnx~l^kGb-28^n7mKb?Eg9sVo2ch6uOAJb+;pfEsuhYIL`Mzf;hBIA70gP&o2n z)Z2-!<+)EK`CYE_oIqsmGsTn4muF`8NTkl!QEd_D%3AuwLrY5@JgtW@7SnSTErz}F zh~Gg4>L9Z8@P%Z4rTI=z2J%v|rN>3S>N|wht?QLzSCGTco<@HWRgqe%h3)0LLN80c zQ^mh`fwP#wxI2O$U2U}+i!vkb`rVl#abA;y?}eoJJBnK=JG54sN~)T!Z7c2@O4b#4 z8W>+vy{G!5Y$Qg$V-R^}PJ?+zPDKSf6Jh3d@y4Cayl4JTxaed|F5m3na`4bMg5D-v zuByWYR|MMb<{k##$_ZbI=S4u^juLp!Zn4+_= z;PBTY^|{bUCuhDs@nE#nXp^o`c;npORio3-O*-~&)E_h4-*=oDzrM^rHuoC7^H4m= zptWlvU%STC!*%L}zu=qQ=+2f_HP+3~-p093bm2jXryRMbC&16RA58aQB>i$JKgi&# znO_5lO4`wYD1e7zPugkfW`j#vY^a(&-bcoK6GeP|!3pV^i1cbn)QOkd-+F}Zkz+#IiQ!?_lY@i|0hYN^H8a`O(jL>)L)`wpMxPY&nz zg~vGZwZ|AJS<0(yt9Yk~+}@fN0Cu#5L?@N|H&cs?ibJ3K%+OtA)yg9Ooc@|D(@K|3 zxcNjtqjXUAXv;6N_OVyml7isQku?Ld*lgLY?P-{(s@&SXH{@MZ^i$RD1e2NVJ`=mX zqKrtN;_?LVH&9h#=Pd4M0d;&@1X#R#92Ng%3~{2;{{uu+t47U7V2RwoPC;bWXF^{w zFd!rzL*nlT@&0o5-+WR3bmsn_3+6y8LGH4CD};f_4Y=r}l!p71p|4m+wA*K`NY~P2 zTr0ToG|T*o(xM{UVF$wW&U&H~6s{iw1w1E~;+*mk6%EzG!LS86MRi%j{QV-}`=s{X zYuV+xt}^i&6BPIg3SaoTL!c6MxlKA{aq)g#eVjWY<Q>nTimYYsFi+ zhhz9;H8DM!nqGXuSI?D|3~qey7L)iQe7BpmG;~)&nlfn$(p>lkN`n+}o2wIPVHf%W z*jByyUK!at(bQ|?nNY%I7ZDtATl6_)Tmz}2Z{ilX%#zhUa-rBpSzq%C9rvSK=(=tJ zfs^2vZG$%HvbqD6*16(`E|ey+4>1$3>?UA^rV}zLMu2{l(9$3q9Stufgptl&&IFSwl;Hh&M0!b%)TLV0Pvo+g&{`Rne{>i}MR7g%> zSN)Ipr+Us7=+R~UJy|3m4sNq-rv>5$7#x?_7qTU6i{#8RF|_8(=sx#hM~D!?6fi_(nL*INs+6NY&Gxa_tH%cB4ZN14otqa@YHrnIPKplT$0=pbERbKo;oixk;vt zD^DDW_p2>0;a};>GmBG`BGq{B8r9<+(Kt56#?1F(op-`ouddn%>N2amnX|q)%-86> zqc^hr0l$oi^+DR29iXZ32@3s|qhcz9;tF17PI4O#Qr18a5{BssRmE2XJ?Qm-{UNnVLuTjPe!-+Bn%6|xy#-6!X~fGu+L!Ii&zgAjI3 zeSjUs7U;=^)!E~R@IA>olsOhm2>e0}SRDtSghDij40(Sp()@bX_znb7tAWyggP6ZU zeTN_IVeJXQ%&5~>fK^lBipXQ2xc?6yi`BW=-}zYh0k|T5@a+Xe3DJbm`!h*hE?Lk_ zaGA%bZ--a3H}<`#_D`8xN!1&oSFU(!Cl31l;5z@&&0I6K4mp8VZxJK}AD4shRSece zT^fht(~g{v?3|t+40%I#XWbYyQaJ0rItFMTs-tIVpPj!^#+()0sF zhrCyVYdw5phrw>dQwIp?$>i)A@c8ozhhKyUB-mHZLzPq^>!^rG$T_ zo$TfV9Ah(yT$U>M2mrT80tWczh)*g%C6oduL11zS)Cgz~^1uGIHz>#Z3t*!c2>EGk zP)q%jGhRWQ9DSo-PaKBs+(Ceeb!GOynjP#e`~=$w9CIGqH=~cP^dAvz1R`;TzRrDY zD;o2shM0p9vIs6w*r5nqex+A={IWl}XVAKm6T4_JP9k&O>wa@%b%?ZbiL8;3h~2dl zMTa|_f;yos95$95tXDG}0Z6h_;;j#vjE=Xk*}Y(W4z;O3IPa|m+s24oaSOg zioj>sH2a97l_&b~;!Ag(_^JofUYu3JY%@h3F%`c0?cc@_Uhep!7I>7H(m+y_}X z)5uZD!$Bvmfy6|b28yWH(ovQ1P;9|AYTko77y=L8@NKeeMw|bi>aH}ZsccK9L_|OZ zlqf0^N-P0U0g*`|0WBHJVipRNpcIn`sDPve5|FVNj8dS21!0ugFhypfOu+$xATr2| zGRTw!2pKPZ@U>4@Tm4qO*S%JE|H!(3a(8m?IeVWyeEZu2!vPPOUH+2`+*;Mv6O&|T z;`jwye2c;DLyj+wOUn+h9wnGKc)VKZVOG=4$D#v8e)g-MQdhL+$Omgq-)NAawM2yl z>3WP69}BE~==r^@vCm_fk7MDtsR<17tHIJ%Gttn28k1x0#O6||n52&;q+fhZ>?K z#LnS@)~vQ!+6dVEBOlz*Ie*sm*^G$=xl<7+3J(Ihs#mOWV9zEdmQ;XdJw`nM(k_U; zQT$XPP8f9mKUyD-1a4$u2yhT&;Sck1FY)~FE2!my6V#$1Q|LTyPVORLskZ3NPh5jF zP(gsAp2aibWO2U30|bMP)!)XN@ms_q5X9mrAVd$W2n<9h<~PlSni1bXuSY)7r+jT= z*z^e^FNp*hE|09!rDQGf`mC?F-DzIZZp^%QNf-7UviZU7<5nIgu- z=Ounwwh|(i{S^(&;CPQJ%f#Lv`x%62lqXKu5rYH`o~!`pqRV?SNJ-00*D@Vnx~MmX zkuGG4GD`Pemv+|HJ;Y4rIv%j(!OuZ78oObO_xaXUAM5uAjbQ&R) z^MkO7^@O%Jl{}R9pR$DwSH8tZw|~J$8~*q3(SX9?Uo{woj`@MccW;)dAfg`!U8)z( z>4yvw8`M8yPqWt9A;maT{IhJ&@h6epVSV_U9hD)*c|iY0NjBT0Px)B8|k6S|ax zH6r(UaMp{2vmMs|;Rj@f{KMMN)_Q|`)L$Ko;2_y|dekczu%#j@Zq`urs= z&H;=2skpUH2mCjRrmt-b8QbGNJH9CJ)6+S35~k_K;JKkvwg5hZ_#JU+isDXcNm}g3 z8ZP%nnL!eY4moQu6(XFU5%i1i7$SV&ea)5aLzios(R~BIif04_4jJ*&4m;t5FEt(5 z5>jsjpSON%b1@PY7j{dnOdN%MGJgw%A7LTs*+`@qFhHJk>klZbA~K#+E0L zI@-NVn*bZrHF|6lBu-xpU(j*!%H|#KK1>xoBF{ZvJZM`kEYEoIq{W#$MG<_mT7Z&T z@d*t;Y|%L*6p7t_Ipts;(`~LMyg`yHd1JBq>g0qw<0*_c5><~)j`h5D)w|hvs%-PC zS_4MNdqwZ64y~CKhgb~YGY-t7y8m&JdS*+J?q;1md=o;<9ZZZ=!JEt4`bq;8Sv$My z*_KaoUm_(F!e<(@y$-J4@(Nm2v0HNEH#@lCpIgKFqT63Nk4IZ1)ga#PBUGH&?cmQlr2Yt!rI!+FVwN6BR~`D`y0-kf;I;dZi?`I zp$?Fay#Ee>`=rx~2xb;%mBl@_6)zJJPC7+cZ|VCH}ciVvYC6=~}2TnvQme z-Ind$g@Khrgp?^ZQ%<8L*iB+OpL zs*y-rXz1uIH5+m{00*;~u@DQ+8kT<+cf3H`vWSW`e9;jh*S=ug34t^SSSAX8qa5$y zdYdX(d&IIhp5a3yPOEV>&A=OVCohH?GQELUiR{9SYykdi^bu3&7Gh9&#R^J8bP#w4 zky4&OJDPL`PSlGrL$S`+gheh4sYGhs68A0ewLb`1X2&`vBrqiV~KV#71%T zpx47kX<{)eX1iH?O{?DW6pIFp19gkTT^;I=PPz^ql|?7dr~c;TEc9~s78tB@Rz~o9 zbA!#}4S~OD`H228hAsYWrXbi#OKnR-+EGGz9!_`bI4HHW&P!L`lD8ry{OIp5m-l_h zGofPnuU|1vf6>KFap-_*Y3niT9WicSmWF&7E7Dc}R*r`Br#-5%=7~*p0KMCS>pX>e zPX46Js(@pBGvK-bS_p-q{=PXRzbq7p)th&MV@F?X0l~>%gPHwHLU5mD?Be9}m%s*W zN!iFKuFrX1mE0yQX;GJOQU+!X^cDkW-k%rcU!+L^8bO)u^=BEGJ-JYPP)Uspw!EJ;=s+L;Wy z#L!VRa;7V<;7!tecY4USA;q&t;GF8^XJ;hMzgzzp-P~iZAV6t2##2M=2s(6}t9`Hg zT=(Rz6t9=~q@$7&r8Hs2drO-ttx<*^oV{4|ya01*bY@t)+q=S)bSXLQpr?n?$IjR3 zt+RkoU?od{`heCVtF(WUP@bGRTRGZsyUU=JrB;4>cR|JE1>0oiplU0tDD7c-X0BNv z691fV2M1Kd6T*%2@GJ{Z5x_^1ry%7vFtY?(!@os=f+xV1K7(-jyD>xG14~lc2{f#7 z755C;1nqyn2Alq5$gpQ{jm~34&}tRSK#duKU9tx#EQ>4$!8x-`Q3M-|=wnL;f00T* z?1!2@!XKfX{tHsjG-5Bh#13#&gE3TX* + +## 1. 简介 + +Paddle 混合精度训练(Auto Mixed Precision, AMP)是指在训练过程中同时使用单精度(FP32)和半精度(FP16),基于NVIDIA GPU提供的Tensor Cores技术,混合精度训练使用FP16和FP32即可达到与使用纯FP32训练相同的准确率,并可加速模型的训练速度。 + +更多关于Paddle AMP的介绍,可以参考[Paddle AMP官网教程](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/01_paddle2.0_introduction/basic_concept/amp_cn.html)。 + +本文档主要介绍飞桨模型在 Linux GPU/CPU 下混合精度训练功能开发。 + + + +## 2. 混合精度训练功能开发 + +Paddle 混合精度训练开发可以分为4个步骤,如下图所示。 + +
+ +
+ +其中设置了2个核验点,分别为: + +* 准备混合精度训练环境 +* 验证混合精度训练正确开启 + + + +### 2.1 准备混合精度训练环境 + +混合精度训练的加速依赖于NVIDIA显卡的Tensor Core,目前Tensor Core仅支持Compute Capability在7.0及其以上的显卡,因此在开发混合精度训练之前,首先检查显卡是否支持混合精度训练,检查方法如下: + +- 确定已安装paddle,通过pip安装linux版本paddle命令如下,更多的版本安装方法可查看飞桨[官网](https://www.paddlepaddle.org.cn/) + +``` +pip install paddlepaddle-gpu==2.2.1.post112 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html +``` + +- 进入python环境,执行如下命令: + +``` +>>> import paddle +>>> paddle.device.cuda.get_device_capability() +``` + +以Tesla V100显卡为例,执行上述命令后将打印出如下形式的结果: + +``` +(7, 0) +``` + +结果显示该显卡Compute Capability为7.0,因此符合混合精度训练所要求的环境。 + +此外,若不预先执行上述检查,混合精度训练依旧可以执行,但无法达到性能提升的效果,且在代码执行混合精度训练前会打印UserWarning,以Tesla K40显卡为例: + +``` +UserWarning: AMP only support NVIDIA GPU with Compute Capability 7.0 or higher, current GPU is: Tesla K40m, with Compute Capability: 3.5. +``` + + + +### 2.2 准备FP32训练模型 + +Paddle 神经网络训练默认采用FP32数据类型,混合精度训练是在FP32训练网络上添加必要的策略,因此,准备好可以正确完成训练的FP32网络即可。 + + + + +### 2.3 开发混合精度训练程序 + +使用飞桨框架提供的API,能够在原始训练代码基础上快速开启自动混合精度训练,即在相关OP的计算中,根据一定的规则,自动选择FP16或FP32计算。 + +依据FP16在模型中的使用程度划分,飞桨的AMP分为两个等级: + +- level = O1:采用黑白名单策略进行混合精度训练,黑名单中的OP将采用FP32计算,白名单中的OP将采用FP16计算,训练过程中框架会自动将白名单OP的输入参数数据类型从FP32 cast FP16,使用FP16与FP32进行计算的OP列表可见该文档。 + +- level = O2:该模式采用了比O1更为激进的策略,除了框架不支持FP16计算的OP,其他全部采用FP16计算,框架会预先将网络参数从FP32转换为FP16,相比O1,训练过程中无需做FP32 cast FP16的操作,训练速度会有更明显的提升,但可能会存在精度问题,为此,框架提供了自定义黑名单,用户可通过该名单指定一些存在精度问题的OP执行FP32运算。 + +通常情况下,O2相比O1训练速度会更快,但是训练精度可能会有所降低,您可以首先尝试开启O2模式,如果精度损失严重,您可以再尝试开启O1模式。下面详细介绍如何使用飞桨框架实现混合精度训练。 + +该小节以MobileNetv3模型为例,该模型混合精度训练的应用代码位于:[train.py](https://github.com/PaddlePaddle/models/blob/release/2.2/tutorials/mobilenetv3_prod/Step6/train.py) + +#### 2.3.1 开发AMP-O1训练代码 + +**【基本流程】** + +在飞桨框架中,使用AMP-O1训练训练,需要在FP32代码的基础上改动三处: + +- Step1:定义`paddle.amp.GradScaler`,用于缩放loss比例,避免浮点数下溢 + +- Step2:使用`paddle.amp.auto_cast`创建AMP训练的上下文环境,在该上下文内,框架会根据框架预设的黑白名单,自动确定每个OP的输入数据类型(FP16或FP32) + +- Step3:在训练代码中使用Step1中定义的`GradScaler`完成loss的缩放,用缩放后的loss进行反向传播,完成训练 + +**【实战】** + +MobileNetv3中添加O1混合精度训练如下所示,参考链接:[train.py](https://github.com/PaddlePaddle/models/blob/release/2.2/tutorials/mobilenetv3_prod/Step6/train.py)。 + +1)在执行训练前,初始化`paddle.amp.GradScaler`类: + +```python +if args.amp_level is not None: + scaler = paddle.amp.GradScaler(init_loss_scaling=1024) +``` + +其中:```amp_level = 'O1'``` + +- ``paddle.amp.GradScaler``使用介绍见[API文档](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/amp/GradScaler_cn.html) + +2)在训练代码中使用`paddle.amp.auto_cast`创建AMP-O1训练的上下文环境,并利用`GradScaler`完成loss的缩放: + +```python +if amp_level is not None: + with paddle.amp.auto_cast(level=amp_level): + output = model(image) + loss = criterion(output, target) + scaled = scaler.scale(loss) + scaled.backward() + scaler.minimize(optimizer, scaled) +``` + +其中:```amp_level = 'O1'``` + +- ``paddle.amp.auto_cast``使用介绍见[API文档](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/amp/auto_cast_cn.html) + +#### 2.3.2 开发AMP-O2训练代码 + +**【基本流程】** + +O2模式采用了比O1更为激进的策略,除了框架不支持FP16计算的OP,其他全部采用FP16计算,需要在训练前将网络参数从FP32转为FP16,在FP32代码的基础上改动四处: + +- Step1: 定义`paddle.amp.GradScaler`,用于缩放loss,避免浮点数下溢 + +- Step2: 使用`paddle.amp.decorate`将网络参数从FP32转换为FP16 + +- Step3: 使用`paddle.amp.auto_cast`创建AMP上下文环境,在该上下文内,框架会将所有支持FP16的OP都采用FP16进行计算(自定义的黑名单除外),其他OP采用FP32进行计算 + +- Step4: 在训练代码中使用Step1中定义的`GradScaler`完成loss的缩放,用缩放后的loss进行反向传播,完成训练 + +**【实战】** + +MobileNetv3中添加O2混合精度训练如下所示,参考链接:[train.py](https://github.com/PaddlePaddle/models/blob/release/2.2/tutorials/mobilenetv3_prod/Step6/train.py)。 + +1)Step1定义`GradScaler`与O1模式一致,Step2通过`decorate`进行参数类型转换的方法如下: + +```python +if args.amp_level is not None: + scaler = paddle.amp.GradScaler(init_loss_scaling=1024) + if args.amp_level == 'O2': + model = paddle.amp.decorate(models=model, level='O2') +``` + +- ``paddle.amp.decorate``使用介绍见[API文档](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/amp/decorate_cn.html) + +2)Step3通过`auto_cast`创建AMP-O2训练的上下文环境与O1模式一致,Step4利用`GradScaler`完成loss的缩放与O1模式一致: + +```python +if amp_level is not None: + with paddle.amp.auto_cast(level=amp_level): + output = model(image) + loss = criterion(output, target) + scaled = scaler.scale(loss) + scaled.backward() + scaler.minimize(optimizer, scaled) +``` + +其中:```amp_level = 'O2'``` + + + + +### 2.4 验证混合精度训练正确开启 + +**【基本流程】** + +开启混合精度后,可以通过开启`GLOG_v=5`查看Op的执行是否满足预期,默认情况下`matmul`、`conv2d`均应执行在FP16下。 + +1)AMP-O1模式下,训练执行过程中打印的日志包括如下内容: + +- `Auto mixed precision run operator: xxx`:表示名字为xxx的op执行在混合精度训练O1模式下 +- `expected_kernel_key:data_type[::paddle::platform::float16]`:在`Auto mixed precision run operator: xxx`的log后,会打印该log,表明op实际执行了FP16 kernel + +2)AMP-O2模式下,训练执行过程中打印的日志包括如下内容: + +- `Pure fp16 run operator: xxx`:表示名字为xxx的op执行在混合精度训练O2模式下 +- `expected_kernel_key:data_type[::paddle::platform::float16]`:在`Auto mixed precision run operator: xxx`的log后,会打印该log,表明op实际执行了FP16 kernel + +**【实战】** + +MobileNetv3中添加O2混合精度训练如下所示,参考链接:[train.py](https://github.com/PaddlePaddle/models/blob/release/2.2/tutorials/mobilenetv3_prod/Step6/train.py)。 + +1)AMP-O1: + +训练代码执行脚本如下: + +``` +GLOG_v=5 python3 train.py --data-path=./ILSVRC2012 --lr=0.1 --batch-size=256 --amp_level=O1 +``` + +以`conv2d`op为例,打印的部分混合精度训练日志示例如下: + +``` +[1] I1228 11:56:41.567701 5904 tracer.cc:162] Trace Op: conv2d +[2] I1228 11:56:41.567713 5904 tracer.cc:190] Auto mixed precision run operator: conv2d +[3] I1228 11:56:41.567716 5904 amp_auto_cast.cc:202] Op(conv2d): Cast Filter from float to float16 +[4] I1228 11:56:41.567724 5904 tracer.h:112] set amp_level to 0 +[5] I1228 11:56:41.567728 5904 tracer.cc:162] Trace Op: cast +[6] I1228 11:56:41.567735 5904 layer.cc:451] Op(cast): Inputs: X{conv2d_33.w_0[LoDTensor]}, Outputs: Out{dygraph_tmp_67[LoDTensor]} +[7] I1228 11:56:41.567746 5904 prepared_operator.cc:161] expected_kernel_key:data_type[float]:data_layout[ANY_LAYOUT]:place[CUDAPlace(0)]:library_type[PLAIN] +[8] I1228 11:56:41.567770 5904 layer.cc:481] Op(cast): Inputs: X{conv2d_33.w_0[LoDTensor]}, Outputs: Out{dygraph_tmp_67[LoDTensor<::paddle::platform::float16, CUDAPlace(0), (40, 144, 1, 1)>]} +[9] I1228 11:56:41.567786 5904 tracer.h:112] set amp_level to 1 +[10] I1228 11:56:41.567790 5904 amp_auto_cast.cc:202] Op(conv2d): Cast Input from float to float16 +[11] I1228 11:56:41.567797 5904 tracer.h:112] set amp_level to 0 +[12] I1228 11:56:41.567801 5904 tracer.cc:162] Trace Op: cast +[13] I1228 11:56:41.567806 5904 layer.cc:451] Op(cast): Inputs: X{auto_228_[LoDTensor]}, Outputs: Out{dygraph_tmp_68[LoDTensor]} +[14] I1228 11:56:41.567814 5904 prepared_operator.cc:161] expected_kernel_key:data_type[float]:data_layout[ANY_LAYOUT]:place[CUDAPlace(0)]:library_type[PLAIN] +[15] I1228 11:56:41.567828 5904 layer.cc:481] Op(cast): Inputs: X{auto_228_[LoDTensor]}, Outputs: Out{dygraph_tmp_68[LoDTensor<::paddle::platform::float16, CUDAPlace(0), (16, 144, 1, 1)>]} +[16] I1228 11:56:41.567842 5904 tracer.h:112] set amp_level to 1 +[17] I1228 11:56:41.567847 5904 layer.cc:451] Op(conv2d): Inputs: Filter{dygraph_tmp_67[LoDTensor<::paddle::platform::float16, CUDAPlace(0), (40, 144, 1, 1)>]}, Input{dygraph_tmp_68[LoDTensor<::paddle::platform::float16, CUDAPlace(0), (16, 144, 1, 1)>]}, Outputs: Output{auto_229_[LoDTensor]} +[18] I1228 11:56:41.567858 5904 prepared_operator.cc:161] expected_kernel_key:data_type[::paddle::platform::float16]:data_layout[ANY_LAYOUT]:place[CUDAPlace(0)]:library_type[CUDNN] +[19] I1228 11:56:41.567873 5904 conv_cudnn_op.cu:96] Compute ConvOp with cuDNN: data_format=NCHW compute_format=NCHW +``` + +- 第1行:表明即将执行`conv2d` + +- 第2行:表明以混合精度训练O1模式执行`conv2d` + +- 第5、12行:调用`cast` op将conv2d的输入参数转为FP16 + +- 第17行:打印了`conv2d`的输入、输出数据类型、维度等信息 + +- 第18行:表明`conv2d`实际执行的kernel类型为FP16 + + +2)AMP-O2: + +训练代码执行脚本如下: + +``` +GLOG_v=5 python3 train.py --data-path=./ILSVRC2012 --lr=0.1 --batch-size=256 --amp_level=O2 +``` + +以`conv2d`op为例,打印的部分混合精度训练日志示例如下: + +``` +[1] I1228 12:00:45.373405 7576 tracer.cc:162] Trace Op: conv2d +[2] I1228 12:00:45.373430 7576 tracer.cc:193] Pure fp16 run operator: conv2d +[3] I1228 12:00:45.373438 7576 amp_auto_cast.cc:283] Op(conv2d): Cast Filter from ::paddle::platform::float16 to ::paddle::platform::float16 +[4] I1228 12:00:45.373447 7576 amp_auto_cast.cc:283] Op(conv2d): Cast Input from ::paddle::platform::float16 to ::paddle::platform::float16 +[5] I1228 12:00:45.373461 7576 layer.cc:451] Op(conv2d): Inputs: Filter{conv2d_5.w_0[LoDTensor<::paddle::platform::float16, CUDAPlace(0), (72, 16, 1, 1)>]}, Input{auto_1196_[LoDTensor<::paddle::platform::float16, CUDAPlace(0), (16, 16, 56, 56)>]}, Outputs: Output{auto_1200_[LoDTensor]} +[6] I1228 12:00:45.373487 7576 prepared_operator.cc:161] expected_kernel_key:data_type[::paddle::platform::float16]:data_layout[ANY_LAYOUT]:place[CUDAPlace(0)]:library_type[CUDNN] +[7] I1228 12:00:45.373535 7576 conv_cudnn_op.cu:96] Compute ConvOp with cuDNN: data_format=NCHW compute_format=NCHW +``` + +- 第1行:表明即将执行`conv2d` + +- 第2行:表明以混合精度训练O2模式执行`conv2d` + +- 第5行:打印了`conv2d`的输入、输出数据类型、维度等信息 + +- 第6行:表明`conv2d`实际执行的kernel类型为FP16 + + + +## 3. FAQ + +如果您在使用该文档完成混合精度训练的过程中遇到问题,可以给在[这里](https://github.com/PaddlePaddle/Paddle/issues/new/choose)提一个ISSUE,我们会高优跟进。 + + + +### 3.1 通用问题 + +* 混合精度训练存在如下报错:某个op期望输入数据为FP32,但输入的输入为FP16 + +该问题出现的原因可能是您在组网过程中手动调用了`paddle.cast()`或者`astype()`接口将参数强制进行了数据类型转换,使得混合精度训练失去了对该参数数据类型的管理。建议由框架自动对参数类型进行管理。 + +* 训练结果存在nan/inf: + +由于FP16可表示的数据动态范围较小(最大65504),首先需要确定组网过程中不要出现将参数乘较大系数情况。如果没有上述问题,可通过官网提供的[check_nan_inf](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/flags/check_nan_inf_cn.html)工具进行初步检查,确定出现nan/inf的op,而后可以尝试将上述op添加到`paddle.amp.autocast()`的自定义黑名单中,使该op使用FP32 kernel进行计算。 +`paddle.amp.auto_cast`使用介绍见[API文档](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/amp/auto_cast_cn.html)。 + -- GitLab