Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
PaddlePaddle
book
提交
05002b87
B
book
项目概览
PaddlePaddle
/
book
通知
16
Star
4
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
40
列表
看板
标记
里程碑
合并请求
37
Wiki
5
Wiki
分析
仓库
DevOps
项目成员
Pages
B
book
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
40
Issue
40
列表
看板
标记
里程碑
合并请求
37
合并请求
37
Pages
分析
分析
仓库分析
DevOps
Wiki
5
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
05002b87
编写于
1月 22, 2017
作者:
T
Tao Luo
提交者：
GitHub
1月 22, 2017
浏览文件
操作
浏览文件
下载
差异文件
Merge pull request #51 from luotao1/bug
bug fix
上级
d594c57d
13274fa9
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
5 addition
and
5 deletion
+5
-5
label_semantic_roles/image/stacked_lstm.png
label_semantic_roles/image/stacked_lstm.png
+0
-0
understand_sentiment/README.html
understand_sentiment/README.html
+5
-5
未找到文件。
label_semantic_roles/image/stacked_lstm.png
查看替换文件 @
d594c57d
浏览文件 @
05002b87
245.7 KB
|
W:
|
H:
251.0 KB
|
W:
|
H:
2-up
Swipe
Onion skin
understand_sentiment/README.html
浏览文件 @
05002b87
...
...
@@ -11,7 +11,7 @@
}
</style>
</head>
<body
class=
"theme theme-white"
>
<div
style=
"visibility: hidden; overflow: hidden; position: absolute; top: 0px; height: 1px; width: auto; padding: 0px; border: 0px; margin: 0px; text-align: left; text-indent: 0px; text-transform: none; line-height: normal; letter-spacing: normal; word-spacing: normal;"
><div
id=
"MathJax_SVG_Hidden"
></div><svg><defs
id=
"MathJax_SVG_glyphs"
><path
id=
"MJMATHI-6E"
stroke-width=
"1"
d=
"M21 287Q22 293 24 303T36 341T56 388T89 425T135 442Q171 442 195 424T225 390T231 369Q231 367 232 367L243 378Q304 442 382 442Q436 442 469 415T503 336T465 179T427 52Q427 26 444 26Q450 26 453 27Q482 32 505 65T540 145Q542 153 560 153Q580 153 580 145Q580 144 576 130Q568 101 554 73T508 17T439 -10Q392 -10 371 17T350 73Q350 92 386 193T423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 180T152 343Q153 348 153 366Q153 405 129 405Q91 405 66 305Q60 285 60 284Q58 278 41 278H27Q21 284 21 287Z"
></path><path
id=
"MJMATHI-69"
stroke-width=
"1"
d=
"M184 600Q184 624 203 642T247 661Q265 661 277 649T290 619Q290 596 270 577T226 557Q211 557 198 567T184 600ZM21 287Q21 295 30 318T54 369T98 420T158 442Q197 442 223 419T250 357Q250 340 236 301T196 196T154 83Q149 61 149 51Q149 26 166 26Q175 26 185 29T208 43T235 78T260 137Q263 149 265 151T282 153Q302 153 302 143Q302 135 293 112T268 61T223 11T161 -11Q129 -11 102 10T74 74Q74 91 79 106T122 220Q160 321 166 341T173 380Q173 404 156 404H154Q124 404 99 371T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Z"
></path><path
id=
"MJMATHI-78"
stroke-width=
"1"
d=
"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z"
></path><path
id=
"MJMAIN-2208"
stroke-width=
"1"
d=
"M84 250Q84 372 166 450T360 539Q361 539 377 539T419 540T469 540H568Q583 532 583 520Q583 511 570 501L466 500Q355 499 329 494Q280 482 242 458T183 409T147 354T129 306T124 272V270H568Q583 262 583 250T568 230H124V228Q124 207 134 177T167 112T231 48T328 7Q355 1 466 0H570Q583 -10 583 -20Q583 -32 568 -40H471Q464 -40 446 -40T417 -41Q262 -41 172 45Q84 127 84 250Z"
></path><path
id=
"MJAMS-52"
stroke-width=
"1"
d=
"M17 665Q17 672 28 683H221Q415 681 439 677Q461 673 481 667T516 654T544 639T566 623T584 607T597 592T607 578T614 565T618 554L621 548Q626 530 626 497Q626 447 613 419Q578 348 473 326L455 321Q462 310 473 292T517 226T578 141T637 72T686 35Q705 30 705 16Q705 7 693 -1H510Q503 6 404 159L306 310H268V183Q270 67 271 59Q274 42 291 38Q295 37 319 35Q344 35 353 28Q362 17 353 3L346 -1H28Q16 5 16 16Q16 35 55 35Q96 38 101 52Q106 60 106 341T101 632Q95 645 55 648Q17 648 17 665ZM241 35Q238 42 237 45T235 78T233 163T233 337V621L237 635L244 648H133Q136 641 137 638T139 603T141 517T141 341Q141 131 140 89T134 37Q133 36 133 35H241ZM457 496Q457 540 449 570T425 615T400 634T377 643Q374 643 339 648Q300 648 281 635Q271 628 270 610T268 481V346H284Q327 346 375 352Q421 364 439 392T457 496ZM492 537T492 496T488 427T478 389T469 371T464 361Q464 360 465 360Q469 360 497 370Q593 400 593 495Q593 592 477 630L457 637L461 626Q474 611 488 561Q492 537 492 496ZM464 243Q411 317 410 317Q404 317 401 315Q384 315 370 312H346L526 35H619L606 50Q553 109 464 243Z"
></path><path
id=
"MJMATHI-6B"
stroke-width=
"1"
d=
"M121 647Q121 657 125 670T137 683Q138 683 209 688T282 694Q294 694 294 686Q294 679 244 477Q194 279 194 272Q213 282 223 291Q247 309 292 354T362 415Q402 442 438 442Q468 442 485 423T503 369Q503 344 496 327T477 302T456 291T438 288Q418 288 406 299T394 328Q394 353 410 369T442 390L458 393Q446 405 434 405H430Q398 402 367 380T294 316T228 255Q230 254 243 252T267 246T293 238T320 224T342 206T359 180T365 147Q365 130 360 106T354 66Q354 26 381 26Q429 26 459 145Q461 153 479 153H483Q499 153 499 144Q499 139 496 130Q455 -11 378 -11Q333 -11 305 15T277 90Q277 108 280 121T283 145Q283 167 269 183T234 206T200 217T182 220H180Q168 178 159 139T145 81T136 44T129 20T122 7T111 -2Q98 -11 83 -11Q66 -11 57 -1T48 16Q48 26 85 176T158 471L195 616Q196 629 188 632T149 637H144Q134 637 131 637T124 640T121 647Z"
></path><path
id=
"MJMATHI-68"
stroke-width=
"1"
d=
"M137 683Q138 683 209 688T282 694Q294 694 294 685Q294 674 258 534Q220 386 220 383Q220 381 227 388Q288 442 357 442Q411 442 444 415T478 336Q478 285 440 178T402 50Q403 36 407 31T422 26Q450 26 474 56T513 138Q516 149 519 151T535 153Q555 153 555 145Q555 144 551 130Q535 71 500 33Q466 -10 419 -10H414Q367 -10 346 17T325 74Q325 90 361 192T398 345Q398 404 354 404H349Q266 404 205 306L198 293L164 158Q132 28 127 16Q114 -11 83 -11Q69 -11 59 -2T48 16Q48 30 121 320L195 616Q195 629 188 632T149 637H128Q122 643 122 645T124 664Q129 683 137 683Z"
></path><path
id=
"MJMAIN-3A"
stroke-width=
"1"
d=
"M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z"
></path><path
id=
"MJMAIN-2B"
stroke-width=
"1"
d=
"M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"
></path><path
id=
"MJMAIN-2212"
stroke-width=
"1"
d=
"M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"
></path><path
id=
"MJMAIN-31"
stroke-width=
"1"
d=
"M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"
></path><path
id=
"MJMAIN-2C"
stroke-width=
"1"
d=
"M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"
></path><path
id=
"MJMAIN-2026"
stroke-width=
"1"
d=
"M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60ZM525 60Q525 84 542 102T585 120Q609 120 627 104T646 61Q646 36 629 18T586 0T543 17T525 60ZM972 60Q972 84 989 102T1032 120Q1056 120 1074 104T1093 61Q1093 36 1076 18T1033 0T990 17T972 60Z"
></path><path
id=
"MJMATHI-77"
stroke-width=
"1"
d=
"M580 385Q580 406 599 424T641 443Q659 443 674 425T690 368Q690 339 671 253Q656 197 644 161T609 80T554 12T482 -11Q438 -11 404 5T355 48Q354 47 352 44Q311 -11 252 -11Q226 -11 202 -5T155 14T118 53T104 116Q104 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Q21 293 29 315T52 366T96 418T161 441Q204 441 227 416T250 358Q250 340 217 250T184 111Q184 65 205 46T258 26Q301 26 334 87L339 96V119Q339 122 339 128T340 136T341 143T342 152T345 165T348 182T354 206T362 238T373 281Q402 395 406 404Q419 431 449 431Q468 431 475 421T483 402Q483 389 454 274T422 142Q420 131 420 107V100Q420 85 423 71T442 42T487 26Q558 26 600 148Q609 171 620 213T632 273Q632 306 619 325T593 357T580 385Z"
></path><path
id=
"MJMATHI-63"
stroke-width=
"1"
d=
"M34 159Q34 268 120 355T306 442Q362 442 394 418T427 355Q427 326 408 306T360 285Q341 285 330 295T319 325T330 359T352 380T366 386H367Q367 388 361 392T340 400T306 404Q276 404 249 390Q228 381 206 359Q162 315 142 235T121 119Q121 73 147 50Q169 26 205 26H209Q321 26 394 111Q403 121 406 121Q410 121 419 112T429 98T420 83T391 55T346 25T282 0T202 -11Q127 -11 81 37T34 159Z"
></path><path
id=
"MJMAIN-3D"
stroke-width=
"1"
d=
"M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"
></path><path
id=
"MJMATHI-66"
stroke-width=
"1"
d=
"M118 -162Q120 -162 124 -164T135 -167T147 -168Q160 -168 171 -155T187 -126Q197 -99 221 27T267 267T289 382V385H242Q195 385 192 387Q188 390 188 397L195 425Q197 430 203 430T250 431Q298 431 298 432Q298 434 307 482T319 540Q356 705 465 705Q502 703 526 683T550 630Q550 594 529 578T487 561Q443 561 443 603Q443 622 454 636T478 657L487 662Q471 668 457 668Q445 668 434 658T419 630Q412 601 403 552T387 469T380 433Q380 431 435 431Q480 431 487 430T498 424Q499 420 496 407T491 391Q489 386 482 386T428 385H372L349 263Q301 15 282 -47Q255 -132 212 -173Q175 -205 139 -205Q107 -205 81 -186T55 -132Q55 -95 76 -78T118 -61Q162 -61 162 -103Q162 -122 151 -136T127 -157L118 -162Z"
></path><path
id=
"MJMAIN-28"
stroke-width=
"1"
d=
"M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"
></path><path
id=
"MJMAIN-22C5"
stroke-width=
"1"
d=
"M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250Z"
></path><path
id=
"MJMATHI-62"
stroke-width=
"1"
d=
"M73 647Q73 657 77 670T89 683Q90 683 161 688T234 694Q246 694 246 685T212 542Q204 508 195 472T180 418L176 399Q176 396 182 402Q231 442 283 442Q345 442 383 396T422 280Q422 169 343 79T173 -11Q123 -11 82 27T40 150V159Q40 180 48 217T97 414Q147 611 147 623T109 637Q104 637 101 637H96Q86 637 83 637T76 640T73 647ZM336 325V331Q336 405 275 405Q258 405 240 397T207 376T181 352T163 330L157 322L136 236Q114 150 114 114Q114 66 138 42Q154 26 178 26Q211 26 245 58Q270 81 285 114T318 219Q336 291 336 325Z"
></path><path
id=
"MJMAIN-29"
stroke-width=
"1"
d=
"M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"
></path><path
id=
"MJMATHI-73"
stroke-width=
"1"
d=
"M131 289Q131 321 147 354T203 415T300 442Q362 442 390 415T419 355Q419 323 402 308T364 292Q351 292 340 300T328 326Q328 342 337 354T354 372T367 378Q368 378 368 379Q368 382 361 388T336 399T297 405Q249 405 227 379T204 326Q204 301 223 291T278 274T330 259Q396 230 396 163Q396 135 385 107T352 51T289 7T195 -10Q118 -10 86 19T53 87Q53 126 74 143T118 160Q133 160 146 151T160 120Q160 94 142 76T111 58Q109 57 108 57T107 55Q108 52 115 47T146 34T201 27Q237 27 263 38T301 66T318 97T323 122Q323 150 302 164T254 181T195 196T148 231Q131 256 131 289Z"
></path><path
id=
"MJMATHI-67"
stroke-width=
"1"
d=
"M311 43Q296 30 267 15T206 0Q143 0 105 45T66 160Q66 265 143 353T314 442Q361 442 401 394L404 398Q406 401 409 404T418 412T431 419T447 422Q461 422 470 413T480 394Q480 379 423 152T363 -80Q345 -134 286 -169T151 -205Q10 -205 10 -137Q10 -111 28 -91T74 -71Q89 -71 102 -80T116 -111Q116 -121 114 -130T107 -144T99 -154T92 -162L90 -164H91Q101 -167 151 -167Q189 -167 211 -155Q234 -144 254 -122T282 -75Q288 -56 298 -13Q311 35 311 43ZM384 328L380 339Q377 350 375 354T369 368T359 382T346 393T328 402T306 405Q262 405 221 352Q191 313 171 233T151 117Q151 38 213 38Q269 38 323 108L331 118L384 328Z"
></path><path
id=
"MJMATHI-6D"
stroke-width=
"1"
d=
"M21 287Q22 293 24 303T36 341T56 388T88 425T132 442T175 435T205 417T221 395T229 376L231 369Q231 367 232 367L243 378Q303 442 384 442Q401 442 415 440T441 433T460 423T475 411T485 398T493 385T497 373T500 364T502 357L510 367Q573 442 659 442Q713 442 746 415T780 336Q780 285 742 178T704 50Q705 36 709 31T724 26Q752 26 776 56T815 138Q818 149 821 151T837 153Q857 153 857 145Q857 144 853 130Q845 101 831 73T785 17T716 -10Q669 -10 648 17T627 73Q627 92 663 193T700 345Q700 404 656 404H651Q565 404 506 303L499 291L466 157Q433 26 428 16Q415 -11 385 -11Q372 -11 364 -4T353 8T350 18Q350 29 384 161L420 307Q423 322 423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 181Q151 335 151 342Q154 357 154 369Q154 405 129 405Q107 405 92 377T69 316T57 280Q55 278 41 278H27Q21 284 21 287Z"
></path><path
id=
"MJMATHI-6F"
stroke-width=
"1"
d=
"M201 -11Q126 -11 80 38T34 156Q34 221 64 279T146 380Q222 441 301 441Q333 441 341 440Q354 437 367 433T402 417T438 387T464 338T476 268Q476 161 390 75T201 -11ZM121 120Q121 70 147 48T206 26Q250 26 289 58T351 142Q360 163 374 216T388 308Q388 352 370 375Q346 405 306 405Q243 405 195 347Q158 303 140 230T121 120Z"
></path><path
id=
"MJMATHI-64"
stroke-width=
"1"
d=
"M366 683Q367 683 438 688T511 694Q523 694 523 686Q523 679 450 384T375 83T374 68Q374 26 402 26Q411 27 422 35Q443 55 463 131Q469 151 473 152Q475 153 483 153H487H491Q506 153 506 145Q506 140 503 129Q490 79 473 48T445 8T417 -8Q409 -10 393 -10Q359 -10 336 5T306 36L300 51Q299 52 296 50Q294 48 292 46Q233 -10 172 -10Q117 -10 75 30T33 157Q33 205 53 255T101 341Q148 398 195 420T280 442Q336 442 364 400Q369 394 369 396Q370 400 396 505T424 616Q424 629 417 632T378 637H357Q351 643 351 645T353 664Q358 683 366 683ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"
></path><path
id=
"MJMAIN-32"
stroke-width=
"1"
d=
"M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z"
></path><path
id=
"MJMAIN-5B"
stroke-width=
"1"
d=
"M118 -250V750H255V710H158V-210H255V-250H118Z"
></path><path
id=
"MJMAIN-5D"
stroke-width=
"1"
d=
"M22 710V750H159V-250H22V-210H119V710H22Z"
></path><path
id=
"MJMAIN-5E"
stroke-width=
"1"
d=
"M112 560L249 694L257 686Q387 562 387 560L361 531Q359 532 303 581L250 627L195 580Q182 569 169 557T148 538L140 532Q138 530 125 546L112 560Z"
></path><path
id=
"MJMATHI-61"
stroke-width=
"1"
d=
"M33 157Q33 258 109 349T280 441Q331 441 370 392Q386 422 416 422Q429 422 439 414T449 394Q449 381 412 234T374 68Q374 43 381 35T402 26Q411 27 422 35Q443 55 463 131Q469 151 473 152Q475 153 483 153H487Q506 153 506 144Q506 138 501 117T481 63T449 13Q436 0 417 -8Q409 -10 393 -10Q359 -10 336 5T306 36L300 51Q299 52 296 50Q294 48 292 46Q233 -10 172 -10Q117 -10 75 30T33 157ZM351 328Q351 334 346 350T323 385T277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q217 26 254 59T298 110Q300 114 325 217T351 328Z"
></path><path
id=
"MJMATHI-74"
stroke-width=
"1"
d=
"M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"
></path><path
id=
"MJMAIN-30"
stroke-width=
"1"
d=
"M96 585Q152 666 249 666Q297 666 345 640T423 548Q460 465 460 320Q460 165 417 83Q397 41 362 16T301 -15T250 -22Q224 -22 198 -16T137 16T82 83Q39 165 39 320Q39 494 96 585ZM321 597Q291 629 250 629Q208 629 178 597Q153 571 145 525T137 333Q137 175 145 125T181 46Q209 16 250 16Q290 16 318 46Q347 76 354 130T362 333Q362 478 354 524T321 597Z"
></path><path
id=
"MJMATHI-3C3"
stroke-width=
"1"
d=
"M184 -11Q116 -11 74 34T31 147Q31 247 104 333T274 430Q275 431 414 431H552Q553 430 555 429T559 427T562 425T565 422T567 420T569 416T570 412T571 407T572 401Q572 357 507 357Q500 357 490 357T476 358H416L421 348Q439 310 439 263Q439 153 359 71T184 -11ZM361 278Q361 358 276 358Q152 358 115 184Q114 180 114 178Q106 141 106 117Q106 67 131 47T188 26Q242 26 287 73Q316 103 334 153T356 233T361 278Z"
></path><path
id=
"MJMATHI-57"
stroke-width=
"1"
d=
"M436 683Q450 683 486 682T553 680Q604 680 638 681T677 682Q695 682 695 674Q695 670 692 659Q687 641 683 639T661 637Q636 636 621 632T600 624T597 615Q597 603 613 377T629 138L631 141Q633 144 637 151T649 170T666 200T690 241T720 295T759 362Q863 546 877 572T892 604Q892 619 873 628T831 637Q817 637 817 647Q817 650 819 660Q823 676 825 679T839 682Q842 682 856 682T895 682T949 681Q1015 681 1034 683Q1048 683 1048 672Q1048 666 1045 655T1038 640T1028 637Q1006 637 988 631T958 617T939 600T927 584L923 578L754 282Q586 -14 585 -15Q579 -22 561 -22Q546 -22 542 -17Q539 -14 523 229T506 480L494 462Q472 425 366 239Q222 -13 220 -15T215 -19Q210 -22 197 -22Q178 -22 176 -15Q176 -12 154 304T131 622Q129 631 121 633T82 637H58Q51 644 51 648Q52 671 64 683H76Q118 680 176 680Q301 680 313 683H323Q329 677 329 674T327 656Q322 641 318 637H297Q236 634 232 620Q262 160 266 136L501 550L499 587Q496 629 489 632Q483 636 447 637Q428 637 422 639T416 648Q416 650 418 660Q419 664 420 669T421 676T424 680T428 682T436 683Z"
></path><path
id=
"MJMATHI-46"
stroke-width=
"1"
d=
"M48 1Q31 1 31 11Q31 13 34 25Q38 41 42 43T65 46Q92 46 125 49Q139 52 144 61Q146 66 215 342T285 622Q285 629 281 629Q273 632 228 634H197Q191 640 191 642T193 659Q197 676 203 680H742Q749 676 749 669Q749 664 736 557T722 447Q720 440 702 440H690Q683 445 683 453Q683 454 686 477T689 530Q689 560 682 579T663 610T626 626T575 633T503 634H480Q398 633 393 631Q388 629 386 623Q385 622 352 492L320 363H375Q378 363 398 363T426 364T448 367T472 374T489 386Q502 398 511 419T524 457T529 475Q532 480 548 480H560Q567 475 567 470Q567 467 536 339T502 207Q500 200 482 200H470Q463 206 463 212Q463 215 468 234T473 274Q473 303 453 310T364 317H309L277 190Q245 66 245 60Q245 46 334 46H359Q365 40 365 39T363 19Q359 6 353 0H336Q295 2 185 2Q120 2 86 2T48 1Z"
></path><path
id=
"MJMAIN-
33"
stroke-width=
"1"
d=
"M127 463Q100 463 85 480T69 524Q69 579 117 622T233 665Q268 665 277 664Q351 652 390 611T430 522Q430 470 396 421T302 350L299 348Q299 347 308 345T337 336T375 315Q457 262 457 175Q457 96 395 37T238 -22Q158 -22 100 21T42 130Q42 158 60 175T105 193Q133 193 151 175T169 130Q169 119 166 110T159 94T148 82T136 74T126 70T118 67L114 66Q165 21 238 21Q293 21 321 74Q338 107 338 175V195Q338 290 274 322Q259 328 213 329L171 330L168 332Q166 335 166 348Q166 366 174 366Q202 366 232 371Q266 376 294 413T322 525V533Q322 590 287 612Q265 626 240 626Q208 626 181 615T143 592T132 580H135Q138 579 143 578T153 573T165 566T175 555T183 540T186 520Q186 498 172 481T127 463Z"
></path><path
id=
"MJMAIN-34"
stroke-width=
"1"
d=
"M462 0Q444 3 333 3Q217 3 199 0H190V46H221Q241 46 248 46T265 48T279 53T286 61Q287 63 287 115V165H28V211L179 442Q332 674 334 675Q336 677 355 677H373L379 671V211H471V165H379V114Q379 73 379 66T385 54Q393 47 442 46H471V0H462ZM293 211V545L74 212L183 211H293Z"
></path><path
id=
"MJMAIN-35"
stroke-width=
"1"
d=
"M164 157Q164 133 148 117T109 101H102Q148 22 224 22Q294 22 326 82Q345 115 345 210Q345 313 318 349Q292 382 260 382H254Q176 382 136 314Q132 307 129 306T114 304Q97 304 95 310Q93 314 93 485V614Q93 664 98 664Q100 666 102 666Q103 666 123 658T178 642T253 634Q324 634 389 662Q397 666 402 666Q410 666 410 648V635Q328 538 205 538Q174 538 149 544L139 546V374Q158 388 169 396T205 412T256 420Q337 420 393 355T449 201Q449 109 385 44T229 -22Q148 -22 99 32T50 154Q50 178 61 192T84 210T107 214Q132 214 148 197T164 157Z"
></path><path
id=
"MJMAIN-2299"
stroke-width=
"1"
d=
"M56 250Q56 394 156 488T384 583Q530 583 626 485T722 250Q722 110 625 14T390 -83Q249 -83 153 14T56 250ZM682 250Q682 322 649 387T546 497T381 542Q272 542 184 459T95 250Q95 132 178 45T389 -42Q515 -42 598 45T682 250ZM311 250Q311 285 332 304T375 328Q376 328 382 328T392 329Q424 326 445 305T466 250Q466 217 445 195T389 172Q354 172 333 195T311 250Z"
></path><path
id=
"MJMAIN-36"
stroke-width=
"1"
d=
"M42 313Q42 476 123 571T303 666Q372 666 402 630T432 550Q432 525 418 510T379 495Q356 495 341 509T326 548Q326 592 373 601Q351 623 311 626Q240 626 194 566Q147 500 147 364L148 360Q153 366 156 373Q197 433 263 433H267Q313 433 348 414Q372 400 396 374T435 317Q456 268 456 210V192Q456 169 451 149Q440 90 387 34T253 -22Q225 -22 199 -14T143 16T92 75T56 172T42 313ZM257 397Q227 397 205 380T171 335T154 278T148 216Q148 133 160 97T198 39Q222 21 251 21Q302 21 329 59Q342 77 347 104T352 209Q352 289 347 316T329 361Q302 397 257 397Z"
></path><path
id=
"MJMAIN-37"
stroke-width=
"1"
d=
"M55 458Q56 460 72 567L88 674Q88 676 108 676H128V672Q128 662 143 655T195 646T364 644H485V605L417 512Q408 500 387 472T360 435T339 403T319 367T305 330T292 284T284 230T278 162T275 80Q275 66 275 52T274 28V19Q270 2 255 -10T221 -22Q210 -22 200 -19T179 0T168 40Q168 198 265 368Q285 400 349 489L395 552H302Q128 552 119 546Q113 543 108 522T98 479L95 458V455H55V458Z"
></path><path
id=
"MJMAIN-38"
stroke-width=
"1"
d=
"M70 417T70 494T124 618T248 666Q319 666 374 624T429 515Q429 485 418 459T392 417T361 389T335 371T324 363L338 354Q352 344 366 334T382 323Q457 264 457 174Q457 95 399 37T249 -22Q159 -22 101 29T43 155Q43 263 172 335L154 348Q133 361 127 368Q70 417 70 494ZM286 386L292 390Q298 394 301 396T311 403T323 413T334 425T345 438T355 454T364 471T369 491T371 513Q371 556 342 586T275 624Q268 625 242 625Q201 625 165 599T128 534Q128 511 141 492T167 463T217 431Q224 426 228 424L286 386ZM250 21Q308 21 350 55T392 137Q392 154 387 169T375 194T353 216T330 234T301 253T274 270Q260 279 244 289T218 306L210 311Q204 311 181 294T133 239T107 157Q107 98 150 60T250 21Z"
></path><path
id=
"MJMAIN-39"
stroke-width=
"1"
d=
"M352 287Q304 211 232 211Q154 211 104 270T44 396Q42 412 42 436V444Q42 537 111 606Q171 666 243 666Q245 666 249 666T257 665H261Q273 665 286 663T323 651T370 619T413 560Q456 472 456 334Q456 194 396 97Q361 41 312 10T208 -22Q147 -22 108 7T68 93T121 149Q143 149 158 135T173 96Q173 78 164 65T148 49T135 44L131 43Q131 41 138 37T164 27T206 22H212Q272 22 313 86Q352 142 352 280V287ZM244 248Q292 248 321 297T351 430Q351 508 343 542Q341 552 337 562T323 588T293 615T246 625Q208 625 181 598Q160 576 154 546T147 441Q147 358 152 329T172 282Q197 248 244 248
Z"
></path><path
id=
"MJMATHI-52"
stroke-width=
"1"
d=
"M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"
></path><path
id=
"MJMATHI-65"
stroke-width=
"1"
d=
"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z"
></path><path
id=
"MJMATHI-72"
stroke-width=
"1"
d=
"M21 287Q22 290 23 295T28 317T38 348T53 381T73 411T99 433T132 442Q161 442 183 430T214 408T225 388Q227 382 228 382T236 389Q284 441 347 441H350Q398 441 422 400Q430 381 430 363Q430 333 417 315T391 292T366 288Q346 288 334 299T322 328Q322 376 378 392Q356 405 342 405Q286 405 239 331Q229 315 224 298T190 165Q156 25 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 114 189T154 366Q154 405 128 405Q107 405 92 377T68 316T57 280Q55 278 41 278H27Q21 284 21 287Z"
></path><path
id=
"MJMATHI-75"
stroke-width=
"1"
d=
"M21 287Q21 295 30 318T55 370T99 420T158 442Q204 442 227 417T250 358Q250 340 216 246T182 105Q182 62 196 45T238 27T291 44T328 78L339 95Q341 99 377 247Q407 367 413 387T427 416Q444 431 463 431Q480 431 488 421T496 402L420 84Q419 79 419 68Q419 43 426 35T447 26Q469 29 482 57T512 145Q514 153 532 153Q551 153 551 144Q550 139 549 130T540 98T523 55T498 17T462 -8Q454 -10 438 -10Q372 -10 347 46Q345 45 336 36T318 21T296 6T267 -6T233 -11Q189 -11 155 7Q103 38 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z"
></path></defs></svg></div><div
id=
"wmd-preview"
class=
"wmd-preview wmd-preview-full-reader"
><div
class=
"md-section-divider"
></div><div
class=
"md-section-divider"
></div><h1
data-anchor-id=
"woa1"
id=
"情感分析"
>
情感分析
</h1><div
class=
"md-section-divider"
></div><h2
data-anchor-id=
"6sb6"
id=
"背景介绍"
>
背景介绍
</h2><p
data-anchor-id=
"p24n"
>
在自然语言处理中，情感分析一般是指判断一段文本所表达的情绪状态。其中，一段文本可以是一个句子，一个段落或一个文档。情绪状态可以是两类，如（正面，负面），（高兴，悲伤）；也可以是三类，如（积极，消极，中性）等等。情感分析的应用场景十分广泛，如把用户在购物网站（亚马逊、天猫、淘宝等）、旅游网站、电影评论网站上发表的评论分成正面评论和负面评论；或为了分析用户对于某一产品的整体使用感受，抓取产品的用户评论并进行情感分析等等。表格1展示了对电影评论进行情感分析的例子：
</p><table
data-anchor-id=
"ydgg"
class=
"table table-striped-white table-bordered"
>
<div
style=
"visibility: hidden; overflow: hidden; position: absolute; top: 0px; height: 1px; width: auto; padding: 0px; border: 0px; margin: 0px; text-align: left; text-indent: 0px; text-transform: none; line-height: normal; letter-spacing: normal; word-spacing: normal;"
><div
id=
"MathJax_SVG_Hidden"
></div><svg><defs
id=
"MathJax_SVG_glyphs"
><path
id=
"MJMATHI-6E"
stroke-width=
"1"
d=
"M21 287Q22 293 24 303T36 341T56 388T89 425T135 442Q171 442 195 424T225 390T231 369Q231 367 232 367L243 378Q304 442 382 442Q436 442 469 415T503 336T465 179T427 52Q427 26 444 26Q450 26 453 27Q482 32 505 65T540 145Q542 153 560 153Q580 153 580 145Q580 144 576 130Q568 101 554 73T508 17T439 -10Q392 -10 371 17T350 73Q350 92 386 193T423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 180T152 343Q153 348 153 366Q153 405 129 405Q91 405 66 305Q60 285 60 284Q58 278 41 278H27Q21 284 21 287Z"
></path><path
id=
"MJMATHI-69"
stroke-width=
"1"
d=
"M184 600Q184 624 203 642T247 661Q265 661 277 649T290 619Q290 596 270 577T226 557Q211 557 198 567T184 600ZM21 287Q21 295 30 318T54 369T98 420T158 442Q197 442 223 419T250 357Q250 340 236 301T196 196T154 83Q149 61 149 51Q149 26 166 26Q175 26 185 29T208 43T235 78T260 137Q263 149 265 151T282 153Q302 153 302 143Q302 135 293 112T268 61T223 11T161 -11Q129 -11 102 10T74 74Q74 91 79 106T122 220Q160 321 166 341T173 380Q173 404 156 404H154Q124 404 99 371T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Z"
></path><path
id=
"MJMATHI-78"
stroke-width=
"1"
d=
"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z"
></path><path
id=
"MJMAIN-2208"
stroke-width=
"1"
d=
"M84 250Q84 372 166 450T360 539Q361 539 377 539T419 540T469 540H568Q583 532 583 520Q583 511 570 501L466 500Q355 499 329 494Q280 482 242 458T183 409T147 354T129 306T124 272V270H568Q583 262 583 250T568 230H124V228Q124 207 134 177T167 112T231 48T328 7Q355 1 466 0H570Q583 -10 583 -20Q583 -32 568 -40H471Q464 -40 446 -40T417 -41Q262 -41 172 45Q84 127 84 250Z"
></path><path
id=
"MJAMS-52"
stroke-width=
"1"
d=
"M17 665Q17 672 28 683H221Q415 681 439 677Q461 673 481 667T516 654T544 639T566 623T584 607T597 592T607 578T614 565T618 554L621 548Q626 530 626 497Q626 447 613 419Q578 348 473 326L455 321Q462 310 473 292T517 226T578 141T637 72T686 35Q705 30 705 16Q705 7 693 -1H510Q503 6 404 159L306 310H268V183Q270 67 271 59Q274 42 291 38Q295 37 319 35Q344 35 353 28Q362 17 353 3L346 -1H28Q16 5 16 16Q16 35 55 35Q96 38 101 52Q106 60 106 341T101 632Q95 645 55 648Q17 648 17 665ZM241 35Q238 42 237 45T235 78T233 163T233 337V621L237 635L244 648H133Q136 641 137 638T139 603T141 517T141 341Q141 131 140 89T134 37Q133 36 133 35H241ZM457 496Q457 540 449 570T425 615T400 634T377 643Q374 643 339 648Q300 648 281 635Q271 628 270 610T268 481V346H284Q327 346 375 352Q421 364 439 392T457 496ZM492 537T492 496T488 427T478 389T469 371T464 361Q464 360 465 360Q469 360 497 370Q593 400 593 495Q593 592 477 630L457 637L461 626Q474 611 488 561Q492 537 492 496ZM464 243Q411 317 410 317Q404 317 401 315Q384 315 370 312H346L526 35H619L606 50Q553 109 464 243Z"
></path><path
id=
"MJMATHI-6B"
stroke-width=
"1"
d=
"M121 647Q121 657 125 670T137 683Q138 683 209 688T282 694Q294 694 294 686Q294 679 244 477Q194 279 194 272Q213 282 223 291Q247 309 292 354T362 415Q402 442 438 442Q468 442 485 423T503 369Q503 344 496 327T477 302T456 291T438 288Q418 288 406 299T394 328Q394 353 410 369T442 390L458 393Q446 405 434 405H430Q398 402 367 380T294 316T228 255Q230 254 243 252T267 246T293 238T320 224T342 206T359 180T365 147Q365 130 360 106T354 66Q354 26 381 26Q429 26 459 145Q461 153 479 153H483Q499 153 499 144Q499 139 496 130Q455 -11 378 -11Q333 -11 305 15T277 90Q277 108 280 121T283 145Q283 167 269 183T234 206T200 217T182 220H180Q168 178 159 139T145 81T136 44T129 20T122 7T111 -2Q98 -11 83 -11Q66 -11 57 -1T48 16Q48 26 85 176T158 471L195 616Q196 629 188 632T149 637H144Q134 637 131 637T124 640T121 647Z"
></path><path
id=
"MJMATHI-68"
stroke-width=
"1"
d=
"M137 683Q138 683 209 688T282 694Q294 694 294 685Q294 674 258 534Q220 386 220 383Q220 381 227 388Q288 442 357 442Q411 442 444 415T478 336Q478 285 440 178T402 50Q403 36 407 31T422 26Q450 26 474 56T513 138Q516 149 519 151T535 153Q555 153 555 145Q555 144 551 130Q535 71 500 33Q466 -10 419 -10H414Q367 -10 346 17T325 74Q325 90 361 192T398 345Q398 404 354 404H349Q266 404 205 306L198 293L164 158Q132 28 127 16Q114 -11 83 -11Q69 -11 59 -2T48 16Q48 30 121 320L195 616Q195 629 188 632T149 637H128Q122 643 122 645T124 664Q129 683 137 683Z"
></path><path
id=
"MJMAIN-3A"
stroke-width=
"1"
d=
"M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z"
></path><path
id=
"MJMAIN-2B"
stroke-width=
"1"
d=
"M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"
></path><path
id=
"MJMAIN-2212"
stroke-width=
"1"
d=
"M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"
></path><path
id=
"MJMAIN-31"
stroke-width=
"1"
d=
"M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"
></path><path
id=
"MJMAIN-2C"
stroke-width=
"1"
d=
"M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"
></path><path
id=
"MJMAIN-2026"
stroke-width=
"1"
d=
"M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60ZM525 60Q525 84 542 102T585 120Q609 120 627 104T646 61Q646 36 629 18T586 0T543 17T525 60ZM972 60Q972 84 989 102T1032 120Q1056 120 1074 104T1093 61Q1093 36 1076 18T1033 0T990 17T972 60Z"
></path><path
id=
"MJMATHI-77"
stroke-width=
"1"
d=
"M580 385Q580 406 599 424T641 443Q659 443 674 425T690 368Q690 339 671 253Q656 197 644 161T609 80T554 12T482 -11Q438 -11 404 5T355 48Q354 47 352 44Q311 -11 252 -11Q226 -11 202 -5T155 14T118 53T104 116Q104 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Q21 293 29 315T52 366T96 418T161 441Q204 441 227 416T250 358Q250 340 217 250T184 111Q184 65 205 46T258 26Q301 26 334 87L339 96V119Q339 122 339 128T340 136T341 143T342 152T345 165T348 182T354 206T362 238T373 281Q402 395 406 404Q419 431 449 431Q468 431 475 421T483 402Q483 389 454 274T422 142Q420 131 420 107V100Q420 85 423 71T442 42T487 26Q558 26 600 148Q609 171 620 213T632 273Q632 306 619 325T593 357T580 385Z"
></path><path
id=
"MJMATHI-63"
stroke-width=
"1"
d=
"M34 159Q34 268 120 355T306 442Q362 442 394 418T427 355Q427 326 408 306T360 285Q341 285 330 295T319 325T330 359T352 380T366 386H367Q367 388 361 392T340 400T306 404Q276 404 249 390Q228 381 206 359Q162 315 142 235T121 119Q121 73 147 50Q169 26 205 26H209Q321 26 394 111Q403 121 406 121Q410 121 419 112T429 98T420 83T391 55T346 25T282 0T202 -11Q127 -11 81 37T34 159Z"
></path><path
id=
"MJMAIN-3D"
stroke-width=
"1"
d=
"M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"
></path><path
id=
"MJMATHI-66"
stroke-width=
"1"
d=
"M118 -162Q120 -162 124 -164T135 -167T147 -168Q160 -168 171 -155T187 -126Q197 -99 221 27T267 267T289 382V385H242Q195 385 192 387Q188 390 188 397L195 425Q197 430 203 430T250 431Q298 431 298 432Q298 434 307 482T319 540Q356 705 465 705Q502 703 526 683T550 630Q550 594 529 578T487 561Q443 561 443 603Q443 622 454 636T478 657L487 662Q471 668 457 668Q445 668 434 658T419 630Q412 601 403 552T387 469T380 433Q380 431 435 431Q480 431 487 430T498 424Q499 420 496 407T491 391Q489 386 482 386T428 385H372L349 263Q301 15 282 -47Q255 -132 212 -173Q175 -205 139 -205Q107 -205 81 -186T55 -132Q55 -95 76 -78T118 -61Q162 -61 162 -103Q162 -122 151 -136T127 -157L118 -162Z"
></path><path
id=
"MJMAIN-28"
stroke-width=
"1"
d=
"M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"
></path><path
id=
"MJMAIN-22C5"
stroke-width=
"1"
d=
"M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250Z"
></path><path
id=
"MJMATHI-62"
stroke-width=
"1"
d=
"M73 647Q73 657 77 670T89 683Q90 683 161 688T234 694Q246 694 246 685T212 542Q204 508 195 472T180 418L176 399Q176 396 182 402Q231 442 283 442Q345 442 383 396T422 280Q422 169 343 79T173 -11Q123 -11 82 27T40 150V159Q40 180 48 217T97 414Q147 611 147 623T109 637Q104 637 101 637H96Q86 637 83 637T76 640T73 647ZM336 325V331Q336 405 275 405Q258 405 240 397T207 376T181 352T163 330L157 322L136 236Q114 150 114 114Q114 66 138 42Q154 26 178 26Q211 26 245 58Q270 81 285 114T318 219Q336 291 336 325Z"
></path><path
id=
"MJMAIN-29"
stroke-width=
"1"
d=
"M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"
></path><path
id=
"MJMATHI-73"
stroke-width=
"1"
d=
"M131 289Q131 321 147 354T203 415T300 442Q362 442 390 415T419 355Q419 323 402 308T364 292Q351 292 340 300T328 326Q328 342 337 354T354 372T367 378Q368 378 368 379Q368 382 361 388T336 399T297 405Q249 405 227 379T204 326Q204 301 223 291T278 274T330 259Q396 230 396 163Q396 135 385 107T352 51T289 7T195 -10Q118 -10 86 19T53 87Q53 126 74 143T118 160Q133 160 146 151T160 120Q160 94 142 76T111 58Q109 57 108 57T107 55Q108 52 115 47T146 34T201 27Q237 27 263 38T301 66T318 97T323 122Q323 150 302 164T254 181T195 196T148 231Q131 256 131 289Z"
></path><path
id=
"MJMATHI-67"
stroke-width=
"1"
d=
"M311 43Q296 30 267 15T206 0Q143 0 105 45T66 160Q66 265 143 353T314 442Q361 442 401 394L404 398Q406 401 409 404T418 412T431 419T447 422Q461 422 470 413T480 394Q480 379 423 152T363 -80Q345 -134 286 -169T151 -205Q10 -205 10 -137Q10 -111 28 -91T74 -71Q89 -71 102 -80T116 -111Q116 -121 114 -130T107 -144T99 -154T92 -162L90 -164H91Q101 -167 151 -167Q189 -167 211 -155Q234 -144 254 -122T282 -75Q288 -56 298 -13Q311 35 311 43ZM384 328L380 339Q377 350 375 354T369 368T359 382T346 393T328 402T306 405Q262 405 221 352Q191 313 171 233T151 117Q151 38 213 38Q269 38 323 108L331 118L384 328Z"
></path><path
id=
"MJMATHI-6D"
stroke-width=
"1"
d=
"M21 287Q22 293 24 303T36 341T56 388T88 425T132 442T175 435T205 417T221 395T229 376L231 369Q231 367 232 367L243 378Q303 442 384 442Q401 442 415 440T441 433T460 423T475 411T485 398T493 385T497 373T500 364T502 357L510 367Q573 442 659 442Q713 442 746 415T780 336Q780 285 742 178T704 50Q705 36 709 31T724 26Q752 26 776 56T815 138Q818 149 821 151T837 153Q857 153 857 145Q857 144 853 130Q845 101 831 73T785 17T716 -10Q669 -10 648 17T627 73Q627 92 663 193T700 345Q700 404 656 404H651Q565 404 506 303L499 291L466 157Q433 26 428 16Q415 -11 385 -11Q372 -11 364 -4T353 8T350 18Q350 29 384 161L420 307Q423 322 423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 181Q151 335 151 342Q154 357 154 369Q154 405 129 405Q107 405 92 377T69 316T57 280Q55 278 41 278H27Q21 284 21 287Z"
></path><path
id=
"MJMATHI-6F"
stroke-width=
"1"
d=
"M201 -11Q126 -11 80 38T34 156Q34 221 64 279T146 380Q222 441 301 441Q333 441 341 440Q354 437 367 433T402 417T438 387T464 338T476 268Q476 161 390 75T201 -11ZM121 120Q121 70 147 48T206 26Q250 26 289 58T351 142Q360 163 374 216T388 308Q388 352 370 375Q346 405 306 405Q243 405 195 347Q158 303 140 230T121 120Z"
></path><path
id=
"MJMATHI-64"
stroke-width=
"1"
d=
"M366 683Q367 683 438 688T511 694Q523 694 523 686Q523 679 450 384T375 83T374 68Q374 26 402 26Q411 27 422 35Q443 55 463 131Q469 151 473 152Q475 153 483 153H487H491Q506 153 506 145Q506 140 503 129Q490 79 473 48T445 8T417 -8Q409 -10 393 -10Q359 -10 336 5T306 36L300 51Q299 52 296 50Q294 48 292 46Q233 -10 172 -10Q117 -10 75 30T33 157Q33 205 53 255T101 341Q148 398 195 420T280 442Q336 442 364 400Q369 394 369 396Q370 400 396 505T424 616Q424 629 417 632T378 637H357Q351 643 351 645T353 664Q358 683 366 683ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"
></path><path
id=
"MJMAIN-32"
stroke-width=
"1"
d=
"M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z"
></path><path
id=
"MJMAIN-5B"
stroke-width=
"1"
d=
"M118 -250V750H255V710H158V-210H255V-250H118Z"
></path><path
id=
"MJMAIN-5D"
stroke-width=
"1"
d=
"M22 710V750H159V-250H22V-210H119V710H22Z"
></path><path
id=
"MJMAIN-5E"
stroke-width=
"1"
d=
"M112 560L249 694L257 686Q387 562 387 560L361 531Q359 532 303 581L250 627L195 580Q182 569 169 557T148 538L140 532Q138 530 125 546L112 560Z"
></path><path
id=
"MJMATHI-61"
stroke-width=
"1"
d=
"M33 157Q33 258 109 349T280 441Q331 441 370 392Q386 422 416 422Q429 422 439 414T449 394Q449 381 412 234T374 68Q374 43 381 35T402 26Q411 27 422 35Q443 55 463 131Q469 151 473 152Q475 153 483 153H487Q506 153 506 144Q506 138 501 117T481 63T449 13Q436 0 417 -8Q409 -10 393 -10Q359 -10 336 5T306 36L300 51Q299 52 296 50Q294 48 292 46Q233 -10 172 -10Q117 -10 75 30T33 157ZM351 328Q351 334 346 350T323 385T277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q217 26 254 59T298 110Q300 114 325 217T351 328Z"
></path><path
id=
"MJMATHI-74"
stroke-width=
"1"
d=
"M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"
></path><path
id=
"MJMAIN-30"
stroke-width=
"1"
d=
"M96 585Q152 666 249 666Q297 666 345 640T423 548Q460 465 460 320Q460 165 417 83Q397 41 362 16T301 -15T250 -22Q224 -22 198 -16T137 16T82 83Q39 165 39 320Q39 494 96 585ZM321 597Q291 629 250 629Q208 629 178 597Q153 571 145 525T137 333Q137 175 145 125T181 46Q209 16 250 16Q290 16 318 46Q347 76 354 130T362 333Q362 478 354 524T321 597Z"
></path><path
id=
"MJMATHI-3C3"
stroke-width=
"1"
d=
"M184 -11Q116 -11 74 34T31 147Q31 247 104 333T274 430Q275 431 414 431H552Q553 430 555 429T559 427T562 425T565 422T567 420T569 416T570 412T571 407T572 401Q572 357 507 357Q500 357 490 357T476 358H416L421 348Q439 310 439 263Q439 153 359 71T184 -11ZM361 278Q361 358 276 358Q152 358 115 184Q114 180 114 178Q106 141 106 117Q106 67 131 47T188 26Q242 26 287 73Q316 103 334 153T356 233T361 278Z"
></path><path
id=
"MJMATHI-57"
stroke-width=
"1"
d=
"M436 683Q450 683 486 682T553 680Q604 680 638 681T677 682Q695 682 695 674Q695 670 692 659Q687 641 683 639T661 637Q636 636 621 632T600 624T597 615Q597 603 613 377T629 138L631 141Q633 144 637 151T649 170T666 200T690 241T720 295T759 362Q863 546 877 572T892 604Q892 619 873 628T831 637Q817 637 817 647Q817 650 819 660Q823 676 825 679T839 682Q842 682 856 682T895 682T949 681Q1015 681 1034 683Q1048 683 1048 672Q1048 666 1045 655T1038 640T1028 637Q1006 637 988 631T958 617T939 600T927 584L923 578L754 282Q586 -14 585 -15Q579 -22 561 -22Q546 -22 542 -17Q539 -14 523 229T506 480L494 462Q472 425 366 239Q222 -13 220 -15T215 -19Q210 -22 197 -22Q178 -22 176 -15Q176 -12 154 304T131 622Q129 631 121 633T82 637H58Q51 644 51 648Q52 671 64 683H76Q118 680 176 680Q301 680 313 683H323Q329 677 329 674T327 656Q322 641 318 637H297Q236 634 232 620Q262 160 266 136L501 550L499 587Q496 629 489 632Q483 636 447 637Q428 637 422 639T416 648Q416 650 418 660Q419 664 420 669T421 676T424 680T428 682T436 683Z"
></path><path
id=
"MJMATHI-46"
stroke-width=
"1"
d=
"M48 1Q31 1 31 11Q31 13 34 25Q38 41 42 43T65 46Q92 46 125 49Q139 52 144 61Q146 66 215 342T285 622Q285 629 281 629Q273 632 228 634H197Q191 640 191 642T193 659Q197 676 203 680H742Q749 676 749 669Q749 664 736 557T722 447Q720 440 702 440H690Q683 445 683 453Q683 454 686 477T689 530Q689 560 682 579T663 610T626 626T575 633T503 634H480Q398 633 393 631Q388 629 386 623Q385 622 352 492L320 363H375Q378 363 398 363T426 364T448 367T472 374T489 386Q502 398 511 419T524 457T529 475Q532 480 548 480H560Q567 475 567 470Q567 467 536 339T502 207Q500 200 482 200H470Q463 206 463 212Q463 215 468 234T473 274Q473 303 453 310T364 317H309L277 190Q245 66 245 60Q245 46 334 46H359Q365 40 365 39T363 19Q359 6 353 0H336Q295 2 185 2Q120 2 86 2T48 1Z"
></path><path
id=
"MJMAIN-
2299"
stroke-width=
"1"
d=
"M56 250Q56 394 156 488T384 583Q530 583 626 485T722 250Q722 110 625 14T390 -83Q249 -83 153 14T56 250ZM682 250Q682 322 649 387T546 497T381 542Q272 542 184 459T95 250Q95 132 178 45T389 -42Q515 -42 598 45T682 250ZM311 250Q311 285 332 304T375 328Q376 328 382 328T392 329Q424 326 445 305T466 250Q466 217 445 195T389 172Q354 172 333 195T311 250
Z"
></path><path
id=
"MJMATHI-52"
stroke-width=
"1"
d=
"M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"
></path><path
id=
"MJMATHI-65"
stroke-width=
"1"
d=
"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z"
></path><path
id=
"MJMATHI-72"
stroke-width=
"1"
d=
"M21 287Q22 290 23 295T28 317T38 348T53 381T73 411T99 433T132 442Q161 442 183 430T214 408T225 388Q227 382 228 382T236 389Q284 441 347 441H350Q398 441 422 400Q430 381 430 363Q430 333 417 315T391 292T366 288Q346 288 334 299T322 328Q322 376 378 392Q356 405 342 405Q286 405 239 331Q229 315 224 298T190 165Q156 25 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 114 189T154 366Q154 405 128 405Q107 405 92 377T68 316T57 280Q55 278 41 278H27Q21 284 21 287Z"
></path><path
id=
"MJMATHI-75"
stroke-width=
"1"
d=
"M21 287Q21 295 30 318T55 370T99 420T158 442Q204 442 227 417T250 358Q250 340 216 246T182 105Q182 62 196 45T238 27T291 44T328 78L339 95Q341 99 377 247Q407 367 413 387T427 416Q444 431 463 431Q480 431 488 421T496 402L420 84Q419 79 419 68Q419 43 426 35T447 26Q469 29 482 57T512 145Q514 153 532 153Q551 153 551 144Q550 139 549 130T540 98T523 55T498 17T462 -8Q454 -10 438 -10Q372 -10 347 46Q345 45 336 36T318 21T296 6T267 -6T233 -11Q189 -11 155 7Q103 38 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z"
></path></defs></svg></div><div
id=
"wmd-preview"
class=
"wmd-preview wmd-preview-full-reader"
><div
class=
"md-section-divider"
></div><div
class=
"md-section-divider"
></div><h1
data-anchor-id=
"woa1"
id=
"情感分析"
>
情感分析
</h1><div
class=
"md-section-divider"
></div><h2
data-anchor-id=
"6sb6"
id=
"背景介绍"
>
背景介绍
</h2><p
data-anchor-id=
"p24n"
>
在自然语言处理中，情感分析一般是指判断一段文本所表达的情绪状态。其中，一段文本可以是一个句子，一个段落或一个文档。情绪状态可以是两类，如（正面，负面），（高兴，悲伤）；也可以是三类，如（积极，消极，中性）等等。情感分析的应用场景十分广泛，如把用户在购物网站（亚马逊、天猫、淘宝等）、旅游网站、电影评论网站上发表的评论分成正面评论和负面评论；或为了分析用户对于某一产品的整体使用感受，抓取产品的用户评论并进行情感分析等等。表格1展示了对电影评论进行情感分析的例子：
</p><table
data-anchor-id=
"ydgg"
class=
"table table-striped-white table-bordered"
>
<thead>
<tr>
<th>
电影评论
</th>
...
...
@@ -34,20 +34,20 @@
<td>
剧情四星。但是圆镜视角加上婺源的风景整个非常有中国写意山水画的感觉，看得实在太舒服了。。
</td>
<td>
正面
</td>
</tr>
</tbody></table><p
align=
"center"
data-anchor-id=
"zxk7"
>
表格 1 电影评论情感分析
</p><div
class=
"md-section-divider"
></div><div
class=
"md-section-divider"
></div
><p
align=
"center"
data-anchor-id=
"jos4"
>
</tbody></table><p
data-anchor-id=
"vben"
>
在自然语言处理中，情感分析属于典型的
<strong>
文本分类
</strong>
问题，即把需要进行情感分析的文本划分为其所属类别。文本分类涉及文本表示和分类方法两个问题。在深度学习的方法出现之前，主流的文本表示方法为词袋模型BOW(bag of words)，话题模型等等；分类方法有SVM(support vector machine), LR(logistic regression)等等。
</p><p
data-anchor-id=
"g3bc"
>
对于一段文本，BOW表示会忽略其词顺序、语法和句法，将这段文本仅仅看做是一个词集合，因此BOW方法并不能充分表示文本的语义信息。例如，句子“这部电影糟糕透了”和“一个乏味，空洞，没有内涵的作品”在情感分析中具有很高的语义相似度，但是它们的BOW表示的相似度为0。又如，句子“一个空洞，没有内涵的作品”和“一个不空洞而且有内涵的作品”的BOW相似度很高，但实际上它们的意思很不一样。
</p><p
data-anchor-id=
"9b8i"
>
本章我们所要介绍的深度学习模型克服了BOW表示的上述缺陷，它在考虑词顺序的基础上把文本映射到低维度的语义空间，并且以端对端（end to end）的方式进行文本表示及分类，其性能相对于传统方法有显著的提升[
<a
href=
"#参考文献"
>
1
</a>
]。
</p><div
class=
"md-section-divider"
></div><h2
data-anchor-id=
"o6dd"
id=
"模型概览"
>
模型概览
</h2><p
data-anchor-id=
"pfp7"
>
本章所使用的文本表示模型为卷积神经网络（Convolutional Neural Networks）和循环神经网络(Recurrent Neural Networks)及其扩展。下面依次介绍这几个模型。
</p><div
class=
"md-section-divider"
></div><h3
data-anchor-id=
"3ho4"
id=
"文本卷积神经网络cnn"
>
文本卷积神经网络（CNN）
</h3><p
data-anchor-id=
"tb0x"
>
卷积神经网络经常用来处理具有类似网格拓扑结构（grid-like topology）的数据。例如，图像可以视为二维网格的像素点，自然语言可以视为一维的词序列。卷积神经网络可以提取多种局部特征，并对其进行组合抽象得到更高级的特征表示。实验表明，卷积神经网络能高效地对图像及文本问题进行建模处理。
</p><p
data-anchor-id=
"3mtf"
>
卷积神经网络主要由卷积（convolution）和池化（pooling）操作构成，其应用及组合方式灵活多变，种类繁多。本小结我们以一种简单的文本分类卷积神经网络为例进行讲解[
<a
href=
"#参考文献"
>
1
</a>
]，如图1所示：
</p
><p
align=
"center"
data-anchor-id=
"jos4"
>
<img
src=
"https://raw.githubusercontent.com/PaddlePaddle/book/develop/understand_sentiment/image/text_cnn.png"
width=
"700"
><br>
图1. 卷积神经网络文本分类模型
</p><p
data-anchor-id=
"514w"
>
假设待处理句子的长度为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-1-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -464.0516853480245 600.5 496.10337069604896"
style=
"width: 1.39ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-1"
>
n
</script>
，其中第
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-2-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -682.0516853480245 345.5 714.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-2"
>
i
</script>
个词的词向量（word embedding）为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-3-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -922.4713132022996 3331.117134844292 1101.0082799245627"
style=
"width: 7.761ex; height: 2.548ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"809"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2208"
x=
"1194"
y=
"0"
></use><g
transform=
"translate(2139,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJAMS-52"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6B"
x=
"1021"
y=
"581"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-3"
>
x_i
\
in
\
mathbb
{
R
}
^
k
</script>
，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-4-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 521.5 747.103370696049"
style=
"width: 1.158ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6B"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-4"
>
k
</script>
为维度大小。
</p><p
data-anchor-id=
"gvwb"
>
首先，进行词向量的拼接操作：将每
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-5-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 576.5 747.103370696049"
style=
"width: 1.39ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-5"
>
h
</script>
个词拼接起来形成一个大小为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-6-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 576.5 747.103370696049"
style=
"width: 1.39ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-6"
>
h
</script>
的词窗口，记为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-7-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 3220.5592860057245 691.7932335345324"
style=
"width: 7.529ex; height: 1.622ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><g
transform=
"translate(572,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3A"
x=
"345"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"624"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"969"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1747"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"2324"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"3103"
y=
"0"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-7"
>
x_
{
i
:
i
+
h
-
1
}
</script>
，它表示词序列
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-8-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 8191.991680149484 691.7932335345324"
style=
"width: 18.996ex; height: 1.622ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"809"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"916"
y=
"0"
></use><g
transform=
"translate(1361,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><g
transform=
"translate(572,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"345"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1124"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"3183"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2026"
x=
"3628"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"4967"
y=
"0"
></use><g
transform=
"translate(5412,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><g
transform=
"translate(572,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"345"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1124"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"1700"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"2479"
y=
"0"
></use></g></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-8"
>
x_
{
i
},
x_
{
i
+
1
},
\
ldots
,
x_
{
i
+
h
-
1
}
</script>
的拼接，其中，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-9-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -682.0516853480245 345.5 714.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-9"
>
i
</script>
表示词窗口中第一个词在整个句子中的位置，取值范围从
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-10-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -687.0516853480245 500.5 708.103370696049"
style=
"width: 1.158ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-10"
>
1
</script>
到
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-11-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 4123.388888888889 818.103370696049"
style=
"width: 9.614ex; height: 1.853ex; vertical-align: -0.347ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"822"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1823"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"2622"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"3622"
y=
"0"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-11"
>
n
-
h
+
1
</script>
，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-12-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -922.4713132022996 6042.518087304109 1151.2128613888076"
style=
"width: 14.015ex; height: 2.664ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><g
transform=
"translate(572,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3A"
x=
"345"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"624"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"969"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1747"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"2324"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"3103"
y=
"0"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2208"
x=
"3498"
y=
"0"
></use><g
transform=
"translate(4443,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJAMS-52"
></use><g
transform=
"translate(722,410)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6B"
x=
"576"
y=
"0"
></use></g></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-12"
>
x_
{
i
:
i
+
h
-
1
}
\
in
\
mathbb
{
R
}
^
{
hk
}
</script>
。
</p><p
data-anchor-id=
"6eru"
>
其次，进行卷积操作：把卷积核(kernel)
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-13-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -922.4713132022996 3538.458801298385 984.522998550324"
style=
"width: 8.224ex; height: 2.317ex; vertical-align: -0.232ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-77"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2208"
x=
"994"
y=
"0"
></use><g
transform=
"translate(1939,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJAMS-52"
></use><g
transform=
"translate(722,410)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6B"
x=
"576"
y=
"0"
></use></g></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-13"
>
w
\
in
\
mathbb
{
R
}
^
{
hk
}
</script>
应用于包含
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-14-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 576.5 747.103370696049"
style=
"width: 1.39ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-14"
>
h
</script>
个词的窗口
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-15-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 3220.5592860057245 691.7932335345324"
style=
"width: 7.529ex; height: 1.622ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><g
transform=
"translate(572,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3A"
x=
"345"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"624"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"969"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1747"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"2324"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"3103"
y=
"0"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-15"
>
x_
{
i
:
i
+
h
-
1
}
</script>
，得到特征
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-16-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -771.0516853480245 9753.809123350122 1042.103370696049"
style=
"width: 22.703ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"613"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1055"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"2111"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"2662"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-77"
x=
"3051"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-22C5"
x=
"3990"
y=
"0"
></use><g
transform=
"translate(4491,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><g
transform=
"translate(572,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3A"
x=
"345"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"624"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"969"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1747"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"2324"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"3103"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"7934"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
x=
"8934"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"9364"
y=
"0"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-16"
>
c_i
=
f
(
w
\
cdot
x_
{
i
:
i
+
h
-
1
}
+
b
)
</script>
，其中
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-17-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 2375.0555555555557 777.103370696049"
style=
"width: 5.56ex; height: 1.853ex; vertical-align: -0.232ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2208"
x=
"707"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJAMS-52"
x=
"1652"
y=
"0"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-17"
>
b
\
in
\
mathbb
{
R
}
</script>
为偏置项（bias），
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-18-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -726.0516853480245 550.5 952.103370696049"
style=
"width: 1.274ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-18"
>
f
</script>
为非线性激活函数，如
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-19-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 3528.5 941.103370696049"
style=
"width: 8.224ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-73"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"469"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-67"
x=
"815"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6D"
x=
"1295"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"2174"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"2659"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-64"
x=
"3005"
y=
"0"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-19"
>
sigmoid
</script>
。将卷积核应用于句子中所有的词窗口
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-20-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 10422.206468011855 691.7932335345324"
style=
"width: 24.208ex; height: 1.622ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><g
transform=
"translate(572,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3A"
x=
"500"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"779"
y=
"0"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"1630"
y=
"0"
></use><g
transform=
"translate(2076,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><g
transform=
"translate(572,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-32"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3A"
x=
"500"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"779"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"1355"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"2134"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"4611"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2026"
x=
"5056"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"6395"
y=
"0"
></use><g
transform=
"translate(6841,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><g
transform=
"translate(572,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"600"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1379"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"1955"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"2734"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3A"
x=
"3234"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
x=
"3513"
y=
"0"
></use></g></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-20"
>
{
x_
{
1
:
h
},
x_
{
2
:
h
+
1
},
\
ldots
,
x_
{
n
-
h
+
1
:
n
}}
</script>
，产生一个特征图（feature map）：
</p><div
class=
"md-section-divider"
></div><p
data-anchor-id=
"adaa"
><span
class=
"MathJax_Preview"
></span><div
class=
"MathJax_SVG_Display"
role=
"textbox"
aria-readonly=
"true"
style=
"text-align: center;"
><span
class=
"MathJax_SVG"
id=
"MathJax-Element-21-Frame"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -924.3828982726751 14806.032099907956 1195.4345836206996"
style=
"width: 34.402ex; height: 2.78ex; vertical-align: -0.695ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"711"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-5B"
x=
"1767"
y=
"0"
></use><g
transform=
"translate(2046,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"613"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"2933"
y=
"0"
></use><g
transform=
"translate(3378,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-32"
x=
"613"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"4266"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2026"
x=
"4711"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"6050"
y=
"0"
></use><g
transform=
"translate(6495,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><g
transform=
"translate(433,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"600"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1379"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"1955"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"2734"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-5D"
x=
"9316"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"9594"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"10039"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2208"
x=
"10751"
y=
"0"
></use><g
transform=
"translate(11696,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJAMS-52"
></use><g
transform=
"translate(722,412)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"600"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1379"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"1955"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"2734"
y=
"0"
></use></g></g></g></svg></span></div><script
type=
"math/tex; mode=display"
id=
"MathJax-Element-21"
>
c
=
[
c_1
,
c_2
,
\
ldots
,
c_
{
n
-
h
+
1
}],
c
\
in
\
mathbb
{
R
}
^
{
n
-
h
+
1
}
</script></p><p
data-anchor-id=
"vkpd"
>
接下来，对特征图采用时间维度上的最大池化（max pooling over time）操作得到此卷积核对应的整句话的特征
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-22-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -702.8583648847041 556.1 735.9100502327285"
style=
"width: 1.274ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"33"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-5E"
x=
"55"
y=
"-13"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-22"
>
\
hat
c
</script>
，它是特征图中所有元素的最大值：
</p><div
class=
"md-section-divider"
></div><p
data-anchor-id=
"od9l"
><span
class=
"MathJax_Preview"
></span><div
class=
"MathJax_SVG_Display"
role=
"textbox"
aria-readonly=
"true"
style=
"text-align: center;"
><span
class=
"MathJax_SVG"
id=
"MathJax-Element-23-Frame"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -771.0516853480245 5083.155555555555 1042.103370696049"
style=
"width: 11.815ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"33"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-5E"
x=
"55"
y=
"-13"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"833"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6D"
x=
"1890"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-61"
x=
"2768"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
x=
"3298"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"3870"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"4260"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"4693"
y=
"0"
></use></g></svg></span></div><script
type=
"math/tex; mode=display"
id=
"MathJax-Element-23"
>
\
hat
c
=
max
(
c
)
</script></p><p
data-anchor-id=
"tmo2"
>
在实际应用中，我们会使用多个卷积核来处理句子，窗口大小相同的卷积核堆叠起来形成一个矩阵（上文中的单个卷积核参数
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-24-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -464.0516853480245 716.5 496.10337069604896"
style=
"width: 1.622ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-77"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-24"
>
w
</script>
相当于矩阵的某一行），这样可以更高效的完成运算。另外，我们也可使用窗口大小不同的卷积核来处理句子（图1作为示意画了四个卷积核，不同颜色表示不同大小的卷积核操作）。
</p><p
data-anchor-id=
"pbpk"
>
最后，将所有卷积核得到的特征拼接起来即为文本的定长向量表示，对于文本分类问题，将其连接至softmax即构建出完整的模型。
</p><p
data-anchor-id=
"5gyc"
>
对于一般的短文本分类问题，上文所述的简单的文本卷积网络即可达到很高的正确率[
<a
href=
"#参考文献"
>
1
</a>
]。若想得到更抽象更高级的文本特征表示，可以构建深层文本卷积神经网络[
<a
href=
"#参考文献"
>
2
</a>
,
<a
href=
"#参考文献"
>
3
</a>
]。
</p><div
class=
"md-section-divider"
></div><h3
data-anchor-id=
"76da"
id=
"循环神经网络rnn"
>
循环神经网络（RNN）
</h3><p
data-anchor-id=
"jix2"
>
循环神经网络是一种能对序列数据进行精确建模的有力工具。实际上，循环神经网络的理论计算能力是图灵完备的[
<a
href=
"#参考文献"
>
4
</a>
]。自然语言是一种典型的序列数据（词序列），近年来，循环神经网络及其变体（如long short term memory[
<a
href=
"#参考文献"
>
5
</a>
]等）在自然语言处理的多个领域，如语言模型、句法解析、语义角色标注（或一般的序列标注）、语义表示、图文生成、对话、机器翻译等任务上均表现优异甚至成为目前效果最好的方法。
</p><p
align=
"center"
data-anchor-id=
"5qsd"
>
<img
src=
"https://raw.githubusercontent.com/PaddlePaddle/book/develop/understand_sentiment/image/rnn.png"
width=
"400"
><br>
图2. 循环神经网络按时间展开的示意图
</p><p
data-anchor-id=
"rwui"
>
循环神经网络按时间展开后如图2所示：在第
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-25-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -647.0516853480245 361.5 679.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-25"
>
t
</script>
时刻，网络读入第
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-26-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -647.0516853480245 361.5 679.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-26"
>
t
</script>
个输入
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-27-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 928.1191013989369 641.5886520702876"
style=
"width: 2.201ex; height: 1.506ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-27"
>
x_t
</script>
（向量表示）及前一时刻隐层的状态值
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-28-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 1836.5086745365313 893.5886520702876"
style=
"width: 4.286ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-28"
>
h_
{
t
-
1
}
</script>
（向量表示，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-29-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 1030.406943983867 901.3668266633396"
style=
"width: 2.432ex; height: 2.085ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-30"
x=
"815"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-29"
>
h_0
</script>
一般初始化为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-30-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -687.0516853480245 500.5 730.103370696049"
style=
"width: 1.158ex; height: 1.737ex; vertical-align: -0.232ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-30"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-30"
>
0
</script>
向量），计算得出本时刻隐层的状态值
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-31-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 932.1191013989369 893.5886520702876"
style=
"width: 2.201ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-31"
>
h_t
</script>
，重复这一步骤直至读完所有输入。如果将循环神经网络所表示的函数记为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-32-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -726.0516853480245 550.5 952.103370696049"
style=
"width: 1.274ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-32"
>
f
</script>
，则其公式可表示为：
</p><div
class=
"md-section-divider"
></div><p
data-anchor-id=
"ffs1"
><span
class=
"MathJax_Preview"
></span><div
class=
"MathJax_SVG_Display"
role=
"textbox"
aria-readonly=
"true"
style=
"text-align: center;"
><span
class=
"MathJax_SVG"
id=
"MathJax-Element-33-Frame"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -771.0516853480245 19507.476147537123 1042.103370696049"
style=
"width: 45.29ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1209"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"2266"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"2816"
y=
"0"
></use><g
transform=
"translate(3206,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"4134"
y=
"0"
></use><g
transform=
"translate(4579,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"6415"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"7083"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
x=
"8139"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"8712"
y=
"0"
></use><g
transform=
"translate(9101,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(10958,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"12108"
y=
"0"
></use><g
transform=
"translate(13109,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(14969,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"17180"
y=
"0"
></use><g
transform=
"translate(18180,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"607"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"19117"
y=
"0"
></use></g></svg></span></div><script
type=
"math/tex; mode=display"
id=
"MathJax-Element-33"
>
h_t
=
f
(
x_t
,
h_
{
t
-
1
})
=
\
sigma
(
W_
{
xh
}
x_t
+
W_
{
hh
}
h_
{
h
-
1
}
+
b_h
)
</script></p><p
data-anchor-id=
"863e"
>
其中
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-34-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -704.0516853480245 1856.9656915833432 882.5886520702876"
style=
"width: 4.286ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"572"
y=
"0"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-34"
>
W_
{
xh
}
</script>
是输入到隐层的矩阵参数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-35-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -704.0516853480245 1859.7941187080894 882.5886520702876"
style=
"width: 4.286ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"576"
y=
"0"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-35"
>
W_
{
hh
}
</script>
是隐层到隐层的矩阵参数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-36-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 937.1470593540447 893.5886520702876"
style=
"width: 2.201ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"607"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-36"
>
b_h
</script>
为隐层的偏置向量（bias）参数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-37-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -452.0516853480245 572.5 484.10337069604896"
style=
"width: 1.274ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-37"
>
\
sigma
</script>
为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-38-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 3528.5 941.103370696049"
style=
"width: 8.224ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-73"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"469"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-67"
x=
"815"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6D"
x=
"1295"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"2174"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"2659"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-64"
x=
"3005"
y=
"0"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-38"
>
sigmoid
</script>
函数。
</p><p
data-anchor-id=
"7ac9"
>
在处理自然语言时，一般会先将词（one-hot表示）映射为其词向量（word embedding）表示，然后再作为循环神经网络每一时刻的输入
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-39-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 928.1191013989369 641.5886520702876"
style=
"width: 2.201ex; height: 1.506ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-39"
>
x_t
</script>
。此外，可以根据实际需要的不同在循环神经网络的隐层上连接其它层。如，可以把一个循环神经网络的隐层输出连接至下一个循环神经网络的输入构建深层（deep or stacked）循环神经网络，或者提取最后一个时刻的隐层状态作为句子表示进而使用分类模型等等。
</p><div
class=
"md-section-divider"
></div><h3
data-anchor-id=
"opm4"
id=
"长短期记忆网络lstm"
>
长短期记忆网络（LSTM）
</h3><p
data-anchor-id=
"puym"
>
对于较长的序列数据，循环神经网络的训练过程中容易出现梯度消失或爆炸现象[
<a
href=
"#参考文献"
>
6
</a>
]。为了解决这一问题，Hochreiter S, Schmidhuber J. (1997)提出了LSTM(long short term memory[
<a
href=
"#参考文献"
>
5
</a>
])。
</p><p
data-anchor-id=
"ornj"
>
相比于简单的循环神经网络，LSTM增加了记忆单元
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-40-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 433.5 496.10337069604896"
style=
"width: 1.042ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-40"
>
c
</script>
、输入门
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-41-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -682.0516853480245 345.5 714.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-41"
>
i
</script>
、遗忘门
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-42-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -726.0516853480245 550.5 952.103370696049"
style=
"width: 1.274ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-42"
>
f
</script>
及输出门
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-43-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -462.0516853480245 485.5 494.10337069604896"
style=
"width: 1.158ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-43"
>
o
</script>
。这些门及记忆单元组合起来大大提升了循环神经网络处理长序列数据的能力。若将基于LSTM的循环神经网络表示的函数记为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-44-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -701.0516853480245 749.5 722.103370696049"
style=
"width: 1.737ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-46"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-44"
>
F
</script>
，则其公式为：
</p><div
class=
"md-section-divider"
></div><p
data-anchor-id=
"5cdo"
><span
class=
"MathJax_Preview"
></span><div
class=
"MathJax_SVG_Display"
role=
"textbox"
aria-readonly=
"true"
style=
"text-align: center;"
><span
class=
"MathJax_SVG"
id=
"MathJax-Element-45-Frame"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -771.0516853480245 7004.469099556628 1042.103370696049"
style=
"width: 16.216ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1209"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-46"
x=
"2266"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"3015"
y=
"0"
></use><g
transform=
"translate(3405,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"4333"
y=
"0"
></use><g
transform=
"translate(4778,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"6614"
y=
"0"
></use></g></svg></span></div><script
type=
"math/tex; mode=display"
id=
"MathJax-Element-45"
>
h_t
=
F
(
x_t
,
h_
{
t
-
1
})
</script></p><p
data-anchor-id=
"51da"
><span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-
61-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -701.0516853480245 749.5 722.103370696049"
style=
"width: 1.737ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-46"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-61"
>
F
</script>
由下列公式组合而成[
<a
href=
"#参考文献"
>
7
</a>
]：
</p><div
class=
"md-section-divider"
></div><p
data-anchor-id=
"x9vt"
><span
class=
"MathJax_Preview"
></span><div
class=
"MathJax_SVG_Display"
role=
"textbox"
aria-readonly=
"true"
><span
class=
"MathJax_SVG"
id=
"MathJax-Element-147-Frame"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -771.0516853480245 39497.926795366795 6821.757555138181"
style=
"width: 91.737ex; height: 15.869ex; vertical-align: -14.131ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><g
transform=
"translate(11036,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"488"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"978"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
x=
"2035"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"2607"
y=
"0"
></use><g
transform=
"translate(2997,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(4690,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"5841"
y=
"0"
></use><g
transform=
"translate(6841,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(8538,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"10749"
y=
"0"
></use><g
transform=
"translate(11749,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"433"
y=
"0"
></use></g></g><g
transform=
"translate(13345,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><g
transform=
"translate(433,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"15260"
y=
"0"
></use><g
transform=
"translate(16261,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"607"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"17035"
y=
"0"
></use></g><g
transform=
"translate(10674,-1433)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"693"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1123"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
x=
"2180"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"2752"
y=
"0"
></use><g
transform=
"translate(3142,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-155)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(4980,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"6131"
y=
"0"
></use><g
transform=
"translate(7131,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-155)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(8973,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"11183"
y=
"0"
></use><g
transform=
"translate(12184,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-155)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"433"
y=
"0"
></use></g></g><g
transform=
"translate(13925,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><g
transform=
"translate(433,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"15840"
y=
"0"
></use><g
transform=
"translate(16841,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"607"
y=
"-219"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"17760"
y=
"0"
></use></g><g
transform=
"translate(8952,-2915)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"613"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1066"
y=
"0"
></use><g
transform=
"translate(2123,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"693"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2299"
x=
"3191"
y=
"0"
></use><g
transform=
"translate(4192,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><g
transform=
"translate(433,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"6107"
y=
"0"
></use><g
transform=
"translate(7108,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"488"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2299"
x=
"8032"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"9032"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-61"
x=
"9394"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
x=
"9923"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"10524"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"11100"
y=
"0"
></use><g
transform=
"translate(11490,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(13246,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"14396"
y=
"0"
></use><g
transform=
"translate(15397,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(17155,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"19366"
y=
"0"
></use><g
transform=
"translate(20367,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"607"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"21203"
y=
"0"
></use></g><g
transform=
"translate(11220,-4348)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"686"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1118"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
x=
"2175"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"2747"
y=
"0"
></use><g
transform=
"translate(3137,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(4929,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"6080"
y=
"0"
></use><g
transform=
"translate(7080,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(8876,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"11087"
y=
"0"
></use><g
transform=
"translate(12087,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"433"
y=
"0"
></use></g></g><g
transform=
"translate(13782,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"613"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"14793"
y=
"0"
></use><g
transform=
"translate(15794,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"607"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"16666"
y=
"0"
></use></g><g
transform=
"translate(15765,-5780)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1209"
y=
"0"
></use><g
transform=
"translate(2266,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"686"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2299"
x=
"3329"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"4330"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-61"
x=
"4691"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
x=
"5221"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"5821"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"6398"
y=
"0"
></use><g
transform=
"translate(6787,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"613"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"7576"
y=
"0"
></use></g></g></svg></span></div><script
type=
"math/tex; mode=display"
id=
"MathJax-Element-1
47"
>
i_t
=
\
sigma
(
W_
{
xi
}
x_t
+
W_
{
hi
}
h_
{
h
-
1
}
+
W_
{
ci
}
c_
{
t
-
1
}
+
b_i
)
\
\
</p><p
data-anchor-id=
"rwui"
>
循环神经网络按时间展开后如图2所示：在第
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-25-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -647.0516853480245 361.5 679.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-25"
>
t
</script>
时刻，网络读入第
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-26-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -647.0516853480245 361.5 679.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-26"
>
t
</script>
个输入
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-27-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 928.1191013989369 641.5886520702876"
style=
"width: 2.201ex; height: 1.506ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-27"
>
x_t
</script>
（向量表示）及前一时刻隐层的状态值
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-28-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 1836.5086745365313 893.5886520702876"
style=
"width: 4.286ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-28"
>
h_
{
t
-
1
}
</script>
（向量表示，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-29-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 1030.406943983867 901.3668266633396"
style=
"width: 2.432ex; height: 2.085ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-30"
x=
"815"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-29"
>
h_0
</script>
一般初始化为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-30-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -687.0516853480245 500.5 730.103370696049"
style=
"width: 1.158ex; height: 1.737ex; vertical-align: -0.232ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-30"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-30"
>
0
</script>
向量），计算得出本时刻隐层的状态值
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-31-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 932.1191013989369 893.5886520702876"
style=
"width: 2.201ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-31"
>
h_t
</script>
，重复这一步骤直至读完所有输入。如果将循环神经网络所表示的函数记为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-32-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -726.0516853480245 550.5 952.103370696049"
style=
"width: 1.274ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-32"
>
f
</script>
，则其公式可表示为：
</p><div
class=
"md-section-divider"
></div><p
data-anchor-id=
"ffs1"
><span
class=
"MathJax_Preview"
></span><div
class=
"MathJax_SVG_Display"
role=
"textbox"
aria-readonly=
"true"
style=
"text-align: center;"
><span
class=
"MathJax_SVG"
id=
"MathJax-Element-33-Frame"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -771.0516853480245 19507.476147537123 1042.103370696049"
style=
"width: 45.29ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1209"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"2266"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"2816"
y=
"0"
></use><g
transform=
"translate(3206,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"4134"
y=
"0"
></use><g
transform=
"translate(4579,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"6415"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"7083"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
x=
"8139"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"8712"
y=
"0"
></use><g
transform=
"translate(9101,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(10958,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"12108"
y=
"0"
></use><g
transform=
"translate(13109,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(14969,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"17180"
y=
"0"
></use><g
transform=
"translate(18180,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"607"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"19117"
y=
"0"
></use></g></svg></span></div><script
type=
"math/tex; mode=display"
id=
"MathJax-Element-33"
>
h_t
=
f
(
x_t
,
h_
{
t
-
1
})
=
\
sigma
(
W_
{
xh
}
x_t
+
W_
{
hh
}
h_
{
h
-
1
}
+
b_h
)
</script></p><p
data-anchor-id=
"863e"
>
其中
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-34-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -704.0516853480245 1856.9656915833432 882.5886520702876"
style=
"width: 4.286ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"572"
y=
"0"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-34"
>
W_
{
xh
}
</script>
是输入到隐层的矩阵参数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-35-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -704.0516853480245 1859.7941187080894 882.5886520702876"
style=
"width: 4.286ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"576"
y=
"0"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-35"
>
W_
{
hh
}
</script>
是隐层到隐层的矩阵参数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-36-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 937.1470593540447 893.5886520702876"
style=
"width: 2.201ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"607"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-36"
>
b_h
</script>
为隐层的偏置向量（bias）参数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-37-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -452.0516853480245 572.5 484.10337069604896"
style=
"width: 1.274ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-37"
>
\
sigma
</script>
为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-38-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 3528.5 941.103370696049"
style=
"width: 8.224ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-73"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"469"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-67"
x=
"815"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6D"
x=
"1295"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"2174"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"2659"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-64"
x=
"3005"
y=
"0"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-38"
>
sigmoid
</script>
函数。
</p><p
data-anchor-id=
"7ac9"
>
在处理自然语言时，一般会先将词（one-hot表示）映射为其词向量（word embedding）表示，然后再作为循环神经网络每一时刻的输入
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-39-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 928.1191013989369 641.5886520702876"
style=
"width: 2.201ex; height: 1.506ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-39"
>
x_t
</script>
。此外，可以根据实际需要的不同在循环神经网络的隐层上连接其它层。如，可以把一个循环神经网络的隐层输出连接至下一个循环神经网络的输入构建深层（deep or stacked）循环神经网络，或者提取最后一个时刻的隐层状态作为句子表示进而使用分类模型等等。
</p><div
class=
"md-section-divider"
></div><h3
data-anchor-id=
"opm4"
id=
"长短期记忆网络lstm"
>
长短期记忆网络（LSTM）
</h3><p
data-anchor-id=
"puym"
>
对于较长的序列数据，循环神经网络的训练过程中容易出现梯度消失或爆炸现象[
<a
href=
"#参考文献"
>
6
</a>
]。为了解决这一问题，Hochreiter S, Schmidhuber J. (1997)提出了LSTM(long short term memory[
<a
href=
"#参考文献"
>
5
</a>
])。
</p><p
data-anchor-id=
"ornj"
>
相比于简单的循环神经网络，LSTM增加了记忆单元
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-40-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 433.5 496.10337069604896"
style=
"width: 1.042ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-40"
>
c
</script>
、输入门
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-41-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -682.0516853480245 345.5 714.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-41"
>
i
</script>
、遗忘门
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-42-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -726.0516853480245 550.5 952.103370696049"
style=
"width: 1.274ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-42"
>
f
</script>
及输出门
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-43-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -462.0516853480245 485.5 494.10337069604896"
style=
"width: 1.158ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-43"
>
o
</script>
。这些门及记忆单元组合起来大大提升了循环神经网络处理长序列数据的能力。若将基于LSTM的循环神经网络表示的函数记为
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-44-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -701.0516853480245 749.5 722.103370696049"
style=
"width: 1.737ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-46"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-44"
>
F
</script>
，则其公式为：
</p><div
class=
"md-section-divider"
></div><p
data-anchor-id=
"5cdo"
><span
class=
"MathJax_Preview"
></span><div
class=
"MathJax_SVG_Display"
role=
"textbox"
aria-readonly=
"true"
style=
"text-align: center;"
><span
class=
"MathJax_SVG"
id=
"MathJax-Element-45-Frame"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -771.0516853480245 7004.469099556628 1042.103370696049"
style=
"width: 16.216ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1209"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-46"
x=
"2266"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"3015"
y=
"0"
></use><g
transform=
"translate(3405,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"4333"
y=
"0"
></use><g
transform=
"translate(4778,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"6614"
y=
"0"
></use></g></svg></span></div><script
type=
"math/tex; mode=display"
id=
"MathJax-Element-45"
>
h_t
=
F
(
x_t
,
h_
{
t
-
1
})
</script></p><p
data-anchor-id=
"51da"
><span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-
46-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -701.0516853480245 749.5 722.103370696049"
style=
"width: 1.737ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-46"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-46"
>
F
</script>
由下列公式组合而成[
<a
href=
"#参考文献"
>
7
</a>
]：
</p><div
class=
"md-section-divider"
></div><p
data-anchor-id=
"x9vt"
><span
class=
"MathJax_Preview"
></span><div
class=
"MathJax_SVG_Display"
role=
"textbox"
aria-readonly=
"true"
><span
class=
"MathJax_SVG"
id=
"MathJax-Element-47-Frame"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -771.0516853480245 39896.895752895754 6821.757555138181"
style=
"width: 92.664ex; height: 15.869ex; vertical-align: -14.131ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><g
transform=
"translate(11236,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"488"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"978"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
x=
"2035"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"2607"
y=
"0"
></use><g
transform=
"translate(2997,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(4690,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"5841"
y=
"0"
></use><g
transform=
"translate(6841,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(8538,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"10749"
y=
"0"
></use><g
transform=
"translate(11749,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"433"
y=
"0"
></use></g></g><g
transform=
"translate(13345,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><g
transform=
"translate(433,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"15260"
y=
"0"
></use><g
transform=
"translate(16261,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
x=
"607"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"17035"
y=
"0"
></use></g><g
transform=
"translate(10873,-1433)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"693"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1123"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
x=
"2180"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"2752"
y=
"0"
></use><g
transform=
"translate(3142,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-155)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(4980,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"6131"
y=
"0"
></use><g
transform=
"translate(7131,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-155)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(8973,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"11183"
y=
"0"
></use><g
transform=
"translate(12184,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-155)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"433"
y=
"0"
></use></g></g><g
transform=
"translate(13925,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><g
transform=
"translate(433,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"15840"
y=
"0"
></use><g
transform=
"translate(16841,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
x=
"607"
y=
"-219"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"17760"
y=
"0"
></use></g><g
transform=
"translate(9152,-2915)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"613"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1066"
y=
"0"
></use><g
transform=
"translate(2123,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"693"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2299"
x=
"3191"
y=
"0"
></use><g
transform=
"translate(4192,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><g
transform=
"translate(433,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"6107"
y=
"0"
></use><g
transform=
"translate(7108,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"488"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2299"
x=
"8032"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"9032"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-61"
x=
"9394"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
x=
"9923"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"10524"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"11100"
y=
"0"
></use><g
transform=
"translate(11490,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(13246,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"14396"
y=
"0"
></use><g
transform=
"translate(15397,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(17155,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"19366"
y=
"0"
></use><g
transform=
"translate(20367,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"607"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"21203"
y=
"0"
></use></g><g
transform=
"translate(11420,-4348)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"686"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1118"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-3C3"
x=
"2175"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"2747"
y=
"0"
></use><g
transform=
"translate(3137,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"572"
y=
"0"
></use></g></g><g
transform=
"translate(4929,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"6080"
y=
"0"
></use><g
transform=
"translate(7080,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"576"
y=
"0"
></use></g></g><g
transform=
"translate(8876,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"576"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1355"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"11087"
y=
"0"
></use><g
transform=
"translate(12087,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use><g
transform=
"translate(944,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"433"
y=
"0"
></use></g></g><g
transform=
"translate(13782,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"613"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2B"
x=
"14793"
y=
"0"
></use><g
transform=
"translate(15794,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
x=
"607"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"16666"
y=
"0"
></use></g><g
transform=
"translate(15965,-5780)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1209"
y=
"0"
></use><g
transform=
"translate(2266,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"686"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2299"
x=
"3329"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"4330"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-61"
x=
"4691"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
x=
"5221"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"5821"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"6398"
y=
"0"
></use><g
transform=
"translate(6787,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"613"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"7576"
y=
"0"
></use></g></g></svg></span></div><script
type=
"math/tex; mode=display"
id=
"MathJax-Element-
47"
>
i_t
=
\
sigma
(
W_
{
xi
}
x_t
+
W_
{
hi
}
h_
{
h
-
1
}
+
W_
{
ci
}
c_
{
t
-
1
}
+
b_i
)
\
\
f_t
=
\
sigma
(
W_
{
xf
}
x_t
+
W_
{
hf
}
h_
{
h
-
1
}
+
W_
{
cf
}
c_
{
t
-
1
}
+
b_f
)
\
\
c_t
=
f_t
\
odot
c_
{
t
-
1
}
+
i_t
\
odot
tanh
(
W_
{
xc
}
x_t
+
W_
{
hc
}
h_
{
h
-
1
}
+
b_c
)
\
\
o_t
=
\
sigma
(
W_
{
xo
}
x_t
+
W_
{
ho
}
h_
{
h
-
1
}
+
W_
{
co
}
c_
{
t
}
+
b_o
)
\
\
h_t
=
o_t
\
odot
tanh
(
c_t
)
</script>
<br>
其中，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-
148-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -726.0516853480245 4512.976405595748 952.103370696049"
style=
"width: 10.425ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"488"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"701"
y=
"0"
></use><g
transform=
"translate(1146,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"693"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"1992"
y=
"0"
></use><g
transform=
"translate(2437,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"613"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"3226"
y=
"0"
></use><g
transform=
"translate(3671,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"686"
y=
"-213"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-148"
>
i_t
,
f_t
,
c_t
,
o_t
</script>
分别表示输入门，遗忘门，记忆单元及输出门的向量值，带角标的
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-149-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -704.0516853480245 1048.5 747.103370696049"
style=
"width: 2.432ex; height: 1.737ex; vertical-align: -0.232ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-149"
>
W
</script>
及
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-150-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 429.5 747.103370696049"
style=
"width: 1.042ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-150"
>
b
</script>
为模型参数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-151-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 2068 747.103370696049"
style=
"width: 4.749ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-61"
x=
"361"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
x=
"891"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1491"
y=
"0"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-151"
>
tanh
</script>
为双曲正切函数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-152-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -604.0516853480245 778.5 708.103370696049"
style=
"width: 1.853ex; height: 1.622ex; vertical-align: -0.347ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2299"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-152"
>
\
odot
</script>
表示逐元素（elementwise）的乘法操作。输入门控制着新输入进入记忆单元
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-153-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 433.5 496.10337069604896"
style=
"width: 1.042ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-153"
>
c
</script>
的强度，遗忘门控制着记忆单元维持上一时刻值的强度，输出门控制着输出记忆单元的强度。三种门的计算方式类似，但有着完全不同的参数，它们各自以不同的方式控制着记忆单元
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-154-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 433.5 496.10337069604896"
style=
"width: 1.042ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-1
54"
>
c
</script>
，如图3所示：
</p><p
align=
"center"
data-anchor-id=
"gztx"
>
其中，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-
48-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -726.0516853480245 4512.976405595748 952.103370696049"
style=
"width: 10.425ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-69"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"488"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"701"
y=
"0"
></use><g
transform=
"translate(1146,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-66"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"693"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"1992"
y=
"0"
></use><g
transform=
"translate(2437,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"613"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"3226"
y=
"0"
></use><g
transform=
"translate(3671,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6F"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"686"
y=
"-213"
></use></g></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-48"
>
i_t
,
f_t
,
c_t
,
o_t
</script>
分别表示输入门，遗忘门，记忆单元及输出门的向量值，带角标的
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-49-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -704.0516853480245 1048.5 747.103370696049"
style=
"width: 2.432ex; height: 1.737ex; vertical-align: -0.232ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-57"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-49"
>
W
</script>
及
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-50-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 429.5 747.103370696049"
style=
"width: 1.042ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-62"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-50"
>
b
</script>
为模型参数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-51-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 2068 747.103370696049"
style=
"width: 4.749ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-61"
x=
"361"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
x=
"891"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
x=
"1491"
y=
"0"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-51"
>
tanh
</script>
为双曲正切函数，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-52-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -604.0516853480245 778.5 708.103370696049"
style=
"width: 1.853ex; height: 1.622ex; vertical-align: -0.347ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2299"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-52"
>
\
odot
</script>
表示逐元素（elementwise）的乘法操作。输入门控制着新输入进入记忆单元
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-53-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 433.5 496.10337069604896"
style=
"width: 1.042ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-53"
>
c
</script>
的强度，遗忘门控制着记忆单元维持上一时刻值的强度，输出门控制着输出记忆单元的强度。三种门的计算方式类似，但有着完全不同的参数，它们各自以不同的方式控制着记忆单元
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-54-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -463.0516853480245 433.5 496.10337069604896"
style=
"width: 1.042ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-
54"
>
c
</script>
，如图3所示：
</p><p
align=
"center"
data-anchor-id=
"gztx"
>
<img
src=
"https://raw.githubusercontent.com/PaddlePaddle/book/develop/understand_sentiment/image/lstm.png"
width=
"600"
><br>
图3. 时刻
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-
60-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -647.0516853480245 361.5 679.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-60
"
>
t
</script>
的LSTM [7]
图3. 时刻
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-
55-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -647.0516853480245 361.5 679.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-55
"
>
t
</script>
的LSTM [7]
</p><p
data-anchor-id=
"jbb7"
>
LSTM通过给简单的循环神经网络增加记忆及控制门的方式，增强了其处理远距离依赖问题的能力。类似原理的改进还有Gated Recurrent Unit (GRU)[
<a
href=
"#参考文献"
>
8
</a>
]，其设计更为简洁一些。
<strong>
这些改进虽然各有不同，但是它们的宏观描述却与简单的循环神经网络一样（如图2所示），即隐状态依据当前输入及前一时刻的隐状态来改变，不断地循环这一过程直至输入处理完毕：
</strong></p><div
class=
"md-section-divider"
></div><p
data-anchor-id=
"p2ku"
><span
class=
"MathJax_Preview"
></span><div
class=
"MathJax_SVG_Display"
role=
"textbox"
aria-readonly=
"true"
style=
"text-align: center;"
><span
class=
"MathJax_SVG"
id=
"MathJax-Element-56-Frame"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -771.0516853480245 10818.469099556627 1042.103370696049"
style=
"width: 25.135ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-3D"
x=
"1209"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-52"
x=
"2266"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-65"
x=
"3025"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"3492"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-72"
x=
"3925"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-75"
x=
"4377"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-72"
x=
"4949"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-65"
x=
"5401"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
x=
"5867"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"6468"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-28"
x=
"6829"
y=
"0"
></use><g
transform=
"translate(7219,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-78"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"809"
y=
"-213"
></use></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2C"
x=
"8147"
y=
"0"
></use><g
transform=
"translate(8592,0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><g
transform=
"translate(576,-150)"
><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-2212"
x=
"361"
y=
"0"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-31"
x=
"1140"
y=
"0"
></use></g></g><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMAIN-29"
x=
"10428"
y=
"0"
></use></g></svg></span></div><script
type=
"math/tex; mode=display"
id=
"MathJax-Element-56"
>
h_t
=
Recrurent
(
x_t
,
h_
{
t
-
1
})
</script></p><p
data-anchor-id=
"dmbf"
>
其中，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-57-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -704.0516853480245 4563.5 746.103370696049"
style=
"width: 10.656ex; height: 1.737ex; vertical-align: -0.232ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-52"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-65"
x=
"759"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-63"
x=
"1226"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-72"
x=
"1659"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-75"
x=
"2111"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-72"
x=
"2683"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-65"
x=
"3135"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-6E"
x=
"3601"
y=
"0"
></use><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"4202"
y=
"0"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-57"
>
Recrurent
</script>
可以表示简单的循环神经网络、GRU或LSTM。
</p><div
class=
"md-section-divider"
></div><h3
data-anchor-id=
"ub28"
id=
"栈式双向lstmstacked-bidirectional-lstm"
>
栈式双向LSTM（Stacked Bidirectional LSTM）
</h3><p
data-anchor-id=
"af1z"
>
对于正常顺序的循环神经网络，
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-58-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -715.0516853480245 932.1191013989369 893.5886520702876"
style=
"width: 2.201ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-68"
></use><use
transform=
"scale(0.7071067811865476)"
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
x=
"815"
y=
"-213"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-58"
>
h_t
</script>
包含了
<span
class=
"MathJax_Preview"
></span><span
class=
"MathJax_SVG"
id=
"MathJax-Element-59-Frame"
role=
"textbox"
aria-readonly=
"true"
style=
"font-size: 100%; display: inline-block;"
><svg
xmlns:xlink=
"http://www.w3.org/1999/xlink"
viewBox=
"0 -647.0516853480245 361.5 679.103370696049"
style=
"width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"
><g
stroke=
"black"
fill=
"black"
stroke-width=
"0"
transform=
"matrix(1 0 0 -1 0 0)"
><use
xmlns:xlink=
"http://www.w3.org/1999/xlink"
xlink:href=
"#MJMATHI-74"
></use></g></svg></span><script
type=
"math/tex"
id=
"MathJax-Element-59"
>
t
</script>
时刻之前的输入信息，也就是上文信息。同样，为了得到下文信息，我们可以使用反方向（将输入逆序处理）的循环神经网络。结合构建深层循环神经网络的方法（深层神经网络往往能得到更抽象和高级的特征表示），我们可以通过构建更加强有力的基于LSTM的栈式双向循环神经网络[
<a
href=
"#参考文献"
>
9
</a>
]，来对时序数据进行建模。
</p><p
data-anchor-id=
"ozsi"
>
如图4所示（以三层为例），奇数层LSTM正向，偶数层LSTM反向，高一层的LSTM使用低一层LSTM及之前所有层的信息作为输入，对最高层LSTM序列使用时间维度上的最大池化即可得到文本的定长向量表示（这一表示充分融合了文本的上下文信息，并且对文本进行了深层次抽象），最后我们将文本表示连接至softmax构建分类模型。
</p><p
align=
"center"
data-anchor-id=
"9fkt"
>
<img
src=
"https://raw.githubusercontent.com/PaddlePaddle/book/develop/understand_sentiment/image/stacked_lstm.jpg"
width=
"450"
><br>
图4. 栈式双向LSTM用于文本分类
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑！
取消
想要评论请
注册
或
登录
