README.html 171.7 KB
Newer Older
L
Luo Tao 已提交
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182
<!DOCTYPE html>
<html class="theme theme-white">
<head>
<meta charset="utf-8">
<title>语义角色标注</title>
<link href="https://www.zybuluo.com/static/assets/template-theme-white.css" rel="stylesheet" media="screen">
<style type="text/css">

#wmd-preview h1  {
    color: #0077bb; /* 将标题改为蓝色 */
}</style>
</head>
<body class="theme theme-white">
<div style="visibility: hidden; overflow: hidden; position: absolute; top: 0px; height: 1px; width: auto; padding: 0px; border: 0px; margin: 0px; text-align: left; text-indent: 0px; text-transform: none; line-height: normal; letter-spacing: normal; word-spacing: normal;"><div id="MathJax_SVG_Hidden"></div><svg><defs id="MathJax_SVG_glyphs"><path id="MJMATHI-6E" stroke-width="1" d="M21 287Q22 293 24 303T36 341T56 388T89 425T135 442Q171 442 195 424T225 390T231 369Q231 367 232 367L243 378Q304 442 382 442Q436 442 469 415T503 336T465 179T427 52Q427 26 444 26Q450 26 453 27Q482 32 505 65T540 145Q542 153 560 153Q580 153 580 145Q580 144 576 130Q568 101 554 73T508 17T439 -10Q392 -10 371 17T350 73Q350 92 386 193T423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 180T152 343Q153 348 153 366Q153 405 129 405Q91 405 66 305Q60 285 60 284Q58 278 41 278H27Q21 284 21 287Z"></path><path id="MJMATHI-4C" stroke-width="1" d="M228 637Q194 637 192 641Q191 643 191 649Q191 673 202 682Q204 683 217 683Q271 680 344 680Q485 680 506 683H518Q524 677 524 674T522 656Q517 641 513 637H475Q406 636 394 628Q387 624 380 600T313 336Q297 271 279 198T252 88L243 52Q243 48 252 48T311 46H328Q360 46 379 47T428 54T478 72T522 106T564 161Q580 191 594 228T611 270Q616 273 628 273H641Q647 264 647 262T627 203T583 83T557 9Q555 4 553 3T537 0T494 -1Q483 -1 418 -1T294 0H116Q32 0 32 10Q32 17 34 24Q39 43 44 45Q48 46 59 46H65Q92 46 125 49Q139 52 144 61Q147 65 216 339T285 628Q285 635 228 637Z"></path><path id="MJMAIN-32" stroke-width="1" d="M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z"></path><path id="MJMATHI-4E" stroke-width="1" d="M234 637Q231 637 226 637Q201 637 196 638T191 649Q191 676 202 682Q204 683 299 683Q376 683 387 683T401 677Q612 181 616 168L670 381Q723 592 723 606Q723 633 659 637Q635 637 635 648Q635 650 637 660Q641 676 643 679T653 683Q656 683 684 682T767 680Q817 680 843 681T873 682Q888 682 888 672Q888 650 880 642Q878 637 858 637Q787 633 769 597L620 7Q618 0 599 0Q585 0 582 2Q579 5 453 305L326 604L261 344Q196 88 196 79Q201 46 268 46H278Q284 41 284 38T282 19Q278 6 272 0H259Q228 2 151 2Q123 2 100 2T63 2T46 1Q31 1 31 10Q31 14 34 26T39 40Q41 46 62 46Q130 49 150 85Q154 91 221 362L289 634Q287 635 234 637Z"></path><path id="MJMATHI-74" stroke-width="1" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path id="MJMAIN-2212" stroke-width="1" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path id="MJMAIN-31" stroke-width="1" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path id="MJMATHI-78" stroke-width="1" d="M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z"></path><path id="MJMATHI-50" stroke-width="1" d="M287 628Q287 635 230 637Q206 637 199 638T192 648Q192 649 194 659Q200 679 203 681T397 683Q587 682 600 680Q664 669 707 631T751 530Q751 453 685 389Q616 321 507 303Q500 302 402 301H307L277 182Q247 66 247 59Q247 55 248 54T255 50T272 48T305 46H336Q342 37 342 35Q342 19 335 5Q330 0 319 0Q316 0 282 1T182 2Q120 2 87 2T51 1Q33 1 33 11Q33 13 36 25Q40 41 44 43T67 46Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628ZM645 554Q645 567 643 575T634 597T609 619T560 635Q553 636 480 637Q463 637 445 637T416 636T404 636Q391 635 386 627Q384 621 367 550T332 412T314 344Q314 342 395 342H407H430Q542 342 590 392Q617 419 631 471T645 554Z"></path><path id="MJMAIN-28" stroke-width="1" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path id="MJMATHI-58" stroke-width="1" d="M42 0H40Q26 0 26 11Q26 15 29 27Q33 41 36 43T55 46Q141 49 190 98Q200 108 306 224T411 342Q302 620 297 625Q288 636 234 637H206Q200 643 200 645T202 664Q206 677 212 683H226Q260 681 347 681Q380 681 408 681T453 682T473 682Q490 682 490 671Q490 670 488 658Q484 643 481 640T465 637Q434 634 411 620L488 426L541 485Q646 598 646 610Q646 628 622 635Q617 635 609 637Q594 637 594 648Q594 650 596 664Q600 677 606 683H618Q619 683 643 683T697 681T738 680Q828 680 837 683H845Q852 676 852 672Q850 647 840 637H824Q790 636 763 628T722 611T698 593L687 584Q687 585 592 480L505 384Q505 383 536 304T601 142T638 56Q648 47 699 46Q734 46 734 37Q734 35 732 23Q728 7 725 4T711 1Q708 1 678 1T589 2Q528 2 496 2T461 1Q444 1 444 10Q444 11 446 25Q448 35 450 39T455 44T464 46T480 47T506 54Q523 62 523 64Q522 64 476 181L429 299Q241 95 236 84Q232 76 232 72Q232 53 261 47Q262 47 267 47T273 46Q276 46 277 46T280 45T283 42T284 35Q284 26 282 19Q279 6 276 4T261 1Q258 1 243 1T201 2T142 2Q64 2 42 0Z"></path><path id="MJMAIN-7C" stroke-width="1" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path id="MJMATHI-59" stroke-width="1" d="M66 637Q54 637 49 637T39 638T32 641T30 647T33 664T42 682Q44 683 56 683Q104 680 165 680Q288 680 306 683H316Q322 677 322 674T320 656Q316 643 310 637H298Q242 637 242 624Q242 619 292 477T343 333L346 336Q350 340 358 349T379 373T411 410T454 461Q546 568 561 587T577 618Q577 634 545 637Q528 637 528 647Q528 649 530 661Q533 676 535 679T549 683Q551 683 578 682T657 680Q684 680 713 681T746 682Q763 682 763 673Q763 669 760 657T755 643Q753 637 734 637Q662 632 617 587Q608 578 477 424L348 273L322 169Q295 62 295 57Q295 46 363 46Q379 46 384 45T390 35Q390 33 388 23Q384 6 382 4T366 1Q361 1 324 1T232 2Q170 2 138 2T102 1Q84 1 84 9Q84 14 87 24Q88 27 89 30T90 35T91 39T93 42T96 44T101 45T107 45T116 46T129 46Q168 47 180 50T198 63Q201 68 227 171L252 274L129 623Q128 624 127 625T125 627T122 629T118 631T113 633T105 634T96 635T83 636T66 637Z"></path><path id="MJMAIN-29" stroke-width="1" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path id="MJMAIN-3D" stroke-width="1" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path id="MJMAIN-2C" stroke-width="1" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path id="MJMAIN-2E" stroke-width="1" d="M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z"></path><path id="MJMATHI-79" stroke-width="1" d="M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z"></path><path id="MJMAIN-2217" stroke-width="1" d="M229 286Q216 420 216 436Q216 454 240 464Q241 464 245 464T251 465Q263 464 273 456T283 436Q283 419 277 356T270 286L328 328Q384 369 389 372T399 375Q412 375 423 365T435 338Q435 325 425 315Q420 312 357 282T289 250L355 219L425 184Q434 175 434 161Q434 146 425 136T401 125Q393 125 383 131T328 171L270 213Q283 79 283 63Q283 53 276 44T250 35Q231 35 224 44T216 63Q216 80 222 143T229 213L171 171Q115 130 110 127Q106 124 100 124Q87 124 76 134T64 161Q64 166 64 169T67 175T72 181T81 188T94 195T113 204T138 215T170 230T210 250L74 315Q65 324 65 338Q65 353 74 363T98 374Q106 374 116 368T171 328L229 286Z"></path><path id="MJMAIN-61" stroke-width="1" d="M137 305T115 305T78 320T63 359Q63 394 97 421T218 448Q291 448 336 416T396 340Q401 326 401 309T402 194V124Q402 76 407 58T428 40Q443 40 448 56T453 109V145H493V106Q492 66 490 59Q481 29 455 12T400 -6T353 12T329 54V58L327 55Q325 52 322 49T314 40T302 29T287 17T269 6T247 -2T221 -8T190 -11Q130 -11 82 20T34 107Q34 128 41 147T68 188T116 225T194 253T304 268H318V290Q318 324 312 340Q290 411 215 411Q197 411 181 410T156 406T148 403Q170 388 170 359Q170 334 154 320ZM126 106Q126 75 150 51T209 26Q247 26 276 49T315 109Q317 116 318 175Q318 233 317 233Q309 233 296 232T251 223T193 203T147 166T126 106Z"></path><path id="MJMAIN-72" stroke-width="1" d="M36 46H50Q89 46 97 60V68Q97 77 97 91T98 122T98 161T98 203Q98 234 98 269T98 328L97 351Q94 370 83 376T38 385H20V408Q20 431 22 431L32 432Q42 433 60 434T96 436Q112 437 131 438T160 441T171 442H174V373Q213 441 271 441H277Q322 441 343 419T364 373Q364 352 351 337T313 322Q288 322 276 338T263 372Q263 381 265 388T270 400T273 405Q271 407 250 401Q234 393 226 386Q179 341 179 207V154Q179 141 179 127T179 101T180 81T180 66V61Q181 59 183 57T188 54T193 51T200 49T207 48T216 47T225 47T235 46T245 46H276V0H267Q249 3 140 3Q37 3 28 0H20V46H36Z"></path><path id="MJMAIN-67" stroke-width="1" d="M329 409Q373 453 429 453Q459 453 472 434T485 396Q485 382 476 371T449 360Q416 360 412 390Q410 404 415 411Q415 412 416 414V415Q388 412 363 393Q355 388 355 386Q355 385 359 381T368 369T379 351T388 325T392 292Q392 230 343 187T222 143Q172 143 123 171Q112 153 112 133Q112 98 138 81Q147 75 155 75T227 73Q311 72 335 67Q396 58 431 26Q470 -13 470 -72Q470 -139 392 -175Q332 -206 250 -206Q167 -206 107 -175Q29 -140 29 -75Q29 -39 50 -15T92 18L103 24Q67 55 67 108Q67 155 96 193Q52 237 52 292Q52 355 102 398T223 442Q274 442 318 416L329 409ZM299 343Q294 371 273 387T221 404Q192 404 171 388T145 343Q142 326 142 292Q142 248 149 227T179 192Q196 182 222 182Q244 182 260 189T283 207T294 227T299 242Q302 258 302 292T299 343ZM403 -75Q403 -50 389 -34T348 -11T299 -2T245 0H218Q151 0 138 -6Q118 -15 107 -34T95 -74Q95 -84 101 -97T122 -127T170 -155T250 -167Q319 -167 361 -139T403 -75Z"></path><path id="MJMAIN-20" stroke-width="1"></path><path id="MJMAIN-6D" stroke-width="1" d="M41 46H55Q94 46 102 60V68Q102 77 102 91T102 122T103 161T103 203Q103 234 103 269T102 328V351Q99 370 88 376T43 385H25V408Q25 431 27 431L37 432Q47 433 65 434T102 436Q119 437 138 438T167 441T178 442H181V402Q181 364 182 364T187 369T199 384T218 402T247 421T285 437Q305 442 336 442Q351 442 364 440T387 434T406 426T421 417T432 406T441 395T448 384T452 374T455 366L457 361L460 365Q463 369 466 373T475 384T488 397T503 410T523 422T546 432T572 439T603 442Q729 442 740 329Q741 322 741 190V104Q741 66 743 59T754 49Q775 46 803 46H819V0H811L788 1Q764 2 737 2T699 3Q596 3 587 0H579V46H595Q656 46 656 62Q657 64 657 200Q656 335 655 343Q649 371 635 385T611 402T585 404Q540 404 506 370Q479 343 472 315T464 232V168V108Q464 78 465 68T468 55T477 49Q498 46 526 46H542V0H534L510 1Q487 2 460 2T422 3Q319 3 310 0H302V46H318Q379 46 379 62Q380 64 380 200Q379 335 378 343Q372 371 358 385T334 402T308 404Q263 404 229 370Q202 343 195 315T187 232V168V108Q187 78 188 68T191 55T200 49Q221 46 249 46H265V0H257L234 1Q210 2 183 2T145 3Q42 3 33 0H25V46H41Z"></path><path id="MJMAIN-78" stroke-width="1" d="M201 0Q189 3 102 3Q26 3 17 0H11V46H25Q48 47 67 52T96 61T121 78T139 96T160 122T180 150L226 210L168 288Q159 301 149 315T133 336T122 351T113 363T107 370T100 376T94 379T88 381T80 383Q74 383 44 385H16V431H23Q59 429 126 429Q219 429 229 431H237V385Q201 381 201 369Q201 367 211 353T239 315T268 274L272 270L297 304Q329 345 329 358Q329 364 327 369T322 376T317 380T310 384L307 385H302V431H309Q324 428 408 428Q487 428 493 431H499V385H492Q443 385 411 368Q394 360 377 341T312 257L296 236L358 151Q424 61 429 57T446 50Q464 46 499 46H516V0H510H502Q494 1 482 1T457 2T432 2T414 3Q403 3 377 3T327 1L304 0H295V46H298Q309 46 320 51T331 63Q331 65 291 120L250 175Q249 174 219 133T185 88Q181 83 181 74Q181 63 188 55T206 46Q208 46 208 23V0H201Z"></path><path id="MJMATHI-70" stroke-width="1" d="M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z"></path><path id="MJMATHI-5A" stroke-width="1" d="M58 8Q58 23 64 35Q64 36 329 334T596 635L586 637Q575 637 512 637H500H476Q442 637 420 635T365 624T311 598T266 548T228 469Q227 466 226 463T224 458T223 453T222 450L221 448Q218 443 202 443Q185 443 182 453L214 561Q228 606 241 651Q249 679 253 681Q256 683 487 683H718Q723 678 723 675Q723 673 717 649Q189 54 188 52L185 49H274Q369 50 377 51Q452 60 500 100T579 247Q587 272 590 277T603 282H607Q628 282 628 271Q547 5 541 2Q538 0 300 0H124Q58 0 58 8Z"></path><path id="MJMAIN-65" stroke-width="1" d="M28 218Q28 273 48 318T98 391T163 433T229 448Q282 448 320 430T378 380T406 316T415 245Q415 238 408 231H126V216Q126 68 226 36Q246 30 270 30Q312 30 342 62Q359 79 369 104L379 128Q382 131 395 131H398Q415 131 415 121Q415 117 412 108Q393 53 349 21T250 -11Q155 -11 92 58T28 218ZM333 275Q322 403 238 411H236Q228 411 220 410T195 402T166 381T143 340T127 274V267H333V275Z"></path><path id="MJMAIN-70" stroke-width="1" d="M36 -148H50Q89 -148 97 -134V-126Q97 -119 97 -107T97 -77T98 -38T98 6T98 55T98 106Q98 140 98 177T98 243T98 296T97 335T97 351Q94 370 83 376T38 385H20V408Q20 431 22 431L32 432Q42 433 61 434T98 436Q115 437 135 438T165 441T176 442H179V416L180 390L188 397Q247 441 326 441Q407 441 464 377T522 216Q522 115 457 52T310 -11Q242 -11 190 33L182 40V-45V-101Q182 -128 184 -134T195 -145Q216 -148 244 -148H260V-194H252L228 -193Q205 -192 178 -192T140 -191Q37 -191 28 -194H20V-148H36ZM424 218Q424 292 390 347T305 402Q234 402 182 337V98Q222 26 294 26Q345 26 384 80T424 218Z"></path><path id="MJSZ2-2211" stroke-width="1" d="M60 948Q63 950 665 950H1267L1325 815Q1384 677 1388 669H1348L1341 683Q1320 724 1285 761Q1235 809 1174 838T1033 881T882 898T699 902H574H543H251L259 891Q722 258 724 252Q725 250 724 246Q721 243 460 -56L196 -356Q196 -357 407 -357Q459 -357 548 -357T676 -358Q812 -358 896 -353T1063 -332T1204 -283T1307 -196Q1328 -170 1348 -124H1388Q1388 -125 1381 -145T1356 -210T1325 -294L1267 -449L666 -450Q64 -450 61 -448Q55 -446 55 -439Q55 -437 57 -433L590 177Q590 178 557 222T452 366T322 544L56 909L55 924Q55 945 60 948Z"></path><path id="MJMATHI-69" stroke-width="1" d="M184 600Q184 624 203 642T247 661Q265 661 277 649T290 619Q290 596 270 577T226 557Q211 557 198 567T184 600ZM21 287Q21 295 30 318T54 369T98 420T158 442Q197 442 223 419T250 357Q250 340 236 301T196 196T154 83Q149 61 149 51Q149 26 166 26Q175 26 185 29T208 43T235 78T260 137Q263 149 265 151T282 153Q302 153 302 143Q302 135 293 112T268 61T223 11T161 -11Q129 -11 102 10T74 74Q74 91 79 106T122 220Q160 321 166 341T173 380Q173 404 156 404H154Q124 404 99 371T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Z"></path><path id="MJMATHI-6A" stroke-width="1" d="M297 596Q297 627 318 644T361 661Q378 661 389 651T403 623Q403 595 384 576T340 557Q322 557 310 567T297 596ZM288 376Q288 405 262 405Q240 405 220 393T185 362T161 325T144 293L137 279Q135 278 121 278H107Q101 284 101 286T105 299Q126 348 164 391T252 441Q253 441 260 441T272 442Q296 441 316 432Q341 418 354 401T367 348V332L318 133Q267 -67 264 -75Q246 -125 194 -164T75 -204Q25 -204 7 -183T-12 -137Q-12 -110 7 -91T53 -71Q70 -71 82 -81T95 -112Q95 -148 63 -167Q69 -168 77 -168Q111 -168 139 -140T182 -74L193 -32Q204 11 219 72T251 197T278 308T289 365Q289 372 288 376Z"></path><path id="MJMATHI-3BB" stroke-width="1" d="M166 673Q166 685 183 694H202Q292 691 316 644Q322 629 373 486T474 207T524 67Q531 47 537 34T546 15T551 6T555 2T556 -2T550 -11H482Q457 3 450 18T399 152L354 277L340 262Q327 246 293 207T236 141Q211 112 174 69Q123 9 111 -1T83 -12Q47 -12 47 20Q47 37 61 52T199 187Q229 216 266 252T321 306L338 322Q338 323 288 462T234 612Q214 657 183 657Q166 657 166 673Z"></path><path id="MJMAIN-2B" stroke-width="1" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path id="MJMATHI-6B" stroke-width="1" d="M121 647Q121 657 125 670T137 683Q138 683 209 688T282 694Q294 694 294 686Q294 679 244 477Q194 279 194 272Q213 282 223 291Q247 309 292 354T362 415Q402 442 438 442Q468 442 485 423T503 369Q503 344 496 327T477 302T456 291T438 288Q418 288 406 299T394 328Q394 353 410 369T442 390L458 393Q446 405 434 405H430Q398 402 367 380T294 316T228 255Q230 254 243 252T267 246T293 238T320 224T342 206T359 180T365 147Q365 130 360 106T354 66Q354 26 381 26Q429 26 459 145Q461 153 479 153H483Q499 153 499 144Q499 139 496 130Q455 -11 378 -11Q333 -11 305 15T277 90Q277 108 280 121T283 145Q283 167 269 183T234 206T200 217T182 220H180Q168 178 159 139T145 81T136 44T129 20T122 7T111 -2Q98 -11 83 -11Q66 -11 57 -1T48 16Q48 26 85 176T158 471L195 616Q196 629 188 632T149 637H144Q134 637 131 637T124 640T121 647Z"></path><path id="MJMATHI-3BC" stroke-width="1" d="M58 -216Q44 -216 34 -208T23 -186Q23 -176 96 116T173 414Q186 442 219 442Q231 441 239 435T249 423T251 413Q251 401 220 279T187 142Q185 131 185 107V99Q185 26 252 26Q261 26 270 27T287 31T302 38T315 45T327 55T338 65T348 77T356 88T365 100L372 110L408 253Q444 395 448 404Q461 431 491 431Q504 431 512 424T523 412T525 402L449 84Q448 79 448 68Q448 43 455 35T476 26Q485 27 496 35Q517 55 537 131Q543 151 547 152Q549 153 557 153H561Q580 153 580 144Q580 138 575 117T555 63T523 13Q510 0 491 -8Q483 -10 467 -10Q446 -10 429 -4T402 11T385 29T376 44T374 51L368 45Q362 39 350 30T324 12T288 -4T246 -11Q199 -11 153 12L129 -85Q108 -167 104 -180T92 -202Q76 -216 58 -216Z"></path><path id="MJMATHI-73" stroke-width="1" d="M131 289Q131 321 147 354T203 415T300 442Q362 442 390 415T419 355Q419 323 402 308T364 292Q351 292 340 300T328 326Q328 342 337 354T354 372T367 378Q368 378 368 379Q368 382 361 388T336 399T297 405Q249 405 227 379T204 326Q204 301 223 291T278 274T330 259Q396 230 396 163Q396 135 385 107T352 51T289 7T195 -10Q118 -10 86 19T53 87Q53 126 74 143T118 160Q133 160 146 151T160 120Q160 94 142 76T111 58Q109 57 108 57T107 55Q108 52 115 47T146 34T201 27Q237 27 263 38T301 66T318 97T323 122Q323 150 302 164T254 181T195 196T148 231Q131 256 131 289Z"></path><path id="MJSZ4-239B" stroke-width="1" d="M837 1154Q843 1148 843 1145Q843 1141 818 1106T753 1002T667 841T574 604T494 299Q417 -84 417 -609Q417 -641 416 -647T411 -654Q409 -655 366 -655Q299 -655 297 -654Q292 -652 292 -643T291 -583Q293 -400 304 -242T347 110T432 470T574 813T785 1136Q787 1139 790 1142T794 1147T796 1150T799 1152T802 1153T807 1154T813 1154H819H837Z"></path><path id="MJSZ4-239D" stroke-width="1" d="M843 -635Q843 -638 837 -644H820Q801 -644 800 -643Q792 -635 785 -626Q684 -503 605 -363T473 -75T385 216T330 518T302 809T291 1093Q291 1144 291 1153T296 1164Q298 1165 366 1165Q409 1165 411 1164Q415 1163 416 1157T417 1119Q417 529 517 109T833 -617Q843 -631 843 -635Z"></path><path id="MJSZ4-239E" stroke-width="1" d="M31 1143Q31 1154 49 1154H59Q72 1154 75 1152T89 1136Q190 1013 269 873T401 585T489 294T544 -8T572 -299T583 -583Q583 -634 583 -643T577 -654Q575 -655 508 -655Q465 -655 463 -654Q459 -653 458 -647T457 -609Q457 -58 371 340T100 1037Q87 1059 61 1098T31 1143Z"></path><path id="MJSZ4-23A0" stroke-width="1" d="M56 -644H50Q31 -644 31 -635Q31 -632 37 -622Q69 -579 100 -527Q286 -228 371 170T457 1119Q457 1161 462 1164Q464 1165 520 1165Q575 1165 577 1164Q582 1162 582 1153T583 1093Q581 910 570 752T527 400T442 40T300 -303T89 -626Q78 -640 75 -642T61 -644H56Z"></path><path id="MJMATHI-66" stroke-width="1" d="M118 -162Q120 -162 124 -164T135 -167T147 -168Q160 -168 171 -155T187 -126Q197 -99 221 27T267 267T289 382V385H242Q195 385 192 387Q188 390 188 397L195 425Q197 430 203 430T250 431Q298 431 298 432Q298 434 307 482T319 540Q356 705 465 705Q502 703 526 683T550 630Q550 594 529 578T487 561Q443 561 443 603Q443 622 454 636T478 657L487 662Q471 668 457 668Q445 668 434 658T419 630Q412 601 403 552T387 469T380 433Q380 431 435 431Q480 431 487 430T498 424Q499 420 496 407T491 391Q489 386 482 386T428 385H372L349 263Q301 15 282 -47Q255 -132 212 -173Q175 -205 139 -205Q107 -205 81 -186T55 -132Q55 -95 76 -78T118 -61Q162 -61 162 -103Q162 -122 151 -136T127 -157L118 -162Z"></path><path id="MJSZ1-2211" stroke-width="1" d="M61 748Q64 750 489 750H913L954 640Q965 609 976 579T993 533T999 516H979L959 517Q936 579 886 621T777 682Q724 700 655 705T436 710H319Q183 710 183 709Q186 706 348 484T511 259Q517 250 513 244L490 216Q466 188 420 134T330 27L149 -187Q149 -188 362 -188Q388 -188 436 -188T506 -189Q679 -189 778 -162T936 -43Q946 -27 959 6H999L913 -249L489 -250Q65 -250 62 -248Q56 -246 56 -239Q56 -234 118 -161Q186 -81 245 -11L428 206Q428 207 242 462L57 717L56 728Q56 744 61 748Z"></path><path id="MJMATHI-57" stroke-width="1" d="M436 683Q450 683 486 682T553 680Q604 680 638 681T677 682Q695 682 695 674Q695 670 692 659Q687 641 683 639T661 637Q636 636 621 632T600 624T597 615Q597 603 613 377T629 138L631 141Q633 144 637 151T649 170T666 200T690 241T720 295T759 362Q863 546 877 572T892 604Q892 619 873 628T831 637Q817 637 817 647Q817 650 819 660Q823 676 825 679T839 682Q842 682 856 682T895 682T949 681Q1015 681 1034 683Q1048 683 1048 672Q1048 666 1045 655T1038 640T1028 637Q1006 637 988 631T958 617T939 600T927 584L923 578L754 282Q586 -14 585 -15Q579 -22 561 -22Q546 -22 542 -17Q539 -14 523 229T506 480L494 462Q472 425 366 239Q222 -13 220 -15T215 -19Q210 -22 197 -22Q178 -22 176 -15Q176 -12 154 304T131 622Q129 631 121 633T82 637H58Q51 644 51 648Q52 671 64 683H76Q118 680 176 680Q301 680 313 683H323Q329 677 329 674T327 656Q322 641 318 637H297Q236 634 232 620Q262 160 266 136L501 550L499 587Q496 629 489 632Q483 636 447 637Q428 637 422 639T416 648Q416 650 418 660Q419 664 420 669T421 676T424 680T428 682T436 683Z"></path><path id="MJMATHI-3C9" stroke-width="1" d="M495 384Q495 406 514 424T555 443Q574 443 589 425T604 364Q604 334 592 278T555 155T483 38T377 -11Q297 -11 267 66Q266 68 260 61Q201 -11 125 -11Q15 -11 15 139Q15 230 56 325T123 434Q135 441 147 436Q160 429 160 418Q160 406 140 379T94 306T62 208Q61 202 61 187Q61 124 85 100T143 76Q201 76 245 129L253 137V156Q258 297 317 297Q348 297 348 261Q348 243 338 213T318 158L308 135Q309 133 310 129T318 115T334 97T358 83T393 76Q456 76 501 148T546 274Q546 305 533 325T508 357T495 384Z"></path><path id="MJMATHI-44" stroke-width="1" d="M287 628Q287 635 230 637Q207 637 200 638T193 647Q193 655 197 667T204 682Q206 683 403 683Q570 682 590 682T630 676Q702 659 752 597T803 431Q803 275 696 151T444 3L430 1L236 0H125H72Q48 0 41 2T33 11Q33 13 36 25Q40 41 44 43T67 46Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628ZM703 469Q703 507 692 537T666 584T629 613T590 629T555 636Q553 636 541 636T512 636T479 637H436Q392 637 386 627Q384 623 313 339T242 52Q242 48 253 48T330 47Q335 47 349 47T373 46Q499 46 581 128Q617 164 640 212T683 339T703 469Z"></path><path id="MJMAIN-5B" stroke-width="1" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path id="MJMAIN-5D" stroke-width="1" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path><path id="MJMAIN-6C" stroke-width="1" d="M42 46H56Q95 46 103 60V68Q103 77 103 91T103 124T104 167T104 217T104 272T104 329Q104 366 104 407T104 482T104 542T103 586T103 603Q100 622 89 628T44 637H26V660Q26 683 28 683L38 684Q48 685 67 686T104 688Q121 689 141 690T171 693T182 694H185V379Q185 62 186 60Q190 52 198 49Q219 46 247 46H263V0H255L232 1Q209 2 183 2T145 3T107 3T57 1L34 0H26V46H42Z"></path><path id="MJMAIN-6F" stroke-width="1" d="M28 214Q28 309 93 378T250 448Q340 448 405 380T471 215Q471 120 407 55T250 -10Q153 -10 91 57T28 214ZM250 30Q372 30 372 193V225V250Q372 272 371 288T364 326T348 362T317 390T268 410Q263 411 252 411Q222 411 195 399Q152 377 139 338T126 246V226Q126 130 145 91Q177 30 250 30Z"></path><path id="MJSZ2-220F" stroke-width="1" d="M220 812Q220 813 218 819T214 829T208 840T199 853T185 866T166 878T140 887T107 893T66 896H56V950H1221V896H1211Q1080 896 1058 812V-311Q1076 -396 1211 -396H1221V-450H725V-396H735Q864 -396 888 -314Q889 -312 889 -311V896H388V292L389 -311Q405 -396 542 -396H552V-450H56V-396H66Q195 -396 219 -314Q220 -312 220 -311V812Z"></path><path id="MJMATHI-6D" stroke-width="1" d="M21 287Q22 293 24 303T36 341T56 388T88 425T132 442T175 435T205 417T221 395T229 376L231 369Q231 367 232 367L243 378Q303 442 384 442Q401 442 415 440T441 433T460 423T475 411T485 398T493 385T497 373T500 364T502 357L510 367Q573 442 659 442Q713 442 746 415T780 336Q780 285 742 178T704 50Q705 36 709 31T724 26Q752 26 776 56T815 138Q818 149 821 151T837 153Q857 153 857 145Q857 144 853 130Q845 101 831 73T785 17T716 -10Q669 -10 648 17T627 73Q627 92 663 193T700 345Q700 404 656 404H651Q565 404 506 303L499 291L466 157Q433 26 428 16Q415 -11 385 -11Q372 -11 364 -4T353 8T350 18Q350 29 384 161L420 307Q423 322 423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 181Q151 335 151 342Q154 357 154 369Q154 405 129 405Q107 405 92 377T69 316T57 280Q55 278 41 278H27Q21 284 21 287Z"></path><path id="MJSZ4-28" stroke-width="1" d="M758 -1237T758 -1240T752 -1249H736Q718 -1249 717 -1248Q711 -1245 672 -1199Q237 -706 237 251T672 1700Q697 1730 716 1749Q718 1750 735 1750H752Q758 1744 758 1741Q758 1737 740 1713T689 1644T619 1537T540 1380T463 1176Q348 802 348 251Q348 -242 441 -599T744 -1218Q758 -1237 758 -1240Z"></path><path id="MJSZ4-29" stroke-width="1" d="M33 1741Q33 1750 51 1750H60H65Q73 1750 81 1743T119 1700Q554 1207 554 251Q554 -707 119 -1199Q76 -1250 66 -1250Q65 -1250 62 -1250T56 -1249Q55 -1249 53 -1249T49 -1250Q33 -1250 33 -1239Q33 -1236 50 -1214T98 -1150T163 -1052T238 -910T311 -727Q443 -335 443 251Q443 402 436 532T405 831T339 1142T224 1438T50 1716Q33 1737 33 1741Z"></path><path id="MJMATHI-43" stroke-width="1" d="M50 252Q50 367 117 473T286 641T490 704Q580 704 633 653Q642 643 648 636T656 626L657 623Q660 623 684 649Q691 655 699 663T715 679T725 690L740 705H746Q760 705 760 698Q760 694 728 561Q692 422 692 421Q690 416 687 415T669 413H653Q647 419 647 422Q647 423 648 429T650 449T651 481Q651 552 619 605T510 659Q484 659 454 652T382 628T299 572T226 479Q194 422 175 346T156 222Q156 108 232 58Q280 24 350 24Q441 24 512 92T606 240Q610 253 612 255T628 257Q648 257 648 248Q648 243 647 239Q618 132 523 55T319 -22Q206 -22 128 53T50 252Z"></path><path id="MJMAIN-2225" stroke-width="1" d="M133 736Q138 750 153 750Q164 750 170 739Q172 735 172 250T170 -239Q164 -250 152 -250Q144 -250 138 -244L137 -243Q133 -241 133 -179T132 250Q132 731 133 736ZM329 739Q334 750 346 750Q353 750 361 744L362 743Q366 741 366 679T367 250T367 -178T362 -243L361 -244Q355 -250 347 -250Q335 -250 329 -239Q327 -235 327 250T329 739Z"></path><path id="MJMAIN-AF" stroke-width="1" d="M69 544V590H430V544H69Z"></path></defs></svg></div><div id="wmd-preview" class="wmd-preview wmd-preview-full-reader"><div class="md-section-divider"></div><div class="md-section-divider"></div><div class="md-section-divider"></div><div class="md-section-divider"></div><div align="center" data-anchor-id="2nu7">
<img src="https://raw.githubusercontent.com/PaddlePaddle/book/develop/label_semantic_roles/image/dependency_parsing.png" width="800"><br>
图1. 依存句法分析句法树示例
</div><p data-anchor-id="iayl">然而,完全句法分析需要确定句子所包含的全部句法信息,并确定句子各成分之间的关系,是一个非常困难的任务,目前技术下的句法分析准确率并不高,句法分析的细微错误都会导致SRL的错误。为了降低问题的复杂度,同时获得一定的句法结构信息,“浅层句法分析”的思想应运而生。浅层句法分析也称为部分句法分析(partial parsing)或语块划分(chunking)。和完全句法分析得到一颗完整的句法树不同,浅层句法分析只需要识别句子中某些结构相对简单的独立成分,例如:动词短语,这些被识别出来的结构称为语块。为了回避 “无法获得准确率较高的句法树” 所带来的困难,一些研究[<a href="#参考文献">1</a>]也提出了基于语块(chunk)的SRL方法。基于语块的SRL方法将SRL作为一个序列标注问题来解决。序列标注任务一般都会采用BIO表示方式来定义序列标注的标签集,我们先来介绍这种表示方法。在BIO表示法中,B代表语块的开始,I代表语块的中间,O代表语块结束。通过B、I、O 三种标记将不同的语块赋予不同的标签,例如:对于一个角色为A的论元,将它所包含的第一个语块赋予标签B-A,将它所包含的其它语块赋予标签I-A,不属于任何论元的语块赋予标签O。</p><p data-anchor-id="i9sl">我们继续以上面的这句话为例,图1展示了BIO表示方法。</p><div align="center" data-anchor-id="hvcd">
<img src="https://raw.githubusercontent.com/PaddlePaddle/book/develop/label_semantic_roles/image/bio_example.png" width="900"><br>
图2. BIO标注方法示例
</div><p data-anchor-id="3tp9">从上面的例子可以看到,根据序列标注结果可以直接得到论元的语义角色标注结果,是一个相对简单的过程。这种简单性体现在:(1)依赖浅层句法分析,降低了句法分析的要求和难度;(2)没有了候选论元剪除这一步骤;(3)论元的识别和论元标注是同时实现的。这种一体化处理论元识别和论元标注的方法,简化了流程,降低了错误累积的风险,往往能够取得更好的结果。</p><p data-anchor-id="24uv">与基于语块的SRL方法类似,在本教程中我们也将SRL看作一个序列标注问题,不同的是,我们只依赖输入文本序列,不依赖任何额外的语法解析结果或是复杂的人造特征,利用深度神经网络构建一个端到端学习的SRL系统。我们以<a href="http://www.cs.upc.edu/~srlconll/" target="_blank">CoNLL-2004 and CoNLL-2005 Shared Tasks</a>任务中SRL任务的公开数据集为例,实践下面的任务:给定一句话和这句话里的一个谓词,通过序列标注的方式,从句子中找到谓词对应的论元,同时标注它们的语义角色。</p><div class="md-section-divider"></div><h2 data-anchor-id="ykjr" id="模型概览">模型概览</h2><p data-anchor-id="3eo1">循环神经网络(Recurrent Neural Network)是一种对序列建模的重要模型,在自然语言处理任务中有着广泛地应用。不同于前馈神经网络(Feed-forward Neural Network),RNN能够处理输入之间前后关联的问题。LSTM是RNN的一种重要变种,常用来学习长序列中蕴含的长程依赖关系,我们在<a href="https://github.com/PaddlePaddle/book/tree/develop/understand_sentiment" target="_blank">情感分析</a>一篇中已经介绍过,这一篇中我们依然利用LSTM来解决SRL问题。</p><div class="md-section-divider"></div><h3 data-anchor-id="0w86" id="栈式循环神经网络stacked-recurrent-neural-network">栈式循环神经网络(Stacked Recurrent Neural Network)</h3><p data-anchor-id="yadi">深层网络有助于形成层次化特征,网络上层在下层已经学习到的初级特征基础上,形成更复杂的高级特征。尽管LSTM沿时间轴展开后等价于一个非常“深”的前馈网络,但由于LSTM各个时间步参数共享,<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-7-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -687.0516853480245 2084.9444444444443 719.103370696049" style="width: 4.865ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2212" x="583" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="1584" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-7">t-1</script>时刻状态到<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-8-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -647.0516853480245 361.5 679.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use></g></svg></span><script type="math/tex" id="MathJax-Element-8">t</script>时刻的映射,始终只经过了一次非线性映射,也就是说单层LSTM对状态转移的建模是 “浅” 的。堆叠多个LSTM单元,令前一个LSTM<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-9-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -647.0516853480245 361.5 679.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use></g></svg></span><script type="math/tex" id="MathJax-Element-9">t</script>时刻的输出,成为下一个LSTM单元<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-10-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -647.0516853480245 361.5 679.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use></g></svg></span><script type="math/tex" id="MathJax-Element-10">t</script>时刻的输入,帮助我们构建起一个深层网络,我们把它称为第一个版本的栈式循环神经网络。深层网络提高了模型拟合复杂模式的能力,能够更好地建模跨不同时间步的模式[<a href="#参考文献">2</a>]。</p><p data-anchor-id="y9i2">然而,训练一个深层LSTM网络并非易事。纵向堆叠多个LSTM单元可能遇到梯度在纵向深度上传播受阻的问题。通常,堆叠4层LSTM单元可以正常训练,当层数达到4~8层时,会出现性能衰减,这时必须考虑一些新的结构以保证梯度纵向顺畅传播,这是训练深层LSTM网络必须解决的问题。我们可以借鉴LSTM解决 “梯度消失梯度爆炸” 问题的智慧之一:在记忆单元(Memory Cell)这条信息传播的路线上没有非线性映射,当梯度反向传播时既不会衰减、也不会爆炸。因此,深层LSTM模型也可以在纵向上添加一条保证梯度顺畅传播的路径。</p><p data-anchor-id="dvha">一个LSTM单元完成的运算可以被分为三部分:(1)输入到隐层的映射(input-to-hidden) :每个时间步输入信息<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-11-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -463.0516853480245 572.5 495.10337069604896" style="width: 1.274ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-78"></use></g></svg></span><script type="math/tex" id="MathJax-Element-11">x</script>会首先经过一个矩阵映射,再作为遗忘门,输入门,记忆单元,输出门的输入,注意,这一次映射没有引入非线性激活;(2)隐层到隐层的映射(hidden-to-hidden):这一步是LSTM计算的主体,包括遗忘门,输入门,记忆单元更新,输出门的计算;(3)隐层到输出的映射(hidden-to-output):通常是简单的对隐层向量进行激活。我们在第一个版本的栈式网络的基础上,加入一条新的路径:除上一层LSTM输出之外,将前层LSTM的输入到隐层的映射作为的一个新的输入,同时加入一个线性映射去学习一个新的变换。</p><p data-anchor-id="bhtq">图3是最终得到的栈式循环神经网络结构示意图。</p><p align="center" data-anchor-id="jqz2">    
<img src="https://raw.githubusercontent.com/PaddlePaddle/book/develop/label_semantic_roles/image/stacked_lstm.png" width="400"><br>
图3. 基于LSTM的栈式循环神经网络结构示意图
</p><div class="md-section-divider"></div><h3 data-anchor-id="y4pb" id="双向循环神经网络bidirectional-recurrent-neural-network">双向循环神经网络(Bidirectional Recurrent Neural Network)</h3><p data-anchor-id="6cxf">在LSTM中,<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-12-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -647.0516853480245 361.5 679.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use></g></svg></span><script type="math/tex" id="MathJax-Element-12">t</script>时刻的隐藏层向量编码了到<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-13-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -647.0516853480245 361.5 679.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use></g></svg></span><script type="math/tex" id="MathJax-Element-13">t</script>时刻为止所有输入的信息,但<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-14-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -647.0516853480245 361.5 679.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use></g></svg></span><script type="math/tex" id="MathJax-Element-14">t</script>时刻的LSTM可以看到历史,却无法看到未来。在绝大多数自然语言处理任务中,我们几乎总是能拿到整个句子。这种情况下,如果能够像获取历史信息一样,得到未来的信息,对序列学习任务会有很大的帮助。</p><p data-anchor-id="rupi">为了克服这一缺陷,我们可以设计一种双向循环网络单元,它的思想简单且直接:对上一节的栈式循环神经网络进行一个小小的修改,堆叠多个LSTM单元,让每一层LSTM单元分别以:正向、反向、正向 …… 的顺序学习上一层的输出序列。于是,从第2层开始,<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-15-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -647.0516853480245 361.5 679.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use></g></svg></span><script type="math/tex" id="MathJax-Element-15">t</script>时刻我们的LSTM单元便总是可以看到历史和未来的信息。图4是基于LSTM的双向循环神经网络结构示意图。</p><p align="center" data-anchor-id="85x1">    
<img src="https://raw.githubusercontent.com/PaddlePaddle/book/develop/label_semantic_roles/image/bidirectional_stacked_lstm.png" width="500"><br>
图4. 基于LSTM的双向循环神经网络结构示意图
</p><p data-anchor-id="yond">需要说明的是,这种双向RNN结构和Bengio等人在机器翻译任务中使用的双向RNN结构[<a href="#参考文献">3</a>, <a href="#参考文献">4</a>] 并不相同,我们会在后续<a href="https://github.com/PaddlePaddle/book/blob/develop/machine_translation/README.md" target="_blank">机器翻译</a>任务中,介绍另一种双向循环神经网络。</p><div class="md-section-divider"></div><h3 data-anchor-id="bygv" id="条件随机场-conditional-random-field">条件随机场 (Conditional Random Field)</h3><p data-anchor-id="qybp">使用神经网络模型解决问题的思路通常是:前层网络学习输入的特征表示,网络的最后一层在特征基础上完成最终的任务。在SRL任务中,深层LSTM网络学习输入的特征表示,条件随机场(Conditional Random Filed, CRF)在特征的基础上完成序列标注,处于整个网络的末端。</p><p data-anchor-id="nhmk">CRF是一种概率化结构模型,可以看作是一个概率无向图模型,结点表示随机变量,边表示随机变量之间的概率依赖关系。简单来讲,CRF学习条件概率<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-16-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -771.0516853480245 3425 1042.103370696049" style="width: 7.992ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-50"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="751" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="1141" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-7C" x="1993" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="2272" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="3035" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-16">P(X|Y)</script>,其中 <span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-17-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -771.0516853480245 8786.487065625814 1042.103370696049" style="width: 20.386ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="1130" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="2186" y="0"></use><g transform="translate(2576,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-78"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="809" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="3602" y="0"></use><g transform="translate(4047,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-78"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-32" x="809" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="5074" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2E" x="5519" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2E" x="5964" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2E" x="6409" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="6854" y="0"></use><g transform="translate(7299,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-78"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E" x="809" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="8396" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-17">X = (x_1, x_2, ... , x_n)</script> 是输入序列,<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-18-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -771.0516853480245 8451.487065625814 1042.103370696049" style="width: 19.575ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="1041" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="2097" y="0"></use><g transform="translate(2487,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-79"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="693" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="3431" y="0"></use><g transform="translate(3876,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-79"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-32" x="693" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="4821" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2E" x="5266" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2E" x="5711" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2E" x="6156" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="6601" y="0"></use><g transform="translate(7046,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-79"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E" x="693" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="8061" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-18">Y = (y_1, y_2, ... , y_n)</script> 是标记序列;解码过程是给定 <span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-19-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 852.5 725.103370696049" style="width: 1.969ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use></g></svg></span><script type="math/tex" id="MathJax-Element-19">X</script>序列求解令<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-20-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -771.0516853480245 3425 1042.103370696049" style="width: 7.992ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-50"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="751" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="1141" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-7C" x="1904" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="2183" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="3035" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-20">P(Y|X)</script>最大的<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-21-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 763.5 724.103370696049" style="width: 1.737ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59"></use></g></svg></span><script type="math/tex" id="MathJax-Element-21">Y</script>序列,即<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-22-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -771.0516853480245 10177.488526975352 1042.103370696049" style="width: 23.629ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2217" x="1157" y="513"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="1549" y="0"></use><g transform="translate(2606,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-61"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-72" x="500" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-67" x="893" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-20" x="1393" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-6D" x="1644" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-61" x="2477" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-78" x="2978" y="0"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="4958" y="-343"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-50" x="6752" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="7503" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="7893" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-7C" x="8656" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="8935" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="9787" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-22">Y^* = \mbox{arg max}_{Y} P(Y | X)</script></p><p data-anchor-id="c62u">序列标注任务只需要考虑输入和输出都是一个线性序列,并且由于我们只是将输入序列作为条件,不做任何条件独立假设,因此输入序列的元素之间并不存在图结构。综上,在序列标注任务中使用的是如图5所示的定义在链式图上的CRF,称之为线性链条件随机场(Linear Chain Conditional Random Field)。</p><p align="center" data-anchor-id="9is4">    
<img src="https://raw.githubusercontent.com/PaddlePaddle/book/develop/label_semantic_roles/image/linear_chain_crf.png" width="400"><br>
图5. 序列标注任务中使用的线性链条件随机场
</p><p data-anchor-id="r8qo">根据线性链条件随机场上的因子分解定理[<a href="#参考文献">5</a>],在给定观测序列<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-23-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 852.5 725.103370696049" style="width: 1.969ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use></g></svg></span><script type="math/tex" id="MathJax-Element-23">X</script>时,一个特定标记序列<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-24-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 763.5 724.103370696049" style="width: 1.737ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59"></use></g></svg></span><script type="math/tex" id="MathJax-Element-24">Y</script>的概率可以定义为:</p><div class="md-section-divider"></div><p data-anchor-id="tdcj"><span class="MathJax_Preview"></span><div class="MathJax_SVG_Display" role="textbox" aria-readonly="true" style="text-align: center;"><span class="MathJax_SVG" id="MathJax-Element-25-Frame" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="-38.5 -1850.7792646367611 31680.014552427256 3201.5585292735223" style="width: 73.552ex; height: 7.413ex; vertical-align: -3.243ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-70"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="503" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="893" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-7C" x="1656" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="1935" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="2787" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="3454" y="0"></use><g transform="translate(4631,0)"><rect stroke="none" width="2475" height="60" x="0" y="220"></rect><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="987" y="676"></use><g transform="translate(60,-716)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-5A"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="723" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="1113" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="1965" y="0"></use></g></g><g transform="translate(7392,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-65"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-78" x="444" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-70" x="973" y="0"></use></g><g transform="translate(8922,0)"><g transform="translate(0,1830)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-239B" x="0" y="-1156"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-239D" x="0" y="-2517"></use></g><g transform="translate(875,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ2-2211"></use><g transform="translate(147,-1090)"><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="345" y="0"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="1124" y="0"></use></g><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E" x="721" y="1627"></use></g><g transform="translate(2320,0)"><g transform="translate(0,1830)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-239B" x="0" y="-1156"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-239D" x="0" y="-2517"></use></g><g transform="translate(875,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ2-2211"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6A" x="815" y="-1536"></use></g><g transform="translate(2486,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-3BB"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6A" x="825" y="-213"></use></g><g transform="translate(3461,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6A" x="511" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="4215" y="0"></use><g transform="translate(4604,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-79"></use><g transform="translate(490,-150)"><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2212" x="345" y="0"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="1124" y="0"></use></g></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="6343" y="0"></use><g transform="translate(6788,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-79"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69" x="693" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="7623" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="8068" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="8921" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69" x="9366" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="9712" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2B" x="10323" y="0"></use><g transform="translate(11324,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ2-2211"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="760" y="-1569"></use></g><g transform="translate(12935,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-3BC"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="853" y="-213"></use></g><g transform="translate(14007,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-73"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="663" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="14946" y="0"></use><g transform="translate(15335,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-79"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69" x="693" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="16170" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="16615" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="17468" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69" x="17913" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="18258" y="0"></use><g transform="translate(18648,1830)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-239E" x="0" y="-1155"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-23A0" x="0" y="-2517"></use></g></g><g transform="translate(21843,1830)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-239E" x="0" y="-1155"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-23A0" x="0" y="-2517"></use></g></g></g></svg></span></div><script type="math/tex; mode=display" id="MathJax-Element-25">p(Y | X) = \frac{1}{Z(X)} \text{exp}\left(\sum_{i=1}^{n}\left(\sum_{j}\lambda_{j}t_{j} (y_{i - 1}, y_{i}, X, i) + \sum_{k} \mu_k s_k (y_i, X, i)\right)\right)</script></p><p data-anchor-id="ned9">其中<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-26-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -771.0516853480245 2355 1042.103370696049" style="width: 5.444ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-5A"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="723" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="1113" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="1965" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-26">Z(X)</script>是归一化因子,<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-27-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -647.0516853480245 753.1815472394509 962.0602608392912" style="width: 1.737ex; height: 2.201ex; vertical-align: -0.811ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6A" x="511" y="-213"></use></g></svg></span><script type="math/tex" id="MathJax-Element-27">t_j</script> 是定义在边上的特征函数,依赖于当前和前一个位置,称为转移特征,表示对于输入序列<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-28-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 852.5 725.103370696049" style="width: 1.969ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use></g></svg></span><script type="math/tex" id="MathJax-Element-28">X</script>及其标注序列在 <span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-29-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -682.0516853480245 345.5 714.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69"></use></g></svg></span><script type="math/tex" id="MathJax-Element-29">i</script><span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-30-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -687.0516853480245 2068.9444444444443 719.103370696049" style="width: 4.749ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2212" x="567" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="1568" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-30">i - 1</script>位置上标记的转移概率。<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-31-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 938.2561863887845 642.5886520702876" style="width: 2.201ex; height: 1.506ex; vertical-align: -0.463ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-73"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="663" y="-213"></use></g></svg></span><script type="math/tex" id="MathJax-Element-31">s_k</script>是定义在结点上的特征函数,称为状态特征,依赖于当前位置,表示对于观察序列<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-32-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 852.5 725.103370696049" style="width: 1.969ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use></g></svg></span><script type="math/tex" id="MathJax-Element-32">X</script>及其<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-33-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -682.0516853480245 345.5 714.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69"></use></g></svg></span><script type="math/tex" id="MathJax-Element-33">i</script>位置的标记概率。<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-34-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -715.0516853480245 975.1815472394509 1030.0602608392912" style="width: 2.317ex; height: 2.432ex; vertical-align: -0.811ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-3BB"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6A" x="825" y="-213"></use></g></svg></span><script type="math/tex" id="MathJax-Element-34">\lambda_j</script><span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-35-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -463.0516853480245 1072.2561863887845 700.103370696049" style="width: 2.548ex; height: 1.622ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-3BC"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="853" y="-213"></use></g></svg></span><script type="math/tex" id="MathJax-Element-35">\mu_k</script> 分别是转移特征函数和状态特征函数对应的权值。实际上,<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-36-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -647.0516853480245 361.5 679.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-74"></use></g></svg></span><script type="math/tex" id="MathJax-Element-36">t</script><span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-37-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 469.5 495.10337069604896" style="width: 1.042ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-73"></use></g></svg></span><script type="math/tex" id="MathJax-Element-37">s</script>可以用相同的数学形式表示,再对转移特征和状态特在各个位置<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-38-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -682.0516853480245 345.5 714.103370696049" style="width: 0.811ex; height: 1.622ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69"></use></g></svg></span><script type="math/tex" id="MathJax-Element-38">i</script>求和有:<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-39-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -811.9875111244761 14451.096586641503 1126.8798169060667" style="width: 33.591ex; height: 2.664ex; vertical-align: -0.811ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-66"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="693" y="-213"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="959" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="1348" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="2112" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="2557" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="3409" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="4077" y="0"></use><g transform="translate(5133,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ1-2211"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E" x="1494" y="675"></use><g transform="translate(1056,-287)"><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="345" y="0"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="1124" y="0"></use></g></g><g transform="translate(7605,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-66"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="693" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="8564" y="0"></use><g transform="translate(8954,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-79"></use><g transform="translate(490,-150)"><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2212" x="345" y="0"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="1124" y="0"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="1739" y="0"></use><g transform="translate(2184,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-79"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69" x="693" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="3019" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="3464" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="4316" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-69" x="4762" y="0"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="14061" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-39">f_{k}(Y, X) = \sum_{i=1}^{n}f_k({y_{i - 1}, y_i, X, i})</script>,把<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-40-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -726.0516853480245 550.5 952.103370696049" style="width: 1.274ex; height: 2.201ex; vertical-align: -0.579ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-66"></use></g></svg></span><script type="math/tex" id="MathJax-Element-40">f</script>统称为特征函数,于是<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-41-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -771.0516853480245 3425 1042.103370696049" style="width: 7.992ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-50"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="751" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="1141" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-7C" x="1904" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="2183" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="3035" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-41">P(Y|X)</script>可表示为:</p><div class="md-section-divider"></div><p data-anchor-id="55mz"><span class="MathJax_Preview"></span><div class="MathJax_SVG_Display" role="textbox" aria-readonly="true" style="text-align: center;"><span class="MathJax_SVG" id="MathJax-Element-42-Frame" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="-38.5 -1363.5596853480245 17122.901261666455 2601.2018649949378" style="width: 39.73ex; height: 6.023ex; vertical-align: -3.012ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-70"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="503" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="893" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-7C" x="1656" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="1935" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="2787" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-57" x="3232" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="4281" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="4948" y="0"></use><g transform="translate(6124,0)"><rect stroke="none" width="2475" height="60" x="0" y="220"></rect><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="987" y="676"></use><g transform="translate(60,-716)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-5A"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="723" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="1113" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="1965" y="0"></use></g></g><g transform="translate(8886,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-65"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-78" x="444" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-70" x="973" y="0"></use></g><g transform="translate(10582,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ2-2211"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="760" y="-1569"></use></g><g transform="translate(12193,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-3C9"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="880" y="-213"></use></g><g transform="translate(13284,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-66"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6B" x="693" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="14244" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="14633" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="15397" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="15842" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="16694" y="0"></use></g></svg></span></div><script type="math/tex; mode=display" id="MathJax-Element-42">p(Y|X, W) = \frac{1}{Z(X)}\text{exp}\sum_{k}\omega_{k}f_{k}(Y, X)</script></p><p data-anchor-id="j1lk"><span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-43-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 622.5 497.10337069604896" style="width: 1.39ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-3C9"></use></g></svg></span><script type="math/tex" id="MathJax-Element-43">\omega</script>是特征函数对应的权值,是CRF模型要学习的参数。训练时,对于给定的输入序列和对应的标记序列集合<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-44-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -771.0516853480245 16565.212081659516 1042.103370696049" style="width: 38.456ex; height: 2.432ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-44"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="1106" y="0"></use><g transform="translate(2162,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-5B"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="278" y="0"></use><g transform="translate(668,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="1171" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="1950" y="0"></use><g transform="translate(2395,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="822" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="3430" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="3820" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="4265" y="0"></use><g transform="translate(4655,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-32" x="1171" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="5937" y="0"></use><g transform="translate(6382,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-32" x="822" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="7418" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="7807" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2E" x="8252" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2E" x="8697" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2E" x="9143" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="9588" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="10033" y="0"></use><g transform="translate(10422,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-4E" x="1171" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="11979" y="0"></use><g transform="translate(12424,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-4E" x="822" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="13734" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-5D" x="14124" y="0"></use></g></g></svg></span><script type="math/tex" id="MathJax-Element-44">D = \left[(X_1,  Y_1), (X_2 , Y_2) , ... , (X_N, Y_N)\right]</script> ,通过正则化的极大似然估计,求解如下优化目标:</p><div class="md-section-divider"></div><p data-anchor-id="8cxb"><span class="MathJax_Preview"></span><div class="MathJax_SVG_Display" role="textbox" aria-readonly="true" style="text-align: center;"><span class="MathJax_SVG" id="MathJax-Element-45-Frame" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -1771.0516853480244 21574.709772271945 3042.103370696049" style="width: 50.154ex; height: 7.066ex; vertical-align: -3.012ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-4C"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="681" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-3BB" x="1071" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="1654" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-44" x="2099" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="2928" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="3595" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2212" x="4651" y="0"></use><g transform="translate(5430,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-6C"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-6F" x="278" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-67" x="779" y="0"></use></g><g transform="translate(6709,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-28"></use><g transform="translate(792,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ2-220F" x="123" y="0"></use><g transform="translate(0,-1090)"><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6D"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-3D" x="878" y="0"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="1657" y="0"></use></g><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-4E" x="634" y="1627"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-70" x="2484" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="2988" y="0"></use><g transform="translate(3377,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6D" x="822" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-7C" x="4680" y="0"></use><g transform="translate(4958,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6D" x="1171" y="-213"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2C" x="6508" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-57" x="6953" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="8002" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJSZ4-29" x="8391" y="0"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2B" x="16116" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-43" x="17116" y="0"></use><g transform="translate(18164,0)"><rect stroke="none" width="620" height="60" x="0" y="220"></rect><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-31" x="60" y="676"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-32" x="60" y="-686"></use></g><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2225" x="19071" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-57" x="19571" y="0"></use><g transform="translate(20620,0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-2225"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-32" x="707" y="583"></use></g></g></svg></span></div><script type="math/tex; mode=display" id="MathJax-Element-45">L(\lambda, D) = - \text{log}\left(\prod_{m=1}^{N}p(Y_m|X_m, W)\right) + C \frac{1}{2}\lVert W\rVert^{2}</script></p><p data-anchor-id="n3ra">这个优化目标可以通过反向传播算法和整个神经网络一起求解。解码时,对于给定的输入序列<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-46-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 852.5 725.103370696049" style="width: 1.969ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58"></use></g></svg></span><script type="math/tex" id="MathJax-Element-46">X</script>,通过解码算法(通常有:维特比算法、Beam Search)求令出条件概率<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-47-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -826.8583648847041 3453.675 1097.9100502327285" style="width: 7.992ex; height: 2.548ex; vertical-align: -0.695ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-50"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-AF" x="279" y="215"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-28" x="780" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59" x="1169" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-7C" x="1933" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-58" x="2211" y="0"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-29" x="3064" y="0"></use></g></svg></span><script type="math/tex" id="MathJax-Element-47">\bar{P}(Y|X)</script>最大的输出序列 <span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-48-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -826.8583648847041 763.5 846.9100502327285" style="width: 1.737ex; height: 1.969ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-59"></use><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-AF" x="249" y="215"></use></g></svg></span><script type="math/tex" id="MathJax-Element-48">\bar{Y}</script></p><div class="md-section-divider"></div><h3 data-anchor-id="14xq" id="深度双向lstmdb-lstmsrl模型">深度双向LSTM(DB-LSTM)SRL模型</h3><p data-anchor-id="ilbe">在SRL任务中,输入是 “谓词” 和 “一句话”,目标是从这句话中找到谓词的论元,并标注论元的语义角色。如果一个句子含有<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-49-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 600.5 496.10337069604896" style="width: 1.39ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E"></use></g></svg></span><script type="math/tex" id="MathJax-Element-49">n</script>个谓词,这个句子会被处理<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-50-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 600.5 496.10337069604896" style="width: 1.39ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E"></use></g></svg></span><script type="math/tex" id="MathJax-Element-50">n</script>次。一个最为直接的模型是下面这样:</p><ol data-anchor-id="gcqi">
<li>构造输入; <br>
<ul><li>输入1是谓词,输入2是句子</li>
<li>将输入1扩展成和输入2一样长的序列,用one-hot方式表示;</li></ul></li>
<li>one-hot方式的谓词序列和句子序列通过词表,转换为实向量表示的词向量序列;</li>
<li>将步骤2中的2个词向量序列作为双向LSTM的输入,学习输入序列的特征表示;</li>
<li>CRF以步骤3中模型学习到的特征为输入,以标记序列为监督信号,实现序列标注;</li>
</ol><p data-anchor-id="u97f">大家可以尝试上面这种方法。这里,我们提出一些改进,引入两个简单但对提高系统性能非常有效的特征:</p><ul data-anchor-id="gay3">
<li>谓词上下文:上面的方法中,只用到了谓词的词向量表达谓词相关的所有信息,这种方法始终是非常弱的,特别是如果谓词在句子中出现多次,有可能引起一定的歧义。从经验出发,谓词前后若干个词的一个小片段,能够提供更丰富的信息,帮助消解歧义。于是,我们把这样的经验也添加到模型中,为每个谓词同时抽取一个“谓词上下文” 片段,也就是从这个谓词前后各取<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-51-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 600.5 496.10337069604896" style="width: 1.39ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E"></use></g></svg></span><script type="math/tex" id="MathJax-Element-51">n</script>个词构成的一个窗口片段;</li>
<li>谓词上下文区域标记:为句子中的每一个词引入一个0-1二值变量,表示它们是否在“谓词上下文”片段中;</li>
</ul><p data-anchor-id="1zhr">修改后的模型如下(图6是一个深度为4的模型结构示意图):</p><ol data-anchor-id="1dhg">
<li>构造输入 <br>
<ul><li>输入1是句子序列,输入2是谓词序列,输入3是谓词上下文,从句子中抽取这个谓词前后各<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-52-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 600.5 496.10337069604896" style="width: 1.39ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E"></use></g></svg></span><script type="math/tex" id="MathJax-Element-52">n</script>个词,构成谓词上下文,用one-hot方式表示,输入4是谓词上下文区域标记,标记了句子中每一个词是否在谓词上下文中;</li>
<li>将输入2~3均扩展为和输入1一样长的序列;</li></ul></li>
<li>输入1~4均通过词表取词向量转换为实向量表示的词向量序列;其中输入1、3共享同一个词表,输入2和4各自独有词表;</li>
<li>第2步的4个词向量序列作为双向LSTM模型的输入;LSTM模型学习输入序列的特征表示,得到新的特性表示序列;</li>
<li>CRF以第3步中LSTM学习到的特征为输入,以标记序列为监督信号,完成序列标注;</li>
</ol><p align="center" data-anchor-id="ued5">    
<img src="https://raw.githubusercontent.com/PaddlePaddle/book/develop/label_semantic_roles/image/db_lstm_network.png" width="500"><br>
图6. SRL任务上的深层双向LSTM模型
</p><div class="md-section-divider"></div><h2 data-anchor-id="phrn" id="数据准备">数据准备</h2><div class="md-section-divider"></div><h3 data-anchor-id="pgof" id="数据介绍与下载">数据介绍与下载</h3><p data-anchor-id="6teo">在此教程中,我们选用<a href="http://www.cs.upc.edu/~srlconll/" target="_blank">CoNLL 2005</a>SRL任务开放出的数据集作为示例。运行 <code>sh ./get_data.sh</code> 会自动从官方网站上下载原始数据。需要特别说明的是,CoNLL 2005 SRL任务的训练数集和开发集在比赛之后并非免费进行公开,目前,能够获取到的只有测试集,包括Wall Street Journal的23节和Brown语料集中的3节。在本教程中,我们以测试集中的WSJ数据为训练集来讲解模型。但是,由于测试集中样本的数量远远不够,如果希望训练一个可用的神经网络SRL系统,请考虑付费获取全量数据。</p><p data-anchor-id="6usr">原始数据中同时包括了词性标注、命名实体识别、语法解析树等多种信息。本教程中,我们使用test.wsj文件夹中的数据进行训练和测试,并只会用到words文件夹(文本序列)和props文件夹(标注结果)下的数据。本教程使用的数据目录如下:</p><div class="md-section-divider"></div><pre class="prettyprint linenums prettyprinted" data-anchor-id="421q"><ol class="linenums"><li class="L0"><code class="language-text"><span class="pln">conll05st</span><span class="pun">-</span><span class="pln">release</span><span class="pun">/</span></code></li><li class="L1"><code class="language-text"><span class="pun">└──</span><span class="pln"> test</span><span class="pun">.</span><span class="pln">wsj</span></code></li><li class="L2"><code class="language-text"><span class="pln">    </span><span class="pun">├──</span><span class="pln"> props  </span><span class="com"># 标注结果</span></code></li><li class="L3"><code class="language-text"><span class="pln">    </span><span class="pun">└──</span><span class="pln"> words  </span><span class="com"># 输入文本序列</span></code></li></ol></pre><p data-anchor-id="cmqs">标注信息源自Penn TreeBank[<a href="#参考文献">7</a>]和PropBank[<a href="#参考文献">8</a>]的标注结果。PropBank标注结果的标签和我们在文章一开始示例中使用的标注结果标签不同,但原理是相同的,关于标注结果标签含义的说明,请参考论文[<a href="#参考文献">9</a>]。</p><p data-anchor-id="7om6">除数据之外,<code>get_data.sh</code>同时下载了以下资源:</p><table data-anchor-id="fycd" class="table table-striped-white table-bordered">
<thead>
<tr>
 <th>文件名称</th>
 <th>说明</th>
</tr>
</thead>
<tbody><tr>
 <td>word_dict</td>
 <td>输入句子的词典,共计44068个词</td>
</tr>
<tr>
 <td>label_dict</td>
 <td>标记的词典,共计106个标记</td>
</tr>
<tr>
 <td>predicate_dict</td>
 <td>谓词的词典,共计3162个词</td>
</tr>
<tr>
 <td>emb</td>
 <td>一个训练好的词表,32维</td>
</tr>
</tbody></table><p data-anchor-id="3og1">我们在英文维基百科上训练语言模型得到了一份词向量用来初始化SRL模型。在SRL模型训练过程中,词向量不再被更新。关于语言模型和词向量可以参考<a href="https://github.com/PaddlePaddle/book/blob/develop/word2vec/README.md" target="_blank">词向量</a> 这篇教程。我们训练语言模型的语料共有995,000,000个token,词典大小控制为4900,000词。CoNLL 2005训练语料中有5%的词不在这4900,000个词中,我们将它们全部看作未登录词,用<code>&lt;unk&gt;</code>表示。</p><div class="md-section-divider"></div><h3 data-anchor-id="byyx" id="数据预处理">数据预处理</h3><p data-anchor-id="usf0">脚本在下载数据之后,又调用了<code>extract_pair.py</code><code>extract_dict_feature.py</code>两个子脚本进行数据预处理,前者完成了下面的第1步,后者完成了下面的2~4步:</p><ol data-anchor-id="hsaq">
<li>将文本序列和标记序列其合并到一条记录中;</li>
<li>一个句子如果含有<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-1-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 600.5 496.10337069604896" style="width: 1.39ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E"></use></g></svg></span><script type="math/tex" id="MathJax-Element-1">n</script>个谓词,这个句子会被处理<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-2-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 600.5 496.10337069604896" style="width: 1.39ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E"></use></g></svg></span><script type="math/tex" id="MathJax-Element-2">n</script>次,变成<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-3-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -464.0516853480245 600.5 496.10337069604896" style="width: 1.39ex; height: 1.158ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-6E"></use></g></svg></span><script type="math/tex" id="MathJax-Element-3">n</script>条独立的训练样本,每个样本一个不同的谓词;</li>
<li>抽取谓词上下文和构造谓词上下文区域标记;</li>
<li>构造以BIO法表示的标记;</li>
</ol><p data-anchor-id="bgzy"><code>data/feature</code>文件是处理好的模型输入,一行是一条训练样本,以"\t"分隔,共9列,分别是:句子序列、谓词、谓词上下文(占 5 列)、谓词上下区域标志、标注序列。下表是一条训练样本的示例。</p><table data-anchor-id="k68g" class="table table-striped-white table-bordered">
<thead>
<tr>
 <th>句子序列</th>
 <th>谓词</th>
 <th>谓词上下文(窗口 = 5)</th>
 <th>谓词上下文区域标记</th>
 <th>标注序列</th>
</tr>
</thead>
<tbody><tr>
 <td>A</td>
 <td>set</td>
 <td>n't been set . ×</td>
 <td>0</td>
 <td>B-A1</td>
</tr>
<tr>
 <td>record</td>
 <td>set</td>
 <td>n't been set . ×</td>
 <td>0</td>
 <td>I-A1</td>
</tr>
<tr>
 <td>date</td>
 <td>set</td>
 <td>n't been set . ×</td>
 <td>0</td>
 <td>I-A1</td>
</tr>
<tr>
 <td>has</td>
 <td>set</td>
 <td>n't been set . ×</td>
 <td>0</td>
 <td>O</td>
</tr>
<tr>
 <td>n't</td>
 <td>set</td>
 <td>n't been set . ×</td>
 <td>1</td>
 <td>B-AM-NEG</td>
</tr>
<tr>
 <td>been</td>
 <td>set</td>
 <td>n't been set . ×</td>
 <td>1</td>
 <td>O</td>
</tr>
<tr>
 <td>set</td>
 <td>set</td>
 <td>n't been set . ×</td>
 <td>1</td>
 <td>B-V</td>
</tr>
<tr>
 <td>.</td>
 <td>set</td>
 <td>n't been set . ×</td>
 <td>1</td>
 <td>O</td>
</tr>
</tbody></table><div class="md-section-divider"></div><h3 data-anchor-id="u6r3" id="提供数据给-paddlepaddle">提供数据给 PaddlePaddle</h3><ol data-anchor-id="d87o">
<li><p>使用hook函数进行PaddlePaddle输入字段的格式定义。</p>

<pre class="prettyprint linenums prettyprinted"><ol class="linenums"><li class="L0"><code class="language-python"><span class="kwd">def</span><span class="pln"> hook</span><span class="pun">(</span><span class="pln">settings</span><span class="pun">,</span><span class="pln"> word_dict</span><span class="pun">,</span><span class="pln"> label_dict</span><span class="pun">,</span><span class="pln"> predicate_dict</span><span class="pun">,</span><span class="pln"> </span><span class="pun">**</span><span class="pln">kwargs</span><span class="pun">):</span></code></li><li class="L1"><code class="language-python"><span class="pln">    settings</span><span class="pun">.</span><span class="pln">word_dict </span><span class="pun">=</span><span class="pln"> word_dict   </span><span class="com"># 获取句子序列的字典</span></code></li><li class="L2"><code class="language-python"><span class="pln">    settings</span><span class="pun">.</span><span class="pln">label_dict </span><span class="pun">=</span><span class="pln"> label_dict  </span><span class="com"># 获取标记序列的字典</span></code></li><li class="L3"><code class="language-python"><span class="pln">    settings</span><span class="pun">.</span><span class="pln">predicate_dict </span><span class="pun">=</span><span class="pln"> predicate_dict  </span><span class="com"># 获取谓词的字典</span></code></li><li class="L4"><code class="language-python"></code></li><li class="L5"><code class="language-python"><span class="pln">    </span><span class="com">#  所有输入特征都是使用one-hot表示序列,在PaddlePaddle中是interger_value_sequence类型</span></code></li><li class="L6"><code class="language-python"><span class="pln">    </span><span class="com">#  input_types是一个字典,字典中每个元素对应着配置中的一个data_layer,key恰好就是data_layer的名字</span></code></li><li class="L7"><code class="language-python"></code></li><li class="L8"><code class="language-python"><span class="pln">    settings</span><span class="pun">.</span><span class="pln">input_types </span><span class="pun">=</span><span class="pln"> </span><span class="pun">{</span></code></li><li class="L9"><code class="language-python"><span class="pln">            </span><span class="str">'word_data'</span><span class="pun">:</span><span class="pln"> integer_value_sequence</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">word_dict</span><span class="pun">)),</span><span class="pln">    </span><span class="com"># 句子序列</span></code></li><li class="L0"><code class="language-python"><span class="pln">            </span><span class="str">'ctx_n2_data'</span><span class="pun">:</span><span class="pln"> integer_value_sequence</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">word_dict</span><span class="pun">)),</span><span class="pln">  </span><span class="com"># 谓词上下文中的第1个词</span></code></li><li class="L1"><code class="language-python"><span class="pln">            </span><span class="str">'ctx_n1_data'</span><span class="pun">:</span><span class="pln"> integer_value_sequence</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">word_dict</span><span class="pun">)),</span><span class="pln">  </span><span class="com"># 谓词上下文中的第2个词</span></code></li><li class="L2"><code class="language-python"><span class="pln">            </span><span class="str">'ctx_0_data'</span><span class="pun">:</span><span class="pln"> integer_value_sequence</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">word_dict</span><span class="pun">)),</span><span class="pln">   </span><span class="com"># 谓词上下文中的第3个词</span></code></li><li class="L3"><code class="language-python"><span class="pln">            </span><span class="str">'ctx_p1_data'</span><span class="pun">:</span><span class="pln"> integer_value_sequence</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">word_dict</span><span class="pun">)),</span><span class="pln">  </span><span class="com"># 谓词上下文中的第4个词</span></code></li><li class="L4"><code class="language-python"><span class="pln">            </span><span class="str">'ctx_p2_data'</span><span class="pun">:</span><span class="pln"> integer_value_sequence</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">word_dict</span><span class="pun">)),</span><span class="pln">  </span><span class="com"># 谓词上下文中的第5个词</span></code></li><li class="L5"><code class="language-python"><span class="pln">            </span><span class="str">'verb_data'</span><span class="pun">:</span><span class="pln"> integer_value_sequence</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">predicate_dict</span><span class="pun">)),</span><span class="pln">  </span><span class="com"># 谓词</span></code></li><li class="L6"><code class="language-python"><span class="pln">            </span><span class="str">'mark_data'</span><span class="pun">:</span><span class="pln"> integer_value_sequence</span><span class="pun">(</span><span class="lit">2</span><span class="pun">),</span><span class="pln">  </span><span class="com"># 谓词上下文区域标记</span></code></li><li class="L7"><code class="language-python"><span class="pln">            </span><span class="str">'target'</span><span class="pun">:</span><span class="pln"> integer_value_sequence</span><span class="pun">(</span><span class="pln">len</span><span class="pun">(</span><span class="pln">label_dict</span><span class="pun">))</span><span class="pln">  </span><span class="com"># 标记序列</span></code></li><li class="L8"><code class="language-python"><span class="pln">    </span><span class="pun">}</span></code></li></ol></pre></li>
<li><p>使用process将数据逐一提供给PaddlePaddle,只需要考虑如何从原始数据文件中返回一条训练样本。</p>

<pre class="prettyprint linenums prettyprinted"><ol class="linenums"><li class="L0"><code class="language-python"><span class="kwd">def</span><span class="pln"> process</span><span class="pun">(</span><span class="pln">settings</span><span class="pun">,</span><span class="pln"> file_name</span><span class="pun">):</span></code></li><li class="L1"><code class="language-python"><span class="pln">    </span><span class="kwd">with</span><span class="pln"> open</span><span class="pun">(</span><span class="pln">file_name</span><span class="pun">,</span><span class="pln"> </span><span class="str">'r'</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">as</span><span class="pln"> fdata</span><span class="pun">:</span></code></li><li class="L2"><code class="language-python"><span class="pln">        </span><span class="kwd">for</span><span class="pln"> line </span><span class="kwd">in</span><span class="pln"> fdata</span><span class="pun">:</span></code></li><li class="L3"><code class="language-python"><span class="pln">            sentence</span><span class="pun">,</span><span class="pln"> predicate</span><span class="pun">,</span><span class="pln"> ctx_n2</span><span class="pun">,</span><span class="pln"> ctx_n1</span><span class="pun">,</span><span class="pln"> ctx_0</span><span class="pun">,</span><span class="pln"> ctx_p1</span><span class="pun">,</span><span class="pln"> ctx_p2</span><span class="pun">,</span><span class="pln">  mark</span><span class="pun">,</span><span class="pln"> label </span><span class="pun">=</span><span class="pln"> \</span></code></li><li class="L4"><code class="language-python"><span class="pln">                line</span><span class="pun">.</span><span class="pln">strip</span><span class="pun">().</span><span class="pln">split</span><span class="pun">(</span><span class="str">'\t'</span><span class="pun">)</span></code></li><li class="L5"><code class="language-python"></code></li><li class="L6"><code class="language-python"><span class="pln">            </span><span class="com"># 句子文本</span></code></li><li class="L7"><code class="language-python"><span class="pln">            words </span><span class="pun">=</span><span class="pln"> sentence</span><span class="pun">.</span><span class="pln">split</span><span class="pun">()</span></code></li><li class="L8"><code class="language-python"><span class="pln">            sen_len </span><span class="pun">=</span><span class="pln"> len</span><span class="pun">(</span><span class="pln">words</span><span class="pun">)</span></code></li><li class="L9"><code class="language-python"><span class="pln">            word_slot </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">settings</span><span class="pun">.</span><span class="pln">word_dict</span><span class="pun">.</span><span class="pln">get</span><span class="pun">(</span><span class="pln">w</span><span class="pun">,</span><span class="pln"> UNK_IDX</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">for</span><span class="pln"> w </span><span class="kwd">in</span><span class="pln"> words</span><span class="pun">]</span></code></li><li class="L0"><code class="language-python"></code></li><li class="L1"><code class="language-python"><span class="pln">            </span><span class="com"># 一个谓词,这里将谓词扩展成一个和句子一样长的序列</span></code></li><li class="L2"><code class="language-python"><span class="pln">            predicate_slot </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">settings</span><span class="pun">.</span><span class="pln">predicate_dict</span><span class="pun">.</span><span class="pln">get</span><span class="pun">(</span><span class="pln">predicate</span><span class="pun">)]</span><span class="pln"> </span><span class="pun">*</span><span class="pln"> sen_len</span></code></li><li class="L3"><code class="language-python"></code></li><li class="L4"><code class="language-python"><span class="pln">            </span><span class="com"># 在教程中,我们使用一个窗口为 5 的谓词上下文窗口:谓词和这个谓词前后隔两个词</span></code></li><li class="L5"><code class="language-python"><span class="pln">            </span><span class="com"># 这里会将窗口中的每一个词,扩展成和输入句子一样长的序列</span></code></li><li class="L6"><code class="language-python"><span class="pln">            ctx_n2_slot </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">settings</span><span class="pun">.</span><span class="pln">word_dict</span><span class="pun">.</span><span class="pln">get</span><span class="pun">(</span><span class="pln">ctx_n2</span><span class="pun">,</span><span class="pln"> UNK_IDX</span><span class="pun">)]</span><span class="pln"> </span><span class="pun">*</span><span class="pln"> sen_len</span></code></li><li class="L7"><code class="language-python"><span class="pln">            ctx_n1_slot </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">settings</span><span class="pun">.</span><span class="pln">word_dict</span><span class="pun">.</span><span class="pln">get</span><span class="pun">(</span><span class="pln">ctx_n1</span><span class="pun">,</span><span class="pln"> UNK_IDX</span><span class="pun">)]</span><span class="pln"> </span><span class="pun">*</span><span class="pln"> sen_len</span></code></li><li class="L8"><code class="language-python"><span class="pln">            ctx_0_slot </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">settings</span><span class="pun">.</span><span class="pln">word_dict</span><span class="pun">.</span><span class="pln">get</span><span class="pun">(</span><span class="pln">ctx_0</span><span class="pun">,</span><span class="pln"> UNK_IDX</span><span class="pun">)]</span><span class="pln"> </span><span class="pun">*</span><span class="pln"> sen_len</span></code></li><li class="L9"><code class="language-python"><span class="pln">            ctx_p1_slot </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">settings</span><span class="pun">.</span><span class="pln">word_dict</span><span class="pun">.</span><span class="pln">get</span><span class="pun">(</span><span class="pln">ctx_p1</span><span class="pun">,</span><span class="pln"> UNK_IDX</span><span class="pun">)]</span><span class="pln"> </span><span class="pun">*</span><span class="pln"> sen_len</span></code></li><li class="L0"><code class="language-python"><span class="pln">            ctx_p2_slot </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">settings</span><span class="pun">.</span><span class="pln">word_dict</span><span class="pun">.</span><span class="pln">get</span><span class="pun">(</span><span class="pln">ctx_p2</span><span class="pun">,</span><span class="pln"> UNK_IDX</span><span class="pun">)]</span><span class="pln"> </span><span class="pun">*</span><span class="pln"> sen_len</span></code></li><li class="L1"><code class="language-python"></code></li><li class="L2"><code class="language-python"><span class="pln">            </span><span class="com"># 谓词上下文区域标记,是一个二值特征</span></code></li><li class="L3"><code class="language-python"><span class="pln">            marks </span><span class="pun">=</span><span class="pln"> mark</span><span class="pun">.</span><span class="pln">split</span><span class="pun">()</span></code></li><li class="L4"><code class="language-python"><span class="pln">            mark_slot </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">int</span><span class="pun">(</span><span class="pln">w</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">for</span><span class="pln"> w </span><span class="kwd">in</span><span class="pln"> marks</span><span class="pun">]</span></code></li><li class="L5"><code class="language-python"></code></li><li class="L6"><code class="language-python"><span class="pln">            label_list </span><span class="pun">=</span><span class="pln"> label</span><span class="pun">.</span><span class="pln">split</span><span class="pun">()</span></code></li><li class="L7"><code class="language-python"><span class="pln">            label_slot </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">settings</span><span class="pun">.</span><span class="pln">label_dict</span><span class="pun">.</span><span class="pln">get</span><span class="pun">(</span><span class="pln">w</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">for</span><span class="pln"> w </span><span class="kwd">in</span><span class="pln"> label_list</span><span class="pun">]</span></code></li><li class="L8"><code class="language-python"><span class="pln">            </span><span class="kwd">yield</span><span class="pln"> </span><span class="pun">{</span></code></li><li class="L9"><code class="language-python"><span class="pln">                </span><span class="str">'word_data'</span><span class="pun">:</span><span class="pln"> word_slot</span><span class="pun">,</span></code></li><li class="L0"><code class="language-python"><span class="pln">                </span><span class="str">'ctx_n2_data'</span><span class="pun">:</span><span class="pln"> ctx_n2_slot</span><span class="pun">,</span></code></li><li class="L1"><code class="language-python"><span class="pln">                </span><span class="str">'ctx_n1_data'</span><span class="pun">:</span><span class="pln"> ctx_n1_slot</span><span class="pun">,</span></code></li><li class="L2"><code class="language-python"><span class="pln">                </span><span class="str">'ctx_0_data'</span><span class="pun">:</span><span class="pln"> ctx_0_slot</span><span class="pun">,</span></code></li><li class="L3"><code class="language-python"><span class="pln">                </span><span class="str">'ctx_p1_data'</span><span class="pun">:</span><span class="pln"> ctx_p1_slot</span><span class="pun">,</span></code></li><li class="L4"><code class="language-python"><span class="pln">                </span><span class="str">'ctx_p2_data'</span><span class="pun">:</span><span class="pln"> ctx_p2_slot</span><span class="pun">,</span></code></li><li class="L5"><code class="language-python"><span class="pln">                </span><span class="str">'verb_data'</span><span class="pun">:</span><span class="pln"> predicate_slot</span><span class="pun">,</span></code></li><li class="L6"><code class="language-python"><span class="pln">                </span><span class="str">'mark_data'</span><span class="pun">:</span><span class="pln"> mark_slot</span><span class="pun">,</span></code></li><li class="L7"><code class="language-python"><span class="pln">                </span><span class="str">'target'</span><span class="pun">:</span><span class="pln"> label_slot</span></code></li><li class="L8"><code class="language-python"><span class="pln">            </span><span class="pun">}</span><span class="pln">   </span></code></li></ol></pre></li>
</ol><div class="md-section-divider"></div><h2 data-anchor-id="sy5y" id="模型配置说明">模型配置说明</h2><div class="md-section-divider"></div><h3 data-anchor-id="22j1" id="数据定义">数据定义</h3><p data-anchor-id="9w8p">首先通过 define_py_data_sources2 从dataprovider中读入数据。配置文件中会读取三个字典:输入文本序列的字典、标记的字典、谓词的字典,并传给data provider,data provider会利用这三个字典,将相应的文本输入转换成one-hot序列。</p><div class="md-section-divider"></div><pre class="prettyprint linenums prettyprinted" data-anchor-id="7pq1"><ol class="linenums"><li class="L0"><code class="language-python"><span class="pln">define_py_data_sources2</span><span class="pun">(</span></code></li><li class="L1"><code class="language-python"><span class="pln">        train_list</span><span class="pun">=</span><span class="pln">train_list_file</span><span class="pun">,</span></code></li><li class="L2"><code class="language-python"><span class="pln">        test_list</span><span class="pun">=</span><span class="pln">test_list_file</span><span class="pun">,</span></code></li><li class="L3"><code class="language-python"><span class="pln">        module</span><span class="pun">=</span><span class="str">'dataprovider'</span><span class="pun">,</span></code></li><li class="L4"><code class="language-python"><span class="pln">        obj</span><span class="pun">=</span><span class="str">'process'</span><span class="pun">,</span></code></li><li class="L5"><code class="language-python"><span class="pln">        args</span><span class="pun">={</span></code></li><li class="L6"><code class="language-python"><span class="pln">            </span><span class="str">'word_dict'</span><span class="pun">:</span><span class="pln"> word_dict</span><span class="pun">,</span><span class="pln">   </span><span class="com"># 输入文本序列的字典</span></code></li><li class="L7"><code class="language-python"><span class="pln">            </span><span class="str">'label_dict'</span><span class="pun">:</span><span class="pln"> label_dict</span><span class="pun">,</span><span class="pln"> </span><span class="com"># 标记的字典</span></code></li><li class="L8"><code class="language-python"><span class="pln">            </span><span class="str">'predicate_dict'</span><span class="pun">:</span><span class="pln"> predicate_dict  </span><span class="com"># 谓词的词典</span></code></li><li class="L9"><code class="language-python"><span class="pln">        </span><span class="pun">}</span></code></li><li class="L0"><code class="language-python"><span class="pun">)</span></code></li></ol></pre><div class="md-section-divider"></div><h3 data-anchor-id="7i1q" id="算法配置">算法配置</h3><p data-anchor-id="vb6u">在这里,我们指定了模型的训练参数,选择了<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-4-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 1135.406943983867 874.8104774772355" style="width: 2.664ex; height: 2.085ex; vertical-align: -0.463ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-4C"></use><use transform="scale(0.7071067811865476)" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMAIN-32" x="963" y="-213"></use></g></svg></span><script type="math/tex" id="MathJax-Element-4">L_2</script>正则、学习率和batch size,并使用带Momentum的随机梯度下降法作为优化算法。</p><div class="md-section-divider"></div><pre class="prettyprint linenums prettyprinted" data-anchor-id="tsp4"><ol class="linenums"><li class="L0"><code class="language-python"><span class="pln">settings</span><span class="pun">(</span></code></li><li class="L1"><code class="language-python"><span class="pln">    batch_size</span><span class="pun">=</span><span class="lit">150</span><span class="pun">,</span></code></li><li class="L2"><code class="language-python"><span class="pln">    learning_method</span><span class="pun">=</span><span class="typ">MomentumOptimizer</span><span class="pun">(</span><span class="pln">momentum</span><span class="pun">=</span><span class="lit">0</span><span class="pun">),</span></code></li><li class="L3"><code class="language-python"><span class="pln">    learning_rate</span><span class="pun">=</span><span class="lit">2e-2</span><span class="pun">,</span></code></li><li class="L4"><code class="language-python"><span class="pln">    regularization</span><span class="pun">=</span><span class="pln">L2Regularization</span><span class="pun">(</span><span class="lit">8e-4</span><span class="pun">),</span></code></li><li class="L5"><code class="language-python"><span class="pln">    model_average</span><span class="pun">=</span><span class="typ">ModelAverage</span><span class="pun">(</span><span class="pln">average_window</span><span class="pun">=</span><span class="lit">0.5</span><span class="pun">,</span><span class="pln"> max_average_window</span><span class="pun">=</span><span class="lit">10000</span><span class="pun">)</span></code></li><li class="L6"><code class="language-python"><span class="pun">)</span></code></li></ol></pre><div class="md-section-divider"></div><h3 data-anchor-id="19yq" id="模型结构">模型结构</h3><ol data-anchor-id="043w">
<li><p>定义输入数据维度及模型超参数。</p>

<pre class="prettyprint linenums prettyprinted"><ol class="linenums"><li class="L0"><code class="language-python"><span class="pln">mark_dict_len </span><span class="pun">=</span><span class="pln"> </span><span class="lit">2</span><span class="pln">    </span><span class="com"># 谓上下文区域标志的维度,是一个0-1 2值特征,因此维度为2</span></code></li><li class="L1"><code class="language-python"><span class="pln">word_dim </span><span class="pun">=</span><span class="pln"> </span><span class="lit">32</span><span class="pln">        </span><span class="com"># 词向量维度</span></code></li><li class="L2"><code class="language-python"><span class="pln">mark_dim </span><span class="pun">=</span><span class="pln"> </span><span class="lit">5</span><span class="pln">         </span><span class="com"># 谓词上下文区域通过词表被映射为一个实向量,这个是相邻的维度</span></code></li><li class="L3"><code class="language-python"><span class="pln">hidden_dim </span><span class="pun">=</span><span class="pln"> </span><span class="lit">512</span><span class="pln">     </span><span class="com"># LSTM隐层向量的维度 : 512 / 4</span></code></li><li class="L4"><code class="language-python"><span class="pln">depth </span><span class="pun">=</span><span class="pln"> </span><span class="lit">8</span><span class="pln">            </span><span class="com"># 栈式LSTM的深度</span></code></li><li class="L5"><code class="language-python"></code></li><li class="L6"><code class="language-python"><span class="pln">word </span><span class="pun">=</span><span class="pln"> data_layer</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'word_data'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">word_dict_len</span><span class="pun">)</span></code></li><li class="L7"><code class="language-python"><span class="pln">predicate </span><span class="pun">=</span><span class="pln"> data_layer</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'verb_data'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">pred_len</span><span class="pun">)</span></code></li><li class="L8"><code class="language-python"></code></li><li class="L9"><code class="language-python"><span class="pln">ctx_n2 </span><span class="pun">=</span><span class="pln"> data_layer</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'ctx_n2_data'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">word_dict_len</span><span class="pun">)</span></code></li><li class="L0"><code class="language-python"><span class="pln">ctx_n1 </span><span class="pun">=</span><span class="pln"> data_layer</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'ctx_n1_data'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">word_dict_len</span><span class="pun">)</span></code></li><li class="L1"><code class="language-python"><span class="pln">ctx_0 </span><span class="pun">=</span><span class="pln"> data_layer</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'ctx_0_data'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">word_dict_len</span><span class="pun">)</span></code></li><li class="L2"><code class="language-python"><span class="pln">ctx_p1 </span><span class="pun">=</span><span class="pln"> data_layer</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'ctx_p1_data'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">word_dict_len</span><span class="pun">)</span></code></li><li class="L3"><code class="language-python"><span class="pln">ctx_p2 </span><span class="pun">=</span><span class="pln"> data_layer</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'ctx_p2_data'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">word_dict_len</span><span class="pun">)</span></code></li><li class="L4"><code class="language-python"><span class="pln">mark </span><span class="pun">=</span><span class="pln"> data_layer</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'mark_data'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">mark_dict_len</span><span class="pun">)</span></code></li><li class="L5"><code class="language-python"></code></li><li class="L6"><code class="language-python"><span class="kwd">if</span><span class="pln"> </span><span class="kwd">not</span><span class="pln"> is_predict</span><span class="pun">:</span></code></li><li class="L7"><code class="language-python"><span class="pln">    target </span><span class="pun">=</span><span class="pln"> data_layer</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'target'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">label_dict_len</span><span class="pun">)</span><span class="pln">    </span><span class="com"># 标记序列只在训练和测试流程中定义</span></code></li></ol></pre>

<p>这里需要特别说明的是hidden_dim = 512指定了LSTM隐层向量的维度为128维,关于这一点请参考PaddlePaddle官方文档中<a href="http://www.paddlepaddle.org/doc/ui/api/trainer_config_helpers/layers.html#lstmemory" target="_blank">lstmemory</a>的说明。</p></li>
<li><p>将句子序列、谓词、谓词上下文、谓词上下文区域标记通过词表,转换为实向量表示的词向量序列。</p>

<pre class="prettyprint linenums prettyprinted"><ol class="linenums"><li class="L0"><code class="language-python"></code></li><li class="L1"><code class="language-python"></code></li><li class="L2"><code class="language-python"><span class="com"># 在本教程中,我们加载了预训练的词向量,这里设置了:is_static=True</span></code></li><li class="L3"><code class="language-python"></code></li><li class="L4"><code class="language-python"></code></li><li class="L5"><code class="language-python"><span class="com"># is_static 为 True 时保证了在训练 SRL 模型过程中,词表不再更新</span></code></li><li class="L6"><code class="language-python"></code></li><li class="L7"><code class="language-python"><span class="pln">emb_para </span><span class="pun">=</span><span class="pln"> </span><span class="typ">ParameterAttribute</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'emb'</span><span class="pun">,</span><span class="pln"> initial_std</span><span class="pun">=</span><span class="lit">0.</span><span class="pun">,</span><span class="pln"> is_static</span><span class="pun">=</span><span class="kwd">True</span><span class="pun">)</span></code></li><li class="L8"><code class="language-python"></code></li><li class="L9"><code class="language-python"><span class="pln">word_input </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">word</span><span class="pun">,</span><span class="pln"> ctx_n2</span><span class="pun">,</span><span class="pln"> ctx_n1</span><span class="pun">,</span><span class="pln"> ctx_0</span><span class="pun">,</span><span class="pln"> ctx_p1</span><span class="pun">,</span><span class="pln"> ctx_p2</span><span class="pun">]</span></code></li><li class="L0"><code class="language-python"><span class="pln">emb_layers </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span></code></li><li class="L1"><code class="language-python"><span class="pln">        embedding_layer</span><span class="pun">(</span></code></li><li class="L2"><code class="language-python"><span class="pln">            size</span><span class="pun">=</span><span class="pln">word_dim</span><span class="pun">,</span><span class="pln"> input</span><span class="pun">=</span><span class="pln">x</span><span class="pun">,</span><span class="pln"> param_attr</span><span class="pun">=</span><span class="pln">emb_para</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">for</span><span class="pln"> x </span><span class="kwd">in</span><span class="pln"> word_input</span></code></li><li class="L3"><code class="language-python"><span class="pun">]</span></code></li><li class="L4"><code class="language-python"><span class="pln">emb_layers</span><span class="pun">.</span><span class="pln">append</span><span class="pun">(</span><span class="pln">predicate_embedding</span><span class="pun">)</span></code></li><li class="L5"><code class="language-python"><span class="pln">mark_embedding </span><span class="pun">=</span><span class="pln"> embedding_layer</span><span class="pun">(</span></code></li><li class="L6"><code class="language-python"><span class="pln">        name</span><span class="pun">=</span><span class="str">'word_ctx-in_embedding'</span><span class="pun">,</span><span class="pln"> size</span><span class="pun">=</span><span class="pln">mark_dim</span><span class="pun">,</span><span class="pln"> input</span><span class="pun">=</span><span class="pln">mark</span><span class="pun">,</span><span class="pln"> param_attr</span><span class="pun">=</span><span class="pln">std_0</span><span class="pun">)</span></code></li><li class="L7"><code class="language-python"><span class="pln">emb_layers</span><span class="pun">.</span><span class="pln">append</span><span class="pun">(</span><span class="pln">mark_embedding</span><span class="pun">)</span></code></li></ol></pre></li>
<li><p>8个LSTM单元以“正向/反向”的顺序对所有输入序列进行学习。</p>

<pre class="prettyprint linenums prettyprinted"><ol class="linenums"><li class="L0"><code class="language-python"></code></li><li class="L1"><code class="language-python"><span class="com">#  std_0 指定的参数以均值为0的高斯分布初始化,用在LSTM的bias初始化中  </span></code></li><li class="L2"><code class="language-python"></code></li><li class="L3"><code class="language-python"><span class="pln">std_0 </span><span class="pun">=</span><span class="pln"> </span><span class="typ">ParameterAttribute</span><span class="pun">(</span><span class="pln">initial_std</span><span class="pun">=</span><span class="lit">0.</span><span class="pun">)</span></code></li><li class="L4"><code class="language-python"></code></li><li class="L5"><code class="language-python"><span class="pln">hidden_0 </span><span class="pun">=</span><span class="pln"> mixed_layer</span><span class="pun">(</span></code></li><li class="L6"><code class="language-python"><span class="pln">    name</span><span class="pun">=</span><span class="str">'hidden0'</span><span class="pun">,</span></code></li><li class="L7"><code class="language-python"><span class="pln">    size</span><span class="pun">=</span><span class="pln">hidden_dim</span><span class="pun">,</span></code></li><li class="L8"><code class="language-python"><span class="pln">    bias_attr</span><span class="pun">=</span><span class="pln">std_default</span><span class="pun">,</span></code></li><li class="L9"><code class="language-python"><span class="pln">    input</span><span class="pun">=[</span></code></li><li class="L0"><code class="language-python"><span class="pln">        full_matrix_projection</span><span class="pun">(</span></code></li><li class="L1"><code class="language-python"><span class="pln">            input</span><span class="pun">=</span><span class="pln">emb</span><span class="pun">,</span><span class="pln"> param_attr</span><span class="pun">=</span><span class="pln">std_default</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">for</span><span class="pln"> emb </span><span class="kwd">in</span><span class="pln"> emb_layers</span></code></li><li class="L2"><code class="language-python"><span class="pln">    </span><span class="pun">])</span></code></li><li class="L3"><code class="language-python"><span class="pln">lstm_0 </span><span class="pun">=</span><span class="pln"> lstmemory</span><span class="pun">(</span></code></li><li class="L4"><code class="language-python"><span class="pln">    name</span><span class="pun">=</span><span class="str">'lstm0'</span><span class="pun">,</span></code></li><li class="L5"><code class="language-python"><span class="pln">    input</span><span class="pun">=</span><span class="pln">hidden_0</span><span class="pun">,</span></code></li><li class="L6"><code class="language-python"><span class="pln">    act</span><span class="pun">=</span><span class="typ">ReluActivation</span><span class="pun">(),</span></code></li><li class="L7"><code class="language-python"><span class="pln">    gate_act</span><span class="pun">=</span><span class="typ">SigmoidActivation</span><span class="pun">(),</span></code></li><li class="L8"><code class="language-python"><span class="pln">    state_act</span><span class="pun">=</span><span class="typ">SigmoidActivation</span><span class="pun">(),</span></code></li><li class="L9"><code class="language-python"><span class="pln">    bias_attr</span><span class="pun">=</span><span class="pln">std_0</span><span class="pun">,</span></code></li><li class="L0"><code class="language-python"><span class="pln">    param_attr</span><span class="pun">=</span><span class="pln">lstm_para_attr</span><span class="pun">)</span></code></li><li class="L1"><code class="language-python"><span class="pln">    input_tmp </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">hidden_0</span><span class="pun">,</span><span class="pln"> lstm_0</span><span class="pun">]</span></code></li><li class="L2"><code class="language-python"></code></li><li class="L3"><code class="language-python"><span class="kwd">for</span><span class="pln"> i </span><span class="kwd">in</span><span class="pln"> range</span><span class="pun">(</span><span class="lit">1</span><span class="pun">,</span><span class="pln"> depth</span><span class="pun">):</span></code></li><li class="L4"><code class="language-python"><span class="pln">    mix_hidden </span><span class="pun">=</span><span class="pln"> mixed_layer</span><span class="pun">(</span></code></li><li class="L5"><code class="language-python"><span class="pln">        name</span><span class="pun">=</span><span class="str">'hidden'</span><span class="pln"> </span><span class="pun">+</span><span class="pln"> str</span><span class="pun">(</span><span class="pln">i</span><span class="pun">),</span></code></li><li class="L6"><code class="language-python"><span class="pln">        size</span><span class="pun">=</span><span class="pln">hidden_dim</span><span class="pun">,</span></code></li><li class="L7"><code class="language-python"><span class="pln">        bias_attr</span><span class="pun">=</span><span class="pln">std_default</span><span class="pun">,</span></code></li><li class="L8"><code class="language-python"><span class="pln">        input</span><span class="pun">=[</span></code></li><li class="L9"><code class="language-python"><span class="pln">            full_matrix_projection</span><span class="pun">(</span></code></li><li class="L0"><code class="language-python"><span class="pln">                input</span><span class="pun">=</span><span class="pln">input_tmp</span><span class="pun">[</span><span class="lit">0</span><span class="pun">],</span><span class="pln"> param_attr</span><span class="pun">=</span><span class="pln">hidden_para_attr</span><span class="pun">),</span></code></li><li class="L1"><code class="language-python"><span class="pln">            full_matrix_projection</span><span class="pun">(</span></code></li><li class="L2"><code class="language-python"><span class="pln">                input</span><span class="pun">=</span><span class="pln">input_tmp</span><span class="pun">[</span><span class="lit">1</span><span class="pun">],</span><span class="pln"> param_attr</span><span class="pun">=</span><span class="pln">lstm_para_attr</span><span class="pun">)</span></code></li><li class="L3"><code class="language-python"><span class="pln">        </span><span class="pun">])</span></code></li><li class="L4"><code class="language-python"><span class="pln">    lstm </span><span class="pun">=</span><span class="pln"> lstmemory</span><span class="pun">(</span></code></li><li class="L5"><code class="language-python"><span class="pln">        name</span><span class="pun">=</span><span class="str">'lstm'</span><span class="pln"> </span><span class="pun">+</span><span class="pln"> str</span><span class="pun">(</span><span class="pln">i</span><span class="pun">),</span></code></li><li class="L6"><code class="language-python"><span class="pln">        input</span><span class="pun">=</span><span class="pln">mix_hidden</span><span class="pun">,</span></code></li><li class="L7"><code class="language-python"><span class="pln">        act</span><span class="pun">=</span><span class="typ">ReluActivation</span><span class="pun">(),</span></code></li><li class="L8"><code class="language-python"><span class="pln">        gate_act</span><span class="pun">=</span><span class="typ">SigmoidActivation</span><span class="pun">(),</span></code></li><li class="L9"><code class="language-python"><span class="pln">        state_act</span><span class="pun">=</span><span class="typ">SigmoidActivation</span><span class="pun">(),</span></code></li><li class="L0"><code class="language-python"><span class="pln">        reverse</span><span class="pun">=((</span><span class="pln">i </span><span class="pun">%</span><span class="pln"> </span><span class="lit">2</span><span class="pun">)</span><span class="pln"> </span><span class="pun">==</span><span class="pln"> </span><span class="lit">1</span><span class="pun">),</span></code></li><li class="L1"><code class="language-python"><span class="pln">        bias_attr</span><span class="pun">=</span><span class="pln">std_0</span><span class="pun">,</span></code></li><li class="L2"><code class="language-python"><span class="pln">        param_attr</span><span class="pun">=</span><span class="pln">lstm_para_attr</span><span class="pun">)</span></code></li><li class="L3"><code class="language-python"></code></li><li class="L4"><code class="language-python"><span class="pln">    input_tmp </span><span class="pun">=</span><span class="pln"> </span><span class="pun">[</span><span class="pln">mix_hidden</span><span class="pun">,</span><span class="pln"> lstm</span><span class="pun">]</span></code></li></ol></pre></li>
<li><p>取最后一个栈式LSTM的输出和这个LSTM单元的输入到隐层映射,经过一个全连接层映射到标记字典的维度,得到最终的特征向量表示。</p>

<pre class="prettyprint linenums prettyprinted"><ol class="linenums"><li class="L0"><code class="language-python"><span class="pln">feature_out </span><span class="pun">=</span><span class="pln"> mixed_layer</span><span class="pun">(</span></code></li><li class="L1"><code class="language-python"><span class="pln">    name</span><span class="pun">=</span><span class="str">'output'</span><span class="pun">,</span></code></li><li class="L2"><code class="language-python"><span class="pln">    size</span><span class="pun">=</span><span class="pln">label_dict_len</span><span class="pun">,</span></code></li><li class="L3"><code class="language-python"><span class="pln">    bias_attr</span><span class="pun">=</span><span class="pln">std_default</span><span class="pun">,</span></code></li><li class="L4"><code class="language-python"><span class="pln">    input</span><span class="pun">=[</span></code></li><li class="L5"><code class="language-python"><span class="pln">        full_matrix_projection</span><span class="pun">(</span></code></li><li class="L6"><code class="language-python"><span class="pln">            input</span><span class="pun">=</span><span class="pln">input_tmp</span><span class="pun">[</span><span class="lit">0</span><span class="pun">],</span><span class="pln"> param_attr</span><span class="pun">=</span><span class="pln">hidden_para_attr</span><span class="pun">),</span></code></li><li class="L7"><code class="language-python"><span class="pln">        full_matrix_projection</span><span class="pun">(</span></code></li><li class="L8"><code class="language-python"><span class="pln">            input</span><span class="pun">=</span><span class="pln">input_tmp</span><span class="pun">[</span><span class="lit">1</span><span class="pun">],</span><span class="pln"> param_attr</span><span class="pun">=</span><span class="pln">lstm_para_attr</span><span class="pun">)</span></code></li><li class="L9"><code class="language-python"><span class="pln">    </span><span class="pun">],</span><span class="pln"> </span><span class="pun">)</span><span class="pln"> </span></code></li></ol></pre></li>
<li><p>CRF层在网络的末端,完成序列标注。</p>

<pre class="prettyprint linenums prettyprinted"><ol class="linenums"><li class="L0"><code class="language-python"><span class="pln">crf_l </span><span class="pun">=</span><span class="pln"> crf_layer</span><span class="pun">(</span></code></li><li class="L1"><code class="language-python"><span class="pln">        name</span><span class="pun">=</span><span class="str">'crf'</span><span class="pun">,</span></code></li><li class="L2"><code class="language-python"><span class="pln">        size</span><span class="pun">=</span><span class="pln">label_dict_len</span><span class="pun">,</span></code></li><li class="L3"><code class="language-python"><span class="pln">        input</span><span class="pun">=</span><span class="pln">feature_out</span><span class="pun">,</span></code></li><li class="L4"><code class="language-python"><span class="pln">        label</span><span class="pun">=</span><span class="pln">target</span><span class="pun">,</span></code></li><li class="L5"><code class="language-python"><span class="pln">        param_attr</span><span class="pun">=</span><span class="typ">ParameterAttribute</span><span class="pun">(</span></code></li><li class="L6"><code class="language-python"><span class="pln">            name</span><span class="pun">=</span><span class="str">'crfw'</span><span class="pun">,</span><span class="pln"> initial_std</span><span class="pun">=</span><span class="pln">default_std</span><span class="pun">,</span><span class="pln"> learning_rate</span><span class="pun">=</span><span class="pln">mix_hidden_lr</span><span class="pun">))</span></code></li></ol></pre></li>
</ol><div class="md-section-divider"></div><h2 data-anchor-id="ynqw" id="训练模型">训练模型</h2><p data-anchor-id="uaau">执行<code>sh train.sh</code>进行模型的训练,其中指定了总共需要训练150个pass。</p><div class="md-section-divider"></div><pre class="prettyprint linenums prettyprinted" data-anchor-id="ltum"><ol class="linenums"><li class="L0"><code class="language-bash"><span class="pln">paddle train \</span></code></li><li class="L1"><code class="language-bash"><span class="pln">  </span><span class="pun">--</span><span class="pln">config</span><span class="pun">=./</span><span class="pln">db_lstm</span><span class="pun">.</span><span class="pln">py \</span></code></li><li class="L2"><code class="language-bash"><span class="pln">  </span><span class="pun">--</span><span class="pln">save_dir</span><span class="pun">=./</span><span class="pln">output \</span></code></li><li class="L3"><code class="language-bash"><span class="pln">  </span><span class="pun">--</span><span class="pln">trainer_count</span><span class="pun">=</span><span class="lit">1</span><span class="pln"> \</span></code></li><li class="L4"><code class="language-bash"><span class="pln">  </span><span class="pun">--</span><span class="pln">dot_period</span><span class="pun">=</span><span class="lit">500</span><span class="pln"> \</span></code></li><li class="L5"><code class="language-bash"><span class="pln">  </span><span class="pun">--</span><span class="pln">log_period</span><span class="pun">=</span><span class="lit">10</span><span class="pln"> \</span></code></li><li class="L6"><code class="language-bash"><span class="pln">  </span><span class="pun">--</span><span class="pln">num_passes</span><span class="pun">=</span><span class="lit">200</span><span class="pln"> \</span></code></li><li class="L7"><code class="language-bash"><span class="pln">  </span><span class="pun">--</span><span class="pln">use_gpu</span><span class="pun">=</span><span class="pln">false \</span></code></li><li class="L8"><code class="language-bash"><span class="pln">  </span><span class="pun">--</span><span class="pln">show_parameter_stats_period</span><span class="pun">=</span><span class="lit">10</span><span class="pln"> \</span></code></li><li class="L9"><code class="language-bash"><span class="pln">  </span><span class="pun">--</span><span class="pln">test_all_data_in_one_period</span><span class="pun">=</span><span class="lit">1</span><span class="pln"> \</span></code></li><li class="L0"><code class="language-bash"><span class="lit">2</span><span class="pun">&gt;&amp;</span><span class="lit">1</span><span class="pln"> </span><span class="pun">|</span><span class="pln"> tee </span><span class="str">'train.log'</span></code></li></ol></pre><p data-anchor-id="56he">训练日志示例如下。</p><div class="md-section-divider"></div><pre class="prettyprint linenums prettyprinted" data-anchor-id="alxt"><ol class="linenums"><li class="L0"><code class="language-text"><span class="pln">I1224 </span><span class="lit">18</span><span class="pun">:</span><span class="lit">11</span><span class="pun">:</span><span class="lit">53.661479</span><span class="pln">  </span><span class="lit">1433</span><span class="pln"> </span><span class="typ">TrainerInternal</span><span class="pun">.</span><span class="pln">cpp</span><span class="pun">:</span><span class="lit">165</span><span class="pun">]</span><span class="pln">  </span><span class="typ">Batch</span><span class="pun">=</span><span class="lit">880</span><span class="pln"> samples</span><span class="pun">=</span><span class="lit">145305</span><span class="pln"> </span><span class="typ">AvgCost</span><span class="pun">=</span><span class="lit">2.11541</span><span class="pln"> </span><span class="typ">CurrentCost</span><span class="pun">=</span><span class="lit">1.8645</span><span class="pln"> </span><span class="typ">Eval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.607942</span><span class="pln">  </span><span class="typ">CurrentEval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.59322</span></code></li><li class="L1"><code class="language-text"><span class="pln">I1224 </span><span class="lit">18</span><span class="pun">:</span><span class="lit">11</span><span class="pun">:</span><span class="lit">55.254021</span><span class="pln">  </span><span class="lit">1433</span><span class="pln"> </span><span class="typ">TrainerInternal</span><span class="pun">.</span><span class="pln">cpp</span><span class="pun">:</span><span class="lit">165</span><span class="pun">]</span><span class="pln">  </span><span class="typ">Batch</span><span class="pun">=</span><span class="lit">885</span><span class="pln"> samples</span><span class="pun">=</span><span class="lit">146134</span><span class="pln"> </span><span class="typ">AvgCost</span><span class="pun">=</span><span class="lit">2.11408</span><span class="pln"> </span><span class="typ">CurrentCost</span><span class="pun">=</span><span class="lit">1.88156</span><span class="pln"> </span><span class="typ">Eval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.607299</span><span class="pln">  </span><span class="typ">CurrentEval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.494572</span></code></li><li class="L2"><code class="language-text"><span class="pln">I1224 </span><span class="lit">18</span><span class="pun">:</span><span class="lit">11</span><span class="pun">:</span><span class="lit">56.867604</span><span class="pln">  </span><span class="lit">1433</span><span class="pln"> </span><span class="typ">TrainerInternal</span><span class="pun">.</span><span class="pln">cpp</span><span class="pun">:</span><span class="lit">165</span><span class="pun">]</span><span class="pln">  </span><span class="typ">Batch</span><span class="pun">=</span><span class="lit">890</span><span class="pln"> samples</span><span class="pun">=</span><span class="lit">146987</span><span class="pln"> </span><span class="typ">AvgCost</span><span class="pun">=</span><span class="lit">2.11277</span><span class="pln"> </span><span class="typ">CurrentCost</span><span class="pun">=</span><span class="lit">1.88839</span><span class="pln"> </span><span class="typ">Eval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.607203</span><span class="pln">  </span><span class="typ">CurrentEval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.590856</span></code></li><li class="L3"><code class="language-text"><span class="pln">I1224 </span><span class="lit">18</span><span class="pun">:</span><span class="lit">11</span><span class="pun">:</span><span class="lit">58.424069</span><span class="pln">  </span><span class="lit">1433</span><span class="pln"> </span><span class="typ">TrainerInternal</span><span class="pun">.</span><span class="pln">cpp</span><span class="pun">:</span><span class="lit">165</span><span class="pun">]</span><span class="pln">  </span><span class="typ">Batch</span><span class="pun">=</span><span class="lit">895</span><span class="pln"> samples</span><span class="pun">=</span><span class="lit">147793</span><span class="pln"> </span><span class="typ">AvgCost</span><span class="pun">=</span><span class="lit">2.11129</span><span class="pln"> </span><span class="typ">CurrentCost</span><span class="pun">=</span><span class="lit">1.84247</span><span class="pln"> </span><span class="typ">Eval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.607099</span><span class="pln">  </span><span class="typ">CurrentEval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.588089</span></code></li><li class="L4"><code class="language-text"><span class="pln">I1224 </span><span class="lit">18</span><span class="pun">:</span><span class="lit">12</span><span class="pun">:</span><span class="lit">00.006893</span><span class="pln">  </span><span class="lit">1433</span><span class="pln"> </span><span class="typ">TrainerInternal</span><span class="pun">.</span><span class="pln">cpp</span><span class="pun">:</span><span class="lit">165</span><span class="pun">]</span><span class="pln">  </span><span class="typ">Batch</span><span class="pun">=</span><span class="lit">900</span><span class="pln"> samples</span><span class="pun">=</span><span class="lit">148611</span><span class="pln"> </span><span class="typ">AvgCost</span><span class="pun">=</span><span class="lit">2.11148</span><span class="pln"> </span><span class="typ">CurrentCost</span><span class="pun">=</span><span class="lit">2.14526</span><span class="pln"> </span><span class="typ">Eval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.607882</span><span class="pln">  </span><span class="typ">CurrentEval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.749389</span></code></li><li class="L5"><code class="language-text"><span class="pln">I1224 </span><span class="lit">18</span><span class="pun">:</span><span class="lit">12</span><span class="pun">:</span><span class="lit">00.164089</span><span class="pln">  </span><span class="lit">1433</span><span class="pln"> </span><span class="typ">TrainerInternal</span><span class="pun">.</span><span class="pln">cpp</span><span class="pun">:</span><span class="lit">181</span><span class="pun">]</span><span class="pln">  </span><span class="typ">Pass</span><span class="pun">=</span><span class="lit">0</span><span class="pln"> </span><span class="typ">Batch</span><span class="pun">=</span><span class="lit">901</span><span class="pln"> samples</span><span class="pun">=</span><span class="lit">148647</span><span class="pln"> </span><span class="typ">AvgCost</span><span class="pun">=</span><span class="lit">2.11195</span><span class="pln"> </span><span class="typ">Eval</span><span class="pun">:</span><span class="pln"> __sum_evaluator_0__</span><span class="pun">=</span><span class="lit">0.60793</span></code></li></ol></pre><p data-anchor-id="ieyd">经过150个 pass 后,得到平均 error 约为 0.0516055。</p><div class="md-section-divider"></div><h2 data-anchor-id="ve23" id="应用模型">应用模型</h2><p data-anchor-id="gb0e">训练好的<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-5-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 888.5 725.103370696049" style="width: 2.085ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-4E"></use></g></svg></span><script type="math/tex" id="MathJax-Element-5">N</script>个pass,会得到<span class="MathJax_Preview"></span><span class="MathJax_SVG" id="MathJax-Element-6-Frame" role="textbox" aria-readonly="true" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 -704.0516853480245 888.5 725.103370696049" style="width: 2.085ex; height: 1.737ex; vertical-align: -0.116ex; margin: 1px 0px;"><g stroke="black" fill="black" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#MJMATHI-4E"></use></g></svg></span><script type="math/tex" id="MathJax-Element-6">N</script>个模型,我们需要从中选择一个最优模型进行预测。通常做法是在开发集上进行调参,并基于我们关心的某个性能指标选择最优模型。本教程的<code>predict.sh</code>脚本简单地选择了测试集上标记错误最少的那个pass(这里是pass-00100)用于预测。</p><p data-anchor-id="y3vb">预测时,我们需要将配置中的 <code>crf_layer</code> 删掉,替换为 <code>crf_decoding_layer</code>,如下所示:</p><div class="md-section-divider"></div><pre class="prettyprint linenums prettyprinted" data-anchor-id="siiw"><ol class="linenums"><li class="L0"><code class="language-python"><span class="pln">crf_dec_l </span><span class="pun">=</span><span class="pln"> crf_decoding_layer</span><span class="pun">(</span></code></li><li class="L1"><code class="language-python"><span class="pln">        name</span><span class="pun">=</span><span class="str">'crf_dec_l'</span><span class="pun">,</span></code></li><li class="L2"><code class="language-python"><span class="pln">        size</span><span class="pun">=</span><span class="pln">label_dict_len</span><span class="pun">,</span></code></li><li class="L3"><code class="language-python"><span class="pln">        input</span><span class="pun">=</span><span class="pln">feature_out</span><span class="pun">,</span></code></li><li class="L4"><code class="language-python"><span class="pln">        param_attr</span><span class="pun">=</span><span class="typ">ParameterAttribute</span><span class="pun">(</span><span class="pln">name</span><span class="pun">=</span><span class="str">'crfw'</span><span class="pun">))</span></code></li></ol></pre><p data-anchor-id="tr5o">运行<code>python predict.py</code>脚本,便可使用指定的模型进行预测。</p><div class="md-section-divider"></div><pre class="prettyprint linenums prettyprinted" data-anchor-id="joiw"><ol class="linenums"><li class="L0"><code class="language-bash"><span class="pln">python predict</span><span class="pun">.</span><span class="pln">py</span></code></li><li class="L1"><code class="language-bash"><span class="pln">     </span><span class="pun">-</span><span class="pln">c db_lstm</span><span class="pun">.</span><span class="pln">py  </span><span class="com"># 指定配置文件</span></code></li><li class="L2"><code class="language-bash"><span class="pln">     </span><span class="pun">-</span><span class="pln">w output</span><span class="pun">/</span><span class="pln">pass</span><span class="pun">-</span><span class="lit">00100</span><span class="pln">  </span><span class="com"># 指定预测使用的模型所在的路径</span></code></li><li class="L3"><code class="language-bash"><span class="pln">     </span><span class="pun">-</span><span class="pln">l data</span><span class="pun">/</span><span class="pln">targetDict</span><span class="pun">.</span><span class="pln">txt  </span><span class="com"># 指定标记的字典</span></code></li><li class="L4"><code class="language-bash"><span class="pln">     </span><span class="pun">-</span><span class="pln">p data</span><span class="pun">/</span><span class="pln">verbDict</span><span class="pun">.</span><span class="pln">txt  </span><span class="com"># 指定谓词的词典</span></code></li><li class="L5"><code class="language-bash"><span class="pln">     </span><span class="pun">-</span><span class="pln">d data</span><span class="pun">/</span><span class="pln">wordDict</span><span class="pun">.</span><span class="pln">txt </span><span class="com"># 指定输入文本序列的字典</span></code></li><li class="L6"><code class="language-bash"><span class="pln">     </span><span class="pun">-</span><span class="pln">i data</span><span class="pun">/</span><span class="pln">feature  </span><span class="com"># 指定输入数据的路径</span></code></li><li class="L7"><code class="language-bash"><span class="pln">     </span><span class="pun">-</span><span class="pln">o predict</span><span class="pun">.</span><span class="pln">res  </span><span class="com"># 指定标记结果输出到文件的路径</span></code></li></ol></pre><p data-anchor-id="wmwz">预测结束后,在 - o 参数所指定的标记结果文件中,我们会得到如下格式的输出:每行是一条样本,以 “\t” 分隔的 2 列,第一列是输入文本,第二列是标记的结果。通过BIO标记可以直接得到论元的语义角色标签。</p><div class="md-section-divider"></div><pre class="prettyprint linenums prettyprinted" data-anchor-id="40k3"><ol class="linenums"><li class="L0"><code class="language-text"><span class="typ">The</span><span class="pln"> interest</span><span class="pun">-</span><span class="pln">only securities were priced at </span><span class="lit">35</span><span class="pln"> </span><span class="lit">1</span><span class="pln">\/</span><span class="lit">2</span><span class="pln"> to </span><span class="kwd">yield</span><span class="pln"> </span><span class="lit">10.72</span><span class="pln"> </span><span class="pun">%</span><span class="pln"> </span><span class="pun">.</span><span class="pln">  B</span><span class="pun">-</span><span class="pln">A0 I</span><span class="pun">-</span><span class="pln">A0 I</span><span class="pun">-</span><span class="pln">A0 O O O O O O B</span><span class="pun">-</span><span class="pln">V B</span><span class="pun">-</span><span class="pln">A1 I</span><span class="pun">-</span><span class="pln">A1 O</span></code></li></ol></pre><div class="md-section-divider"></div><h2 data-anchor-id="y1o3" id="总结">总结</h2><p data-anchor-id="1fp2">语义角色标注是许多自然语言理解任务的重要中间步骤。这篇教程中我们以语义角色标注任务为例,介绍如何利用PaddlePaddle进行序列标注任务。教程中所介绍的模型来自我们发表的论文[<a href="#参考文献">10</a>]。由于 CoNLL 2005 SRL任务的训练数据目前并非完全开放,教程中只使用测试数据作为示例。在这个过程中,我们希望减少对其它自然语言处理工具的依赖,利用神经网络数据驱动、端到端学习的能力,得到一个和传统方法可比、甚至更好的模型。在论文中我们证实了这种可能性。关于模型更多的信息和讨论可以在论文中找到。</p><div class="md-section-divider"></div><h2 data-anchor-id="8njh" id="参考文献">参考文献</h2><ol data-anchor-id="k346">
<li>Sun W, Sui Z, Wang M, et al. <a href="http://www.aclweb.org/anthology/D09-1#page=1513" target="_blank">Chinese semantic role labeling with shallow parsing</a>[C]//Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3-Volume 3. Association for Computational Linguistics, 2009: 1475-1483.</li>
<li>Pascanu R, Gulcehre C, Cho K, et al. <a href="https://arxiv.org/abs/1312.6026" target="_blank">How to construct deep recurrent neural networks</a>[J]. arXiv preprint arXiv:1312.6026, 2013.</li>
<li>Cho K, Van Merriënboer B, Gulcehre C, et al. <a href="https://arxiv.org/abs/1406.1078" target="_blank">Learning phrase representations using RNN encoder-decoder for statistical machine translation</a>[J]. arXiv preprint arXiv:1406.1078, 2014.</li>
<li>Bahdanau D, Cho K, Bengio Y. <a href="https://arxiv.org/abs/1409.0473" target="_blank">Neural machine translation by jointly learning to align and translate</a>[J]. arXiv preprint arXiv:1409.0473, 2014.</li>
<li>Lafferty J, McCallum A, Pereira F. <a href="http://www.jmlr.org/papers/volume15/doppa14a/source/biblio.bib.old" target="_blank">Conditional random fields: Probabilistic models for segmenting and labeling sequence data</a>[C]//Proceedings of the eighteenth international conference on machine learning, ICML. 2001, 1: 282-289.</li>
<li>李航. 统计学习方法[J]. 清华大学出版社, 北京, 2012.</li>
<li>Marcus M P, Marcinkiewicz M A, Santorini B. <a href="http://repository.upenn.edu/cgi/viewcontent.cgi?article=1246&amp;context=cis_reports" target="_blank">Building a large annotated corpus of English: The Penn Treebank</a>[J]. Computational linguistics, 1993, 19(2): 313-330.</li>
<li>Palmer M, Gildea D, Kingsbury P. <a href="http://www.mitpressjournals.org/doi/pdfplus/10.1162/0891201053630264" target="_blank">The proposition bank: An annotated corpus of semantic roles</a>[J]. Computational linguistics, 2005, 31(1): 71-106.</li>
<li>Carreras X, Màrquez L. <a href="http://www.cs.upc.edu/~srlconll/st05/papers/intro.pdf" target="_blank">Introduction to the CoNLL-2005 shared task: Semantic role labeling</a>[C]//Proceedings of the Ninth Conference on Computational Natural Language Learning. Association for Computational Linguistics, 2005: 152-164.</li>
<li>Zhou J, Xu W. <a href="http://www.aclweb.org/anthology/P/P15/P15-1109.pdf" target="_blank">End-to-end learning of semantic role labeling using recurrent neural networks</a>[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.</li>
</ol><p data-anchor-id="96b7"><br> <br>
<img src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" alt="知识共享许可协议"></p><p data-anchor-id="s4l3">本教程由<a href="http://book.paddlepaddle.org" target="_blank">PaddlePaddle</a>创作,采用<a href="http://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。</p></div>
</body>
</html>