diff --git a/demo2_config.yaml b/config_demo2.yaml similarity index 70% rename from demo2_config.yaml rename to config_demo2.yaml index 23e9d86fcb527090470316a4841fb4a91969f8ab..a033d5b5fdda7a0752104ae8c4ca60182028b1f9 100644 --- a/demo2_config.yaml +++ b/config_demo2.yaml @@ -1,6 +1,6 @@ -task_instance: "mrqa, match4mrqa" -target_tag: 1, 0 -mix_ratio: 1.0, 0.5 +task_instance: "mrqa, mlm4mrqa, match4mrqa" +target_tag: 1, 0, 0 +mix_ratio: 0.5, 1.0, 0.5 save_path: "output_model/secondrun" @@ -12,9 +12,10 @@ do_lower_case: True max_seq_len: 512 batch_size: 5 -num_epochs: 2 +num_epochs: 5 optimizer: "adam" learning_rate: 3e-5 warmup_proportion: 0.1 weight_decay: 0.1 +print_every_n_steps: 1 diff --git a/data/mlm4mrqa/train.txt b/data/mlm4mrqa/train.txt new file mode 100644 index 0000000000000000000000000000000000000000..eb0914450a79c2dca808d227ae86a67bbbad451a --- /dev/null +++ b/data/mlm4mrqa/train.txt @@ -0,0 +1,121 @@ +text_a +From what work of Durkheim's was interaction ritual theory derived? Subsequent to these developments, Randall Collins (2004) formulated his interaction ritual theory by drawing on Durkheim's work on totemic rituals that was extended by Goffman (1964/2013; 1967) into everyday focused encounters. Based on interaction ritual theory, we experience different levels +where is port au prince located in haiti are near the water , while residential neighborhoods are located on the hills above . Its population is difficult to ascertain due to the rapid growth of slums in the hillsides above the +What is the world’s first-ever pilsner type blond lager, the company also awarded the Master Homebrewer Competition held in San Francisco to an award-winning brewer who won the prestigious American Homebrewers Associations' Homebrewer of the Year award in 2013? of the Year award in 2013, becoming the first woman in thirty years, and the first African American person ever to ever win the award. After an extensive career with the California State Legislature she began working for PicoBrew, a product development company in Seattle, WA that specializes in automated brewing equipment. In addition to +the gakkel ridge is a boundary between which two tectonic plates Mid-Atlantic Ridge ( MAR ) is a mid-ocean ridge , a divergent tectonic plate or constructive plate boundary located along the floor of the Atlantic Ocean , and part of the longest mountain range in the world . The ridge extends from a junction with the Gakkel Ridge ( Mid-Arctic Ridge ) northeast of Greenland southward to the Bouvet Triple Junction in the South Atlantic . Although the Mid-Atlantic Ridge is mostly an underwater +what is the song new years day about a song by Irish rock band U2 . It is on their 1983 album War and it was released as the album 's lead single in January 1983 . Written about the Polish Solidarity movement , `` New Year 's Day '' is driven by Adam Clayton 's distinctive bassline and the Edge 's piano and guitar playing . It was the band 's first UK hit +Which is the biggest historical sites? the Declaration of Independence was signed, and the Liberty Bell are the city's most famous attractions. Other historic sites include homes for Edgar Allan Poe, Betsy Ross, and Thaddeus Kosciuszko, early government buildings like the First and Second Banks of +Encyclopedia of Afghan Jihad, is a manual of Jihad in eleven volumes, detailing how to make and use explosives and firearms, how to plan and carry out assassinations and other terrorist acts, and much more, it was found in the London residence of which Islamic cleric, who was the imam of Finsbury Park Mosque in London, England, where he preached Islamic fundamentalism and militant Islamism? and much more. It was found in the London residence of Islamic cleric Sheikh Abu Hamza al-Masri in May 2004. During al-Masri's trial, the prosecution referred to the literature as a "blueprint for terror". [PAR] [TLE] Abu Hamza al-Masri [SEP] Mustafa Kamel Mustafa (Arabic: مصطفى كامل مصطفى‎ ‎ ; born 15 April 1958), also known as Abu Hamza al-Masri ( أبو حمزة المصري , "Abū Ḥamzah al-Maṣrī" – literally, the Egyptian father of Hamza), the Hook +How can static analysis be useful with query languages? the *Abstract interpretation framework has been extended to the field of query languages for relational databases as a way to support sound approximation techniques. The +In what year did the Motion Picture Association of America introduce its rating system? format being those only ones presented in color), depending upon the original content of movies, particularly movies released after the 1968 implementation of the Motion Picture Association of America's ratings system and the concurrent disestablishment of the Motion Picture Production Code. +What was the windspeed once the typhoon weakened? in north Luzon, due to high waters there related to the storm, the news agency said. Presidential spokesman Abigail Valte earlier Saturday urged residents of low-lying and mountainous areas that could be hit hard by the storm to evacuate, the state news agency said, citing an interview conducted on a government radio station. World Vision, the Christian humanitarian organization, said Saturday that it had to postpone some of its relief efforts due to Nalgae, with two of three emergency teams set to deploy once the storm passes. Another team is in Bulcan province, most of which is "still submerged" because of Nesat. The group is focusing its post-Nesat efforts on two communities in Manila and three in the northern Isabela and Zambales provinces. Vouchers are being distributed so people can buy needed items, some emergency supplies are being given out directly to citizens and 3,000 Manila children will receive school supplies. Sherbien Dacalanio, a CNN iReporter in the Philippines, described one area of Manila as being devastated by Nesat. "The damage +What is the initial phantom body pain sensation experienced by people with spinal cord damage? paraplegics, phantom body pain in areas of complete sensory loss. This phantom body pain is initially described as burning or tingling but may evolve into severe crushing or pinching pain, or the sensation of fire running down the legs or +Sarah Etonge is a star in the sport that involves running over what landforms? Sarah Etonge [SEP] Sarah Liengu Etonge (born in Buea, Southwest Province, Cameroon) is a current African mountain running star. She has won the annual Mount Cameroon Race of Hope seven times in her native Buea, in the +Den of Thieves stars which Scottish actor? of Thieves (film) [SEP] Den of Thieves is an upcoming American heist action thriller film directed by Christian Gudegast. The film stars Gerard Butler, 50 Cent, Pablo Schreiber, O'Shea Jackson Jr., Evan Jones, Dawn Olivieri, Mo McRae, and Max Holloway. After studying law, Butler turned to +How long does it take for listeria to make a person sick? Food and Drug Administration said it has teamed up with state officials in the effort. "FDA and its state partners are conducting checks at retail stores, wholesalers and distributors to make sure they have +Steve Haworth did body modification on Dennis Avner, who underwent how many surgeries? 1958 – November 5, 2012) was an American man known for his extensive body modifications, which were intended to increase his resemblance to a tigress. For his 14 surgical procedures towards that goal, he held a world record for "most permanent transformations to look like an animal." The name "Stalking Cat" is a Native American name, +The Song, Your Mirror, is featured on Stars an, album by Simply Red that was released in what year? on the US "Billboard" Hot 100. Written by Mick Hucknall, it was featured on the album "Stars" and reached number 17 on the UK chart when released in July +What award did the animator of the semi-sequel spin-off to Kung Fu Panda receive? work at DreamWorks Animation and for his award-winning student film "Le Building". He joined DreamWorks in 2008, after working on 2D animated films in France. The following year, Perifel won an Annie Award for the short film "Secrets of the Furious Five". He received subsequent nominations for "Kung Fu Panda 2" and "Rise of the Guardians". +Who did Madonna turn to for comfort during her mother's illness? siblings resented housekeepers and invariably rebelled against anyone brought into their home ostensibly to take the place of their beloved mother. Madonna later told Vanity +What do people in Haiti use to contact relatives, friends in earthquake zones? rumbled across Haiti on Tuesday evening. "It's 8:44 p.m. and we're still getting aftershocks! Can hear people gathered in the distance singing prayers," wrote Richard Morse, hotel manager at the Oloffson Hotel in the capital, Port-au-Prince. On Twitter, he captured the aftermath of a 7.0-magnitude earthquake that struck off the coast of Haiti, causing widespread destruction. The quake, which happened shortly before 5 p.m., was followed by at least 18 aftershocks, averaging a magnitude of 5.0, according to the U.S. Geological Survey. As the night went on, the singing and praying intensified and then waned, Morse wrote. What he didn't hear in all the commotion were helicopters or ambulances. Frantic callers broadcast pleas for help. "My cousin has a broken leg and she's trying to get to the hospital," one caller said. "She has two kids. ... How can she get to the hospital?" Are you there? Submit an iReport The man gave his cousin's address, and the channel's commentator asked anyone listening who lived nearby to see whether he could help. Many callers expressed their sympathies and prayers for the people of Haiti, while others told of fallen buildings and damaged homes. The first photographs from the area showed collapsed walls and sidewalks covered with cinder block-size chunks of debris. One photograph, taken by members of a American church mission group, showed a dead man in a gray T-shirt and jeans, hunched on the ground. Another broadcaster, Radio-Tele Ginen, collected and posted photographs on its Web site that hinted at the damage and +What types of geese are used for human consumption? roam widely in the knowledge that they will return home by dusk. The Chinese goose is more aggressive and noisy than other geese and can be used as a guard animal to warn of intruders. The flesh of meat geese is dark-coloured and high in protein, but they deposit fat subcutaneously, although this fat +what is name of new video game? of your favorite fictional universe and weaving them into a new story that captures the imagination of fans. In the dark, cinematic "Batman: Arkham Asylum," Batman must escape from a spooky psychiatric hospital. Buzz is building online for the dark, cinematic game, which reimagines a brooding Batman and his most notorious nemesis, the Joker, for an experience that's reminiscent of "The Dark Knight" blockbuster movie. You want mayhem, insanity and brutality? Get ready for all of it. "This is the Batman movie I would have liked to have written," said Dini, who was scheduled to discuss +what is the name of the large city in southeastern nevada tourism , including gambling . The city serves as world headquarters for the world 's two largest Fortune 500 gaming companies , Harrah 's Entertainment and MGM Mirage . +what does somalia want of using force against the pirates, the Russian news agency Interfax reported. "The questions of freeing the ships and crew are being dealt with in line with the corresponding international practices," Interfax quoted Navy spokesman Igor Dygalo as saying. "For understandable reasons, the use of force would be an extreme measure because it could threaten the life of the international crew of the ship." The pirates took over the MV Faina last week off the coast of Somalia and are demanding +when was nepal declared federal democratic republic country has changed Nepal practically into a federal democratic republic by making 7 unnamed states +animal that live both on land and in water are animals that live predominantly or entirely on land ( e.g. , cats , ants , spiders ) , as compared with aquatic animals , which live predominantly or entirely in the water ( e.g.g. , frogs , or newts +Conservative theology was the official position of what? formed by moderate Southern Baptists who disagreed with the direction in which the Southern Baptist Convention was heading: the Alliance of Baptists in 1987 and +What laws did homeland security waive? environmental groups, Defenders of Wildlife and the Sierra Club, have filed appeals with the U.S. Supreme Court, claiming the waivers are unconstitutional and set a dangerous precedent. "National security and environmental protection do not have to be at odds with each other," says Defenders of Wildlife spokesman Matt Clark. "If we can drop this arbitrary deadline for constructing the fence and go through the proper procedures, then there are inevitably ways to minimize environmental impact, but as it is now it's throwing all of those laws out the window." Mountain lion tracker Jack Childs also worries about the impact of the fence on local wildlife, especially the jaguar. +How many PVA soldiers fought in this battle and lost? U.S. 2nd Infantry Division Warrior Division's 23rd Regimental Combat Team with an attached French Battalion was hemmed in by more than 25,000 Chinese Communist forces. United Nations forces had previously retreated in the face of large Communist forces instead of getting +who plays sugar ray leonard in hands of stone Usher Raymond IV as +What is the name of the person titled Drift King who helped with editorial supervision of the sports manga series, Initial D? is centered on the prefecture of Gunma, more specifically on several mountains in the Kantō region and in their surrounding cities and towns. Although some of the names of the locations the characters race in have been fictionalized, all of the locations in the series are based on actual locations in Japan. He is also known as the Drift King (ドリキン , Dorikin ) for his +From what work of Durkheim's was interaction ritual theory derived? Subsequent to these developments, Randall Collins (2004) formulated his interaction ritual theory by drawing on Durkheim's work on totemic rituals that was extended by Goffman (1964/2013; 1967) into everyday focused encounters. Based on interaction ritual theory, we experience different levels +where is port au prince located in haiti are near the water , while residential neighborhoods are located on the hills above . Its population is difficult to ascertain due to the rapid growth of slums in the hillsides above the +What is the world’s first-ever pilsner type blond lager, the company also awarded the Master Homebrewer Competition held in San Francisco to an award-winning brewer who won the prestigious American Homebrewers Associations' Homebrewer of the Year award in 2013? of the Year award in 2013, becoming the first woman in thirty years, and the first African American person ever to ever win the award. After an extensive career with the California State Legislature she began working for PicoBrew, a product development company in Seattle, WA that specializes in automated brewing equipment. In addition to +the gakkel ridge is a boundary between which two tectonic plates Mid-Atlantic Ridge ( MAR ) is a mid-ocean ridge , a divergent tectonic plate or constructive plate boundary located along the floor of the Atlantic Ocean , and part of the longest mountain range in the world . The ridge extends from a junction with the Gakkel Ridge ( Mid-Arctic Ridge ) northeast of Greenland southward to the Bouvet Triple Junction in the South Atlantic . Although the Mid-Atlantic Ridge is mostly an underwater +what is the song new years day about a song by Irish rock band U2 . It is on their 1983 album War and it was released as the album 's lead single in January 1983 . Written about the Polish Solidarity movement , `` New Year 's Day '' is driven by Adam Clayton 's distinctive bassline and the Edge 's piano and guitar playing . It was the band 's first UK hit +Which is the biggest historical sites? the Declaration of Independence was signed, and the Liberty Bell are the city's most famous attractions. Other historic sites include homes for Edgar Allan Poe, Betsy Ross, and Thaddeus Kosciuszko, early government buildings like the First and Second Banks of +Encyclopedia of Afghan Jihad, is a manual of Jihad in eleven volumes, detailing how to make and use explosives and firearms, how to plan and carry out assassinations and other terrorist acts, and much more, it was found in the London residence of which Islamic cleric, who was the imam of Finsbury Park Mosque in London, England, where he preached Islamic fundamentalism and militant Islamism? and much more. It was found in the London residence of Islamic cleric Sheikh Abu Hamza al-Masri in May 2004. During al-Masri's trial, the prosecution referred to the literature as a "blueprint for terror". [PAR] [TLE] Abu Hamza al-Masri [SEP] Mustafa Kamel Mustafa (Arabic: مصطفى كامل مصطفى‎ ‎ ; born 15 April 1958), also known as Abu Hamza al-Masri ( أبو حمزة المصري , "Abū Ḥamzah al-Maṣrī" – literally, the Egyptian father of Hamza), the Hook +How can static analysis be useful with query languages? the *Abstract interpretation framework has been extended to the field of query languages for relational databases as a way to support sound approximation techniques. The +In what year did the Motion Picture Association of America introduce its rating system? format being those only ones presented in color), depending upon the original content of movies, particularly movies released after the 1968 implementation of the Motion Picture Association of America's ratings system and the concurrent disestablishment of the Motion Picture Production Code. +What was the windspeed once the typhoon weakened? in north Luzon, due to high waters there related to the storm, the news agency said. Presidential spokesman Abigail Valte earlier Saturday urged residents of low-lying and mountainous areas that could be hit hard by the storm to evacuate, the state news agency said, citing an interview conducted on a government radio station. World Vision, the Christian humanitarian organization, said Saturday that it had to postpone some of its relief efforts due to Nalgae, with two of three emergency teams set to deploy once the storm passes. Another team is in Bulcan province, most of which is "still submerged" because of Nesat. The group is focusing its post-Nesat efforts on two communities in Manila and three in the northern Isabela and Zambales provinces. Vouchers are being distributed so people can buy needed items, some emergency supplies are being given out directly to citizens and 3,000 Manila children will receive school supplies. Sherbien Dacalanio, a CNN iReporter in the Philippines, described one area of Manila as being devastated by Nesat. "The damage +What is the initial phantom body pain sensation experienced by people with spinal cord damage? paraplegics, phantom body pain in areas of complete sensory loss. This phantom body pain is initially described as burning or tingling but may evolve into severe crushing or pinching pain, or the sensation of fire running down the legs or +Sarah Etonge is a star in the sport that involves running over what landforms? Sarah Etonge [SEP] Sarah Liengu Etonge (born in Buea, Southwest Province, Cameroon) is a current African mountain running star. She has won the annual Mount Cameroon Race of Hope seven times in her native Buea, in the +Den of Thieves stars which Scottish actor? of Thieves (film) [SEP] Den of Thieves is an upcoming American heist action thriller film directed by Christian Gudegast. The film stars Gerard Butler, 50 Cent, Pablo Schreiber, O'Shea Jackson Jr., Evan Jones, Dawn Olivieri, Mo McRae, and Max Holloway. After studying law, Butler turned to +How long does it take for listeria to make a person sick? Food and Drug Administration said it has teamed up with state officials in the effort. "FDA and its state partners are conducting checks at retail stores, wholesalers and distributors to make sure they have +Steve Haworth did body modification on Dennis Avner, who underwent how many surgeries? 1958 – November 5, 2012) was an American man known for his extensive body modifications, which were intended to increase his resemblance to a tigress. For his 14 surgical procedures towards that goal, he held a world record for "most permanent transformations to look like an animal." The name "Stalking Cat" is a Native American name, +The Song, Your Mirror, is featured on Stars an, album by Simply Red that was released in what year? on the US "Billboard" Hot 100. Written by Mick Hucknall, it was featured on the album "Stars" and reached number 17 on the UK chart when released in July +What award did the animator of the semi-sequel spin-off to Kung Fu Panda receive? work at DreamWorks Animation and for his award-winning student film "Le Building". He joined DreamWorks in 2008, after working on 2D animated films in France. The following year, Perifel won an Annie Award for the short film "Secrets of the Furious Five". He received subsequent nominations for "Kung Fu Panda 2" and "Rise of the Guardians". +Who did Madonna turn to for comfort during her mother's illness? siblings resented housekeepers and invariably rebelled against anyone brought into their home ostensibly to take the place of their beloved mother. Madonna later told Vanity +What do people in Haiti use to contact relatives, friends in earthquake zones? rumbled across Haiti on Tuesday evening. "It's 8:44 p.m. and we're still getting aftershocks! Can hear people gathered in the distance singing prayers," wrote Richard Morse, hotel manager at the Oloffson Hotel in the capital, Port-au-Prince. On Twitter, he captured the aftermath of a 7.0-magnitude earthquake that struck off the coast of Haiti, causing widespread destruction. The quake, which happened shortly before 5 p.m., was followed by at least 18 aftershocks, averaging a magnitude of 5.0, according to the U.S. Geological Survey. As the night went on, the singing and praying intensified and then waned, Morse wrote. What he didn't hear in all the commotion were helicopters or ambulances. Frantic callers broadcast pleas for help. "My cousin has a broken leg and she's trying to get to the hospital," one caller said. "She has two kids. ... How can she get to the hospital?" Are you there? Submit an iReport The man gave his cousin's address, and the channel's commentator asked anyone listening who lived nearby to see whether he could help. Many callers expressed their sympathies and prayers for the people of Haiti, while others told of fallen buildings and damaged homes. The first photographs from the area showed collapsed walls and sidewalks covered with cinder block-size chunks of debris. One photograph, taken by members of a American church mission group, showed a dead man in a gray T-shirt and jeans, hunched on the ground. Another broadcaster, Radio-Tele Ginen, collected and posted photographs on its Web site that hinted at the damage and +What types of geese are used for human consumption? roam widely in the knowledge that they will return home by dusk. The Chinese goose is more aggressive and noisy than other geese and can be used as a guard animal to warn of intruders. The flesh of meat geese is dark-coloured and high in protein, but they deposit fat subcutaneously, although this fat +what is name of new video game? of your favorite fictional universe and weaving them into a new story that captures the imagination of fans. In the dark, cinematic "Batman: Arkham Asylum," Batman must escape from a spooky psychiatric hospital. Buzz is building online for the dark, cinematic game, which reimagines a brooding Batman and his most notorious nemesis, the Joker, for an experience that's reminiscent of "The Dark Knight" blockbuster movie. You want mayhem, insanity and brutality? Get ready for all of it. "This is the Batman movie I would have liked to have written," said Dini, who was scheduled to discuss +what is the name of the large city in southeastern nevada tourism , including gambling . The city serves as world headquarters for the world 's two largest Fortune 500 gaming companies , Harrah 's Entertainment and MGM Mirage . +what does somalia want of using force against the pirates, the Russian news agency Interfax reported. "The questions of freeing the ships and crew are being dealt with in line with the corresponding international practices," Interfax quoted Navy spokesman Igor Dygalo as saying. "For understandable reasons, the use of force would be an extreme measure because it could threaten the life of the international crew of the ship." The pirates took over the MV Faina last week off the coast of Somalia and are demanding +when was nepal declared federal democratic republic country has changed Nepal practically into a federal democratic republic by making 7 unnamed states +animal that live both on land and in water are animals that live predominantly or entirely on land ( e.g. , cats , ants , spiders ) , as compared with aquatic animals , which live predominantly or entirely in the water ( e.g.g. , frogs , or newts +Conservative theology was the official position of what? formed by moderate Southern Baptists who disagreed with the direction in which the Southern Baptist Convention was heading: the Alliance of Baptists in 1987 and +What laws did homeland security waive? environmental groups, Defenders of Wildlife and the Sierra Club, have filed appeals with the U.S. Supreme Court, claiming the waivers are unconstitutional and set a dangerous precedent. "National security and environmental protection do not have to be at odds with each other," says Defenders of Wildlife spokesman Matt Clark. "If we can drop this arbitrary deadline for constructing the fence and go through the proper procedures, then there are inevitably ways to minimize environmental impact, but as it is now it's throwing all of those laws out the window." Mountain lion tracker Jack Childs also worries about the impact of the fence on local wildlife, especially the jaguar. +How many PVA soldiers fought in this battle and lost? U.S. 2nd Infantry Division Warrior Division's 23rd Regimental Combat Team with an attached French Battalion was hemmed in by more than 25,000 Chinese Communist forces. United Nations forces had previously retreated in the face of large Communist forces instead of getting +who plays sugar ray leonard in hands of stone Usher Raymond IV as +What is the name of the person titled Drift King who helped with editorial supervision of the sports manga series, Initial D? is centered on the prefecture of Gunma, more specifically on several mountains in the Kantō region and in their surrounding cities and towns. Although some of the names of the locations the characters race in have been fictionalized, all of the locations in the series are based on actual locations in Japan. He is also known as the Drift King (ドリキン , Dorikin ) for his +From what work of Durkheim's was interaction ritual theory derived? Subsequent to these developments, Randall Collins (2004) formulated his interaction ritual theory by drawing on Durkheim's work on totemic rituals that was extended by Goffman (1964/2013; 1967) into everyday focused encounters. Based on interaction ritual theory, we experience different levels +where is port au prince located in haiti are near the water , while residential neighborhoods are located on the hills above . Its population is difficult to ascertain due to the rapid growth of slums in the hillsides above the +What is the world’s first-ever pilsner type blond lager, the company also awarded the Master Homebrewer Competition held in San Francisco to an award-winning brewer who won the prestigious American Homebrewers Associations' Homebrewer of the Year award in 2013? of the Year award in 2013, becoming the first woman in thirty years, and the first African American person ever to ever win the award. After an extensive career with the California State Legislature she began working for PicoBrew, a product development company in Seattle, WA that specializes in automated brewing equipment. In addition to +the gakkel ridge is a boundary between which two tectonic plates Mid-Atlantic Ridge ( MAR ) is a mid-ocean ridge , a divergent tectonic plate or constructive plate boundary located along the floor of the Atlantic Ocean , and part of the longest mountain range in the world . The ridge extends from a junction with the Gakkel Ridge ( Mid-Arctic Ridge ) northeast of Greenland southward to the Bouvet Triple Junction in the South Atlantic . Although the Mid-Atlantic Ridge is mostly an underwater +what is the song new years day about a song by Irish rock band U2 . It is on their 1983 album War and it was released as the album 's lead single in January 1983 . Written about the Polish Solidarity movement , `` New Year 's Day '' is driven by Adam Clayton 's distinctive bassline and the Edge 's piano and guitar playing . It was the band 's first UK hit +Which is the biggest historical sites? the Declaration of Independence was signed, and the Liberty Bell are the city's most famous attractions. Other historic sites include homes for Edgar Allan Poe, Betsy Ross, and Thaddeus Kosciuszko, early government buildings like the First and Second Banks of +Encyclopedia of Afghan Jihad, is a manual of Jihad in eleven volumes, detailing how to make and use explosives and firearms, how to plan and carry out assassinations and other terrorist acts, and much more, it was found in the London residence of which Islamic cleric, who was the imam of Finsbury Park Mosque in London, England, where he preached Islamic fundamentalism and militant Islamism? and much more. It was found in the London residence of Islamic cleric Sheikh Abu Hamza al-Masri in May 2004. During al-Masri's trial, the prosecution referred to the literature as a "blueprint for terror". [PAR] [TLE] Abu Hamza al-Masri [SEP] Mustafa Kamel Mustafa (Arabic: مصطفى كامل مصطفى‎ ‎ ; born 15 April 1958), also known as Abu Hamza al-Masri ( أبو حمزة المصري , "Abū Ḥamzah al-Maṣrī" – literally, the Egyptian father of Hamza), the Hook +How can static analysis be useful with query languages? the *Abstract interpretation framework has been extended to the field of query languages for relational databases as a way to support sound approximation techniques. The +In what year did the Motion Picture Association of America introduce its rating system? format being those only ones presented in color), depending upon the original content of movies, particularly movies released after the 1968 implementation of the Motion Picture Association of America's ratings system and the concurrent disestablishment of the Motion Picture Production Code. +What was the windspeed once the typhoon weakened? in north Luzon, due to high waters there related to the storm, the news agency said. Presidential spokesman Abigail Valte earlier Saturday urged residents of low-lying and mountainous areas that could be hit hard by the storm to evacuate, the state news agency said, citing an interview conducted on a government radio station. World Vision, the Christian humanitarian organization, said Saturday that it had to postpone some of its relief efforts due to Nalgae, with two of three emergency teams set to deploy once the storm passes. Another team is in Bulcan province, most of which is "still submerged" because of Nesat. The group is focusing its post-Nesat efforts on two communities in Manila and three in the northern Isabela and Zambales provinces. Vouchers are being distributed so people can buy needed items, some emergency supplies are being given out directly to citizens and 3,000 Manila children will receive school supplies. Sherbien Dacalanio, a CNN iReporter in the Philippines, described one area of Manila as being devastated by Nesat. "The damage +What is the initial phantom body pain sensation experienced by people with spinal cord damage? paraplegics, phantom body pain in areas of complete sensory loss. This phantom body pain is initially described as burning or tingling but may evolve into severe crushing or pinching pain, or the sensation of fire running down the legs or +Sarah Etonge is a star in the sport that involves running over what landforms? Sarah Etonge [SEP] Sarah Liengu Etonge (born in Buea, Southwest Province, Cameroon) is a current African mountain running star. She has won the annual Mount Cameroon Race of Hope seven times in her native Buea, in the +Den of Thieves stars which Scottish actor? of Thieves (film) [SEP] Den of Thieves is an upcoming American heist action thriller film directed by Christian Gudegast. The film stars Gerard Butler, 50 Cent, Pablo Schreiber, O'Shea Jackson Jr., Evan Jones, Dawn Olivieri, Mo McRae, and Max Holloway. After studying law, Butler turned to +How long does it take for listeria to make a person sick? Food and Drug Administration said it has teamed up with state officials in the effort. "FDA and its state partners are conducting checks at retail stores, wholesalers and distributors to make sure they have +Steve Haworth did body modification on Dennis Avner, who underwent how many surgeries? 1958 – November 5, 2012) was an American man known for his extensive body modifications, which were intended to increase his resemblance to a tigress. For his 14 surgical procedures towards that goal, he held a world record for "most permanent transformations to look like an animal." The name "Stalking Cat" is a Native American name, +The Song, Your Mirror, is featured on Stars an, album by Simply Red that was released in what year? on the US "Billboard" Hot 100. Written by Mick Hucknall, it was featured on the album "Stars" and reached number 17 on the UK chart when released in July +What award did the animator of the semi-sequel spin-off to Kung Fu Panda receive? work at DreamWorks Animation and for his award-winning student film "Le Building". He joined DreamWorks in 2008, after working on 2D animated films in France. The following year, Perifel won an Annie Award for the short film "Secrets of the Furious Five". He received subsequent nominations for "Kung Fu Panda 2" and "Rise of the Guardians". +Who did Madonna turn to for comfort during her mother's illness? siblings resented housekeepers and invariably rebelled against anyone brought into their home ostensibly to take the place of their beloved mother. Madonna later told Vanity +What do people in Haiti use to contact relatives, friends in earthquake zones? rumbled across Haiti on Tuesday evening. "It's 8:44 p.m. and we're still getting aftershocks! Can hear people gathered in the distance singing prayers," wrote Richard Morse, hotel manager at the Oloffson Hotel in the capital, Port-au-Prince. On Twitter, he captured the aftermath of a 7.0-magnitude earthquake that struck off the coast of Haiti, causing widespread destruction. The quake, which happened shortly before 5 p.m., was followed by at least 18 aftershocks, averaging a magnitude of 5.0, according to the U.S. Geological Survey. As the night went on, the singing and praying intensified and then waned, Morse wrote. What he didn't hear in all the commotion were helicopters or ambulances. Frantic callers broadcast pleas for help. "My cousin has a broken leg and she's trying to get to the hospital," one caller said. "She has two kids. ... How can she get to the hospital?" Are you there? Submit an iReport The man gave his cousin's address, and the channel's commentator asked anyone listening who lived nearby to see whether he could help. Many callers expressed their sympathies and prayers for the people of Haiti, while others told of fallen buildings and damaged homes. The first photographs from the area showed collapsed walls and sidewalks covered with cinder block-size chunks of debris. One photograph, taken by members of a American church mission group, showed a dead man in a gray T-shirt and jeans, hunched on the ground. Another broadcaster, Radio-Tele Ginen, collected and posted photographs on its Web site that hinted at the damage and +What types of geese are used for human consumption? roam widely in the knowledge that they will return home by dusk. The Chinese goose is more aggressive and noisy than other geese and can be used as a guard animal to warn of intruders. The flesh of meat geese is dark-coloured and high in protein, but they deposit fat subcutaneously, although this fat +what is name of new video game? of your favorite fictional universe and weaving them into a new story that captures the imagination of fans. In the dark, cinematic "Batman: Arkham Asylum," Batman must escape from a spooky psychiatric hospital. Buzz is building online for the dark, cinematic game, which reimagines a brooding Batman and his most notorious nemesis, the Joker, for an experience that's reminiscent of "The Dark Knight" blockbuster movie. You want mayhem, insanity and brutality? Get ready for all of it. "This is the Batman movie I would have liked to have written," said Dini, who was scheduled to discuss +what is the name of the large city in southeastern nevada tourism , including gambling . The city serves as world headquarters for the world 's two largest Fortune 500 gaming companies , Harrah 's Entertainment and MGM Mirage . +what does somalia want of using force against the pirates, the Russian news agency Interfax reported. "The questions of freeing the ships and crew are being dealt with in line with the corresponding international practices," Interfax quoted Navy spokesman Igor Dygalo as saying. "For understandable reasons, the use of force would be an extreme measure because it could threaten the life of the international crew of the ship." The pirates took over the MV Faina last week off the coast of Somalia and are demanding +when was nepal declared federal democratic republic country has changed Nepal practically into a federal democratic republic by making 7 unnamed states +animal that live both on land and in water are animals that live predominantly or entirely on land ( e.g. , cats , ants , spiders ) , as compared with aquatic animals , which live predominantly or entirely in the water ( e.g.g. , frogs , or newts +Conservative theology was the official position of what? formed by moderate Southern Baptists who disagreed with the direction in which the Southern Baptist Convention was heading: the Alliance of Baptists in 1987 and +What laws did homeland security waive? environmental groups, Defenders of Wildlife and the Sierra Club, have filed appeals with the U.S. Supreme Court, claiming the waivers are unconstitutional and set a dangerous precedent. "National security and environmental protection do not have to be at odds with each other," says Defenders of Wildlife spokesman Matt Clark. "If we can drop this arbitrary deadline for constructing the fence and go through the proper procedures, then there are inevitably ways to minimize environmental impact, but as it is now it's throwing all of those laws out the window." Mountain lion tracker Jack Childs also worries about the impact of the fence on local wildlife, especially the jaguar. +How many PVA soldiers fought in this battle and lost? U.S. 2nd Infantry Division Warrior Division's 23rd Regimental Combat Team with an attached French Battalion was hemmed in by more than 25,000 Chinese Communist forces. United Nations forces had previously retreated in the face of large Communist forces instead of getting +who plays sugar ray leonard in hands of stone Usher Raymond IV as +What is the name of the person titled Drift King who helped with editorial supervision of the sports manga series, Initial D? is centered on the prefecture of Gunma, more specifically on several mountains in the Kantō region and in their surrounding cities and towns. Although some of the names of the locations the characters race in have been fictionalized, all of the locations in the series are based on actual locations in Japan. He is also known as the Drift King (ドリキン , Dorikin ) for his +From what work of Durkheim's was interaction ritual theory derived? Subsequent to these developments, Randall Collins (2004) formulated his interaction ritual theory by drawing on Durkheim's work on totemic rituals that was extended by Goffman (1964/2013; 1967) into everyday focused encounters. Based on interaction ritual theory, we experience different levels +where is port au prince located in haiti are near the water , while residential neighborhoods are located on the hills above . Its population is difficult to ascertain due to the rapid growth of slums in the hillsides above the +What is the world’s first-ever pilsner type blond lager, the company also awarded the Master Homebrewer Competition held in San Francisco to an award-winning brewer who won the prestigious American Homebrewers Associations' Homebrewer of the Year award in 2013? of the Year award in 2013, becoming the first woman in thirty years, and the first African American person ever to ever win the award. After an extensive career with the California State Legislature she began working for PicoBrew, a product development company in Seattle, WA that specializes in automated brewing equipment. In addition to +the gakkel ridge is a boundary between which two tectonic plates Mid-Atlantic Ridge ( MAR ) is a mid-ocean ridge , a divergent tectonic plate or constructive plate boundary located along the floor of the Atlantic Ocean , and part of the longest mountain range in the world . The ridge extends from a junction with the Gakkel Ridge ( Mid-Arctic Ridge ) northeast of Greenland southward to the Bouvet Triple Junction in the South Atlantic . Although the Mid-Atlantic Ridge is mostly an underwater +what is the song new years day about a song by Irish rock band U2 . It is on their 1983 album War and it was released as the album 's lead single in January 1983 . Written about the Polish Solidarity movement , `` New Year 's Day '' is driven by Adam Clayton 's distinctive bassline and the Edge 's piano and guitar playing . It was the band 's first UK hit +Which is the biggest historical sites? the Declaration of Independence was signed, and the Liberty Bell are the city's most famous attractions. Other historic sites include homes for Edgar Allan Poe, Betsy Ross, and Thaddeus Kosciuszko, early government buildings like the First and Second Banks of +Encyclopedia of Afghan Jihad, is a manual of Jihad in eleven volumes, detailing how to make and use explosives and firearms, how to plan and carry out assassinations and other terrorist acts, and much more, it was found in the London residence of which Islamic cleric, who was the imam of Finsbury Park Mosque in London, England, where he preached Islamic fundamentalism and militant Islamism? and much more. It was found in the London residence of Islamic cleric Sheikh Abu Hamza al-Masri in May 2004. During al-Masri's trial, the prosecution referred to the literature as a "blueprint for terror". [PAR] [TLE] Abu Hamza al-Masri [SEP] Mustafa Kamel Mustafa (Arabic: مصطفى كامل مصطفى‎ ‎ ; born 15 April 1958), also known as Abu Hamza al-Masri ( أبو حمزة المصري , "Abū Ḥamzah al-Maṣrī" – literally, the Egyptian father of Hamza), the Hook +How can static analysis be useful with query languages? the *Abstract interpretation framework has been extended to the field of query languages for relational databases as a way to support sound approximation techniques. The +In what year did the Motion Picture Association of America introduce its rating system? format being those only ones presented in color), depending upon the original content of movies, particularly movies released after the 1968 implementation of the Motion Picture Association of America's ratings system and the concurrent disestablishment of the Motion Picture Production Code. +What was the windspeed once the typhoon weakened? in north Luzon, due to high waters there related to the storm, the news agency said. Presidential spokesman Abigail Valte earlier Saturday urged residents of low-lying and mountainous areas that could be hit hard by the storm to evacuate, the state news agency said, citing an interview conducted on a government radio station. World Vision, the Christian humanitarian organization, said Saturday that it had to postpone some of its relief efforts due to Nalgae, with two of three emergency teams set to deploy once the storm passes. Another team is in Bulcan province, most of which is "still submerged" because of Nesat. The group is focusing its post-Nesat efforts on two communities in Manila and three in the northern Isabela and Zambales provinces. Vouchers are being distributed so people can buy needed items, some emergency supplies are being given out directly to citizens and 3,000 Manila children will receive school supplies. Sherbien Dacalanio, a CNN iReporter in the Philippines, described one area of Manila as being devastated by Nesat. "The damage +What is the initial phantom body pain sensation experienced by people with spinal cord damage? paraplegics, phantom body pain in areas of complete sensory loss. This phantom body pain is initially described as burning or tingling but may evolve into severe crushing or pinching pain, or the sensation of fire running down the legs or +Sarah Etonge is a star in the sport that involves running over what landforms? Sarah Etonge [SEP] Sarah Liengu Etonge (born in Buea, Southwest Province, Cameroon) is a current African mountain running star. She has won the annual Mount Cameroon Race of Hope seven times in her native Buea, in the +Den of Thieves stars which Scottish actor? of Thieves (film) [SEP] Den of Thieves is an upcoming American heist action thriller film directed by Christian Gudegast. The film stars Gerard Butler, 50 Cent, Pablo Schreiber, O'Shea Jackson Jr., Evan Jones, Dawn Olivieri, Mo McRae, and Max Holloway. After studying law, Butler turned to +How long does it take for listeria to make a person sick? Food and Drug Administration said it has teamed up with state officials in the effort. "FDA and its state partners are conducting checks at retail stores, wholesalers and distributors to make sure they have +Steve Haworth did body modification on Dennis Avner, who underwent how many surgeries? 1958 – November 5, 2012) was an American man known for his extensive body modifications, which were intended to increase his resemblance to a tigress. For his 14 surgical procedures towards that goal, he held a world record for "most permanent transformations to look like an animal." The name "Stalking Cat" is a Native American name, +The Song, Your Mirror, is featured on Stars an, album by Simply Red that was released in what year? on the US "Billboard" Hot 100. Written by Mick Hucknall, it was featured on the album "Stars" and reached number 17 on the UK chart when released in July +What award did the animator of the semi-sequel spin-off to Kung Fu Panda receive? work at DreamWorks Animation and for his award-winning student film "Le Building". He joined DreamWorks in 2008, after working on 2D animated films in France. The following year, Perifel won an Annie Award for the short film "Secrets of the Furious Five". He received subsequent nominations for "Kung Fu Panda 2" and "Rise of the Guardians". +Who did Madonna turn to for comfort during her mother's illness? siblings resented housekeepers and invariably rebelled against anyone brought into their home ostensibly to take the place of their beloved mother. Madonna later told Vanity +What do people in Haiti use to contact relatives, friends in earthquake zones? rumbled across Haiti on Tuesday evening. "It's 8:44 p.m. and we're still getting aftershocks! Can hear people gathered in the distance singing prayers," wrote Richard Morse, hotel manager at the Oloffson Hotel in the capital, Port-au-Prince. On Twitter, he captured the aftermath of a 7.0-magnitude earthquake that struck off the coast of Haiti, causing widespread destruction. The quake, which happened shortly before 5 p.m., was followed by at least 18 aftershocks, averaging a magnitude of 5.0, according to the U.S. Geological Survey. As the night went on, the singing and praying intensified and then waned, Morse wrote. What he didn't hear in all the commotion were helicopters or ambulances. Frantic callers broadcast pleas for help. "My cousin has a broken leg and she's trying to get to the hospital," one caller said. "She has two kids. ... How can she get to the hospital?" Are you there? Submit an iReport The man gave his cousin's address, and the channel's commentator asked anyone listening who lived nearby to see whether he could help. Many callers expressed their sympathies and prayers for the people of Haiti, while others told of fallen buildings and damaged homes. The first photographs from the area showed collapsed walls and sidewalks covered with cinder block-size chunks of debris. One photograph, taken by members of a American church mission group, showed a dead man in a gray T-shirt and jeans, hunched on the ground. Another broadcaster, Radio-Tele Ginen, collected and posted photographs on its Web site that hinted at the damage and +What types of geese are used for human consumption? roam widely in the knowledge that they will return home by dusk. The Chinese goose is more aggressive and noisy than other geese and can be used as a guard animal to warn of intruders. The flesh of meat geese is dark-coloured and high in protein, but they deposit fat subcutaneously, although this fat +what is name of new video game? of your favorite fictional universe and weaving them into a new story that captures the imagination of fans. In the dark, cinematic "Batman: Arkham Asylum," Batman must escape from a spooky psychiatric hospital. Buzz is building online for the dark, cinematic game, which reimagines a brooding Batman and his most notorious nemesis, the Joker, for an experience that's reminiscent of "The Dark Knight" blockbuster movie. You want mayhem, insanity and brutality? Get ready for all of it. "This is the Batman movie I would have liked to have written," said Dini, who was scheduled to discuss +what is the name of the large city in southeastern nevada tourism , including gambling . The city serves as world headquarters for the world 's two largest Fortune 500 gaming companies , Harrah 's Entertainment and MGM Mirage . +what does somalia want of using force against the pirates, the Russian news agency Interfax reported. "The questions of freeing the ships and crew are being dealt with in line with the corresponding international practices," Interfax quoted Navy spokesman Igor Dygalo as saying. "For understandable reasons, the use of force would be an extreme measure because it could threaten the life of the international crew of the ship." The pirates took over the MV Faina last week off the coast of Somalia and are demanding +when was nepal declared federal democratic republic country has changed Nepal practically into a federal democratic republic by making 7 unnamed states +animal that live both on land and in water are animals that live predominantly or entirely on land ( e.g. , cats , ants , spiders ) , as compared with aquatic animals , which live predominantly or entirely in the water ( e.g.g. , frogs , or newts +Conservative theology was the official position of what? formed by moderate Southern Baptists who disagreed with the direction in which the Southern Baptist Convention was heading: the Alliance of Baptists in 1987 and +What laws did homeland security waive? environmental groups, Defenders of Wildlife and the Sierra Club, have filed appeals with the U.S. Supreme Court, claiming the waivers are unconstitutional and set a dangerous precedent. "National security and environmental protection do not have to be at odds with each other," says Defenders of Wildlife spokesman Matt Clark. "If we can drop this arbitrary deadline for constructing the fence and go through the proper procedures, then there are inevitably ways to minimize environmental impact, but as it is now it's throwing all of those laws out the window." Mountain lion tracker Jack Childs also worries about the impact of the fence on local wildlife, especially the jaguar. +How many PVA soldiers fought in this battle and lost? U.S. 2nd Infantry Division Warrior Division's 23rd Regimental Combat Team with an attached French Battalion was hemmed in by more than 25,000 Chinese Communist forces. United Nations forces had previously retreated in the face of large Communist forces instead of getting +who plays sugar ray leonard in hands of stone Usher Raymond IV as +What is the name of the person titled Drift King who helped with editorial supervision of the sports manga series, Initial D? is centered on the prefecture of Gunma, more specifically on several mountains in the Kantō region and in their surrounding cities and towns. Although some of the names of the locations the characters race in have been fictionalized, all of the locations in the series are based on actual locations in Japan. He is also known as the Drift King (ドリキン , Dorikin ) for his diff --git a/data/user_define/user_define.md b/data/user_define/user_define.md deleted file mode 100644 index 69746fb5959f7aca89a1a8797330e512f67fa1d2..0000000000000000000000000000000000000000 --- a/data/user_define/user_define.md +++ /dev/null @@ -1 +0,0 @@ -user define model dataset diff --git a/demo2.py b/demo2.py index 75a6e77fe01dcd45d0c677dab79eeeb98aca2a10..d1c7b8762e2f384675fd63bee8d72bb8b8c353ed 100644 --- a/demo2.py +++ b/demo2.py @@ -1,10 +1,10 @@ import paddlepalm as palm if __name__ == '__main__': - controller = palm.Controller('demo2_config.yaml', task_dir='demo2_tasks') + controller = palm.Controller('config_demo2.yaml', task_dir='demo2_tasks') controller.load_pretrain('pretrain_model/ernie/params') controller.train() - controller = palm.Controller(config='demo2_config.yaml', task_dir='demo2_tasks', for_train=False) + controller = palm.Controller(config='config_demo2.yaml', task_dir='demo2_tasks', for_train=False) controller.pred('mrqa', inference_model_dir='output_model/secondrun/infer_model') diff --git a/demo2_tasks/mlm4mrqa.yaml b/demo2_tasks/mlm4mrqa.yaml new file mode 100644 index 0000000000000000000000000000000000000000..4d973b6bceafc13d4879a70485cacb3972187f98 --- /dev/null +++ b/demo2_tasks/mlm4mrqa.yaml @@ -0,0 +1,3 @@ +train_file: "data/mlm4mrqa/train.txt" +reader: mlm +paradigm: mlm diff --git a/paddlepalm/backbone/bert.py b/paddlepalm/backbone/bert.py index 05d0af531dc945d4fe0e5d885c0020dc01f5a997..7f481096039bc5f09941e3205cc7bcc753938d7c 100644 --- a/paddlepalm/backbone/bert.py +++ b/paddlepalm/backbone/bert.py @@ -28,9 +28,7 @@ from paddlepalm.interface import backbone class Model(backbone): - def __init__(self, - config, - phase): + def __init__(self, config, phase): # self._is_training = phase == 'train' # backbone一般不用关心运行阶段，因为outputs在任何阶段基本不会变 self._emb_size = config["hidden_size"] @@ -56,16 +54,17 @@ class Model(backbone): @property def inputs_attr(self): - return {"token_ids": [-1, self._max_position_seq_len, 1], 'int64'], - "position_ids": [-1, self._max_position_seq_len, 1], 'int64'], - "segment_ids": [-1, self._max_position_seq_len, 1], 'int64'], - "input_mask": [-1, self._max_position_seq_len, 1], 'float32']} + return {"token_ids": [[-1, -1, 1], 'int64'], + "position_ids": [[-1, -1, 1], 'int64'], + "segment_ids": [[-1, -1, 1], 'int64'], + "input_mask": [[-1, -1, 1], 'float32']} @property def outputs_attr(self): - return {"word_emb": [-1, self._max_position_seq_len, self._emb_size], - "sentence_emb": [-1, self._emb_size], - "sentence_pair_emb": [-1, self._emb_size]} + return {"word_embedding": [[-1, -1, self._emb_size], 'float32'], + "encoder_outputs": [[-1, -1, self._emb_size], 'float32'], + "sentence_embedding": [[-1, self._emb_size], 'float32'], + "sentence_pair_embedding": [[-1, self._emb_size], 'float32']} def build(self, inputs): src_ids = inputs['token_ids'] @@ -146,9 +145,10 @@ class Model(backbone): initializer = self._param_initializer), bias_attr = "pooled_fc.b_0") - return {'word_emb': enc_out, - 'sentence_emb': next_sent_feat, - 'sentence_pair_emb': next_sent_feat} + return {'word_embedding': emb_out, + 'encoder_outputs': enc_out, + 'sentence_embedding': next_sent_feat, + 'sentence_pair_embedding': next_sent_feat} def postprocess(self, rt_outputs): pass diff --git a/paddlepalm/backbone/bow.py b/paddlepalm/backbone/bow.py deleted file mode 100644 index 92689576318fb1b5499b306a8f5e389639c9e00f..0000000000000000000000000000000000000000 --- a/paddlepalm/backbone/bow.py +++ /dev/null @@ -1,63 +0,0 @@ -# -*- coding: UTF-8 -*- -# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -from __future__ import absolute_import -from __future__ import division -from __future__ import print_function - -from paddle import fluid -from paddle.fluid import layers - -class Model(backbone): - - def __init__(self, config, phase): - - # self._is_training = phase == 'train' # backbone一般不用关心运行阶段，因为outputs在任何阶段基本不会变 - - self._emb_size = config["emb_size"] - self._voc_size = config["vocab_size"] - - @property - def inputs_attr(self): - return {"token_ids": [-1, self._max_position_seq_len, 1], 'int64']} - - @property - def outputs_attr(self): - return {"word_emb": [-1, self._max_position_seq_len, self._emb_size], - "sentence_emb": [-1, self._emb_size*2]} - - def build(self, inputs): - - tok_ids = inputs['token_ids'] - - emb_out = layers.embedding( - input=tok_ids, - size=[self._voc_size, self._emb_size], - dtype='float32', - param_attr=fluid.ParamAttr( - name='word_emb', - initializer=fluid.initializer.TruncatedNormal(scale=0.1)), - is_sparse=False) - - sent_emb1 = layers.reduce_mean(emb_out, axis=1) - sent_emb2 = layers.reduce_max(emb_out, axis=1) - sent_emb = layers.concat([sent_emb1, sent_emb2], axis=1) - return {'word_emb': emb_out, - 'sentence_emb': sent_emb} - - def postprocess(self, rt_outputs): - pass - - diff --git a/paddlepalm/backbone/ernie.py b/paddlepalm/backbone/ernie.py index 7a11769b626dd6e5ec0aa03a8d140569d1bb7e0c..4362334ab933ae16ed0917b90b3bb3444de82527 100644 --- a/paddlepalm/backbone/ernie.py +++ b/paddlepalm/backbone/ernie.py @@ -71,6 +71,7 @@ class Model(backbone): @property def outputs_attr(self): return {"word_embedding": [[-1, -1, self._emb_size], 'float32'], + "embedding_table": [[-1, self._voc_size, self._emb_size], 'float32'], "encoder_outputs": [[-1, -1, self._emb_size], 'float32'], "sentence_embedding": [[-1, self._emb_size], 'float32'], "sentence_pair_embedding": [[-1, self._emb_size], 'float32']} @@ -91,6 +92,9 @@ class Model(backbone): param_attr=fluid.ParamAttr( name=self._word_emb_name, initializer=self._param_initializer), is_sparse=False) + + # fluid.global_scope().find_var('backbone-word_embedding').get_tensor() + embedding_table = fluid.default_main_program().global_block().var(self._word_emb_name) position_emb_out = fluid.layers.embedding( input=pos_ids, @@ -161,7 +165,8 @@ class Model(backbone): name="pooled_fc.w_0", initializer=self._param_initializer), bias_attr="pooled_fc.b_0") - return {'word_embedding': emb_out, + return {'embedding_table': embedding_table, + 'word_embedding': emb_out, 'encoder_outputs': enc_out, 'sentence_embedding': next_sent_feat, 'sentence_pair_embedding': next_sent_feat} diff --git a/paddlepalm/mtl_controller.py b/paddlepalm/mtl_controller.py index 0b9ab1d563f8f1072dd839de3deb0eefe7ea13db..4086390f9bb98d9faf76f556b2ff49eafe2c546d 100755 --- a/paddlepalm/mtl_controller.py +++ b/paddlepalm/mtl_controller.py @@ -422,7 +422,7 @@ class Controller(object): prefixes.append(inst.name) mrs.append(inst.mix_ratio) - joint_iterator_fn = create_joint_iterator_fn(iterators, prefixes, joint_shape_and_dtypes, mrs, name_to_position, dev_count=dev_count, verbose=VERBOSE, batch_size=main_conf['batch_size']) + joint_iterator_fn = create_joint_iterator_fn(iterators, prefixes, joint_shape_and_dtypes, mrs, name_to_position, dev_count=dev_count, verbose=VERBOSE) input_attrs = [[i, j, k] for i, (j,k) in zip(joint_input_names, joint_shape_and_dtypes)] pred_input_attrs = [[i, j, k] for i, (j,k) in zip(pred_joint_input_names, pred_joint_shape_and_dtypes)] @@ -653,6 +653,10 @@ class Controller(object): loss, main_conf.get('print_every_n_steps', 5) / time_cost)) time_begin = time.time() + if cur_task.train_finish and cur_task.cur_train_step + cur_task.cur_train_epoch * cur_task.steps_pur_epoch == cur_task.expected_train_steps: + print(cur_task.name+': train finished!') + cur_task.save() + if 'save_every_n_steps' in main_conf and global_step % main_conf['save_every_n_steps'] == 0: save_path = os.path.join(main_conf['save_path'], "step_" + str(global_step)) diff --git a/paddlepalm/reader/match.py b/paddlepalm/reader/match.py new file mode 100644 index 0000000000000000000000000000000000000000..0ef8fdebcccfac7cd539a3570f301ea20dc5a005 --- /dev/null +++ b/paddlepalm/reader/match.py @@ -0,0 +1,103 @@ +# -*- coding: UTF-8 -*- +# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from paddlepalm.interface import reader +from paddlepalm.reader.utils.reader4ernie import ClassifyReader + +class Reader(reader): + + def __init__(self, config, phase='train', dev_count=1, print_prefix=''): + """ + Args: + phase: train, eval, pred + """ + + self._is_training = phase == 'train' + + reader = ClassifyReader(config['vocab_path'], + max_seq_len=config['max_seq_len'], + do_lower_case=config.get('do_lower_case', False), + for_cn=config.get('for_cn', False), + random_seed=config.get('seed', None)) + self._reader = reader + self._dev_count = dev_count + + self._batch_size = config['batch_size'] + self._max_seq_len = config['max_seq_len'] + if phase == 'train': + self._input_file = config['train_file'] + self._num_epochs = None # 防止iteartor终止 + self._shuffle = config.get('shuffle', False) + self._shuffle_buffer = config.get('shuffle_buffer', 5000) + elif phase == 'eval': + self._input_file = config['dev_file'] + self._num_epochs = 1 + self._shuffle = False + self._batch_size = config.get('pred_batch_size', self._batch_size) + elif phase == 'pred': + self._input_file = config['pred_file'] + self._num_epochs = 1 + self._shuffle = False + self._batch_size = config.get('pred_batch_size', self._batch_size) + + self._phase = phase + # self._batch_size = + self._print_first_n = config.get('print_first_n', 1) + + + @property + def outputs_attr(self): + if self._is_training: + return {"token_ids": [[-1, -1, 1], 'int64'], + "position_ids": [[-1, -1, 1], 'int64'], + "segment_ids": [[-1, -1, 1], 'int64'], + "input_mask": [[-1, -1, 1], 'float32'], + "label_ids": [[-1,1], 'int64'], + "task_ids": [[-1, -1, 1], 'int64'] + } + else: + return {"token_ids": [[-1, -1, 1], 'int64'], + "position_ids": [[-1, -1, 1], 'int64'], + "segment_ids": [[-1, -1, 1], 'int64'], + "task_ids": [[-1, -1, 1], 'int64'], + "input_mask": [[-1, -1, 1], 'float32'] + } + + + def load_data(self): + self._data_generator = self._reader.data_generator(self._input_file, self._batch_size, self._num_epochs, dev_count=self._dev_count, shuffle=self._shuffle, phase=self._phase) + + def iterator(self): + + def list_to_dict(x): + names = ['token_ids', 'segment_ids', 'position_ids', 'task_ids', 'input_mask', + 'label_ids', 'unique_ids'] + outputs = {n: i for n,i in zip(names, x)} + del outputs['unique_ids'] + if not self._is_training: + del outputs['label_ids'] + return outputs + + for batch in self._data_generator(): + yield list_to_dict(batch) + + def get_epoch_outputs(self): + return {'examples': self._reader.get_examples(self._phase), + 'features': self._reader.get_features(self._phase)} + + @property + def num_examples(self): + return self._reader.get_num_examples(phase=self._phase) + diff --git a/paddlepalm/reader/mlm.py b/paddlepalm/reader/mlm.py index 823e5058258d8de2703ccdbe8ab41997722b39d3..338fcf2300267b8f879a68289ea62465f219a060 100644 --- a/paddlepalm/reader/mlm.py +++ b/paddlepalm/reader/mlm.py @@ -14,7 +14,7 @@ # limitations under the License. from paddlepalm.interface import reader -from paddlepalm.reader.utils.reader4ernie import BaseReader +from paddlepalm.reader.utils.reader4ernie import MaskLMReader class Reader(reader): @@ -26,7 +26,7 @@ class Reader(reader): self._is_training = phase == 'train' - reader = ClassifyReader(config['vocab_path'], + reader = MaskLMReader(config['vocab_path'], max_seq_len=config['max_seq_len'], do_lower_case=config.get('do_lower_case', False), for_cn=config.get('for_cn', False), @@ -59,21 +59,14 @@ class Reader(reader): @property def outputs_attr(self): - if self._is_training: - return {"token_ids": [[-1, -1, 1], 'int64'], - "position_ids": [[-1, -1, 1], 'int64'], - "segment_ids": [[-1, -1, 1], 'int64'], - "input_mask": [[-1, -1, 1], 'float32'], - "label_ids": [[-1,1], 'int64'], - "task_ids": [[-1, -1, 1], 'int64'] - } - else: - return {"token_ids": [[-1, -1, 1], 'int64'], - "position_ids": [[-1, -1, 1], 'int64'], - "segment_ids": [[-1, -1, 1], 'int64'], - "task_ids": [[-1, -1, 1], 'int64'], - "input_mask": [[-1, -1, 1], 'float32'] - } + return {"token_ids": [[-1, -1, 1], 'int64'], + "position_ids": [[-1, -1, 1], 'int64'], + "segment_ids": [[-1, -1, 1], 'int64'], + "input_mask": [[-1, -1, 1], 'float32'], + "task_ids": [[-1, -1, 1], 'int64'], + "mask_label": [[-1, 1], 'int64'], + "mask_pos": [[-1, 1], 'int64'] + } def load_data(self): @@ -85,9 +78,6 @@ class Reader(reader): names = ['token_ids', 'position_ids', 'segment_ids', 'input_mask', 'task_ids', 'mask_label', 'mask_pos'] outputs = {n: i for n,i in zip(names, x)} - del outputs['unique_ids'] - if not self._is_training: - del outputs['label_ids'] return outputs for batch in self._data_generator(): diff --git a/paddlepalm/reader/utils/mlm_batching.py b/paddlepalm/reader/utils/mlm_batching.py index 71d4ab944540fd6e9e6a09674f2e21f993fe7b36..991d02d3b50c9b3c10b0cebd3d12f6762cb91f01 100644 --- a/paddlepalm/reader/utils/mlm_batching.py +++ b/paddlepalm/reader/utils/mlm_batching.py @@ -93,6 +93,7 @@ def prepare_batch_data(insts, batch_sent_ids = [inst[1] for inst in insts] batch_pos_ids = [inst[2] for inst in insts] + # 这里是否应该反过来？？？否则在task layer里展开后的word embedding是padding后的，这时候word的index是跟没有padding时的index对不上的？ # First step: do mask without padding out, mask_label, mask_pos = mask( batch_src_ids, @@ -106,6 +107,7 @@ def prepare_batch_data(insts, out, max_len=max_len, pad_idx=pad_id, return_input_mask=True) + pos_id = pad_batch_data( batch_pos_ids, max_len=max_len, diff --git a/paddlepalm/reader/utils/reader4ernie.py b/paddlepalm/reader/utils/reader4ernie.py index 85a42d6db7e1bcc35af7cb80ece57cf647762823..fcf25e75b0074bfbedbba7ae8ebd7ed893674511 100644 --- a/paddlepalm/reader/utils/reader4ernie.py +++ b/paddlepalm/reader/utils/reader4ernie.py @@ -45,11 +45,7 @@ if six.PY3: def csv_reader(fd, delimiter='\t'): def gen(): for i in fd: - slots = i.rstrip('\n').split(delimiter) - if len(slots) == 1: - yield slots, - else: - yield slots + yield i.rstrip('\n').split(delimiter) return gen() @@ -74,6 +70,7 @@ class BaseReader(object): self.pad_id = self.vocab["[PAD]"] self.cls_id = self.vocab["[CLS]"] self.sep_id = self.vocab["[SEP]"] + self.mask_id = self.vocab["[MASK]"] self.in_tokens = in_tokens self.is_inference = is_inference self.for_cn = for_cn @@ -242,7 +239,6 @@ class BaseReader(object): batch_records, max_len = [record], len(record.token_ids) if phase == 'pred' and batch_records: - print('the last batch yielded.') yield self._pad_batch_records(batch_records) def get_num_examples(self, input_file=None, phase=None): @@ -371,31 +367,28 @@ class MaskLMReader(BaseReader): token_ids = tokenizer.convert_tokens_to_ids(tokens) position_ids = list(range(len(token_ids))) - Record = namedtuple('Record', - ['token_ids', 'text_type_ids', 'position_ids']) - record = Record( - token_ids=token_ids, - text_type_ids=text_type_ids, - position_ids=position_ids) + # Record = namedtuple('Record', + # ['token_ids', 'text_type_ids', 'position_ids']) + # record = Record( + # token_ids=token_ids, + # text_type_ids=text_type_ids, + # position_ids=position_ids) - return record + return [token_ids, text_type_ids, position_ids] - def batch_reader(examples, batch_size, in_tokens, phase): - batch, total_token_num, max_len = [], 0, 0 + def batch_reader(self, examples, batch_size, in_tokens, phase): + batch = [] + total_token_num = 0 for e in examples: - token_ids, sent_ids, pos_ids = _convert_example_to_record(e, self.max_seq_len, self.tokenizer) - max_len = max(max_len, len(token_ids)) - if in_tokens: - to_append = (len(batch) + 1) * max_len <= batch_size - else: - to_append = len(batch) < batch_size + parsed_line = self._convert_example_to_record(e, self.max_seq_len, self.tokenizer) + to_append = len(batch) < batch_size if to_append: batch.append(parsed_line) - total_token_num += len(token_ids) + total_token_num += len(parsed_line[0]) else: yield batch, total_token_num - batch, total_token_num, max_len = [parsed_line], len( - token_ids), len(token_ids) + batch = [parsed_line] + total_token_num = len(parsed_line[0]) if len(batch) > 0 and phase == 'pred': yield batch, total_token_num @@ -426,17 +419,17 @@ class MaskLMReader(BaseReader): np.random.shuffle(examples) all_dev_batches = [] - for batch_data, total_token_num in batch_reader(examples, - self.batch_size, self.in_tokens, phase=phase): + for batch_data, num_tokens in self.batch_reader(examples, + batch_size, self.in_tokens, phase=phase): batch_data = prepare_batch_data( batch_data, - total_token_num, - voc_size=self.voc_size, + num_tokens, + voc_size=len(self.vocab), pad_id=self.pad_id, cls_id=self.cls_id, sep_id=self.sep_id, mask_id=self.mask_id, - max_len=self.max_seq_len, + # max_len=self.max_seq_len, # 注意，如果padding到最大长度，会导致mask_pos与实际位置不对应。因为mask pos是基于batch内最大长度来计算的。 return_input_mask=True, return_max_len=False, return_num_token=False) diff --git a/paddlepalm/task_instance.py b/paddlepalm/task_instance.py index 70a338874823a5231f0fea77363c949e21fb10b1..36ad848705545bbbace417dbcec00844d2a3b4af 100644 --- a/paddlepalm/task_instance.py +++ b/paddlepalm/task_instance.py @@ -70,7 +70,11 @@ class TaskInstance(object): def build_task_layer(self, net_inputs, phase): output_vars = self._task_layer[phase].build(net_inputs) if phase == 'pred': - self._pred_fetch_name_list, self._pred_fetch_var_list = zip(*output_vars.items()) + if output_vars is not None: + self._pred_fetch_name_list, self._pred_fetch_var_list = zip(*output_vars.items()) + else: + self._pred_fetch_name_list = [] + self._pred_fetch_var_list = [] return output_vars def postprocess(self, rt_outputs, phase): @@ -234,8 +238,6 @@ class TaskInstance(object): self._cur_train_step = 1 if self._is_target and self._cur_train_step + self._cur_train_epoch * self._steps_pur_epoch >= self._expected_train_steps: self._train_finish = True - print(self._name+': train finished!') - self.save() # fluid.io.save_inference_model(self._save_infermodel_path, ) @property diff --git a/paddlepalm/task_paradigm/mlm.py b/paddlepalm/task_paradigm/mlm.py index 817b009cdf3d8cefd35f601b0d9c2d572145c0f4..53c286694056764b0444bd66f8d4858ac8d4afbe 100644 --- a/paddlepalm/task_paradigm/mlm.py +++ b/paddlepalm/task_paradigm/mlm.py @@ -16,6 +16,7 @@ import paddle.fluid as fluid from paddlepalm.interface import task_paradigm from paddle.fluid import layers +from paddlepalm.backbone.utils.transformer import pre_process_layer class TaskParadigm(task_paradigm): ''' @@ -23,6 +24,7 @@ class TaskParadigm(task_paradigm): ''' def __init__(self, config, phase, backbone_config=None): self._is_training = phase == 'train' + self._emb_size = backbone_config['hidden_size'] self._hidden_size = backbone_config['hidden_size'] self._vocab_size = backbone_config['vocab_size'] self._hidden_act = backbone_config['hidden_act'] @@ -30,11 +32,14 @@ class TaskParadigm(task_paradigm): @property def inputs_attrs(self): - if self._is_training: - reader = {"label_ids": [[-1, 1], 'int64']} - else: - reader = {} - bb = {"encoder_outputs": [[-1, self._hidden_size], 'float32']} + reader = { + "mask_label": [[-1, 1], 'int64'], + "mask_pos": [[-1, 1], 'int64']} + if not self._is_training: + del reader['mask_label'] + bb = { + "encoder_outputs": [[-1, -1, self._hidden_size], 'float32'], + "embedding_table": [[-1, self._vocab_size, self._emb_size], 'float32']} return {'reader': reader, 'backbone': bb} @property @@ -42,12 +47,13 @@ class TaskParadigm(task_paradigm): if self._is_training: return {"loss": [[1], 'float32']} else: - return {"logits": [[-1, 1], 'float32']} + return {"logits": [[-1], 'float32']} def build(self, inputs): - mask_label = inputs["reader"]["mask_label"] + if self._is_training: + mask_label = inputs["reader"]["mask_label"] mask_pos = inputs["reader"]["mask_pos"] - word_emb = inputs["backbone"]["word_embedding"] + word_emb = inputs["backbone"]["embedding_table"] enc_out = inputs["backbone"]["encoder_outputs"] emb_size = word_emb.shape[-1] @@ -62,7 +68,6 @@ class TaskParadigm(task_paradigm): # extract masked tokens' feature mask_feat = fluid.layers.gather(input=reshaped_emb_out, index=mask_pos) - num_seqs = fluid.layers.fill_constant(shape=[1], value=512, dtype='int64') # transform: fc mask_trans_feat = fluid.layers.fc( @@ -99,13 +104,12 @@ class TaskParadigm(task_paradigm): attr=mask_lm_out_bias_attr, is_bias=True) - mask_lm_loss = fluid.layers.softmax_with_cross_entropy( - logits=fc_out, label=mask_label) - loss = fluid.layers.mean(mask_lm_loss) - if self._is_training: + mask_lm_loss = fluid.layers.softmax_with_cross_entropy( + logits=fc_out, label=mask_label) + loss = fluid.layers.mean(mask_lm_loss) return {'loss': loss} else: - return None + return {'logits': fc_out} diff --git a/paddlepalm/utils/reader_helper.py b/paddlepalm/utils/reader_helper.py index e8e396adc5dc27d4637e137344efff019f0e2eb1..c9e5f88dab4dd3c552bfe259acddd59a1b006280 100644 --- a/paddlepalm/utils/reader_helper.py +++ b/paddlepalm/utils/reader_helper.py @@ -22,7 +22,7 @@ from paddle import fluid from paddle.fluid import layers -def _check_and_adapt_shape_dtype(rt_val, attr): +def _check_and_adapt_shape_dtype(rt_val, attr, message=""): if not isinstance(rt_val, np.ndarray): rt_val = np.array(rt_val) assert rt_val.dtype != np.dtype('O'), "yielded data is not a valid tensor(number of elements on some dimension may differ)." @@ -30,12 +30,12 @@ def _check_and_adapt_shape_dtype(rt_val, attr): rt_val = rt_val.astype('float32') shape, dtype = attr - assert rt_val.dtype == np.dtype(dtype), "yielded data type not consistent with attr settings." - assert len(shape) == rt_val.ndim, "yielded data rank(ndim) not consistent with attr settings." + assert rt_val.dtype == np.dtype(dtype), message+"yielded data type not consistent with attr settings. Expect: {}, receive: {}.".format(rt_val.dtype, np.dtype(dtype)) + assert len(shape) == rt_val.ndim, message+"yielded data rank(ndim) not consistent with attr settings. Expect: {}, receive: {}.".format(len(shape), rt_val.ndim) for rt, exp in zip(rt_val.shape, shape): if exp is None or exp < 0: continue - assert rt == exp, "yielded data shape is not consistent with attr settings.\nExpected:{}\nActual:{}".format(exp, rt) + assert rt == exp, "yielded data shape is not consistent with attr settings.Expected:{}Actual:{}".format(exp, rt) return rt_val @@ -107,7 +107,7 @@ def create_iterator_fn(iterator, iterator_prefix, shape_and_dtypes, outname_to_p return iterator -def create_joint_iterator_fn(iterators, iterator_prefixes, joint_shape_and_dtypes, mrs, outname_to_pos, dev_count=1, keep_one_task=True, verbose=0, batch_size=None): +def create_joint_iterator_fn(iterators, iterator_prefixes, joint_shape_and_dtypes, mrs, outname_to_pos, dev_count=1, keep_one_task=True, verbose=0): """ joint_shape_and_dtypes: 本质上是根据bb和parad的attr设定的，并且由reader中的attr自动填充-1（可变）维度得到，因此通过与iterator的校验可以完成runtime的batch正确性检查 """ @@ -130,12 +130,12 @@ def create_joint_iterator_fn(iterators, iterator_prefixes, joint_shape_and_dtype if outname in outname_to_pos: idx = outname_to_pos[outname] - val = _check_and_adapt_shape_dtype(val, joint_shape_and_dtypes[idx]) + val = _check_and_adapt_shape_dtype(val, joint_shape_and_dtypes[idx], message=outname+': ') results[idx] = val if task_outname in outname_to_pos: idx = outname_to_pos[task_outname] - val = _check_and_adapt_shape_dtype(val, joint_shape_and_dtypes[idx]) + val = _check_and_adapt_shape_dtype(val, joint_shape_and_dtypes[idx], message=task_outname+': ') results[idx] = val fake_batch = results @@ -153,7 +153,6 @@ def create_joint_iterator_fn(iterators, iterator_prefixes, joint_shape_and_dtype results[0] = task_id_tensor for i in range(dev_count): - # results = _zero_batch(joint_shape_and_dtypes, batch_size=batch_size) results[0] = task_id_tensor if id in outbuf: outputs = outbuf[id] @@ -171,14 +170,14 @@ def create_joint_iterator_fn(iterators, iterator_prefixes, joint_shape_and_dtype idx = outname_to_pos[outname] if v > 0: print(outname + ' is insert in idx ' + str(idx)) - val = _check_and_adapt_shape_dtype(val, joint_shape_and_dtypes[idx]) + val = _check_and_adapt_shape_dtype(val, joint_shape_and_dtypes[idx], message=outname+': ') results[idx] = val if task_outname in outname_to_pos: idx = outname_to_pos[task_outname] if v > 0: print(task_outname + ' is insert in idx ' + str(idx)) - val = _check_and_adapt_shape_dtype(val, joint_shape_and_dtypes[idx]) + val = _check_and_adapt_shape_dtype(val, joint_shape_and_dtypes[idx], message=task_outname+': ') results[idx] = val if v > 0: diff --git a/run_demo2.sh b/run_demo2.sh index e0a085219ec4026885222d67401d8eb34aeb5197..a4c5471d827e6f2132ee9b996e064a48733b72bb 100755 --- a/run_demo2.sh +++ b/run_demo2.sh @@ -1,4 +1,4 @@ -export CUDA_VISIBLE_DEVICES=0,1,2,3 +export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python demo2.py