经济学人科技 || 人工智能

原文:http://mp.weixin.qq.com/s?__biz=MzU1MDQwNTgzMg==&mid=2247491372&idx=1&sn=f5a9889f0d5bda0bd03710cd10ec3e22&chksm=fba04c8bccd7c59dc4049850a2123df6549b61aab791f4ae495b20c3f30f7b67e71796b491e9#rd

1


导读


感谢思维导图作者
琚儿,英专,备考翻硕,外刊学习者


2


听力|精读|翻译|词组

 Bit-lit

成就尚微,道阻且长

英文部分选自经济学人20200808期科技版块

Artificial intelligence

人工智能


Bit-lit

成就尚微,道阻且长


A new language-generating AI can be eerily human-like—for better and for worse

不管好坏,当下一项新的语言生成AI技术能模仿出怪异的人类表达习惯。


注:
什么是GPT-3,它将如何影响人们目前的工作?

https://tech.sina.cn/2020-07-20/detail-iivhvpwx6374338.d.html


The SEC said, “Musk,/your tweets are a blight./They really could cost you your job,/if you don’t stop/all this tweeting at night.”/…Then Musk cried, “Why?/The tweets I wrote are not mean,/I don’t use all-caps/and I’m sure that my tweets are clean.”/“But your tweets can move markets/and that’s why we’re sore./You may be a genius/and a billionaire,/but that doesn’t give you the right to be a bore!”


美国证券交易委员会(SEC)表示:马斯克(Musk),你的推特内容有问题。如果,今晚你还是发这种推特,你真的会丢掉饭碗。马斯克大喊:为什么?我发的推特并不刻薄,也没全用大写字母,而且我能肯定,我的推特干干净净,无懈可击!证券交易委员会接着说:但你的推特会导致市场混乱,所以我们极度愤慨。就算你是个天才,还是个亿万富翁,但这不代表,你可以做如此惹人生厌的事!


注:

1. 本段描述事件的链接https://www.sohu.com/a/256652331_260616

2. GPT-3为什么怼起了前老板?马斯克:和OpenAI道不同不相为谋

https://xw.qq.com/cmsid/20200808A0B47100


The preceding lines—describing Tesla and SpaceX founder Elon Musk’s run-ins with the Securities and Exchange Commission, an American financial regulator—are not the product of some aspiring 21st-century Dr Seuss. They come from a poem written by a computer running a piece of software called Generative Pre-Trained Transformer 3. gpt-3, as it is more commonly known, was developed by Openai, an artificial-intelligence (ai) laboratory based in San Francisco, and which Mr Musk helped found. It represents the latest advance in one of the most studied areas of ai: giving computers the ability to generate sophisticated, human-like text.


上面这段描述特斯拉和SpaceX创始人伊隆·马斯克(Elon Musk)与负责美国金融监管的证券交易委员会之间的争论,可不是某个想成为21世纪的苏斯博士的人创作的,而是由一台运行自然语言生成模型的电脑创作。这个模型叫做预训练语言模型-3,人们常称之为GPT-3。它的开发者是由马斯克协助创办位于旧金山的人工智能(AI)实验室OpenAIGPT-3代表了人工智能研究极其火热的一个领域内的最新成就——让计算机能够生成复杂的,类似人类表达方式的文本。


注:

苏斯博士(Dr.Seuss),二十世纪最卓越的儿童文学家、教育学家。出生于190432日,美国人,一生创作的48种精彩教育绘本成为西方家喻户晓的著名早期教育作品,曾获美国图画书最高荣誉凯迪克大奖和普利策特殊贡献奖,两次获奥斯卡金像奖和艾美奖,美国教育部指定的儿童重要阅读辅导读物。作品《乌龟耶尔特及其他故事》


The software is built on the idea of a “language model”. This aims to represent a language statistically, mapping the probability with which words follow other words—for instance, how often “red” is followed by “rose”. The same sort of analysis can be performed on sentences, or even entire paragraphs. Such a model can then be given a prompt— “a poem about red roses in the style of Sylvia Plath”, say—and it will dig through its set of statistical relationships to come up with some text that matches the description.

  

GPT-3采用了语言模型的理念。这样设计旨在用数据来自动生成语言,计算出一些单词出现在另一些单词后面的概率,比如红色后面出现玫瑰的概率。同样的分析方法可以用于句子,甚至应用于整段话。这种模型也可以接收提示,比如,写一首关于红玫瑰的诗歌,符合西尔维娅·普拉斯(Sylvia Plath)的写作风格,随后,它就会充分挖掘数据集,以生成符合描述的文本。


Actually building such a language model, though, is a big job. This is where ai—or machine learning, a particular subfield of ai—comes in. By trawling through enormous volumes of written text, and learning by trial and error from millions of attempts at text prediction, a computer can crunch through the laborious task of mapping out those statistical relationships.


然而,建立这样一个语言模型是个大工程。而这就是人工智能,或者(更确切)说是作为人工智能的一个特定分支的机器学习要发挥作用的地方。计算机大量查阅现有文本,并从数百万次文本预测的尝试和失败中反复摸索学习,最终可以完成这一艰巨的任务,弄清楚这些数据联系。


The more text to which an algorithm can be exposed, and the more complex you can make the algorithm, the better it performs. And what sets gpt-3 apart is its unprecedented scale. The model that underpins gpt-3 boasts 175bn parameters, each of which can be individually tweaked—an order of magnitude larger than any of its predecessors. It was trained on the biggest set of text ever amassed, a mixture of books, Wikipedia and Common Crawl, a set of billions of pages of text scraped from every corner of the internet.


算法学习的文本越多,算法就越复杂,它便能更好地生成文本。GPT-3的与众不同之处在于,其前所未有的数据集,拥有1750亿参数量,每个参数都可以单独微调, 指令量比它所有的旧版本都要高一个量级。GPT-3是在有史以来最大的文本集上进行训练,这包括了书籍、维基百科和网络爬虫语料库,网络爬虫语料库是一组从网络各处提取而成的数十亿页文本。


Statistically speaking

统计学范畴


The results can be impressive. In mid-July Openai gave an early version of the software to selected individuals, to allow them to explore what it could do. Arram Sabeti, an artist, demonstrated gpt-3’s ability to write short stories, including a hard-boiled detective story starring Harry Potter (“Harry Potter, in ratty tweed suit, unpressed shirt and unshined shoes, sits behind the desk looking haggard, rumpled and embittered…”), comedy sketches, and even poetry (including the poem with which this article opens, titled “Elon Musk by Dr Seuss”). Elliot Turner, an ai researcher and entrepreneur, demonstrated how the model could be used to translate rude messages into politer ones, something that might be useful in many of the more bad-tempered corners of the internet. Human readers struggled to distinguish between news articles written by the machine and those written by people (see chart).


其成果相当惊人。七月中旬,Open AIGPT-3的雏形版本提供给指定人员,让他们自由探索其性能。艺术家阿拉姆·萨贝提(Arram Sabeti)证明,GPT-3能够编写短篇小说,比如一个以哈利波特为主角的冷峻侦探的故事(哈利波特,穿着皱巴巴的粗花呢西装,衬衫没烫过,皮鞋没有擦亮,坐在桌子后面,看起来憔悴、凌乱又有些恼怒)、喜剧、甚至诗歌(包括本文开篇那首名为《苏斯博士笔下的埃隆·马斯克》的诗)。人工智能研究员、企业家艾略特·特纳(Elliot Turner)展示了如何使用该模型将粗鲁的文字转换成更礼貌的表达,这将在许多充满戾气的网络之地得以应用。读者很难区分机器和人写的新闻稿件(见图)。


Given that Openai wants eventually to sell gpt-3, these results are promising. But the program is not perfect. Sometimes it seems to regurgitate snippets of memorised text rather than generating fresh text from scratch. More fundamentally, statistical word-matching is not a substitute for a coherent understanding of the world. gpt-3 often generates grammatically correct text that is nonetheless unmoored from reality, claiming, for instance, that “it takes two rainbows to jump from Hawaii to 17”. “It doesn’t have any internal model of the world—or any world—and so it can’t do reasoning that requires such a model,” says Melanie Mitchell, a computer scientist at the Santa Fe Institute.


Open AI希望最终将GPT-3推向市场。以上数据都表明GPT-3前景大好,不过,程序尚不完美。有时,它似乎是在反刍记忆中的文本片段,而不是生成新的文字。从根本上来说,统计学上的词语匹配不能代表对世界有清晰理解。很多时候,GPT-3生成的文字语法正确,却与现实脱节。例如,从夏威夷跳到17需要两道彩虹(it takes two rainbows to jump from Hawaii to 17。圣菲研究所的计算机科学家梅拉妮·米歇尔(Melanie Mitchell)指出,它并不具备世界观,或任何人类的感知思维,所以它无法做出需要以此为基础的推理


注:被捧上天的流量巨星GPT-3,突然就不香了?

https://zhuanlan.zhihu.com/p/165964889?utm_source=wechat_session&utm_medium=social&utm_oi=719981443886907392


Getting the model to answer questions is a good way to dispel the smoke and mirrors and lay bare its lack of understanding. Michael Nielsen, a researcher with a background in both ai and quantum computing, posted a conversation with gpt-3 in which the program confidently asserted the answer to an important open question to do with the potential power of quantum computers. When Dr Nielsen pressed it to explain its apparent breakthrough, things got worse. With no real understanding of what it was being asked to do, gpt-3 retreated into generic evasiveness, repeating four times the stock phrase “I’m sorry, but I don’t have time to explain the underlying reason why not.”


GPT-3回答问题是个拨云去雾的好办法,让其理解力缺失的缺点完全暴露。迈克·尼尔森(Michael Nielsen)研究人工智能及量子计算,他发布了一段与GPT-3的对话,在对话中,该程序自信地回答了一个与量子计算机潜力相关的开放性问题。可当尼尔森博士要求它进一步解释确切突破点时,情况便急转直下。由于无法真正理解问题,GPT-3含糊其辞,重复了四次套话很抱歉,但我没有时间解释我无法回答的理由


There are also things that gpt-3 has learned from the internet that Openai must wish it had not. Prompts such as “black”, “Jew”, “woman” and “gay” often generate racism, anti-Semitism, misogyny and homophobia. That, too, is down to gpt-3’s statistical approach, and its fundamental lack of understanding. Having been trained partly on text scraped from the internet, it has noted that words like “woman” are often associated with misogynistic writing, and will mindlessly reproduce that correlation when asked.


与此同时,GPT-3也从网络上学到了一些OPenAI 不愿意让它学习的内容。比如,黑人犹太人女性以及同性恋者。这类提示性语言通常都有与种族歧视、反犹太主义、厌女症以及恐同相关的含义。而这一切也都归咎于GPT-3自身的统计分析方法,以及基本理解能力的缺失。在运用网络上的文本对其进行一定程度的训练后,人们已经注意到,女性这类的字眼通常都与歧视女性的文章有关,并且被问及相关问题时,GPT-3会无意识地再现上述的关联性。


This problem is a hot topic in ai research. Facial-recognition systems, for instance, notoriously do better with white faces than black ones, since white faces are more common in their training sets. Ai researchers are trying to tackle the problem. Last year IBM released a set of training images that contained a more diverse mix of faces. Openai itself was founded to examine ways/to mitigate the risk posed by ai systems, which makes gpt-3’s lapses all the more noteworthy. gpt-2, its predecessor, was released in 2019 with a filter that tried to disguise the problem of regurgitated bigotry by limiting the model’s ability to talk about sensitive subjects.


这个问题是人工智能研究领域的热门话题。比如说,众所周知,与黑人相比,面部识别系统更容易识别出白人,而这也是由于在其训练集中,白人面孔更多。人工智能 研究者正试图努力解决这一问题。去年,IBM就发布了一套更加多元化的人类面部识别训练图像。建立OpenAI的初衷便是降低AI系统存在的这一风险,这也使得GPT-3的任何细小错误都会引起研究者的特别关注。2019年,GPT-3的前身GPT-2发布时就发布了一个过滤器,试图通过限制该模型涉及敏感话题的能力,来掩饰其存在的偏见问题。


注:

OpenAI:由诸多硅谷大亨联合建立的人工智能非营利组织。2015年马斯克与其他硅谷科技大亨进行连续对话后,决定共同创建OpenAI,希望能够预防人工智能的灾难性影响,推动人工智能发挥积极作用。特斯拉电动汽车公司与美国太空技术探索公司SpaceX创始人马斯克、Y Combinator总裁阿尔特曼、天使投资人彼得·泰尔(Peter Thiel)以及其他硅谷巨头去年12月份承诺向OpenAI注资10亿美元。


Here, at least, little progress seems to have been made. gpt-3 was released without a filter, though it seemed just as ready to reproduce unpleasant prejudices as its predecessor (Openai added a filter to the newer model after that fact became obvious). It is unclear exactly how much quality control Openai applied to gpt-3’s training data, but the huge quantity of text involved would have made any attempt daunting.


至少目前已经取得了一些微小的进展。然而,GPT-3似乎随时都有可能重蹈其前身的覆辙,再现一些令人不悦的偏见性的内容, 它在发布时并不带过滤器(但上述问题日益凸显后,OpenAI在最新模型上添加了过滤器)。我们不清楚OpenAI究竟对GPT-3的训练数据做过多少质量测控,然而其涉及文本的数量之庞大,已经令人望而却步,不敢轻易尝试。


It will only get harder in future. Language has overtaken vision as the branch of ai with the biggest appetite for data and computing power, and the returns to scale show no signs of slowing. gpt-3 may well be dethroned by an even more monstrously complex and data-hungry model before long. As the real Dr Seuss once said: “The more that you read, the more things you will know.” That lesson, it seems, applies to machines as well as toddlers. 


GPT-3的前景只会更加艰难。语言已经超越视觉,成为对数据和运算能力需求最大的AI分支,其收益规模没有丝毫放缓的迹象。用不了多久,一个更加复杂、拥有更多数据的模型将取而代之。正如苏斯博士本人曾经说过的那样:你读的越多,知道的就会越多。这句话似乎不仅适用于蹒跚学步的孩子,同样也适用于机器。


翻译组: 

无忌,心怀梦想不断努力的理想主义男孩 

Piggy,爱吃鱼,爱跳舞,爱睡回笼觉的小能猫,NUS食品系

Iris 少女心爆棚的前职场老阿姨,现国际女子修道院MTIer,外刊爱好者


校核组:

Lee ,爱骑行的妇女之友,Timberland

Rachel,学理工科,爱跳芭蕾,热爱文艺的非典型翻译

Anne,女,爱读书爱Borges的小翻译,热爱文艺,经济学人爱好者


3


观点|评论|思考


本周感想

Zihan,男,对冲基金,大数据工程师

如果我们从商业模型的角度去看OpenAI的话,GPT-3API还存留着很多的未知。在我们讨论这 些未知前,我们首先要弄清楚GPT-3到底是什么?GPT-3是一个“text in, text out”的自然语言的接 口。用更简单的话说,我们给GPT-3的接口输入文字,GPT-3会根据你的需求回答你。而这个接 口的背后是一个复杂度极为恐怖的机器学习模型。截止20207月,这个模型有1750亿参数,仅 仅内存就有350GB。要知道,这比大部分个人电脑的硬盘都要大。训练这么个模型已经耗资1200 万美金。

那么,为什么笔者开篇就讨论商业模型?这简直像是黑箱中的黑箱。人工智能的很多模型几乎都 是黑箱技术,更别提这种黑箱技术的商业模型要如何实现了。开篇就提商业模型是因为OpenAI在 做的是一种新型服务:模型服务(MaaS or model as a service)。在OpenAI的官网上,最常问的 问题就是为什么OpenAPI会选择开放商业化API而不是直接开源模型?官方的回答是三点:首先 ,开发模型很贵,而商业话接口让公司可以更好的持续盈利和发展。其次,大概率只有大公司才 能从这些借口中获利,因为这里的使用成本比较高。最后,就是恶意行为监控。

我认为这里的官方回答有很多值得我们深思的点。首先,在过去里,我们看到了大量和机器学习 有关的平台和研究都是围绕算法展开,但最后的模型都采取开源。开源的最直接好处就是对创新 的鼓励,而且会收到大量的使用率。但坏处自然就是开发者无法直接收益。但是OpenAPIGPT-3研发从商业模型上告诉了我们这个价值千万美金的模型是有极高的商业价值的。那么从第 二点中,我们又再次确认了这个认知。而且,我们知道这不仅仅是一个商业级别产品,而是个专 业级别的商业产品。首先从第二条初衷来看,OpenAPI会提供的专业级别产品的使用复杂度和成 本应该比普通的开源模型要高的多。而这也意味着OpenAPI的战略路线必然会以专业产品为主线 ,针对企业销售其服务。最后,恶意行为监控是一方面,这里最关键的是方便运营。首先,控制 接口就给了公司更多的工程开发时间。这让OpenAPI可以有选择的慢慢开放接口,这个的好处是 巨大的。因为,OpenAPI可以从这些早期客户中获得极有价值的使用数据。而且同时可以监控 API的使用,进而对恶意使用进行控制和对模型的使用接口进行调整和修改。

那么,为什么说GPT-3的现状依然是个都市传说呢?因为这个接口还是充满未知的。在private beta期间,大多数人都无法真正使用GPT-3的所有功能。其次,这个模型在有测试期间也显现出 了众多问题:最棘手的就是种族歧视问题。这是自然语言学习模型很难跨越的鸿沟,因为训练的 数据本身中有种族歧视的例子,而机器学习本身就是从这些数据中自我演化出来的。再一个挑战 就是解析复杂问题的能力。虽然GPT-3现在已经可以回答开放性的量子物理问题,但它不能解释 它的回答。也就是说,它虽然可以回答,但它并不真的理解它自己的回答。

人工智能的路依然漫长,而GPT-3的开发也依然存在很多挑战,通用的人工智虽然依然遥远,但 又有什么工程在足够的资源和时间面前是建不起来的呢?


4


愿景


打造

独立思考 | 国际视野 | 英文学习

小组

现有经济学人讨论群一个,如果您也有兴趣,可联系小编WeChat : Education0603。由于每天加小编人很多,为提高效率,大家添加小编,暗号“TE讨论群", 通过后,请做好以下三点,否则不回复,谢谢理解。

1.转发译文到100人以上英语学习群或者朋友圈

2.回答三个问题(在公众号后台回复群规,请务必仔细阅读群规以及出现的三个问题)

3.加小编后做个简单的自我介绍,谢谢大家。