“I think we can think more thoughtfully and more carefully about safety issues,” says Altman. “Part of our strategy is: Gradual changes in the world are better than sudden changes,” or, as OpenAI VP Mira Murati said, open access to software. When I asked her about the work of the safety team to limit, “Learn how to deploy these powerful technologies. Start when your stakes are very low.”
GPT-3 itself OpenAI runs on 285,000 CPU cores in the Iowa supercomputer cluster and runs in a refurbished luggage factory in the mission district of San Francisco. Last November, I met Ilya Sutskever there and tried to get a layman’s explanation of how GPT-3 actually works.
“This is the underlying idea of GPT-3,” Sutskever enthusiastically leaned against his chair. He has an interesting way to answer the question: some false starts — “I can give you an explanation that closely matches what you asked for” — as if he were planning the entire response A forward interrupted by a long meditative pause, like.
“The underlying idea of GPT-3 is a way to link the concept of intuitive understanding to something that can be measured and understood mechanically,” he said at the end. “And it’s the task of predicting the next word in the text.”’ Other forms of artificial intelligence try to hard-code information about the world. Grandmaster’s chess strategy, a principle of climatology. But GPT- The intelligence of 3 comes from the bottom up if the intelligence is the correct word for it: through the basic act of predicting the next word. To train GPT-3, the model has a “prompt” (eg, for example). , A few sentences or paragraphs of text from newspaper articles, novels, academic papers), and then you are asked to suggest a list of potential words such as: It is possible to complete a ranked sequence with a probability. In the early stages of training, the suggested words are nonsense. Prompt the algorithm with a statement like”. The writer omitted the last word of the first word. .. .. ” And guessing becomes a kind of nonsense flow:”Satellite””Puppy””Seattle””Therefore”But somewhere in the list-perhaps thousands of words in the list— The correct missing word is displayed: “Paragraph” The software then strengthens the random neural connections that generated that particular suggestion and weakens all the connections that generated the wrong guess. Then proceed to the next prompt. Over time, the software learns when it iterates enough.
In January of this year, OpenAI added the ability to give direct instructions to GPT-3 as a prompt, rather than requesting the user to extend the text sample passage. For example, using “instruction” mode, I once prompted GPT-3 to “write an essay discussing the role of metafiction in Italo Calvino’s work.” In return, the software provided five well-built-precision of paragraphs that began as follows:
The Italian writer Italo Calvino is considered a master of metafiction. Metafiction is a genre of writing that breaks the fourth wall for writers to discuss the act of writing themselves. For Carvino, metafiction is a way of exploring the essence of reality and how stories can shape the perceptions of our world. His novels often incorporate a playful maze-like structure that plays at the boundary between reality and fiction. For example, if there are travelers on winter nights, the reader is constantly interrupted by meta-level discussions about the act of reading and the nature of storytelling. .. .. ..
You can give the GPT-3 the exact same prompt over and over, and each time it produces a unique response, some of them are more convincing than others, but almost everything is very clear. Instruction prompts can take any form. “Tell me a list of all the ingredients in Bolognese sauce” “Write a poem about a French coastal village in the style of John Ashbery” “Please explain the Big Bang First this kind of GPT-3 prompt When given several times to, I felt a real quiver run down my spine. The machine was very clear and responsive text, based entirely on the predictive element training of the next word. It seemed almost impossible to generate.
However, AI has a long history of creating illusions of intelligence and understanding without actually delivering products.so Well-discussed papers A group of co-authors with Emily M. Bender, a linguistic professor at the University of Washington, and former Google researcher Timnit Gebru, published last year, declared that the large language model was just a “probabilistic parrot.” In other words, the software is just a randomization to remix human-generated sentences. “What has changed is not to cross the threshold for’AI’,” the vendor recently emailed me. Rather, what has changed is “hardware, software, and economic innovations that enable the storage and processing of vast datasets,” and “the technological culture that people who build and sell such things can get.” “She said. Stop building them on the foundation of uncurated data. “