(The below text version of the notes is for search purposes and convenience. See the PDF version for proper formatting such as bold, italics, etc., and graphics where applicable. Copyright: 2022 Retraice, Inc.)
Re98: News of ChatGPT, Part 2
retraice.com
Wondering about ChatGPT's predictions and keeping it alive.
What is ChatGPT good at? Generating meaningful text in styles; explaining coding errors; it's like an unknown new person in town; What does ChatGPT do? Text responses to text prompts; predicting the next word in a sequence? trying to maximize reward or scores of some sort; its builders and controllers and context; the dangers of certainty about goals; humans keeping ChatGPT `alive'; ChatGPT keeping itself `alive'; crossing ChatGPT with the hypotheses.
Air date: Tuesday, 27th Dec. 2022, 11:00 PM Eastern/US.
Amendments and corrections to Re96
Correction: The Ezra Klein interview of Sam Ultman was a 2021 replay, not recent. Amendment: The MWE of Django code from ChatGPT didn't work. See Retraice (2022/12/26) and https://github.com/retraice/ReMisc for details.
Open question: What is it good at?
It's good a generating meaningful text in styles. The prompt "Write a story, in Shakespearean voice, about a boy and a girl who go on a journey that takes place in Bucharest." generates several paragraphs, beginning with:
"In fair Bucharest, where we lay our scene,
A boy and girl, with hearts full of gleam,
Decided to embark on a journey grand,
To see all the sights, and roam free as sand."^1
Tools are being built to use ChatGPT to explain coding
errors:
* https://github.com/shobrook/stackexplain^2
* https://github.com/fkhan0520/cgpt_exceptions
* Extensive list (see Antonio Cheong below): https://github.com/stars/acheong08/lists/awesome-chatgpt
One programmer, Antonio Cheong, has a "Reverse Engineered ChatGPT" that's popular on Github: https://github.com/acheong08/ChatGPT
Only time will tell what ChatGPT is good and bad at. It's like a new person in the world.^3 This will be true of many systems that come online in the next twenty years. Everyone on Earth is going to have to get used to new AI systems on a regular basis. We'll also have to look for signs of systems that are not publicly known, systems being used by criminals and other nefarious actors.
Open question: What does it do?
Physically, it produces text in response to text prompts. Alternatively: it produces compelling dialogue and confident knowledge work of unchecked quality.
It's a large language model:
"We define a language model as a probability distribution describing the likelihood of any string. Such a model should say that `Do I dare disturb the universe?' has a reasonable probability as a string of English, but `Universe dare the I disturb do?' is extremely unlikely. With a language model, we can predict what words are likely to come next in a text, and thereby suggest completions for an email or text message. We can compute which alterations to a text would make it more probable, and thereby suggest spelling or grammar corrections. With a pair of models, we can compute the most probable translation of a sentence. With some example question/answer pairs as training data, we can compute the most likely answer to a question. So language models are at the heart of a broad range of natural language tasks."^4
It predicts words in a sequence? It's not predicting the next word in an existing sequence. There is no next word yet. Is it predicting the future? Not in the way a weatherman would. It's predicting the next word that would ... maximize its reward? maximize a utility or objective function? please a supervisor?
This explanation seems totally inadequate:
"EZRA KLEIN: And so if I basically understand how GPT-3 works, it's a system that has read a lot of stuff on the internet.
SAM ALTMAN: Yes.
EZRA KLEIN: And it's predicting the next word in the sequence.
SAM ALTMAN: Slightly oversimplified but very close. Yes, it is trying to predict what comes next in a sequence."^5
In the real world, we have to know a lot about what lead to an AI system being deployed, and (if possible) the intentions of the people controlling it, to understand what it's doing. And this is to say nothing of the difference between systems that pursue goals with 100% certainty vs. systems that have some doubt about the fidelity of their representation of a goal, as discussed by Russell (2019). See `fetching coffee' below.
What does it take to keep
(e.g.) ChatGPT `alive'?
To keep ChatGPT going will require much more than just keeping OpenAI going. First there is hardware and software infrastructure, from company to country to economy, and therefore humans, from owners and employees (the `power' yin to the `control' yang^6) to customers, families, friends and foes. Companies, like people and governments are hugely interdependent. Companies, unlike governments, depend on power derived from having customers; countries (nation states) depend on power, in the end, derived from a monopoly on violence granted to them (happily or unhappily) by citizens or subjects.
These are considerations if humans are trying to keep ChatGPT `alive'. What would a system such as ChatGPT prioritize if it were trying to keep itself `alive'? Consider `instrumental convergence' and Omohundro's `basic AI drives':^7
1.
self-improvement;
2.
rationality;
3.
preservation of utility function;
4.
prevention of counterfeit utility;
5.
self-protection;
6.
acquisition and efficient use of resources.
Stuart Russell on fetching coffee:
"If a machine pursuing an incorrect objective sounds bad enough, there's worse. The solution suggested by Alan Turing--turning off the power at strategic moments--may not be available, for a very simple reason: you can't fetch the coffee if you're dead. Let me explain. Suppose a machine has the objective of fetching the coffee. If it is sufficiently intelligent, it will certainly understand that it will fail in its objective if it is switched off before completing its mission. Thus, the objective of fetching coffee creates, as a necessary subgoal, the objective of disabling the off-switch. The same is true for curing cancer or calculating the digits of pi. There's really not a lot you can do once you're dead, so we can expect AI systems to act preemptively to preserve their own existence, given more or less any definite objective."^8
See also Butler (1863).
Next, we'll cross ChatGPT with the hypotheses
On H1-H11, see Retraice (2022/03/07).
H1
Space: `Humans are now technologically capable of living in space.'
H2
Technology: `Human technology risks are growing faster than their mitigation.'
H3
Death: `Human lifespan is being prolonged by new
technologies.'
H4
China: `The U.S. is no longer the only superpower; war is likely.'
H5
Civil War: `The U.S. seems vulnerable to a civil war this decade.'
H6
Environments: `Humans can change environments faster than they can adapt.'
H7
Betterment: `Some things make the future better than the past.'
H8
Intelligence: `There are intelligence differences.'
H9
Darkness: `There is a pervasive darkness in humans, even amongst the good guys.'
H10
Wealth: `The current trend toward concentration of wealth is making human life worse.'
H11
Wildcards: `New technologies, discoveries and deception regularly cause historic changes.'
H12
Computers: `Some humans now control others better, but machinery could take control.'
Here's H12 (an attempt to unify H1-H11) in detail:
Computers,
which are chain-reaction controllers,
and which make AI handling of information
possible,
and which are inherently vulnerable to
hacking,
are causing some humans to know others
better than they know themselves,
and thereby to control them,
though computer-controlled machinery
could take control
if the motivation to control,
which humans have,
were to occur, naturally or by design,
in the chain-reactions.^9
_
References
Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford. First published in 2014. Citations are from the pbk. edition, 2016. ISBN: 978-0198739838. Searches:
https://www.amazon.com/s?k=978-0198739838
https://www.google.com/search?q=isbn+978-0198739838
https://lccn.loc.gov/2015956648
Butler, S. (1863). Darwin among the machines. The Press (Canterbury, New Zealand). Reprinted in ?.
Omohundro, S. (2008). The Basic AI Drives. (pp. 483-492). In Wang et al. (2008).
Retraice (2022). AIMA4e Notes. retraice.com. https://aima4e.retraice.com
Retraice (2022/03/07). Re17: Hypotheses to Eleven. retraice.com.
https://www.retraice.com/segments/re17 Retrieved 17th Mar. 2022.
Retraice (2022/10/19). Re22: Computer Control. retraice.com.
https://www.retraice.com/segments/re22 Retrieved 19th Oct. 2022.
Retraice (2022/11/24). Re60: Complexity, Linear Algebra, Probability (AIMA4e Appendix A). retraice.com.
https://www.retraice.com/segments/re60 Retrieved 25th Nov. 2022.
Retraice (2022/12/26). Re96: News of ChatGPT, Part 1. retraice.com.
https://www.retraice.com/segments/re96 Retrieved 27th Dec. 2022.
Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking. ISBN: 978-0525558613. Searches:
https://www.amazon.com/s?k=978-0525558613
https://www.google.com/search?q=isbn+978-0525558613
https://lccn.loc.gov/2019029688
Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach. Pearson, 4th ed. ISBN: 978-0134610993. Searches:
https://www.amazon.com/s?k=978-0134610993
https://www.google.com/search?q=isbn+978-0134610993
https://lccn.loc.gov/2019047498
Wang, P., Goertzel, B., & Franklin, S. (Eds.) (2008). Artificial General Intelligence 2008: Proceedings of the First AGI Conference. IOS Press. ISBN: 978-1586038335. Searches:
https://www.amazon.com/s?k=9781586038335
https://www.google.com/search?q=isbn+9781586038335
https://lccn.loc.gov/2008900954
Footnotes
^1 https://github.com/retraice/ReMisc/tree/main/Re98-ChatGPT-News-2
^2 The Reddit post by jsonathan mentioned during the livestream: "[P] I made a command-line tool that explains your errors using ChatGPT"
^3 Is it a `stochastic parrot'?
https://en.wikipedia.org/wiki/ChatGPT#Negative_reactions
^4 Russell & Norvig (2020) p. 824. On probability, see Retraice (2022) and Retraice (2022/11/24).
^5 Transcript: Ezra Klein Interviews Sam Altman, June 11th, 2021.
^6 See Re96, Retraice (2022/12/26).
^7 On instrumental convergence, see Bostrom (2014) p. 131 ff. On `basic AI drives', see Omohundro (2008) or https://wiki.lesswrong.com/wiki/Basic_AI_drives. CORRECTION: During the livestream, I said `four drives'; Omohundro actually gives six.
^8 Russell (2019) pp. 140-141.
^9 Retraice (2022/10/19).
