Lists of Figures, Tables, and Boxes | p. xv |
Past developments and present capabilities | p. 1 |
Growth modes and big history | p. 1 |
Great expectations | p. 3 |
Seasons of hope and despair | p. 5 |
State of the art | p. 11 |
Opinions about the future of machine intelligence | p. 18 |
Paths to superintelligence | p. 22 |
Artificial intelligence | p. 23 |
Whole brain emulation | p. 30 |
Biological cognition | p. 36 |
Brain-computer interfaces | p. 44 |
Networks and organizations | p. 48 |
Summary | p. 50 |
Forms of superintelligence | p. 52 |
Speed superintelligence | p. 53 |
Collective superintelligence | p. 54 |
Quality superintelligence | p. 56 |
Direct and indirect reach | p. 58 |
Sources of advantage for digital intelligence | p. 59 |
The kinetics of an intelligence explosion | p. 62 |
Timing and speed of the takeoff | p. 62 |
Recalcitrance | p. 66 |
Non-machine intelligence paths | p. 66 |
Emulation and AI paths | p. 68 |
Optimization power and explosivity | p. 73 |
Decisive strategic advantage | p. 78 |
Will the frontrunner get a decisive strategic advantage? | p. 79 |
How large will the successful project be? | p. 83 |
Monitoring | p. 84 |
International collaboration | p. 86 |
From decisive strategic advantage to singleton | p. 87 |
Cognitive superpowers | p. 91 |
Functionalities and superpowers | p. 92 |
An AI takeover scenario | p. 95 |
Power over nature and agents | p. 99 |
The superintendent will | p. 105 |
The relation between intelligence and motivation | p. 105 |
Instrumental convergence | p. 109 |
Self-preservation | p. 109 |
Goal-content integrity | p. 109 |
Cognitive enhancement | p. 111 |
Technological perfection | p. 112 |
Resource acquisition | p. 113 |
Is the default outcome doom? | p. 115 |
Existential catastrophe as the default outcome of an intelligence explosion? | p. 115 |
The treacherous turn | p. 116 |
Malignant failure modes | p. 119 |
Perverse instantiation | p. 120 |
Infra structure profusion | p. 122 |
Mind crime | p. 125 |
The control problem | p. 127 |
Two agency problems | p. 127 |
Capability control methods | p. 129 |
Boxing methods | p. 129 |
Incentive methods | p. 131 |
Stunting | p. 135 |
Tripwires | p. 137 |
Motivation selection methods | p. 138 |
Direct specification | p. 139 |
Domesticity | p. 140 |
Indirect normativity | p. 141 |
Augmentation | p. 142 |
Synopsis | p. 143 |
Oracles, genies, sovereigns, tools | p. 145 |
Oracles | p. 145 |
Genies and sovereigns | p. 148 |
Tool-AIs | p. 151 |
Comparison | p. 155 |
Multipolar scenarios | p. 159 |
Of horses and men | p. 160 |
Wages and unemployment | p. 160 |
Capital and welfare | p. 161 |
The Malthusian principle in a historical perspective | p. 163 |
Population growth and investment | p. 164 |
Life in an algorithmic economy | p. 166 |
Voluntary slavery, casual death | p. 167 |
Would maximally efficient work be fun? | p. 169 |
Unconscious outsourcers? | p. 172 |
Evolution is not necessarily up | p. 173 |
Post-transition formation of a singleton? | p. 176 |
A second transition | p. 177 |
Superorganisms and scale economies | p. 178 |
Unification by treaty | p. 180 |
Acquiring values | p. 185 |
The value-loading problem | p. 185 |
Evolutionary selection | p. 187 |
Reinforcement learning | p. 188 |
Associative value accretion | p. 189 |
Motivational scaffolding | p. 191 |
Value learning | p. 192 |
Emulation modulation | p. 201 |
Institution design | p. 202 |
Synopsis | p. 207 |
Choosing the criteria for choosing | p. 209 |
The need for indirect normativity | p. 209 |
Coherent extrapolated volition | p. 211 |
Some explications | p. 212 |
Rationales for CEV | p. 213 |
Further remarks | p. 216 |
Morality models | p. 217 |
Do What I Mean | p. 220 |
Component list | p. 221 |
Goal content | p. 222 |
Decision theory | p. 223 |
Epistemology | p. 224 |
Ratification | p. 225 |
Getting close enough | p. 227 |
The strategic picture | p. 228 |
Science and technology strategy | p. 228 |
Differential technological development | p. 229 |
Prefer red order of arrival | p. 230 |
Rates of change and cognitive enhancement | p. 233 |
Technology couplings | p. 236 |
Second-guessing | p. 238 |
Pathways and enablers | p. 240 |
Effects of hardware progress | p. 240 |
Should whole brain emulation research be promoted? | p. 242 |
The person-affecting perspective favors speed | p. 245 |
Collaboration | p. 246 |
The race dynamic and its perils | p. 246 |
On the benefits of collaboration | p. 249 |
Working together | p. 253 |
Crunch time | p. 255 |
Philosophy with a deadline | p. 255 |
What is to be done? | p. 256 |
Seeking the strategic light | p. 257 |
Building good capacity | p. 258 |
Particular measures | p. 258 |
Will the best in human nature please stand up | p. 259 |
Notes | p. 261 |
Bibliography | p. 305 |
Index | p. 325 |
Table of Contents provided by Ingram. All Rights Reserved. |