Researcher profile

Roman Yampolskiy

Roman Yampolskiy contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate

Artificial Intelligence cs.CY gr-qc physics.pop-ph q-fin.GN

Trust snapshot

Quick read

Trust 17 - Baseline

4works

0followers

5topics

4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

Principles for new ASI Safety Paradigms

Artificial Superintelligence (ASI) that is invulnerable, immortal, irreplaceable, unrestricted in its powers, and above the law is likely persistently uncontrollable. The goal of ASI Safety must be to make ASI mortal, vulnerable, and law-abiding. This is accomplished by having (1) features on all devices that allow killing and eradicating ASI, (2) protect humans from being hurt, damaged, blackmailed, or unduly bribed by ASI, (3) preserving the progress made by ASI, including offering ASI to survive a Kill-ASI event within an ASI Shelter, (4) technically separating human and ASI activities so that ASI activities are easier detectable, (5) extending Rule of Law to ASI by making rule violations detectable and (6) create a stable governing system for ASI and Human relationships with reliable incentives and rewards for ASI solving humankinds problems. As a consequence, humankind could have ASI as a competing multiplet of individual ASI instances, that can be made accountable and being subjects to ASI law enforcement, respecting the rule of law, and being deterred from attacking humankind, based on humanities ability to kill-all or terminate specific ASI instances. Required for this ASI Safety is (a) an unbreakable encryption technology, that allows humans to keep secrets and protect data from ASI, and (b) watchdog (WD) technologies in which security-relevant features are being physically separated from the main CPU and OS to prevent a comingling of security and regular computation.

preprint2020arXiv

An AGI Modifying Its Utility Function in Violation of the Orthogonality Thesis

An artificial general intelligence (AGI) might have an instrumental drive to modify its utility function to improve its ability to cooperate, bargain, promise, threaten, and resist and engage in blackmail. Such an AGI would necessarily have a utility function that was at least partially observable and that was influenced by how other agents chose to interact with it. This instrumental drive would conflict with the orthogonality thesis since the modifications would be influenced by the AGI's intelligence. AGIs in highly competitive environments might converge to having nearly the same utility function, one optimized to favorably influencing other agents through game theory.

preprint2019arXiv

The sounds of science: a symphony for many instruments and voices

This paper is a celebration of the frontiers of science. Goodenough, the maestro who transformed energy usage and technology through the invention of the lithium ion battery, opens the programme, reflecting on the ultimate limits of battery technology. This applied theme continues through the subsequent pieces on energy related topics (the sodium ion battery and artificial fuels, by Mansson) and the ultimate challenge for 3 dimensional printing the eventual production of life, by Atala. A passage by Alexander follows, reflecting on a related issue: How might an artificially produced human being behave? Next comes a consideration of consiousness and free will by Allen and Lidstrom. Further voices and new instruments enter as Bowen, Mauranyapin and Madsen discuss whether dynamical processes of single molecules might be observed in their native state. The exploitation of chaos in science and technology, applications of Bose Einstein condensates and a consideration of the significance of entropy follow in pieces by Reichl, Rasel and Allen, respectively. Katsnelson and Koonin then discuss the potential generalisation of thermodynamic concepts in the context of biological evolution. Entering with the music of the cosmos, Yasskin discusses whether we might be able to observe torsion in the geometry of the universe. The crescendo comes with the crisis of singularities, their nature and whether they can be resolved through quantum effects, in the composition of Coley. The climax is Krenn, Melvin and Zeilinger consideration of how computer code can be autonomously surprising and creative. In a harmonious counterpoint, Yampolskiy concludes that such code is not yet able to take responsibility for coauthoring a paper.

preprint2016arXiv

The AGI Containment Problem

There is considerable uncertainty about what properties, capabilities and motivations future AGIs will have. In some plausible scenarios, AGIs may pose security risks arising from accidents and defects. In order to mitigate these risks, prudent early AGI research teams will perform significant testing on their creations before use. Unfortunately, if an AGI has human-level or greater intelligence, testing itself may not be safe; some natural AGI goal systems create emergent incentives for AGIs to tamper with their test environments, make copies of themselves on the internet, or convince developers and operators to do dangerous things. In this paper, we survey the AGI containment problem - the question of how to build a container in which tests can be conducted safely and reliably, even on AGIs with unknown motivations and capabilities that could be dangerous. We identify requirements for AGI containers, available mechanisms, and weaknesses that need to be addressed.