Posted on 16-2-2002
Forfeiting
The Future
by Bill Joy from www.resurgence.org
issue 208 (Photo shows Bill Joy)
ACCUSTOMED TO living with routine scientific break-throughs,
we have yet
tocome to terms with the fact that the most compelling new technologies
-
robotics, genetic engineering and nanotechnology - pose a different
kind of
threat than the technologies that came before. Specifically,
robots,
engineered organisms and nanobots share a dangerous amplifying
factor: they
can self-replicate. A bomb is blown up only once, but one altered
gene can
become many, and quickly get out of control.
While replication in a computer or a computer network can be
a nuisance, at
worst it disables a machine or takes down a network or network
service. But
self-replication in the new technologies runs a much greater
risk: a risk
of substantial damage in the physical world.
Each of these new technologies also offers untold promise: the
vision of
near immortality; genetic engineering that may soon provide
treatments, if
not outright cures, for most diseases; and nanotechnology and
nanomedicine
which can apparently address yet more ills. Together these technologies
could significantly extend our average life-span and improve
the quality of
our lives. Yet, with each of these technologies, a sequence
of small,
individually sensible advances leads to an accumulation of great
power,
and, concomitantly, great danger.
What was different in the twentieth century? Certainly, the
technologies
underlying the weapons of mass destruction - nuclear, biological
and
chemical - were and are a powerful and enormous threat. But
building
nuclear weapons required, at least for a time, access to both
rare raw
materials and highly protected information.
The twenty-first-century technologies, however, are within the
grasp of
individuals. They do not require large facilities and rare materials.
Knowledge alone will enable the use of them. Thus, we have the
possibility
of not just weapons of mass destruction, but of knowledge-enabled
mass
destruction, hugely amplified by the power of self-replication.
It is always hard to see the bigger impact of technology whilst
in the
vortex of change, but failing to understand the consequences
of our
inventions while we are in the rapture of discovery and innovation
seems to
be a common fault of scientists and technologists. We have long
been driven
by an overarching desire to know: that is the nature of science's
quest,
not stopping to notice that the progress to newer and more powerful
technologies can take on a life of its own.
Because of the recent rapid and radical progress in molecular
electronics
and related nanoscale technologies, by 2030 we are likely to
be able to
build machines a million times as powerful as the personal computers
of
today. As this enormous computing power is combined with the
manipulative
advances of the physical sciences and the new, deep understandings
in
genetics, enormous transformative power is being unleashed.
These
combinations open up the opportunity to redesign the world completely,
for
better or worse. The replicating and evolving processes that
have been
confined to the natural world are about to become realms of
human endeavour.
Given the incredible power of these new technologies, shouldn't
we proceed
with great caution?
THE DREAM OF ROBOTICS is that intelligent machines can do our
work for us,
allowing us lives of leisure, restoring us to Eden. How soon
could such an
intelligent robot be built? The coming advances in computing
power seem to
make it possible by 2030, and once an intelligent robot exists,
it is only
a small step to a robot species - to an intelligent robot that
can make
evolved copies of itself.
A second dream of robotics is that we will gradually replace
ourselves with
our robotic technology, achieving near immortality by downloading
our
consciousnesses. We are beginning to see intimations of this
in the
implantation of computer devices into the human body. But if
we are
downloaded into our technologies, what are the chances that
we will
thereafter be ourselves or even human?
Genetic engineering promises to revolutionize agriculture by
increasing
crop yields while reducing the use of pesticides; to create
tens of
thousands of novel species of bacteria, plants, viruses and
animals; to
replace reproduction, or supplement it, with cloning; to create
cures for
many diseases, increasing our life-span. We know with certainty
that the
profound changes in the biological sciences are imminent and
will challenge
all our notions of what life is.
Technologies such as human cloning have, in particular, raised
our
awareness of the profound ethical and moral issues we face.
If, for
example, we were to re-engineer ourselves into several separate
and unequal
species using the power of genetic engineering, then we would
threaten the
notion of equality that is the very cornerstone of our democracy.
THE MANY WONDERS of nanotechnology were first imagined by the
Nobel-laureate physicist Richard Feynman in a speech in 1959,
in which he
described how manipulation of matter at the atomic level could
create a
utopian future of abundance, where just about everything could
be made
cheaply, and almost any disease or physical problem could be
solved using
nanotechnology and artificial intelligences.
Imagine some of the changes that might take place in a world
where we had
molecular-level 'assemblers'. Assemblers could make possible
incredibly
low-cost solar power, cure cancer and the common cold by augmentation
of
the human immune system, could clean up the environment, create
inexpensive
pocket supercomputers, and restore extinct species.
The enabling breakthrough to assemblers seems quite likely within
the next
twenty years. Molecular electronics should mature quickly and
become
enormously lucrative within this decade, causing a large incremental
investment in all nanotechnologies.
But we can't simply do our science and not worry about the ethical
issues.
Unfortunately, as with nuclear technology, it is far easier
to create
destructive uses of nanotechnology than constructive ones. Nanotechnology
has clear military and terrorist uses, and you need not be suicidal
to
release a massively destructive nanotechnological device: such
devices
could be built to be selectively destructive, affecting for
example, only a
certain geographical area or a group of people who are genetically
distinct.
An immediate consequence of the Faustian bargain in obtaining
the great
power of nanotechnology is that we run a grave risk - the risk
that we
might destroy the biosphere on which all life depends. For example,
as Eric
Drexler explained in his book, Engines of Creation: "'Plants'
with 'leaves'
no more efficient than today's solar cells could out-compete
real plants,
crowding the biosphere with an inedible foliage. Tough omnivorous
'bacteria' could out-compete real bacteria; they could spread
like pollen
in the wind, replicating swiftly and reducing the biosphere
to dust in a
matter of days.
Dangerous replicators could easily be too rapidly-spreading,
tough and
small to stop. We have trouble enough controlling viruses and
fruit flies.
We cannot afford these kinds of accident with self-replicating
assemblers."
THESE POSSIBILITIES ARE all undesirable. The only realistic
alternative is
relinquishment: to limit development of the technologies that
are too
dangerous, by limiting our pursuit of certain kinds of knowledge.
Although
humankind inherently 'desires to know', if open access to, and
unlimited
development of, knowledge henceforth puts us all in clear danger
of
extinction, then common sense demands that we re-examine our
reverence for
knowledge.
If we could agree, as a species, what we wanted, where we were
headed and
why, then we could make our future much less dangerous - then
we might
understand what we could and should relinquish. If the course
of humanity
could be determined by our collective values, ethics and morals,
and if we
had gained more collective wisdom over the past few thousand
years, then a
dialogue to this end would be practical, and the incredible
powers that we
are about to unleash would not be nearly so troubling.
One would think that we might be driven to such a dialogue by
our instinct
for self-preservation. Individuals clearly have this desire,
yet as a
species our behaviour seems not to be in our favour. The new
Pandora's
boxes of genetics, nanotechnology and robotics are almost open,
yet we seem
hardly to have noticed. Ideas can't be put back in a box: unlike
uranium or
plutonium, they don't need to be mined and refined; they can
be freely
copied. Once they are out, they are out.
Verifying relinquishment will be a difficult problem, but not
an unsolvable
one. We are fortunate to have already done a lot of relevant
work in the
context of the Biological Weapons Convention and other treaties.
Verifying
compliance will also require that scientists, technologists
and engineers
adopt a strong code of ethical conduct, resembling the Hippocratic
oath,
that they cease and desist from work creating, developing and
manufacturing
knowledge-enabled technologies of mass destruction.
Where can we look for a new ethical basis to set our course?
We would do
well to consider a new book by His Holiness the Dalai Lama called
Ethics
for the New Millennium. As is perhaps well-known but little-heeded,
the
Dalai Lama argues that the most important thing is for us to
conduct our
lives with love and compassion for others, and that our societies
need to
develop a stronger notion of universal responsibility and of
our
interdependency. He proposes a standard of positive ethical
conduct for
individuals and societies and further argues that we must understand
what
it is that makes people happy, and acknowledge the strong evidence
that
neither material progress nor the pursuit of the power of knowledge
is the
key - that there are limits to what science and the scientific
pursuit
alone can do. o
A full-length version of this article first appeared in the
April 2000
edition of Wired magazine.
Bill Joy is co-founder and Chief Scientist of Sun Micro-systems
and was
Co-Chair of the Presidential Commission on the Future of IT
Research.
|