How Humanity Survives and Thrives

We human beings have come to dominate this planet, not because of our powerful bodies, but because we do cooperate in large numbers — even with strangers. What allows us to do this? Intelligence and language help: they let us adapt effectively and quickly, and to develop new ways to cooperate as circumstances change. But there's a second human quality just as important: our sense of ethics.

Ethics is the foundation cooperation rests on. When someone breaks the rules of cooperation, we instinctually take careful notice, because we feel our wellbeing and survival is threatened. We pass this preoccupation with cooperation and ethics down deliberately, by preparing children for responsibility through ethical stories. But the sensitivity to ethical conduct isn't only taught — evidence suggests we're born with an instinctive sense of it too. Either way, human survival has always depended on ethics, and still does.

We are now giving machines the capacity to act independently. Done well, this capacity could help build a prosperous future. But machines don't necessarily inherit the implicit guardrails of cooperative culture and instinct that human beings assume. Our human survival instincts keep most of us from crossing the line into harm, whereas autonomous machine systems may behave in uncooperative or harmful ways without hesitation or pause, simply because their training left them oblivious to the approach of cooperation and ethical restraint. This is why we may feel uneasy when considering how autonomous machines will impact our wellbeing and survival as we try use them to create a better future. 

This isn't all hypothetical. Lists of real-world harm from autonomous systems already exist - sometimes in obvious ways, and sometimes the quiet result of a system that works exactly as designed, but within a scope too narrow for the world it was released into. By this I mean how the intrusion of an algorithm into our social networks - the recommendation engine in social media - has reduced trust and cooperation in human societies.

It is the ethical machine, rather than the intelligent machine that we must carefully design and create. Here I will explore how we may train ethically cooperative systems rather than ethically oblivous systems.

**Next:** Self-Aware Machines →