Blog

When Elon Musk announced the team behind his new artificial intelligence company xAI last month, whose mission is reportedly to “understand the true nature of the universe,” it underscored the criticality of answering existential concerns about AI’s promise and peril.

Whether the newly formed company can actually align its behavior to reduce the potential risks of the technology, or whether it’s solely aiming to gain an edge over OpenAI, its formation does elevate important questions about how companies should actually respond to concerns about AI. Specifically:

1. Who internally, especially at the largest foundational model companies, is actually asking questions about both the short- and long-term impacts of the technology they’re building?
2. Are they coming at the issues with an appropriate lens and expertise?
3. Are they adequately balancing technological considerations with social, moral, and epistemological issues?

In college, I majored in computer science and philosophy, which seemed like an incongruous combination at the time. In one classroom, I was surrounded by people thinking deeply about ethics (“What’s right, what’s wrong?”), ontology (“What is there, really?”), and epistemology (“What do we actually know?”). In another, I was surrounded by people who did algorithms, code, and math.

Twenty years later, in a stroke of luck over foresight, the combination is not so inharmonious in the context of how companies need to think about AI. The stakes of AI’s impact are existential, and companies need to make an authentic commitment worthy of those stakes.

This means staffing their leadership teams with stakeholders who are adequately equipped to sort through the consequences of the technology they’re building — which is beyond the natural expertise of engineers who write code and harden APIs.

AI isn’t an exclusively computer science challenge, neuroscience challenge, or optimization challenge. It’s a human challenge. To address it, we need to embrace an enduring version of an “AI meeting of the minds,” equivalent in scope to Oppenheimer’s cross-disciplinary gathering in the New Mexico desert (where I was born) in the early 1940s.

The collision of human desire with AI’s unintended consequences results in what researchers term the “alignment problem,” expertly described in Brian Christian’s book “The Alignment Problem.” Essentially, machines have a way of misinterpreting our most comprehensive instructions, and we, as their alleged masters, have a poor track record of making them fully understand what we think we want them to do.

The net result: Algorithms can advance bias and disinformation and thereby corrode the fabric of our society. In a longer-term, more dystopian scenario, they can take the “treacherous turn” and the algorithms to which we’ve ceded too much control over the operation of our civilization overtake us all.

Unlike Oppenheimer’s challenge, which was scientific, ethical AI requires a deep understanding of what there is, what we want, what we think we know, and how intelligence unfolds. This is an undertaking that is certainly analytic, though not strictly scientific in nature. It requires an integrative approach rooted in critical thinking from both the humanities and the sciences.

Thinkers from different fields need to work closely together, now more than ever. The dream team for a company seeking to get this really right would look something like:

• Chief AI and data ethicist: This person would address short- and long-term issues with data and AI, including but not limited to the articulation and adoption of ethical data principles, the development of reference architectures for ethical data use, citizens’ rights regarding how their data is consumed and used by AI, and protocols for shaping and adequately controlling AI behavior. This should be separate from the chief technology officer, whose role is largely to execute a technology plan rather than address its repercussions. It’s a senior role on the CEO’s staff that bridges the communication gap between internal decision makers and regulators. You can’t separate a data ethicist from a chief AI ethicist: Data is the precondition and the fuel for AI; AI itself begets new data.
• Chief philosopher architect: This role would address the longer-term, existential concerns with a principal focus on the “Alignment Problem”: how to define safeguards, policies, back doors, and kill switches for AI to align it to the maximum extent possible with human needs and objectives.
• Chief neuroscientist: This person would address critical questions of sentience and how intelligence unfolds within AI models, what models of human cognition are most relevant and useful for the development of AI, and what AI can teach us about human cognition.

Critically, to turn the dream team’s output into responsible, effective technology, we need technologists who can translate abstract concepts and questions posed by “The Three” into working software. As with all working technology groups, this depends on the product leader/designer who sees the whole picture.

A new breed of inventive product leader in the “Age of AI” must move comfortably across new layers of the technology stack encompassing model infrastructure for AI, as well as new services for things like fine-tuning and proprietary model development. They need to be inventive enough to imagine and design “Human in the Loop” workflows to implement safeguards, back doors, and kill switches as prescribed by the chief philosopher architect. They need to have a renaissance engineer’s ability to translate the chief AI’s and data ethicist’s policies and protocols into working systems. They need to appreciate the chief neuroscientist’s efforts to move between machines and minds and adequately discern findings with the potential to give rise to smarter, more responsible AI.

Let’s look at OpenAI as one early example of a well-developed, extremely influential, foundational model company struggling with this staffing challenge: They have a chief scientist (who is also their co-founder), a head of global policy, and a general counsel.

However, without the three positions I outline above in executive leadership positions, the biggest questions surrounding the repercussions of their technology remain unaddressed. If Sam Altman is concerned about approaching the treatment and coordination of superintelligence in an expansive, thoughtful way, building a holistic lineup is a good place to start.

We have to build a more responsible future where companies are trusted stewards of people’s data and where AI-driven innovation is synonymous with good. In the past, legal teams carried the water on issues like privacy, but the brightest among them recognize they can’t solve problems of ethical data use in the age of AI by themselves.

Bringing broad-minded, differing perspectives to the table where the decisions are made is the only way to achieve ethical data and AI in the service of human flourishing — while keeping the machines in their place.