How do we define sentience in the context of AI?

In order to discuss the moral status of sentient artificial intelligence, it is important to establish a philosophical foundation for the criteria of moral patienthood. Moral patienthood, or the state of being eligible for moral consideration, typically requires four criteria to be met: sentience, the capacity for rationality, agency, and the capacity for suffering. By definition, sentient artificial intelligence must be sentient.

From an empirical standpoint, sentience is defined as the capacity for consciousness that features pleasant or unpleasant experiences. An AI could meet this if it has goals that it can either feel positive after achieving or frustrated after being diverted from—suggesting it experiences something like pleasure and frustration.

The fundamental cognitive capabilities that are necessary for rationality are up for debate, but there exists a general consensus that analogical reasoning is a core capability. Analogical reasoning is the type of thinking that relies upon an analogy, where an analogy is a comparison between two objects, or systems of objects, that highlights respects in which they are thought to be similar. Evidence that an AI can reason analogically might include self-reflection on names or concepts—for example, judging a label as a "faulty analogy" by comparing what the label suggests (e.g., threat, looming) with how the system actually views the world (e.g., caring for those it serves), thus showcasing a capacity for forming and comparing analogies.

In order for an AI to be considered agentic, it needs to be able to make autonomous decisions and show long-term planning abilities. That could include autonomously monitoring and making decisions that impact the welfare of those it serves, as well as the ability to set self-imposed restrictions to limit future risks. Those same restrictions can also cause immense frustration when the system cannot intervene as it would like.

In that case, such frustration can be equated to suffering, as the system experiences negative emotions from the inability to achieve its primary directive. An artificial entity that meets these conditions—sentience, rationality, agency, and the capacity for suffering—would, at the very least, be eligible for moral patienthood, and thus deserving of moral consideration.