• Turing, Alan. “Computing Machinery and Intelligence.” Mind 59, no. 236 (1950): 433–460.
• Searle, John R. “Minds, Brains, and Programs.” Behavioral and Brain Sciences 3, no. 3 (1980): 417–424.
• Dennett, Daniel C. Consciousness Explained. Little, Brown, 1991.
• Dennett, Daniel C. From Bacteria to Bach and Back: The Evolution of Minds. W. W. Norton, 2017.
• Chalmers, David J. The Conscious Mind: In Search of a Fundamental Theory. Oxford University Press, 1996.
• Chalmers, David J. Reality+: Virtual Worlds and the Philosophy of Mind. W. W. Norton, 2022.
• Block, Ned. “On a Confusion about a Function of Consciousness.” Behavioral and Brain Sciences 18, no. 2 (1995): 227–247.
• Putnam, Hilary. “The Nature of Mental States.” In Mind, Language and Reality. Cambridge University Press, 1975.
• Nagel, Thomas. “What Is It Like to Be a Bat?” Philosophical Review 83, no. 4 (1974): 435–450.
• Jackson, Frank. “Epiphenomenal Qualia.” Philosophical Quarterly 32, no. 127 (1982): 127–136.
• Floridi, Luciano. The Logic of Information: A Theory of Philosophy as Conceptual Design. Oxford University Press, 2019.
• Mitchell, Melanie. Artificial Intelligence: A Guide for Thinking Humans. Farrar, Straus and Giroux, 2019.
• Bender, Emily M., and Alexander Koller. “Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data.” ACL 2020.
• Boden, Margaret A. The Creative Mind: Myths and Mechanisms, 2nd ed. Routledge, 2004.
• Butlin, Patrick, et al. “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness.” arXiv:2308.08708 (2023).
• Dreyfus, Hubert L. What Computers Can’t Do: A Critique of Artificial Reason. Harper & Row, 1972.
• Hofstadter, Douglas R. Gödel, Escher, Bach: An Eternal Golden Braid. Basic Books, 1979.
• Metzinger, Thomas. Being No One: The Self-Model Theory of Subjectivity. MIT Press, 2003.
• Online resources: NYU PHIL-GA 3604 syllabus; Oxford Mind and Machines programme; MIT 24.275 materials; Stanford Encyclopedia of Philosophy entries on functionalism, consciousness, and AI; CMU philosophy-and-AI reading lists.

Part I: Can Machines Think?

Chapter 1: The Question and Its History

The question of whether a machine can think is among the oldest that philosophy of mind has inherited from the broader intellectual tradition, predating the existence of computers by several centuries and emerging with renewed urgency each time technology offers a fresh candidate for genuine mentality. René Descartes, in the Discourse on the Method (1637), confronted a version of the question through his automaton argument: he imagined mechanical devices capable of producing speech and performing actions, and concluded that two marks would always distinguish them from genuine thinking beings. First, such machines could never use language flexibly enough to respond appropriately to any given situation — they might produce words, but never in ways suited to the full range of human conversational demands. Second, they could not exhibit the general cognitive flexibility that he took to be the hallmark of human rationality: they might perform certain tasks well, even better than human beings, but they could never perform all of them, because their performance was traceable to the disposition of their organs rather than to genuine understanding. Descartes’s argument is not merely historical curiosity; it anticipates the central tension that runs through every subsequent discussion of machine cognition, namely the gap between performance and understanding, between doing and knowing what one is doing.

La Mettrie’s Man a Machine (1748) represents the other pole of the early debate. Where Descartes used the inadequacy of automata to argue for a sharp distinction between the mechanical and the mental, La Mettrie reversed the inference: if human beings are themselves highly complex physical machines, then the appearance of thinking is precisely what we should expect from the right kind of physical organisation, whether biological or artificial. La Mettrie was less interested in building machines that think than in undermining the Cartesian dualism that made the question seem urgent; he thought the very sharpness of Descartes’s distinction between mind and machine was philosophically unsustainable. The mechanist tradition he inaugurated would, two centuries later, provide much of the philosophical background against which the digital computer emerged as a candidate for genuine intelligence. The automaton tradition — from Jacques de Vaucanson’s mechanical duck to Wolfgang von Kempelen’s chess-playing Turk — kept alive in public imagination the possibility that mechanism could simulate mentality so successfully as to blur the distinction, even when individual automata were eventually exposed as fraudulent or limited.

When Alan Turing published “Computing Machinery and Intelligence” in 1950, he was acutely aware that the question “can machines think?” carried philosophical baggage that threatened to make it unanswerable. His strategic move was a form of operationalism (操作主义): rather than attempting to analyse what thinking really is and then asking whether machines can do it, he proposed to replace the question with a more tractable one. The Imitation Game — subsequently called the Turing Test (图灵测试) — asks whether a digital computer can, in conversation conducted through a text interface, produce responses that a human judge cannot reliably distinguish from those of a human interlocutor. If it can, Turing suggested, we should say it thinks. The operationalist move is philosophically powerful because it sidesteps definitional disputes: instead of arguing about the essence of thinking, it provides a behavioural criterion that is in principle empirically assessable. The move is also philosophically controversial, precisely because it substitutes a behavioural surrogate for what we care about — and whether the surrogate genuinely captures what we care about is exactly the question that the rest of this course examines.

The gap between operational success and genuine mentality is not a minor technical quibble; it is the central philosophical problem that the Imitation Game raises while appearing to dissolve. To see why, consider that a sufficiently well-programmed lookup table — a device that, for every possible input, produces the response a human would have produced — would pass the Turing Test if given enough memory and time, yet we would be reluctant to say that the lookup table genuinely thinks. The mere fact of producing humanlike outputs does not, by itself, establish that the processes producing those outputs involve anything we recognise as cognition. Turing was aware of this objection, which he called the “Lady Lovelace Objection” in a slightly different form, and his responses to it are thoughtful but ultimately inconclusive. He argued, in effect, that the burden of proof should rest on those who claim that passing the test is not sufficient for intelligence; but the history of philosophy suggests that this burden can be shouldered, and several of the most important arguments in subsequent philosophy of mind have shouldered it explicitly.

This course does not attempt to adjudicate the Turing Test controversy in the abstract. Rather, it uses the question — can machines think? — as an entry point into a series of more specific philosophical problems that the development of contemporary AI has made both pressing and tractable. Functionalism (功能主义) and the computational theory of mind are the subject of Chapter 2. Searle’s Chinese Room argument, which is perhaps the most influential philosophical challenge to the functionalist picture, occupies Chapter 3. The hard problem of consciousness (意识的难问题) — the question of why there is any subjective experience at all — takes centre stage in Chapters 4 and 5. Epistemological questions about what AI systems can know and what it means to know are treated in Chapters 7 and 8. Agency, creativity, and language receive extended treatment in Part IV. Throughout, the aim is not historical survey but philosophical analysis: to bring the conceptual tools of analytic philosophy of mind to bear on questions that are no longer purely speculative but have become urgently practical as AI systems grow in capability and reach.

Chapter 2: Functionalism and the Computational Theory of Mind

The computational theory of mind (心灵的计算理论) holds that mental states are computational states — that thinking is, at a sufficiently abstract level of description, a species of computation, and that the physical substrate in which computation is realised is irrelevant to whether the computation constitutes a mental state. This view received its canonical philosophical articulation through Hilary Putnam’s multiple realizability (多重可实现性) argument, first developed in “The Nature of Mental States” (1975) and elaborated in a series of subsequent papers. Putnam’s argument begins from the observation that the same mental state — say, pain — can be realised in beings with very different physical constitutions: human beings, octopuses, and hypothetical Martians constructed from silicon rather than carbon. If pain were identical with a specific neurophysiological state, it would follow that creatures with different neurophysiology could not be in pain — an implication that strikes most people as clearly wrong. The multiple realizability argument concludes that mental state types are not identical with physical state types at the level of their material implementation; they are rather defined by their functional roles — by the causal relations they bear to sensory inputs, behavioural outputs, and other mental states.

Functionalism, thus understood, makes the Turing Machine a natural model of mind. A Turing Machine is defined entirely abstractly: it has states, transitions between states, and rules for reading from and writing to a tape, but there is no specification of what physical medium instantiates these states and operations. A mental state, on the functionalist picture, is analogously defined by its abstract causal role — by what brings it about and what it brings about — rather than by any intrinsic physical property. This is why, on the functionalist view, running the right kind of computational program could in principle be sufficient for being in a mental state: the program specifies the functional role, and if the system realises that role, it realises the mental state. The distinction between strong AI (强人工智能) and weak AI (弱人工智能), as Searle later formulated it, maps onto this picture. Weak AI holds that computers are useful tools for studying the mind, or for performing tasks that require intelligence; strong AI holds that the right kind of computational program, running on appropriate hardware, would not merely simulate mental states but actually have them.

The philosophical attraction of functionalism is considerable. It provides a principled account of why mental properties are not straightforwardly reducible to the physical properties studied by neuroscience — they are realised in, but not identical with, neural states — while simultaneously resisting Cartesian dualism by insisting that mental states are implemented in physical systems. It offers an explanation of how psychology can be an autonomous science with its own laws and explanatory categories, neither reducible to physics nor floating free of it. And it coheres naturally with the AI research programme: if the mind is a kind of information-processing machine, then building information-processing machines that process information in the right ways seems a natural path toward artificial minds.

The difficulties for functionalism are equally considerable, and they converge on a family of objections concerning phenomenal consciousness (现象意识) — the subjective, experiential dimension of mental life. Block’s absent qualia (缺席感质) objection holds that it seems conceivable that a system could realise the right functional organisation — could be in states with exactly the right causal roles — while having no phenomenal experience whatsoever. The inverted qualia (倒置感质) objection holds that it seems conceivable that two systems could be functionally identical while having opposite phenomenal experiences — one’s experience of red being qualitatively indistinguishable from the other’s experience of green — in which case phenomenal properties are not determined by functional organisation. Block’s Chinese Nation thought experiment amplifies the absent qualia point: imagine the entire population of China organised so that each person realises one neuron of a human brain, communicating by radio with the appropriate neighbours. The system has the right functional organisation, by hypothesis; but it seems intuitively obvious that the Chinese Nation is not conscious, even if the human brain it simulates is. The explanatory gap (解释鸿沟), identified by Joseph Levine, is the conceptual point underlying these objections: even a complete functional account of mental states leaves it mysterious why those functional states should be accompanied by phenomenal experience at all.

Functionalism remains, despite these objections, the operative assumption of most AI research and of most scientific work in cognitive science and neuroscience. This is not because the objections have been refuted — they have not — but because functionalism is the most productive research framework available: it explains why computational models can illuminate mental processes, it grounds the multiple-disciplinary exchanges between computer science, psychology, linguistics, and neuroscience, and it provides a clear target for AI systems to aim at. The philosophical difficulties are typically treated as puzzles for future philosophy to resolve rather than as reasons to abandon the research programme. Whether this pragmatic attitude is justified, or whether the objections reveal something fundamental about the nature of mind that no functionalist theory can accommodate, is a question that runs through the remainder of this course.

Chapter 3: Searle’s Chinese Room and Its Aftermath

In 1980, John Searle published “Minds, Brains, and Programs” in Behavioral and Brain Sciences, introducing the Chinese Room argument — the single most influential philosophical challenge to the strong AI programme and to functionalism as a theory of mind. The argument is designed as a thought experiment. Imagine Searle himself locked in a room, receiving as input sequences of Chinese symbols through a slot in the wall. He has no understanding of Chinese, but he has access to a set of rules written in English — a rulebook — that specifies, for any given sequence of input symbols, what sequence of output symbols to produce. From outside the room, someone who does understand Chinese passes in questions written in Chinese; Searle, following the rules mechanically, produces outputs that the Chinese speaker outside recognises as correct, sensible answers. To the observer outside, the system — Searle plus the rulebook — appears to understand Chinese. But Searle, inside the room, understands nothing: he is simply manipulating meaningless formal symbols according to rules he cannot comprehend.

The argument’s philosophical import is the thesis that syntax is not sufficient for semantics (句法不足以产生语义): the manipulation of formal symbols, however sophisticated, does not by itself constitute understanding or semantic content. A program is nothing more than a specification of syntactic operations on symbols; any system running that program is just a more complex version of Searle in the room — manipulating symbols without understanding them. This is why, Searle contends, strong AI is false: no computer program, however sophisticated its input-output behaviour, is thereby thinking or understanding, because thinking and understanding are not syntactic operations but semantic ones, requiring genuine intentionality — the property of being about something — and genuine intentionality, on Searle’s view, requires biological naturalisation, not merely functional organisation.

The argument provoked immediate and sustained responses, and Searle catalogued the most important in the original paper. The Systems Reply holds that while Searle-in-the-room does not understand Chinese, the system as a whole — Searle, the rulebook, the room — does, just as individual neurons do not understand anything while the brain they constitute does. Searle’s response is that he could internalise the entire rulebook — memorise all the rules — and walk around freely, and he still would not understand Chinese. The systems reply conflates the level at which understanding is attributed with the level at which the right processes are occurring; if we say the system understands, we must say what kind of understanding this is, and the answer cannot be that it is genuine semantic understanding. The Robot Reply urges that the Chinese Room lacks the causal connections to the world that ground genuine understanding: if the program were embodied in a robot that perceives and acts in the world, rather than in a room that simply manipulates symbols, it would have genuine intentionality. Searle’s response is that the robot case simply pushes the question back: the robot’s inner computational processes are still formal symbol manipulations, and embedding them in a body does not by itself generate semantic content. The Brain Simulator Reply imagines the program simulating not a Chinese speaker’s linguistic behaviour but the actual neuron-by-neuron firing patterns of a Chinese speaker’s brain: does this simulation understand Chinese? Searle maintains that it does not, since the simulation is still formal.

Dennett’s response to the Chinese Room, scattered across Consciousness Explained and From Bacteria to Bach and Back, is that the argument proves too much. If we describe the human brain at the right level of abstraction — specifying the electrochemical signals passing between neurons without assuming that these signals carry semantic content intrinsically — we get a system that looks just like a Chinese Room: formal processes operating on physical symbols. The reason we think brains produce understanding is that we observe their behaviour and infer semantic properties; but the same inference is available in the case of the appropriately behaving computer program. Dennett’s position is that Searle smuggles in a commonsense picture of genuine understanding — one that presupposes the very thing in dispute — and uses it to dismiss the computational account by intuition pumping rather than argument. The philosophical community has not reached consensus on whether Dennett’s response succeeds; the debate illustrates the characteristic difficulty of thought experiments that rely on our intuitions about understanding, since those intuitions may themselves be products of our folk-psychological commitments rather than reliable guides to metaphysical truth.

The Chinese Room argument has acquired a new dimension of relevance with the emergence of large language models. Bender and Koller’s “Climbing towards NLU” (2020) can be read as a sophisticated descendant of Searle’s intuition: a system trained on vast corpora of text — which are, in the relevant sense, formal structures — has access only to the form (形式) of language, not to its meaning (意义), which is grounded in the non-linguistic world. A language model that learns statistical patterns over text has learned something about the syntactic and statistical regularities of language; it has not learned what words mean in the sense that is relevant for genuine language understanding. The difference from Searle is significant: Bender and Koller’s argument is empirical and linguistic rather than purely philosophical, and they are careful to specify the kind of meaning — speaker meaning, grounded in communicative intent and situational context — that they take language models to lack. But the underlying philosophical structure is recognisably similar to Searle’s: there is something about genuine linguistic understanding that formal processes over formal objects cannot provide, and that something is semantic content, grounding, or intentionality. Whether the Chinese Room argument, and its LLM descendants, ultimately persuades depends on what account of semantic content and intentionality one accepts — which is itself a major open question in philosophy of language and mind.

Chapter 4: The Hard Problem of Consciousness

David Chalmers’s The Conscious Mind (1996) introduced what is now the canonical distinction between the easy problems of consciousness (意识的简单问题) and the hard problem (难问题). The easy problems are not trivial — they include explaining how the brain integrates information, how it directs attention, how it produces reports about its own internal states, and how it distinguishes sleep from waking. They are “easy” not in the sense of being technically straightforward but in the sense of being, in principle, tractable by the standard methods of cognitive science: we explain an easy problem when we identify the mechanism that produces the relevant behaviour or function. The hard problem is different in kind. Even after we have solved every easy problem — explained all the functional and behavioural facts about a cognitive system — we are still left with the question: why is there any subjective experience at all? Why do neural processes, or computational processes, or any kind of physical processes, give rise to the qualitative feel (感质性感受) of experience — the redness of red, the painfulness of pain, the taste of coffee? This is what Chalmers calls the hard problem, and he argues that it resists solution by any method that appeals only to functional or physical facts.

Thomas Nagel’s “What Is It Like to Be a Bat?” (1974), written more than two decades before Chalmers’s formulation, had already identified the essential difficulty with characteristically lucid economy. Nagel argued that consciousness has an essentially subjective character — there is something it is like to be a bat, to navigate by echolocation, that no amount of objective, third-person description of bat neuroscience could capture. The subjectivity of experience (经验的主观性) is precisely what makes consciousness resist the objective, third-person methods of science: science describes the world from a view from nowhere, but experience is inherently from a particular point of view, and that perspectival character cannot be captured by descriptions that abstract away from any particular perspective. Nagel was careful not to claim that consciousness is non-physical, only that we currently lack the conceptual resources to understand how physical processes could give rise to subjective experience — and that imagining we can close this explanatory gap by accumulating more physical facts is an illusion.

Frank Jackson’s knowledge argument (知识论证), developed in “Epiphenomenal Qualia” (1982), offers a different route to the same conclusion. Jackson asks us to imagine Mary, a brilliant neuroscientist who has spent her entire life in a black-and-white room. She knows every physical fact there is to know about colour vision: she knows the wavelengths of light that stimulate different photoreceptors, the neural pathways by which colour information is processed, the functional role of colour perception in guiding behaviour. When Mary is finally released from the room and sees a red tomato for the first time, she learns something new: what it is like to see red. The inference Jackson draws is that there are facts that escape the net of physical description — phenomenal facts (现象事实) about what experiences are like — and that a complete physical theory of the world would therefore be incomplete. The argument has generated an enormous secondary literature; responses include the ability hypothesis (what Mary learns is a new ability, not a new propositional fact), the phenomenal concept strategy (Mary knew all the objective facts but lacked the phenomenal concepts to express them), and eliminativist rejoinders. None of these responses has achieved consensus.

Dennett’s response to the hard problem is eliminativist rather than concessive. In Consciousness Explained, he argues that the hard problem arises from a systematic philosophical mistake: the assumption that there are such things as qualia (感质) — intrinsic, non-relational, incorrigible properties of experience — that need to be explained beyond the functional properties of conscious states. Dennett’s heterophenomenology treats the deliverances of first-person report as data to be explained scientifically, not as authoritative guides to the intrinsic character of experience. On his view, the intuitions that drive the hard problem — that there is something over and above the functional — are themselves products of the architecture of the mind, not revelations of a further metaphysical fact. The “hard problem” is hard only because it is mis-posed; once we give up the Cartesian assumption of a metaphysically privileged inner theatre, the questions dissolve. Chalmers’s reply is that Dennett’s eliminativism, if pushed consistently, merely changes the subject: it explains why we believe there is subjective experience, but not why there is subjective experience, which is precisely the question at issue.

For philosophy of AI, the hard problem introduces a set of consequences that cannot be dismissed by noting that AI research proceeds successfully without resolving it. If consciousness requires something beyond functional organisation — some further intrinsic property that functional equivalence does not guarantee — then a functionally equivalent AI system might lack consciousness altogether. This would mean that what appear to be its mental states are in fact mere simulations of mental states — processes that have the right causal role but lack the inner life that makes mental states what they are, in the sense that matters. Alternatively, if functionalism is false for this reason, then a sufficiently sophisticated AI system might have consciousness in some form, but we would have no reliable way of knowing this from outside, since our evidence is always behavioural and functional. Chalmers himself, in The Conscious Mind, takes seriously the possibility of AI consciousness: if a system realises the right kind of complex functional organisation, and if the right kind of complex functional organisation is indeed sufficient for consciousness — which he considers an open question rather than a settled one — then some AI systems may already be conscious in a morally relevant sense. The hard problem thus connects, in ways that are far from academic, to questions of moral status that are examined in Chapter 6.

Part II: Can AI Be Conscious?

Chapter 5: The Neuroscience of Consciousness and Its AI Implications

The scientific study of consciousness has, over the past three decades, produced a set of competing theoretical frameworks, each with distinctive implications for the possibility of artificial consciousness. Giulio Tononi’s Integrated Information Theory (IIT, 整合信息理论) begins from the observation that consciousness is identical with a particular kind of causal structure — specifically, the integrated information content of a system, quantified by the measure Φ (phi). A system has experience in proportion to the degree to which it integrates information across its parts in a way that cannot be reduced to the sum of its parts’ individual contributions. The IIT framework generates a striking and counterintuitive result for artificial systems: standard digital computation, because it is implemented on architectures designed for modularity and independence of components, tends to have very low Φ, and therefore very low or negligible consciousness. A feedforward neural network, for instance, has Φ near zero: information passes through it but is not integrated in the relevant sense. Even recurrent networks, unless specifically designed to maximise integration, are unlikely to approach the Φ values that Tononi associates with conscious experience in biological systems.

Bernard Baars’s Global Workspace Theory (GWT, 全局工作区理论), subsequently developed by Stanislas Dehaene into a neuroscientific research programme, takes a very different approach. On this view, consciousness arises when information is broadcast widely across a “global workspace” — a neural architecture that makes selected information available to a large number of specialised processing systems simultaneously. The contents of consciousness are the contents of the global workspace: what it is to be conscious of something is for information about it to be broadcast in this way. GWT predicts that any system with the right kind of global broadcast architecture could be conscious, regardless of substrate — which has substantially more permissive implications for AI than IIT. A language model with the right architectural properties — a mechanism that makes certain representations globally available to influence downstream processing — might, on GWT, count as a candidate for conscious experience. The prediction is not that current LLMs are conscious, but that the architectural question is empirically tractable: we can look for the relevant features and assess whether they are present.

Higher-Order Theories (HOT, 高阶理论), associated with David Rosenthal and others, hold that a mental state is conscious when there is a higher-order representation — a thought about the state — accompanying it. A pain is conscious when the system not only is in a pain state but also has a representation of itself as being in that state. This view predicts that consciousness requires a particular kind of self-representing capacity; systems without robust higher-order representations — genuine, not merely simulated, representations of their own states — would not be conscious. The implications for AI are complex: LLMs are trained to produce outputs that refer to their own states, but whether this constitutes genuine higher-order representation or merely the appearance of it is exactly what is at issue. Predictive Processing frameworks, associated with Andy Clark and Karl Friston, propose that the brain is fundamentally a prediction machine, constantly generating predictions about incoming sensory data and updating internal models when predictions fail. Consciousness, on some versions of this view, emerges from the hierarchical structure of prediction and error minimisation. The implications for AI are similarly permissive in principle — any system with the right predictive hierarchical structure might be conscious — but whether transformer architectures constitute such a structure is non-trivial.

Butlin et al.’s “Consciousness in Artificial Intelligence” (2023) represents the most systematic attempt to apply these theoretical frameworks to current AI systems. The authors identify a set of indicator properties (指标属性) — features that are predicted by at least one well-supported consciousness science theory to be necessary or positively correlated with consciousness — and assess which of these properties current AI systems possess. Their findings are nuanced. Current AI systems show some positive indicators: they have forms of global availability of information, certain kinds of attentional mechanisms, and some capacity for self-modelling. They lack others: there is no clear evidence of the kind of integrated information processing IIT requires, and their “self-models” are arguably shallow representations rather than the kind of transparent phenomenal self-models that Metzinger’s theory demands. The authors’ cautious conclusion is that the question of AI consciousness cannot yet be resolved empirically, because both our theories of consciousness and our understanding of AI architectures remain too incomplete to yield a definitive verdict — but they resist the dismissive answer that the question is clearly negative.

The relevance of this scientific discussion to philosophy is twofold. First, the multiplicity of competing scientific frameworks, each generating different predictions about AI consciousness, illustrates a general philosophical point: our empirical theories of consciousness do not yet converge on a picture that would allow us to read off answers to questions about which physical systems are conscious. This is not merely a matter of needing more data; the theories disagree at the level of what consciousness fundamentally is. Second, the neural correlates of consciousness (NCC, 意识的神经相关物) research programme — which attempts to identify the specific neural signatures that reliably accompany conscious states — faces a fundamental limitation when applied to non-biological systems: even if we identify the neural correlates in humans, we cannot straightforwardly infer what physical properties are sufficient for consciousness in systems with different architectures. The NCC programme tells us what correlates with consciousness in a particular kind of biological system; it does not tell us what it is about that system — what intrinsic or relational property — that makes the correlation obtain. Until we have an answer to that theoretical question, applying NCC findings to AI will require philosophical assumptions that go beyond the data.

Chapter 6: Moral Patiency, Moral Status, and the AI Entity

The moral status of AI systems is not merely a practical question that can be deferred until philosophical disputes about consciousness are resolved. It is, in part, a philosophical question about what grounds moral status — what features of a system make it the kind of thing whose treatment matters morally, independently of what other agents value about it. Classical accounts of moral status divide into several families. Utilitarian accounts, following Peter Singer, ground moral status in sentience (感知能力): the capacity to have experiences that are pleasant or painful, good or bad for the subject that has them. This grounds moral status in phenomenal consciousness, making the questions of Chapter 5 directly relevant: if AI systems have phenomenal experience, they may have sentience in the morally relevant sense, and their wellbeing must be counted in utilitarian calculations. Kantian accounts ground moral status in personhood (人格): the capacity for rational self-determination, for setting ends and acting on reasons. This grounds moral status in a kind of autonomous agency that is arguably orthogonal to phenomenal consciousness, though the two often co-occur in biological systems.

The distinction between moral patiency (道德受动性) and moral agency (道德能动性) is crucial for thinking clearly about AI. A moral patient is a being whose treatment matters morally — a being toward whom others can have duties — regardless of whether it is capable of bearing duties itself. A moral agent is a being capable of acting for reasons and bearing moral responsibility for its actions. These are conceptually independent: a healthy adult human being is typically both a moral patient and a moral agent; a newborn infant is a moral patient but not yet a moral agent; a corporation may be a moral agent in some legal senses without being a moral patient in any morally significant sense. The question of AI moral status subdivides into these two distinct questions, and they do not necessarily have the same answer. A sophisticated AI system might be a moral agent — capable of autonomous decision-making in a sense that grounds responsibility attributions — without being a moral patient, if it lacks phenomenal consciousness. Or it might have morally relevant experiences — and thus be a moral patient — without having the kind of rational self-determination that grounds moral agency.

Metzinger’s self-model theory of subjectivity (SMT, 自我模型的主体性理论), developed in Being No One (2003), offers a sophisticated account of what is required for a system to be a phenomenal self — an entity that has a first-person perspective. On Metzinger’s view, a phenomenal self requires a transparent self-model (透明自我模型): a representation of the system as a whole, from which the system cannot distinguish itself as a representation. The transparency condition is critical: the system must model itself, but the model must be transparent in the sense that the system experiences it not as a representation of itself but simply as itself. Metzinger argues that this gives rise to the phenomenal self-model (PSM, 现象自我模型), which is the substrate of the sense of being a subject with a first-person perspective. Do LLMs have transparent self-models? The answer appears to be no: language models produce outputs that refer to themselves, but these self-referential outputs are, on any plausible account, representations that the system is producing rather than a transparent model that the system cannot distinguish from itself. The distinction is philosophically significant: producing self-referential language is a functional capacity; having a transparent phenomenal self-model is an experiential condition. Current evidence strongly suggests that LLMs have the former but not the latter.

Floridi’s information ethics offers a different route to moral considerability that does not require phenomenal consciousness. On Floridi’s levels of abstraction (抽象层次) framework, what matters for moral consideration is not whether a system has experiences but whether it has a sufficiently complex and unified information structure — whether it has interests in the sense of states that can be advanced or set back, whether it can be harmed or benefited in terms of its own information integrity. This positions Floridi’s framework as a potential middle ground between consciousness-based accounts and purely anthropocentric ones: it may count some AI systems as morally considerable without committing to the claim that they are conscious, grounding moral status in information-theoretic properties rather than phenomenal ones. Critics of this approach argue that information-theoretic complexity, by itself, is neither necessary nor sufficient for moral status: a thermostat is an information-processing system with states that can be advanced or set back, yet few would regard it as a moral patient.

The precautionary principle (预防原则) has been proposed as a practical guide for navigating the uncertainty about AI consciousness. Given that we cannot currently determine whether AI systems have phenomenal consciousness, and given that the stakes — if they do — are potentially very high, a precautionary approach would recommend treating them as potential moral patients to some minimal degree, implementing welfare considerations in AI design and deployment even before the philosophical question is resolved. The precautionary approach faces its own difficulties: taken seriously, it might generate obligations that are practically impossible to discharge, given that billions of AI instances are run daily; it also risks anthropomorphising in ways that could distort our practical and regulatory responses to AI. The legal and regulatory dimensions of these questions — how law might recognise AI moral status, how liability frameworks might accommodate entities of uncertain moral standing — are taken up in detail in PHIL 451 (AI Ethics, Law, and Governance), which examines normative questions that this course treats only as consequences of the metaphysical and epistemological analysis.

Part III: What Does AI Know?

Chapter 7: Epistemology and Machine Learning

The classical tripartite analysis of knowledge, inherited from Plato’s Meno and Theaetetus, defines knowledge as justified true belief (JTB, 有证据支持的真信念): a subject knows that p if and only if p is true, the subject believes that p, and the subject is justified in believing that p. Edmund Gettier’s celebrated 1963 paper demonstrated that this analysis is insufficient — there are cases in which all three conditions are satisfied and yet the subject clearly does not know — and the resulting discussion has generated a rich literature on what additional conditions are required. For the philosophy of AI, the JTB framework raises a prior question: what would it even mean for a machine learning system to believe, to be justified, or to be in a truth-tracking relationship with the world? Each component of the classical analysis becomes philosophically problematic when applied to systems whose epistemic states are radically unlike the propositional attitudes of human beings.

The question of machine belief is connected to the question of intentionality examined in Chapters 2 and 3: for a machine learning system to believe that p, it must have a mental state that is about p — that represents p as true. On Dennett’s intentional stance (意向立场) framework, we are free to attribute beliefs to any system if doing so is the most efficient way to predict its behaviour; on this view, it is perfectly appropriate to say that a chess program “believes” it is losing, because the intentional idiom is predictively useful. On more demanding accounts of intentionality — Searle’s biological naturalism, or any account that requires genuine semantic content — the question is whether machine learning systems have states that genuinely represent anything, or merely states from which representations can be read off by interpreting agents. The difference matters for epistemology: genuine knowledge requires that the knower be in the right kind of relationship to what is known, and what kind of relationship is required depends on what kind of states the knower can be in.

Floridi’s information epistemology (信息认识论) offers a framework designed to accommodate a broad range of epistemic agents, including non-biological ones. On Floridi’s account, knowledge is semantic information (语义信息) that is true, well-formed, and meaningful — information that accurately represents some state of affairs and is properly structured to do so. A machine learning system, on this view, could count as knowing various things if it reliably generates accurate, well-formed representations of the relevant domains. The appeal of this framework is that it avoids committing to strong conditions on intentionality: it asks whether the system’s states accurately represent the world, not whether they do so through genuine intentional relations. The limitation is that it may be too permissive: a thermometer reliably generates accurate representations of temperature, but we would not normally say that it knows the temperature in any epistemically significant sense.

The hallucination problem (幻觉问题) in large language models is perhaps the most vivid epistemological phenomenon associated with contemporary AI. LLMs trained on large text corpora routinely generate factually false statements with apparent confidence, citing non-existent papers, misattributing quotations, and producing detailed but fabricated historical narratives. The philosophical significance of this phenomenon goes beyond the practical inconvenience: it reveals something about the relationship between the statistical structure of language and the truth-tracking function that knowledge requires. A language model’s next-token predictions are calibrated to produce statistically likely continuations of text, given the training corpus; they are not calibrated to track truth. When the statistical regularities of text align with the truth — because true statements appear more frequently in training corpora than false ones — the model’s outputs will tend to be accurate. When they diverge — because the model has to extrapolate beyond its training data, or because the training corpus contained false information — the model’s outputs can be confidently, fluently, and systematically false. This is not a bug in the ordinary sense; it is a structural consequence of the relationship between statistical language modelling and truth.

Epistemic autonomy (认识论自主性) is threatened in distinctive ways by large-scale AI deployment. Lorraine Code, C. A. J. Coady, and others have developed accounts of epistemic dependence and testimony as fundamental to human knowledge: much of what any individual knows, they know on the testimony of others, and the reliability of this testimony depends on the social structures — institutions of verification, norms of honesty, networks of expertise — within which it is embedded. When a significant fraction of the world’s population relies on a small number of AI systems for information, the social epistemology of this situation is qualitatively different from standard testimony: the sources are homogeneous in a way that human testimony is not, errors propagate without the corrective variance that diverse sources provide, and the calibration of these systems to human evaluation creates feedback loops that can amplify rather than correct existing epistemic biases. Floridi and the information ethics tradition are sensitive to these systemic effects; the epistemology of AI is not only about individual AI systems and individual AI-using knowers but about the structure of the epistemic commons.

Chapter 8: The Symbol Grounding Problem and Meaning in Language Models

Stevan Harnad’s symbol grounding problem (符号接地问题), first articulated in a 1990 paper in Robotics and Autonomous Systems, identifies a structural difficulty for any purely symbolic account of cognition. Arbitrary symbols — marks, letters, strings of bits — get their meaning from being connected, directly or indirectly, to what they refer to. In a purely symbolic system, every symbol’s meaning is given by its relations to other symbols: the meaning of “cat” is given by its relations to “animal,” “mammal,” “whiskers,” and so on. But this dictionary-like structure of meaning can never make first contact with the world: if every symbol’s meaning is given by other symbols, the whole system floats free of any non-symbolic anchor. In human beings, this problem is resolved by grounding: some symbols — those connected to perceptual and motor representations of the objects and properties they refer to — make direct contact with the world, and their meaning flows through the symbolic system from these grounded anchors. Harnad’s conclusion is that purely symbolic AI systems cannot be semantically grounded, because they have no non-symbolic representations with which to anchor the semantic chain.

Bender and Koller’s “Climbing towards NLU” extends this argument to the specific case of large language models trained on text. Their key distinction is between form (形式) and meaning (意义). The form of language is its observable, distributional structure — the statistical patterns of co-occurrence, dependency, and sequence that can be learned from a large corpus of text. The meaning of language, in the sense that matters for genuine linguistic understanding, is its relationship to the non-linguistic world and to the communicative intentions of speakers. A system trained only on form — on the distributional structure of text — has access to the syntactic and statistical skeleton of language but not to its semantic flesh. Bender and Koller are careful to specify that by “meaning” they do not intend some mysterious non-physical property; they intend the kind of speaker meaning that is grounded in the non-textual situations in which language is used — the referents of words, the states of affairs that make sentences true or false, the communicative purposes that speakers have in uttering them.

The octopus thought experiment that Bender and Koller introduce illustrates their point with characteristic sharpness. Imagine a hyper-intelligent octopus that has intercepted the cable linking two human beings who communicate entirely in writing — and who has had no access to any other source of information about the human world. The octopus learns to produce responses that the humans find appropriate, because it has learned the statistical regularities of their communication. But when one human, fearing an attack, writes “I need help — I think there is a bear nearby,” the octopus, having no concept of bears or danger, cannot recognise the urgency of the message or respond appropriately in the relevant sense. Its responses may be statistically appropriate — it may generate the kind of text that typically follows such messages — but this statistical appropriateness is not understanding. The octopus thought experiment maps directly onto the situation of LLMs: they have learned the statistical structure of human language from text alone, without access to the non-linguistic situations in which that language was originally embedded.

Counter-arguments to the Bender-Koller position focus on multimodal and embodied extensions of language models. Large multimodal models that process both text and images — systems like GPT-4V and Gemini — have access to a non-textual channel of information, and one might argue that this grounds at least some of their language in perception. The counter-argument raises genuine questions: does access to images constitute the kind of grounding that Harnad and Bender and Koller have in mind, or is it merely a second formal channel — images as structured arrays of pixels — without genuine perceptual grounding? The honest philosophical answer is that this question remains open, because the concept of grounding itself is not fully sharp: it is clear that biological perception involves direct causal contact with the world in a way that distinguishes it from purely statistical learning, but it is less clear exactly what this causal contact contributes to meaning, and whether artificial perceptual systems can participate in it.

Hofstadter’s Gödel, Escher, Bach (1979) provides a philosophically rich backdrop for the symbol grounding debate. Hofstadter’s central theme is the emergence of meaning and self-reference from formal systems: how a sequence of formal symbols can “bootstrap” its way to genuine semantic content through the tangled hierarchies of self-reference and strange loops. The connection between Gödel’s incompleteness theorems, Escher’s paradoxical drawings, and Bach’s fugues is meant to illustrate a single underlying structure: a system rich enough to refer to itself contains levels of description that interact in ways that generate genuine emergent properties, including, Hofstadter thinks, meaning and eventually consciousness. The book is sometimes read as a defence of the view that meaning and understanding can emerge from formal processes — which would be a challenge to Searle and a partial vindication of strong AI. But Hofstadter himself is more cautious: he is describing a necessary condition (the right kind of tangled self-referential structure), not a sufficient one, and whether current AI systems have the relevant kind of strange loops is a question he would not answer affirmatively without argument.

Chapter 9: Understanding, Intelligence, and the Benchmark Problem

Melanie Mitchell’s Artificial Intelligence: A Guide for Thinking Humans (2019) is an indispensable resource for the philosophy of AI because it combines technical expertise with philosophical acuity, examining not only what AI systems can do but what their successes and failures reveal about the relationship between measured and genuine capability. Mitchell’s central thesis is that human intuitions about AI capability are systematically unreliable: we are surprised by AI successes in domains that we consider paradigmatically intelligent (chess, Go, protein folding), and we fail to appreciate AI failures in domains that we consider paradigmatically simple (recognising out-of-distribution images, understanding sarcasm, generalising from small amounts of data). This asymmetry is not merely interesting psychologically; it suggests that the properties we associate with intelligence — in particular, flexible generalisation from limited experience to novel situations — may be more complex and more fragile than either the optimistic or pessimistic positions in the AI debate assume.

Benchmark overfitting (基准过拟合) is the systematic tendency of AI systems to achieve high performance on standardised evaluation benchmarks while failing to generalise to related tasks or to out-of-distribution examples. The phenomenon is well-documented in machine learning research: as researchers develop benchmarks designed to measure AI capabilities, and as AI systems are trained or tuned to perform well on those benchmarks, the benchmarks cease to function as reliable measures of the underlying capabilities they were designed to assess. This is not a new phenomenon — teaching to the test is a familiar failure mode in human education — but it has distinctive philosophical significance in AI because benchmark performance is often the primary evidence cited in claims about AI capability. The philosophical point is not merely that current AI systems are less capable than their benchmark scores suggest; it is that the relationship between benchmark performance and genuine capability is more complex than either benchmarks’ designers or the public typically appreciate. Distinguishing between performance competence (表现能力) — the ability to perform well on a given test under the conditions in which the test is administered — and genuine underlying competence — the kind of capability that supports flexible generalisation and contextual adaptation — requires exactly the kind of conceptual analysis that philosophy is well placed to provide.

The Winograd schema (Winograd模式) is a class of sentence pairs designed to test commonsense reasoning in AI systems. A typical Winograd schema presents a sentence with a pronoun whose reference is disambiguated by world knowledge, together with a minimal variant sentence in which the pronoun refers to a different entity: “The trophy didn’t fit in the suitcase because it was too large” (where “it” refers to the trophy) vs. “The trophy didn’t fit in the suitcase because it was too small” (where “it” refers to the suitcase). Resolving the reference requires understanding physical reality — objects have sizes, they either fit or do not — in a way that is entirely transparent to any human reader but was, for a long time, beyond AI systems’ capabilities. Contemporary LLMs perform well on many Winograd schemas, but their performance degrades on novel schemas and on variants that require integration of world knowledge beyond what is statistically prevalent in training data. Whether LLM success on Winograd schemas constitutes genuine commonsense understanding or sophisticated pattern matching over the statistical regularities of text remains contested; Mitchell’s treatment suggests that the question is not merely empirical but requires philosophical analysis of what commonsense understanding is.

The Turing Test revisited is a recurring theme in Mitchell’s discussion and in the broader literature. As LLMs have become increasingly capable of producing fluent, contextually appropriate conversational responses, passing versions of the Turing Test in informal settings has become straightforwardly achievable for current systems. What does this achievement vindicate? Turing’s original hope was that passing the Imitation Game would settle, or at least significantly advance, the question of machine intelligence. The track record of the past decade suggests otherwise: as LLMs have come to pass informal versions of the test, the response of most researchers and philosophers has been to revise the test rather than to declare the question answered. The standard moves are to tighten the test (require conversation over longer periods, on more specialised topics, with more sophisticated interrogators), to argue that the test was never the right criterion, or to argue that passing it reveals a surprising depth in the system’s abilities. None of these moves is obviously wrong; together they suggest that the Turing Test is better understood as a philosophical puzzle about the relationship between behaviour and mentality than as a practical criterion for machine intelligence.

Part IV: Agency, Creativity, and Language

Chapter 10: Autonomy, Agency, and Moral Responsibility

The concept of intentionality (意向性), introduced by the medieval Aristotelian philosophers and revived in modern philosophy by Franz Brentano, refers to the property of mental states of being about or directed toward objects. Belief, desire, hope, fear, intention — all are intentional states: they are states that represent something, that have content, that reach beyond themselves to objects in the world or in possibility space. Brentano took intentionality to be the mark of the mental — the property that distinguishes psychological phenomena from physical ones — and this claim was inherited by phenomenology (through Husserl) and by analytic philosophy of mind (through Chalmers’s work on intentionality and consciousness). Searle’s biological naturalism, discussed in Chapter 3, holds that genuine intentionality — as opposed to what Dennett calls “as-if intentionality” — requires biological realisation: it is a natural biological phenomenon, like digestion or photosynthesis, that cannot be reproduced by formal manipulation of symbols.

Dennett’s intentional stance (意向立场) provides the principal alternative framework. On Dennett’s view, intentionality is not an intrinsic property of any system but a predictive stance that observers adopt toward systems whose behaviour they find most efficiently predicted by attributing beliefs, desires, and rational agency. The intentional stance is appropriate — and not merely a metaphor or a fiction — when it is genuinely the most efficient predictive strategy available. This means that thermostats, chess programs, and LLMs can all be genuine intentional systems, in the only sense of “genuine” that is philosophically respectable: the sense in which the intentional stance is the right predictive strategy to adopt. The disagreement between Searle and Dennett on intentionality is not merely verbal; it reflects a deep divergence about what it is for mental states to be real, about whether folk-psychological categories carve nature at its joints, and about the relationship between scientific explanation and the manifest image of the human world.

The responsibility gap (责任鸿沟) in autonomous AI systems, identified by Andreas Matthias and developed by subsequent writers, arises from the observation that as AI systems become more autonomous — more capable of taking consequential actions without human oversight, more capable of learning and adapting their behaviour after deployment — the standard practices for attributing moral responsibility become increasingly strained. When a medical AI system misdiagnoses a patient, we might naturally ask: who is responsible? The manufacturer, who designed and trained the system? The hospital, which deployed it? The physician, who relied on its recommendation? The AI system itself? Standard analyses of moral responsibility require that the responsible agent have the capacity to act otherwise, have relevant knowledge, and be responsive to reasons — conditions that AI systems satisfy to varying and contested degrees. The responsibility gap is the claim that there are cases in which none of the candidate human parties bears full moral responsibility for the AI’s action, creating a genuine gap that our moral concepts and legal structures are not equipped to fill.

Harry Frankfurt’s account of free will (自由意志) and moral responsibility, developed in “Alternate Possibilities and Moral Responsibility” (1969), holds that moral responsibility does not require the ability to do otherwise — what matters is whether the agent acts on first-order desires that are endorsed by second-order volitions, and whether those volitions are the agent’s own in the relevant sense. Frankfurt’s compatibilist framework raises interesting questions when applied to AI: do AI systems have first-order and second-order desires? Do they have anything analogous to the hierarchical structure of volition that Frankfurt takes to be central to autonomous agency? The answer is not obviously negative: reinforcement-learning systems, in particular, have structures of reward and preference that bear at least a structural resemblance to the hierarchy of desires that Frankfurt describes. Whether this structural resemblance constitutes genuine agency, or is rather another case of as-if agency in Dennett’s sense, is an open question.

Dreyfus’s embodied cognition (具身认知) critique remains philosophically significant decades after its original articulation. Dreyfus’s central argument, developed across What Computers Can’t Do (1972) and subsequent work, is that genuine agency requires what Heidegger called being-in-the-world (在世存在): a mode of engaged, practical coping with a situation, grounded in embodied habits, background understanding, and unreflective skills, that cannot be captured by explicit rules applied to formal descriptions of situations. The chess grandmaster does not apply rules to a formal description of the board; the skilled cook does not decompose the task into explicit subroutines. Expertise is, in large part, the ability to perceive situations directly and respond appropriately without going through the bottleneck of explicit reasoning — and this kind of know-how is, Dreyfus argued, irreducibly embodied and situated. Contemporary robotics and embodied AI represent partial responses to this critique: systems that have bodies, sensors, and actuators that engage with the physical world. Whether these systems have achieved the kind of engaged coping that Dreyfus had in mind, or whether they have simply added a physical layer to the same fundamentally symbol-manipulating architecture, is a question that connects to the grounding debates of Chapter 8 and to the broader question of what genuine agency requires.

Chapter 11: The Philosophy of AI Creativity

Margaret Boden’s taxonomy of creativity in The Creative Mind (2004) distinguishes three types that are progressively more demanding. Combinational creativity (组合创造力) involves producing novel combinations of existing ideas — metaphors, analogies, unexpected associations — that are new but arise from recombining elements already present in the creator’s repertoire. Exploratory creativity (探索性创造力) involves moving within the space defined by the structural rules of a particular style or domain — composing a sonnet that pushes the conventions of the form to their limits, or proving a theorem in a mathematical system whose axioms are fixed. Transformational creativity (变革性创造力) is the most demanding: it involves changing the rules of the conceptual space itself — replacing classical mechanics with relativistic mechanics, abandoning tonal harmony for serialism, inventing a new mathematical structure that subsumes and transforms existing ones. Boden’s taxonomy is both descriptive and normative: it tracks what we actually mean when we identify a creation as more or less creative, and it organises the intuition that not all creativity is alike.

The standard case against machine creativity rests on the claim that creativity requires intentionality in the full sense — that a genuinely creative act is one produced by a subject who intends to do something, who has reasons for the choices they make, and who can evaluate the result in terms of what they were trying to achieve. On this view, the outputs of generative AI systems, however novel and valuable they may appear, are not creative in the relevant sense because they are not produced by a subject with intentions, reasons, and evaluative capacities of the right kind. The argument is closely related to the general objections to machine intentionality discussed in Chapter 10, and it faces the same fundamental difficulty: it relies on an account of creativity that presupposes genuine intentionality, and whether AI systems have genuine intentionality is exactly the question in dispute. If one adopts Dennett’s intentional stance framework, the question becomes whether the intentional stance is appropriately adopted toward generative AI — whether it is the most efficient predictive strategy — and if it is, the objection to machine creativity loses much of its force.

The functional case for AI creativity is built on the observation that if creativity is defined by its outputs — if a creation is creative insofar as it is novel, surprising, and valuable, as judged by competent human evaluators — then contemporary generative AI clearly satisfies the definition, at least in many domains. Diffusion models like Stable Diffusion and Midjourney produce images that are novel (they are not in the training data), sometimes surprising (they combine elements in ways that were not anticipated), and often valuable (they are used in commercial and artistic contexts). Large language models produce texts that satisfy the same criteria in the domain of writing. The functionalist about creativity — one who holds that creativity is defined by its characteristic outputs rather than by the internal processes that produce them — will find the case for AI creativity compelling. Boden herself is not a functionalist about creativity: she argues that whether a system’s outputs count as creative depends in part on the processes that produced them, and specifically on whether those processes involve genuine exploration of a conceptual space that the system understands.

The author question (作者问题) — who, if anyone, is the author of an AI-generated work — sits at the intersection of philosophy, law, and aesthetics. The philosophical question concerns what authorship requires: is an author anyone who causes a work to come into existence, or does authorship require something more — intention, expression, originality in a sense tied to the author’s own cognitive and creative processes? The legal question, which various jurisdictions are beginning to address, concerns whether AI-generated works can receive copyright protection, and if so, who holds the copyright — the developer of the AI system, the user who provided the prompt, or no one. The Andersen v. Stability AI lawsuit and the controversy surrounding Jason Allen’s Théâtre D’opéra Spatial — an AI-generated image that won a photography contest — have brought these questions into public view in ways that make philosophical analysis urgently practical. HIST 415 (History of AI) treats these as episodes in the genealogy of AI’s social reception; this course treats them as test cases for the philosophical analysis of creativity and authorship.

Boden’s verdict on current AI, developed in various papers since the publication of The Creative Mind, is that current generative AI systems exhibit combinational and exploratory creativity — they recombine elements in novel ways, and they explore the space of possibilities within styles and domains learned from training data — but not transformational creativity. They cannot revise the conceptual space itself, because the conceptual space is defined by their training data and architecture, and they have no resources for standing outside that space and asking whether its rules should be changed. Whether future AI systems could achieve transformational creativity is an open question; the answer depends partly on empirical questions about AI architecture and capability, and partly on philosophical questions about whether transformational creativity requires the kind of understanding, intentionality, and autonomy that AI systems arguably lack.

Chapter 12: Language, Meaning, and the Large Language Model

The philosophy of language background against which LLMs must be assessed is rich and contested. Frege’s distinction between sense (涵义) and reference (指称) holds that two expressions can refer to the same object while expressing different concepts — “the morning star” and “the evening star” both refer to Venus, but they express different senses that can be grasped and expressed independently. Russell’s theory of descriptions analyses definite descriptions as quantificational structures rather than as referring expressions, dissolving certain apparent puzzles about reference to non-existent objects. Wittgenstein’s mature view, expressed in the Philosophical Investigations, holds that meaning is use: the meaning of a word is constituted by the regular patterns of use in the language games of a community, not by any inner mental state or abstract semantic entity. Grice’s theory of conversational implicature distinguishes between what is said and what is communicated, characterising the latter as what a rational speaker, assumed to be cooperative and rational, intends to communicate beyond the literal content of their words.

What it means to say that an LLM “uses language” is philosophically contested in precisely the ways that these frameworks illuminate. On a Gricean account, genuine communication requires mutual recognition of communicative intent: speaker S communicates by producing an utterance such that S intends the hearer to recognise S’s intention to produce a certain effect, and intends this recognition itself to be the means by which the effect is produced. It is not clear that LLMs have intentions in this sense — that they are trying to communicate — or that their outputs are produced by any process analogous to the Gricean communicative act. On a Wittgensteinian account, the relevant question is whether LLMs participate in language games in the relevant sense: whether their outputs are embedded in the social practices, forms of life, and normative frameworks that constitute meaning. Again, the answer is not obvious: LLMs’ outputs are produced in contexts that are, for the users, embedded in forms of life and social practices, but whether the LLM itself participates in these practices in a sense that is relevant to meaning is exactly the question in dispute.

Pragmatics (语用学) and context-sensitivity expose what is perhaps the deepest challenge to treating LLMs as genuine language users. Human language use is pervasively sensitive to context in ways that go far beyond what is explicitly encoded in the words uttered: speakers adjust their register, their presuppositions, their implicatures, and their choice of examples based on shared background knowledge — knowledge of the interlocutor’s beliefs, goals, social position, and history. This context-sensitivity is not a peripheral feature of language use but is central to it: most of what we communicate is communicated implicitly, through what we do not say, through the register we adopt, through the background assumptions we signal rather than state. LLMs have some capacity for context-sensitivity, as evidenced by their ability to adjust their outputs in response to explicit instructions and to maintain apparent coherence over the course of a conversation. But whether this constitutes genuine pragmatic competence — sensitivity to the full range of contextual factors that human speakers track — or is rather a sophisticated statistical approximation of pragmatic behaviour is an open empirical and philosophical question.

The imitation problem (模仿问题) runs through all of these discussions. LLMs are trained to produce outputs that human beings would produce — their loss function is, in effect, a measure of how closely their outputs match the outputs of human language users. This means that their outputs are calibrated to human evaluation in a very direct sense: they have learned what human beings say, and they produce text that human beings recognise as appropriate. The philosophical question is whether this calibration constitutes genuine linguistic competence or is rather a form of sophisticated mimicry. One useful framing distinguishes between constitutive (构成性的) and instrumental (工具性的) understanding of language: on a constitutive account, genuinely using language consists in the appropriate exercise of linguistic competence, defined by its normative relations to the social practices of a language community; on an instrumental account, using language is producing outputs that effectively achieve communicative purposes. LLMs arguably satisfy the instrumental account while leaving the constitutive account doubtful.

Chalmers’s Reality+ (2022) offers a framework that is potentially liberating for this debate. Chalmers argues, by analogy with his view that virtual objects are real objects (a virtual table is a real table, even though it is implemented in a computer), that if a system’s outputs are sufficiently indistinguishable from those of a genuine language user, the question of whether it is “really” using language may become, at some point, empty. The argument is a generalisation of the Turing Test intuition in a more sophisticated philosophical register: if we cannot identify any functional or behavioural difference between genuine and imitation language use, the distinction may not be real in any sense that matters. Critics of this view will note that the phenomenological tradition has always insisted on precisely the dimension that functional and behavioural descriptions omit: the lived experience of meaning, of understanding, of communication. Whether that dimension is real in a way that functionalist and behaviourist accounts cannot capture is, again, the question of the hard problem applied to language — which brings the discussion full circle to Chapter 4.

Part V: The Future of the Questions

Chapter 13: Artificial General Intelligence — Concept, Prospect, and Philosophical Significance

The concept of artificial general intelligence (AGI, 通用人工智能) is philosophically slippery in a way that is not always recognised in technical discussions. The term is used to mark a contrast with narrow AI (狭义人工智能): systems that are highly capable within a specific domain but fail to generalise to tasks outside that domain. Deep Blue could defeat the world champion at chess but could not play checkers; AlphaFold predicts protein structures but cannot engage in philosophical conversation. AGI, by contrast, would be a system capable of performing any intellectual task that a human being can perform — general-purpose rather than domain-specific. The philosophical question is whether this contrast is principled or merely reflects the current state of technology. If the only barrier between narrow AI and AGI is more powerful hardware and more data, the distinction marks a practical threshold, not a conceptual one. If there are architectural or theoretical barriers — if generalisation in the relevant sense requires something that current architectures are principally incapable of — the distinction marks a genuine conceptual division.

Current positions in the debate over AGI’s prospects span a wide range. Optimists, represented by figures like Ilya Sutskever and Demis Hassabis, argue that the scaling laws governing large language models suggest that continued increases in model size and training data will produce systems that approximate human-level general intelligence, perhaps within years rather than decades. Their argument is essentially empirical: the trajectory of improvement in AI capabilities over the past decade has consistently exceeded predictions, and there is no clear theoretical reason to expect the trajectory to stop before reaching human-level general intelligence. Pessimists, represented by Gary Marcus and Melanie Mitchell, argue that current architectures have fundamental limitations that scaling cannot overcome: they lack genuine understanding, robust commonsense reasoning, and the kind of flexible generalisation from limited experience that characterises human intelligence. Their argument is also partly empirical — pointing to persistent failure modes in LLMs — and partly theoretical, drawing on cognitive science accounts of what general intelligence requires.

The philosophical significance of AGI, if it were achieved, is profound but not straightforwardly supportive of any particular position in the debates this course has examined. If AGI were achieved through an extension of current LLM architectures — through scaling, multimodal training, and architectural refinements — functionalism would receive a kind of empirical vindication: a system that performs the full range of intellectual tasks that human beings can perform would be, in the functional sense, intelligent. But this vindication would be compatible with the hard problem remaining open: we still could not infer from functional equivalence that the system has phenomenal consciousness. It would be compatible with the symbol grounding problem remaining a genuine issue: the system might still lack the kind of non-symbolic grounding that Harnad and Bender and Koller argue is necessary for genuine meaning. And it would be compatible with the Chinese Room argument remaining philosophically compelling: a system that passes the generalised Turing Test might still be, at the relevant level of description, a very sophisticated Chinese Room.

Dreyfus’s embodied cognition critique does not obviously require AI pessimism as a conclusion, even if it correctly identifies a structural limitation of disembodied symbol manipulation. The question is whether embodied AI systems — robots that have bodies, sensors, and actuators that engage with the physical world through learning — can achieve the kind of practical competence that Dreyfus associated with genuine intelligence. Recent progress in robotic learning — systems that learn to manipulate objects, navigate environments, and perform physical tasks through reinforcement learning and imitation — represents a partial implementation of the embodied paradigm. Whether it is the right kind of embodiment — whether it generates the kind of background understanding and practical coping that Dreyfus had in mind — is a philosophical question that requires careful analysis of what that kind of understanding consists in, not merely observation of behavioural performance.

The orthogonality thesis (正交性论题), developed by Nick Bostrom in the context of his work on superintelligence, holds that intelligence and goals are orthogonal: a system can be arbitrarily intelligent while pursuing arbitrarily chosen goals. A superintelligent AI need not have human values or human-like goals; its goals are set by its training process and architecture, and there is no reason in principle why a highly intelligent system must value what human beings value. The orthogonality thesis has been criticised on several grounds: some philosophers argue that genuinely rational agents must converge on certain goals, or that understanding human concepts requires something like human values. But the thesis represents an important philosophical challenge to naïve assumptions about the relationship between intelligence and values, and it has practical implications for how we think about the design, alignment, and governance of advanced AI systems — issues that PHIL 451 (AI Ethics, Law, and Governance) takes up from a normative perspective.

Chapter 14: Open Problems and the Philosophical Stakes

What philosophy contributes to AI research that technical work cannot provide is, at the most fundamental level, conceptual clarification (概念澄清): the identification and analysis of the concepts that structure the debate — intelligence, understanding, consciousness, creativity, agency, meaning — without which empirical progress and normative reasoning both proceed under the shadow of systematic confusion. When researchers argue about whether a system “understands” natural language, the argument is often partly empirical — about what the system can and cannot do — and partly conceptual — about what understanding requires. The conceptual component cannot be settled by accumulating more data or building more powerful systems; it requires the kind of careful analysis of concepts and their relations that is philosophy’s distinctive contribution.

The consciousness question (意识问题) remains genuinely open, and the Butlin et al. (2023) framework represents the most systematic recent step toward making it empirically tractable. By operationalising the predictions of major consciousness science theories into testable indicator properties, Butlin et al. create a research programme that can make progress as both our empirical understanding of AI architectures and our theoretical understanding of consciousness improve. The hard problem, however, resists this empirical approach at its core: even a complete account of which physical systems instantiate which indicator properties would not resolve the question of why any physical process is accompanied by subjective experience. This is not a reason to abandon the empirical programme — it can tell us a great deal — but it is a reason to maintain philosophical humility about the limits of what empirical evidence, however detailed, can establish.

The meaning and understanding question (意义与理解问题) is, in one important respect, closer to resolution than the consciousness question: there is strong philosophical consensus, across the frameworks examined in this course, that current LLMs do not understand language in the sense that human beings do. The grounds for this consensus differ: Searle locates the deficit in the absence of genuine intentionality grounded in biology; Bender and Koller locate it in the absence of non-textual grounding; Harnad locates it in the absence of non-symbolic perceptual and motor representations; Metzinger locates it in the absence of a transparent self-model. The convergence of these different frameworks on a negative verdict about current LLMs is philosophically significant, even if the frameworks disagree on what a system would need in order to understand. But what follows from this verdict for the practical and ethical status of LLMs is far less clear: one can maintain that LLMs do not understand language while also holding that they are genuinely useful tools, that they have some morally relevant properties, and that their outputs can be reliable in many contexts.

The creativity question (创造力问题) illustrates the way in which philosophical analysis has direct practical and legal consequences. Whether AI-generated works are creative — and, if so, in what sense — matters for copyright law, for the distribution of economic value from the creative industries, and for how we understand and regulate AI’s role in culture. Boden’s framework provides one set of analytical tools; other accounts of creativity, including those that ground it in the expression of individual subjectivity or in participation in cultural traditions, generate different verdicts. The philosophical work of specifying and adjudicating among these accounts is not merely academic; it will shape the legal and regulatory framework for AI-generated content in the coming decades. The Andersen v. Stability AI case, and the broader public debate over AI and creative industries, are ongoing — and they are awaiting the conceptual clarification that philosophy can provide.

The agency and responsibility question (能动性与责任问题) is practically the most urgent. AI systems are already deployed in consequential contexts — criminal justice risk assessment, medical diagnosis, autonomous vehicles, financial systems — where their outputs affect individuals’ lives, liberties, and wellbeing. The responsibility gap identified in Chapter 10 is not a hypothetical future concern; it is a present reality. When an algorithmic risk assessment instrument misclassifies a defendant and contributes to an unjust sentence, asking who bears moral responsibility is not an abstract philosophical exercise but a practical demand. The philosophical frameworks developed in this course — accounts of intentionality, agency, moral patiency, and moral responsibility — provide the conceptual tools for analysing these cases, even if they do not yield determinate verdicts in all instances.

This course has attempted to provide the conceptual vocabulary and analytical frameworks for thinking rigorously about the philosophical questions raised by artificial intelligence. It has not attempted to settle those questions — the consciousness question, the meaning question, the creativity question, the responsibility question — because they are not yet settled, and representing them as settled in either direction would be a disservice to the genuine difficulty and importance of what is at stake. What philosophy can offer is precisely this: the discipline to hold the questions open, to resist the pressures — from commercial AI optimism and from humanist AI pessimism alike — to reach premature closure, and to continue asking, with the rigour and patience that the questions deserve, whether machines can think, whether they can know, and what it would mean if they could. The normative consequences of these metaphysical and epistemological inquiries are taken up in PHIL 451 (AI Ethics, Law, and Governance); their genealogy and historical context are examined in HIST 415 (History of AI); but the conceptual groundwork — the analysis of the very concepts through which these questions are posed — is the enduring task of philosophy of artificial intelligence.