AI Liability and Human Accountability: Navigating the Legal Frontier of Intelligent Machines
In an era where artificial intelligence (AI) increasingly shapes the fabric of daily life, from self-driving vehicles to medical diagnostic systems, a fundamental question looms large: who is responsible when AI causes harm? As intelligent systems become more autonomous, the legal and ethical frameworks governing their use are being tested like never before. A recent scholarly article by Cai Xin from Shanghai University, published in Brand and People, offers a rigorous examination of this pressing issue, arguing that despite the sophistication of modern AI, it remains a product of human design—and thus, human responsibility must remain central to any regulatory approach.
The rapid integration of AI into consumer and industrial applications has brought undeniable benefits. Automation streamlines manufacturing, AI-powered diagnostics enhance medical accuracy, and intelligent transportation systems promise to reduce traffic fatalities. Yet, alongside these advancements, a growing number of incidents involving AI-related harm have sparked public concern and legal scrutiny. High-profile cases, such as fatal crashes involving autonomous vehicles, have exposed the vulnerabilities embedded within these systems. In 2016, for instance, a Tesla operating on Autopilot failed to distinguish a white tractor-trailer against a bright sky, resulting in a fatal collision. Investigations revealed that the system’s object recognition algorithms were insufficiently robust, and no adequate warnings or recalls were issued. This incident, among others, underscores a critical gap: while technology advances at breakneck speed, the legal infrastructure meant to govern it lags behind.
Cai Xin’s analysis begins with a clear-eyed assessment of the current state of AI development. She emphasizes that despite popular narratives suggesting the emergence of sentient or superintelligent machines, contemporary AI systems remain firmly within the domain of what experts call “weak AI.” These systems operate based on pre-programmed algorithms and machine learning models trained on vast datasets, but they lack consciousness, intentionality, or moral reasoning. Their behavior is not driven by desire or judgment but by code. As Cai points out, a medical robot cannot choose to stop and assist a pedestrian in distress—not because it lacks compassion, but because such a decision falls outside its operational parameters. The so-called “intelligence” in AI is not cognitive; it is computational.
This distinction is crucial when considering liability. Some scholars have proposed granting AI systems legal personhood—either full or limited—to hold them accountable for their actions. Ideas such as “electronic personhood” or “digital legal entities” have gained traction in certain policy circles, particularly in the European Union, where debates over robot rights have occasionally surfaced. However, Cai firmly rejects this line of thinking. She argues that attributing legal responsibility to AI itself is not only philosophically flawed but also practically dangerous. If machines can be deemed responsible, there is a risk that human developers, manufacturers, and operators may evade accountability by shifting blame onto the technology they created.
Moreover, the practical implications of AI personhood are untenable. Legal liability typically involves the capacity to own property, pay damages, or suffer penalties. An AI system, being a non-sentient artifact, cannot possess assets or experience punishment in any meaningful sense. Imposing fines on a machine would be symbolic at best and economically nonsensical at worst. Even in criminal law, where punishment serves both retributive and deterrent purposes, penalizing an algorithm would have no effect on future behavior unless humans modify the underlying code. Therefore, Cai concludes that the pursuit of AI personhood distracts from the real issue: ensuring that the humans behind the technology are held to appropriate standards of care and oversight.
Instead of chasing futuristic legal fictions, Cai advocates for a return to foundational principles of product liability. She asserts that AI should be treated not as a new form of life but as an advanced type of product—one that combines software, hardware, and data-driven functionality. Like any other product, AI systems must meet safety standards, undergo rigorous testing, and be subject to post-market surveillance. When defects lead to harm, the responsibility should fall on the entities that designed, produced, distributed, or misused the system.
This approach aligns with existing legal doctrines, particularly in tort and consumer protection law. In traditional product liability cases, manufacturers can be held liable under three main theories: design defects, manufacturing defects, and failure to warn. Cai argues that these same categories apply to AI. A design defect might involve flawed decision-making algorithms—for example, an autonomous vehicle that consistently misjudges distances in low-light conditions. A manufacturing defect could arise if a sensor is improperly installed, leading to inaccurate environmental perception. And a failure to warn occurs when companies market AI systems as fully autonomous when they are, in fact, only semi-autonomous, thereby misleading users about the level of human supervision required.
One of the most significant contributions of Cai’s work is her emphasis on the role of developers as key liability holders. Unlike traditional manufacturing, where engineers design physical components, AI development involves creating complex software architectures that learn and adapt over time. The developers who write the algorithms, select training data, and define performance metrics wield immense influence over how AI systems behave in real-world scenarios. Because of their specialized knowledge, they are in the best position to anticipate risks and implement safeguards.
Cai calls for stricter professional standards within the AI development community. Just as medical doctors and civil engineers are licensed and held to ethical codes, AI developers should be subject to certification requirements and continuing education. In high-stakes domains such as healthcare, transportation, and defense, the consequences of AI failure can be catastrophic. Therefore, only qualified professionals with proven expertise should be allowed to design and deploy such systems. Furthermore, developers must maintain ongoing responsibility even after deployment. Continuous monitoring, software updates, and prompt responses to emerging risks are essential components of responsible AI stewardship.
Manufacturers, too, have a critical role to play. While developers focus on the software side, manufacturers ensure that the physical components—sensors, processors, power supplies—function reliably and do not interfere with AI operations. A malfunctioning camera or a delayed signal transmission can compromise the entire system, regardless of how well the algorithm performs. Thus, manufacturers must adhere to stringent quality control protocols and verify that their products meet both national and international safety benchmarks.
Sales and marketing channels also bear responsibility. Misleading advertising can create dangerous misconceptions about AI capabilities. For example, promoting a driver-assistance system as “fully autonomous” may lead consumers to disengage from driving tasks, increasing the likelihood of accidents when the system encounters an unexpected situation. Cai stresses that sales representatives and advertising teams must provide accurate, transparent information about the limitations and operational boundaries of AI products. Regulatory bodies should enforce strict guidelines on AI-related claims to prevent deceptive practices.
Users, while generally less knowledgeable than developers or manufacturers, are not entirely absolved of responsibility. In many cases, AI systems require human oversight, especially in transitional phases where automation is partial. If a user ignores clear warnings, disables safety features, or uses the system in ways contrary to instructions, they may share liability for any resulting harm. However, Cai cautions against placing undue burden on end-users, particularly when interfaces are poorly designed or when warnings are buried in technical jargon. The principle of reasonable reliance should guide user expectations: if a product is marketed as safe and easy to use, consumers should not be expected to anticipate hidden flaws.
Perhaps the most forward-looking aspect of Cai’s analysis is her call for public law intervention when AI-related harm reaches a certain threshold of severity. While civil litigation can address individual cases of injury or property damage, it may be insufficient when systemic risks threaten public safety or societal stability. In such instances, criminal law may need to step in. If a company knowingly deploys a defective AI system—say, one with a known flaw in its collision avoidance algorithm—and that flaw leads to multiple fatalities, the executives who authorized the release could face criminal charges for reckless endangerment or manslaughter.
Cai draws on the work of Japanese legal scholar Haruo Nishina to illustrate how criminal liability evolves in response to technological change. When a new technology introduces unprecedented risks, society demands accountability. The fear and uncertainty generated by AI accidents can erode public trust, disrupt markets, and hinder innovation. By establishing clear legal boundaries and enforcing them through criminal sanctions when necessary, governments can restore confidence and guide responsible development.
To support this framework, Cai recommends several concrete policy measures. First, regulatory agencies should establish higher entry barriers for AI development and deployment. This includes mandatory licensing for AI firms, mandatory training for operators in sensitive fields, and standardized testing protocols before market release. The recent adoption of China’s GB/T 40429-2021 standard for automated driving levels is a step in the right direction, providing a clear classification system that helps differentiate between driver assistance and full autonomy. Similar standards should be developed across other sectors.
Second, liability regimes must be clarified to avoid ambiguity. Courts need consistent guidelines for determining fault in AI-related cases. Should the focus be on the developer’s algorithm, the manufacturer’s hardware, the seller’s marketing claims, or the user’s behavior? Cai suggests a tiered liability model, where responsibility is apportioned based on each party’s degree of control and foreseeability of harm. This would allow for fairer outcomes and discourage risk-shifting strategies.
Third, public oversight mechanisms should be strengthened. Independent auditing bodies could review AI systems before and after deployment, ensuring compliance with safety and ethical standards. Whistleblower protections would encourage insiders to report potential dangers without fear of retaliation. And transparency requirements—such as open-access logs of AI decision-making processes—could aid in post-incident investigations.
Ultimately, Cai’s argument is grounded in a deep respect for both technological progress and human dignity. She does not advocate for halting AI development; rather, she calls for a balanced, pragmatic approach that maximizes benefits while minimizing risks. The goal is not to stifle innovation but to channel it in directions that serve the common good. By reaffirming the product nature of AI and anchoring accountability in human actors, society can navigate the complexities of the digital age with greater clarity and justice.
As AI continues to evolve, so too must our legal and ethical frameworks. The questions raised by Cai Xin are not merely academic—they are urgent and practical. From boardrooms to courtrooms, from research labs to living rooms, the way we think about responsibility in the age of machines will shape the future of technology and society. Her work serves as a timely reminder that no matter how intelligent our tools become, the duty to use them wisely remains ours alone.
Cai Xin, Shanghai University, Brand and People, DOI: 10.19653/j.cnki.brand.2021.22.067