Research warns: Artificial intelligence could transform China's social credit system into a surveillance tool

A study from China warns that introducing artificial intelligence (AI) into the existing social credit system could be a mistake. Published in the Journal of Sichuan Normal University, the study indicates that this could turn the scoring system into a form of "mass surveillance tool." Notably, this research is funded by a project supported by the Chinese government aimed at examining social governance and institutional innovation.

Currently, China's social credit scoring system consists of various programs. It combines a financial credit system similar to Western credit scoring or reporting with regulatory blacklists that list individuals or companies sanctioned for violations. The system also includes local behavioral experiments that reward or punish individuals based on established criteria, along with a series of government accountability measures to monitor compliance and conduct of public officials. These programs are not connected into a single national score for each citizen, but collectively they may hinder individuals' opportunities for loans through automated decision-making.

Potential Impact of AI

The study suggests that introducing AI could transform this system into what many Western critics perceive as an Orwellian system. Is combining social credit scores with AI a good idea? According to a report, such AI-driven systems could analyze individuals' activities, financial histories, relationships, and other available data on a massive scale. Consequently, the system might judge someone as being similar to individuals who could pose a problem. In such cases, individuals may not engage in any wrongdoing, but the system could treat them differently. This would shift the system from being primarily reactive to more predictive, reminiscent of the plot of "Minority Report," but in a more extreme form.

Researcher Lin Jianxing warns that this potential shift could move from assessing actual behavior to profiling individuals and making administrative decisions based on algorithmically predicted risks. All these processes occur without the individuals' knowledge, and even without formally breaking the law, individuals might face "punishments" such as reduced opportunities or increased scrutiny. More importantly, this is not purely theoretical. A 2025 study from Stanford University examined existing social credit systems, investigating one city's social credit system in China, where adults start with a score of 1,000, which then fluctuates based on compliance with 389 reward and punishment rules.

These rules include political behaviors, such as appealing to government authorities. The study also found that individuals with the lowest ratings might face police surveillance. Compared to urban residents, rural residents appear to face unfair disadvantages, as they have fewer opportunities for rewards. Overall, the study found that while a nationwide "Big Brother" type system does not exist, certain local systems seem to impose penalties on individuals that go beyond simple criminal offenses. However, the introduction of AI could make everyone's concerns about this a reality.

Future Possibilities and Challenges

It should be noted that there is currently no evidence of plans to introduce a unified nationwide AI-driven social scoring system. However, China is already exploring the application of AI in public administration. The study serves as a warning that if such a situation arises in the future, the outcomes will not be satisfactory. On the other hand, proponents might argue that reliably identifying potential serious offenders could benefit society and help identify those most in need of assistance. If AI is to be introduced, the study emphasizes that it must be done in a way that does not punish the innocent and does not overly interfere.

Is this an impossible task? That is left for the readers to decide.

Risks of Introducing AI into the Social Credit System

Introducing AI into China's social credit system could have profound implications, particularly regarding personal privacy and surveillance. The study points out that this system could shift from reactive to predictive, leading to the punishment of the innocent. While there are currently no plans to implement a nationwide system, the risks in the future cannot be ignored as technology evolves. Balancing technological applications with individual rights will become a crucial issue in China's social governance.