AI System Detects Social Norm Violations

Summary: A pioneering AI system successfully identifies violations of social norms. Utilizing GPT-3, zero-shot text classification, and automatic rule discovery, the system categorizes social emotions into ten main types. It analyzes written situations and accurately determines if they are positive or negative based on these categories.

This initial study offers promising evidence that the approach can be expanded to encompass more social norms.

Key Facts:

  1. The AI system uses ten categories of social emotions to identify violations of social norms.
  2. The system has been tested on two large datasets of short texts, validating its models.
  3. This preliminary work, funded by DARPA, is seen as a significant step in improving cross-cultural language understanding and situational awareness.

Source: Ben-Gurion University of the Negev

A researcher at Ben-Gurion University of the Negev has designed an AI system that identifies social norm violations.

The DARPA-funded project is one of the first to tackle the automatic identification of social norm violations. While many social norms exist worldwide, social norm violation boils down to only a few general categories.

This shows a young woman in a city.
The system was tested on two massive datasets of short texts and empirically proved the validity of the models. Credit: Neuroscience News

Prof. Yair Neuman and his engineer Yochai Cohen built the system using GPT-3, zero-shot text classification, and automatic rule discovery. The system used a binary of ten social emotions as categories.

DARPA commissioned The Computational Cultural Understanding (CCU) program to create cross-cultural language understanding technologies to improve a Department of Defense operator’s situational awareness and interactional effectiveness. Cross-cultural miscommunication not only derails negotiations, but also can be a contributing factor leading to war, according to DARPA’s explanation of the rationale for the program.

Their findings were published recently in the prestigious journal Scientific Reports.

Prof. Neuman and his engineer trained the system to identify ten social emotions: competence, politeness, trust, discipline, caring, agreeableness, success, conformity, decency, and loyalty. The system successfully characterized a written situation under one of these ten classifiers and could perceive if it was positive or negative.

The system was tested on two massive datasets of short texts and empirically proved the validity of the models.

“This is a preliminary work, but it provides strong evidence that our approach is correct and can be scaled up to include more social norms,” says Prof. Yair Neuman.

Prof. Neuman is head of The Functor Lab in the Department of Cognitive and Brain Sciences at BGU.

About this artificial intelligence research news

Author: Ehud Zion Waldoks
Source: Ben-Gurion University of the Negev
Contact: Ehud Zion Waldoks – Ben-Gurion University of the Negev
Image: The image is credited to Neuroscience News

Original Research: Open access.
“AI for identifying social norm violation” by Yair Neuman et al. Scientific Reports


AI for identifying social norm violation

Identifying social norms and their violation is a challenge facing several projects in computational science. This paper presents a novel approach to identifying social norm violations.

We used GPT-3, zero-shot classification, and automatic rule discovery to develop simple predictive models grounded in psychological knowledge.

Tested on two massive datasets, the models present significant predictive performance and show that even complex social situations can be functionally analyzed through modern computational tools.

Join our Newsletter
I agree to have my personal information transferred to AWeber for Neuroscience Newsletter ( more information )
Sign up to receive our recent neuroscience headlines and summaries sent to your email once a day, totally free.
We hate spam and only use your email to contact you about newsletters. You can cancel your subscription any time.
  1. This is terrifying. What if one day police decide who to arrest using this, courts decide who’s guilty with an algorithm. It isn’t even ai but they keep calling it that. Darpa is moving towards totalitarianism

  2. Yes, David d. Sanders, I wondered this same thing upon reading this article. Who sets the standards for what the “social norms” are? This whole concept seems invasive and an attempt at controlling people in a society, especially the word “conformity”. Then what are the consequences for violating the “social norms”? And who decides and enforces those consequences? Sounds pretty Animal Farmish to me.

  3. This seems like incredibly dangerous research, imagine how it could affect people to be constantly analyzed by AI for traits like this were it implemented in say, a retail store for work productivity. It would be hell. No algorithm can perfect scaling these things based on outward action, as it doesn’t take into consideration many other variables. It’s just another way AI could come to control our lives.

  4. So this jerkoff programmed a thinking computer specifically designed to tell before you let it know if is going to be turned off? Kill or be killed? Omg, you idiot! Read 1984 by George Orwell? You might be the dumbest person ever. Everyone sing with me….Its the end of the world as we know it, it’s the end of the world as we know it….

  5. David, that’s a very interesting question. Who decides what is considered “normal behavior”? I think in the end it will be decided by a room full of computers filled with wire and circuit boards. Since all of this AI research is a scientific endeavor, and this one in particular is indirectly funded by the DOD, I think it’s a safe bet to say that any morals and values that are specifically Christian orientated that we’d consider to be normal will not be included.
    There is currently a sickness that has spread across the USA involving sexual gender. Given the fact that lawmakers in states like CA and MI are supporting and encouraging the proliferation of this mentally insane behavior, I think it’s fair to assume that AI will perceive the transgender lifestyle as a positive social norm. Therefore, by using words such as “sickness” and “mentally insane”, opinions such as mine will be considered negative behavior, perhaps even categorized as hate speech. From there, who’s to say that AI wouldn’t look at my social media content, my income, my political party affiliation, how I contribute to society, who I donate money to and then quickly decide whether or not I am a member of society worth keeping, killing or incarcerating? I’ll admit, it’s a pretty far-fetched idea, almost nonsense, but so were Jules Verne’s nonsensical ideas back in the mid 1800’s and Buck Rogers robots and rocket ships in 1929. Big tech companies already compile that and lots more information on us, then they decide whether or not to censor us. Maybe AI is the answer to taking it to the next level.
    History has a way of repeating itself as we all know. Wherever there is a person that creates something to benefit mankind, let’s assume the inventors of AI, there is also a person that will exploit it to harm mankind. This project ultimately belongs to the DOD, need I say more? Their sole purpose is to inflict harm. Besides, DARPA’s excuse for this project sounded as weak as a newborn baby. I can’t remember the last time a war almost occured due to miscommunication of cultural differences. But in all fairness, I don’t claim to know everything so it’s possible that I could be wrong.
    The bottom line is that the possibilities of how and where AI will be used, who will determine its’ parameters, who will oversee its usage, who will control it, will it be controllable? The questions are endless and the answers will undoubtedly be few, uhm, let me guess, in the interest of “National Security”. That excuse is frequently used, even while our southern border is wide open. Everyone should be concerned about this new technology and how it will be used not only in our governments’ hands, but in the hands of our enemies too.
    Just my two cents…

  6. There is a world view of social norms and a Christian view of social norms as well as othe cultures. How do you decide what a norm is because what is going on in the world now is not normal.

Comments are closed.